The DevOps teams have been working hard over the last few months, looking at where we can improve our processes and toolsets to empower engineers at WorldRemit. We aim to enable engineers to reach our goals faster, provide a more enjoyable development experience, and ship higher-quality features for our customers. To achieve this, we have been particularly focusing on; application routing (Netscaler), application deployments (Jenkins) and our documentation.
Before we dive in and unveil our plans, it is useful to remind ourselves that DevOps is not a set of tools or scripts. DevOps is a discipline aimed at shortening the software development lifecycle and helping companies deliver higher quality software, faster. Even with the best tools, DevOps is just another buzzword if you don't have the right principals and culture. At its core, DevOps relies on seven principals*:
- Collaboration between the roles of development and operations
- Empower developers to have end-to-end responsibility
- Encourage continuous improvement
- Be customer focused
- Empower teams to be autonomous
- Reduce silos
With this groundwork laid, we chose to look at the three areas listed above (application routing, application deployments and documentation), for closer review. We looked at how we are performing in each area, where we might be able to improve our processes, and if there are any tools we can add/replace/remove to make our goals possible.
For the last 3+ years, WorldRemit has relied on Citrix Netscaler ADC and Nginx (in Kubernetes) to facilitate application ingress and routing. Neither of these tools suited well to our evolving needs. Netscaler is cumbersome, overly complicated, relies on DevOps for modifications, is unreliable, and lacks many of the newer features our platform is going to soon require. Nginx in Kubernetes is great as it is mostly self-service, but it promotes insecure practices and again lacks a lot of the more complex features our platform is going to soon require.
The goals which were set for our new routing infrastructure were that it would; enable future WorldRemit product needs (namely the IAM and API-first projects), be self-service for all engineers, be more secure than our current solution, require minimal new infrastructure, and promote continuous delivery.
With the help of our IAM partners RAIDIAM, the decision was made to move our routing infrastructure to a modern API gateway solution. Out of the many possible products, Kong was selected as the most suitable.
Kong is an API gateway solution which is already well established in the market and used in some of the largest firms. We have chosen to use the Enterprise version, providing us with all the functionality of Kong as well as support from their team.
Our deployment of Kong replaces Nginx as the way to ingress traffic into our Kubernetes clusters. By sitting at the edge of our clusters, Kong not only removes the need for both Netscaler and Nginx, but it also reduces the amount of extra infrastructure we need to run and allows Kong to be managed using the same methods as any other WorldRemit service.
Kong is the primary enabler for the IAM project (together with Istio), providing functionality to support the OpenID flows and components needed to make this project possible.
From a developer perspective, services can be onboarded to and configured within Kong using Kubernetes YAML manifests similarly to how Nginx ingress was. This allows easy self-service of routing decisions, rewrites, responders, rate-limiting and many other functions.
Kong also exposes an admin UI which will be available soon on Okta. This interface will be read-only (as all updates are pushed as code via the aforementioned YAML manifests) but gives developers useful insights into their services.
For future use, Kong also provides an excellent developer portal solution, allowing developers to publish, locate and access APIs with ease. This will form a core component of the API-first project, and as with the routing rules, can be set up and configured as code by developers; self-service.
Over the last two years, application deployments have come a long way at WorldRemit. We have moved from an unordered collection of TeamCity and Octopus deploy jobs to a small number of unified Jenkins pipelines (with Bitrise for mobile). Jenkins has taken us a long way and has enabled us to grow the number of services we deploy rapidly in the last 12 months. However, we are always aiming to improve and strive for continuous deployments.
The issue we and many companies have with Jenkins is that it was built as a CI system, to host simple pipelines intended for the building and testing of software. It isn't (nor has it ever been) an efficient deployment platform. It is possible to write deployment steps for Jenkins, but these always end up as delicate customisations built of scripts and groovy.
In late summer DevOps started to look for an alternative deployment orchestrator. A tool to provide deployment-specific functionality, empower developers to manage their deployments and foster continuous deployments. We considered several alternatives during this process, and the eventual pick is Harness.
Harness is a continuous deployment tool, it's job is to take the output artifacts generated by CI (which is still Jenkins) and deploy them into our infrastructure. Harness is pipeline and workflow-based, similar to Jenkins. However, Harness focusses on deployments with almost all the functionality we would need available as standard.
Harness has a rich UI which will be available in Okta for all engineers allowing you to trigger, track and debug deployments.
Continuous verification steps mean that Harness can hook into AppDynamics (and other monitoring systems if needed), to determine if a deployment has been successful. When errors or performance issues are detected after deployment, Harness can automatically initiate a rollback recovering system health without input.
Clearer error states and logging should help developers to diagnose deployment issues on their own.
We hope this gives you an idea of the DevOps initiatives coming in 2021. We are excited to see how these new tools and processes empower our engineers to deliver reliable software, faster for our customers.