Why On Earth Did We Choose Jenkins For 2019?
In this article, I’ll try to explain why the hell Rookout, a relatively new SaaS company, chose to use Jenkins, and what are the big advantages that make Jenkins so great even now, eight years in.
In the last few years, the devops world has changed rapidly: We’ve moved from one big monolith architecture into microservices, from bare-metal servers into the cloud, and then into containers, and from there onto Lambda and Kubernetes. Impervious to all of these changes, Jenkins stands tall, with a community that just keeps growing. Jenkins has been around since 2011 (maybe even before, which counts for several lifetimes in this industry) and it ain’t going nowhere.
Here at Rookout, we thought hard about which CI/CD is the right one to use. Should we choose multiple SaaS CI/CD solutions or just use plain old Jenkins?
Spoiler alert: We chose Jenkins (with some CircleCi). Not because the other tools were bad but mainly because of the unmatchable flexibility and code-reuse Jenkins offered us.
The following are some of the Jenkins features and capabilities that played a starring role in our decision:
Don’t repeat yourself (Jenkins shared libraries)
What is it?
An independent repository where you can define chunks of reusable code and import them into your Pipelines as needed.
Why do we need it?
As you adopt Pipelines for more projects across an organization, common patterns are likely to emerge. Creating libraries to enable sharing of discrete parts of pipelines among projects reduces redundancies and keeps code “DRY” and compliant with best practices for software development.
How does it look in real life?
Jenkinsfile examples for two of our microservices:
To add a new microservice in Rookout all you need to do is to write three lines since the basic pipeline code is available in the shared library.
Run one test, two ways
What is it?
The ability to run the same test, either by triggering a commit or by cron.
Why do we need it?
If you have a good quality e2e test, you might want to run it on different occasions.
How does it look in real life?
- Run this test as part of the deploy pipeline
- Run the test every 10 min to make sure the system is working
Using Jenkins, you can just choose the trigger option you want and set it.
Run parameterized jobs
What is it?
Start the same job with different parameters each time.
Why do we need it?
This capability could be valuable in the following cases, as well as many more:
- Running the CD pipeline with a different configuration to remove some safeties or to change flows
- Running a test/cron on a different environment
- When deploying a feature branch
How does it look in real life?
Create complex Pipelines
What is it?
With Jenkins, you can trigger one job directly from another, and even have the second job wait for the first one to finish before it starts.
Why do we need it?
It allows you to automate your workflow. For instance, instead of doing the manual work of running tests each time a new version goes up, you just trigger the test pipeline when the deploy pipeline finishes.
It also offers you a simple way to sync tasks. So, for instance, if a lot of people push to master branch, it will run the deploys to staging one at a time.
How does it look in real life?
As an example, let’s see how multi repo CI/CD looks:
- For each repo, a commit to the master branch builds the docker image and the helm package.
- It then triggers the “deploy to staging pipeline” job, which (not surprisingly) deploys to staging.
- If we pushed to two repos at the same time, for instance, to backend and frontend, then the first job would trigger the “deploy pipeline” while the other would simply wait in the Jenkins queue.
- We can add another job trigger later. If the deployment to staging succeeds, it will deploy to production, and so on.
Rookout “deploy to staging” Pipeline example:
Use code instead of a template language (Groovy vs YAML)
What is it?
When it comes to a choice between a programming language and a DSL, I’d opt for the DSL every time. Of course, others may feel differently.
Why do we need it?
The advantages of using Groovy rather than YAML include:
- Code reuse functionality (DRY again)
- The pipeline is more readable (using function)
- If/else/try/catch loops are part of the code.
How does it look in real life?
Use as a cron server
What is it?
An easy and useful way to schedule jobs.
Why do we need it?
Many, if not most of us, have cron jobs set up to do light housekeeping and other small tasks on the various servers that are scattered in different locations.
While these tasks are relatively minor, they need to be done. Yet cron does not make it easy to keep track of these tasks and make sure they’re getting done: No single point of management covers all servers; there’s no simple, yet reliable logging; and there are no notifications or other ways of easily confirming if tasks succeeded.
How does it look in real life?
Run dynamic agents in a K8S
What is it?
The Jenkins K8s plugin helps you automate scaling agents by letting you run dynamic agents in Kubernetes. It creates a Kubernetes Pod for each agent, as defined by the Docker image, and lets you use your own Docker images as the agent.
Why do we need this?
As we add more and more jobs to Jenkins, a single machine just can’t handle the workload. To address this challenge, Jenkins supports spawning “slaves” — machines other than the Jenkins master that can handle the jobs.
The k8s plugin offers us the ability to super-scale with no need to orchestrate anything manually.
If you’re a novice, check out this super cool starter guide for setting up a Jenkins CI/CD pipeline with K8s.
How does it look in real life?
We run the `sh` command in our own docker-image, which is a pod in K8s
Performance and security
What is it?
The fact the CI is running inside your cloud has some major advantages:
- Docker pull and push is a lot faster. In fact, we saw 10X improvement vs a hosted solution
- Spinning up slaves is fast! If each step in your CI/CD is using a dedicated docker image (slave), then spinning up those docker image slaves (k8s pod for us) is also a lot faster vs hosted CI — a few seconds in Jenkins vs +-30 in hosted solutions
- You can use your cloud provider security policy (IAM rules and similar)
Why do we need this?
Bet you can figure this one out!
How does it look in real life?
Mighty good!
The Jenkins community keeps on giving
The plugins, features, and capabilities above are specific examples of great things you can do with Jenkins. But the real thing that makes Jenkins great is the Jenkins community, which is HUGE and just keeps on growing. And since it is so huge and so active, everything you might ever need has probably already been implemented or tackled by someone else before you. This means that when you need a Jenkins plugin for almost anything, you can probably find it.
And I mean anything!
Let’s sum it all up
So even today, when a lot of good CI-as-a-service solutions are available, Jenkins continues to lead the way with its strength (and the strength of the huge community behind it) to customize almost everything.
The main reason we choose Jenkins and keep using it is its ability to do many things: Jenkins serves as a CI service, a cron service, and a CD service, all using the same language and by reusing a lot of code and logic.
That all means less technology to keep track of. If it weren’t for Jenkins, we would need to use 3 different SaaS, all with its own configs and learning curve.
So now that we have presented the rosy face of Jenkins, it’s important to admit that it presents some serious issues as well, especially for novice users. Stay tuned for our next blog post on Jenkins’ dark side: Why not choose Jenkins in 2019?
Related posts
