Quick Guide to Automating TLS Endpoints in Kubernetes

Setting TLS endpoints in Kubernetes can be a Sisyphean task. What if you could make it a ‘set and forget’ kind of task instead? Well, now you can.

At Rookout, we use k8s to handle all of our services. We wanted to automate the process of adding new services that require SSL and a custom domain name. As part of our CI/CD process, we also wanted to allow developers to deploy a full env from their branch, which entailed seamlessly creating new domains. For example, a developer working on a branch named “best-feature-ever” should test it at best-feature-ever.rookout-test-domain.com.

TL;DR we documented all of our steps so feel free to jump right in: https://github.com/Rookout/k8s-auto-dns-and-tls-guide

What are the benefits of automating TLS endpoints in Kubernetes?

• It allows your DNS to be configured as code (infrastructure as code)
• It allows you to handle SSL automatically
• It allows you to easily create a new environment with a domain, so you can test features/branches more easily, in a more unified way
• It automates multiple processes that everyone hates, making them easy to do.

Follow these 7 steps to automate TLS endpoints

It took me about 2 days to crack the complete process, but this time-saving post should allow you to automate your TLS endpoints in just 15 minutes of work, give or take, with the help of a few open-source tools. Once you complete the process described in this guide, fully configuring a new DNS with a SSL certificate in your Kubernetes cluster will take only a few seconds!

* Note:  We assume that you’re using GKE domains. If not, you might need to change a few things.

1. Clone our demo repo, create a Kubernetes cluster, and add yourself as cluster admin

export EMAIL=YOUR_EMAIL
Git clone https://github.com/rookout/k8s-auto-dns-and-tls-guide
gcloud container clusters create k8s-aut-dns-and-tls-guide –num-nodes=2 –scopes https://www.googleapis.com/auth/ndev.clouddns.readwrite
kubectl create clusterrolebinding cluster-admin-binding –clusterrole cluster-admin –user ${EMAIL}

Make sure you have –scopes https://www.googleapis.com/auth/ndev.clouddns.readwrite

2. Install Helm

Helm is a tool that streamlines installation and management of Kubernetes applications. Think of it as apt/yum/homebrew for Kubernetes.
Helm has two parts: a client (Helm) and a server (Tiller). Tiller runs inside of your Kubernetes cluster and manages releases (installations) of your charts*. Helm runs on your laptop, CI/CD, or wherever you want it to run.

*Charts are curated application definitions for Kubernetes Helm

First, install Helm on your laptop:

curl -o get_helm.sh https://raw.githubusercontent.com/kubernetes/helm/master/scripts/get
chmod +x get_helm.sh
./get_helm.sh

Next, install Tiller on your cluster:

kubectl create serviceaccount –namespace kube-system tiller
kubectl create clusterrolebinding tiller-cluster-rule –clusterrole=cluster-admin –serviceaccount=kube-system:tiller
helm init –service-account tiller –upgrade

3. Install our ‘hello world’ example

helm install hello-world –name hello-world –dep-up -f hello-world/values.yaml

Verify that you’ve installed it:

kubectl get pod

And now, really check that it’s up:

kubectl port-forward POD_NAME 8080:80

4. Install external DNS controller

To simplify matters, we’ve created another Helm chart for the infra stuff — the things that should be created just once for each cluster, not the ones that must be deployed for every new service.

export DOMAIN=YOUR_DOMAIN
export DOMAIN_NAMESPACE=YOU_DOMAIN_NAMESPACE
helm install infra –name infra –set externalDns.enabled=true –set domainNamespace=${DOMAIN_NAMESPACE} –dep-up

*endpoint.fake-domain.com -> DOMAIN=endpoint.fake-domain.com, DOMAIN_NAMESPACE=fake-domain.com

5. Install Nginx ingress controller and config the Nginx

An ingress controller is a daemon, deployed as a Kubernetes Pod, that watches the apiserver’s /ingresses endpoint for updates to the ingress resource. Its job is to satisfy requests for ingresses. To deploy an Nginx controller for each Helm, deploy just as we added it to requirement.yaml:
helm dependency build hello-world

helm upgrade –install hello-world hello-world –set nginx.enabled=true –set domain=${DOMAIN} -f hello-world/values.yaml

This step sets up both the ingress and the controller.

6. Install cert-manager

cert-manager is a Kubernetes add-on that automates management and issuance of TLS certificates from various issuing sources. It periodically ensures that certificates are valid and up to date, and renews certificates at the appropriate time before they expire.

helm install –name cert-manager stable/cert-manager –namespace cert-manager
helm upgrade infra infra –set externalDns.enabled=true –set certManager.enabled=true –set email=${EMAIL} –set domainNamespace=${DOMAIN_NAMESPACE}

7. Wrap if it all up

Once the infra part of the system is deployed (cert manager + external dns + nginx controller), we have all the components needed to create new domains with SSL on the fly. All that’s left to do is to create the certificate and add it to the ingress.

You can add it automatically…

helm upgrade –install hello-world hello-world –set nginx.enabled=true –set domain=${DOMAIN} –set tls.enabled=true -f hello-world/values.yaml

And….. we are done!

Now you can deploy as many services as you want, each with its own custom DNS and SSL, without repeating this process ever again! I hope you find this useful and the steps are easy to follow. The reward is clear — using an automated process instead of a manual one is a habit that most people are probably very quick to adopt! 🙂
Feel free to send me comments or let me know if I missed something — I’d love to hear your feedback.