Some reading material > Kubernetes > Wildcard certs

Wildcard certs

This is NOT representative of what I do at work (mostly for my home hobby cluster).

Once you have NGINX set up, each of your services can sit behind an Ingress with a host resembling something close to myservice.mydomain.com. That’s great and all, but you’ll find that your connections aren’t secured until you configure TLS.

Some understanding

You’ll need to pass a DNS-01 challenge. The challenge itself is run by servers from a certificate authority (CA) such as Let’s Encrypt. Those servers expect your web server to be reachable from whatever IP address your domain points to.

Clusters with a public IP

It’s fairly straightforward to secure your services via cert-manager as the Let’s Encrypt servers will be able to reach your node clusters (which play the role of your web server).

Clusters without a public IP

Solutions like cert-manager won’t work out of the box here. Essentially you’ll need to stand up a web service apart from your cluster that has a static IP for the sole purpose of getting the pem certs requried to generate your TLS Secrets. AWS, Google Cloud, etc., are able to provide here.

Note: the following instructions are tested on a server running Ubuntu 20.04:

Set up a server with a publicly accessible static IP address either in the cloud or some other hosting provider you trust.
Create an A record pointing to its static IP. Below is an example:

sample_dns

Replace the IP address Value with your server’s static IP.

Install the following in the same server:

certbot
- used to get tls cert from Let’s Encrypt
nginx
- used as the installer plugin when combining plugins
kubectl
- used to locally render the secret to copy over to our private cluster

In your publicly available server, create an index.html file that will display something.

# server with publicly available static ip
cd /var/www
sudo mkdir tutorial
cd tutorial
sudo "${EDITOR:-vi}" index.html

<!-- index.html -->
<!-- This is what the nginx server will return -->
<!doctype html>
<html>
<head>
    <meta charset="utf-8">
    <title>Hello, Nginx!</title>
</head>
<body>
    <h1>Hello, Nginx!</h1>
    <p>We have just configured our Nginx web server on Ubuntu Server!</p>
</body>
</html>

In your publicly available server, configure NGINX as shown below:

# server with publicly available static ip
cd /etc/nginx/sites-enabled
sudo "${EDITOR:-vi}" tutorial

server {
       listen 80;
       listen [::]:80;

       server_name *.mydomain.com;

       root /var/www/tutorial;
       index index.html;

       location / {
               try_files $uri $uri/ =404;
       }
}

The above config is just a simple nginx server that listens on a wildcard path returning a simple response.

In your publicly available server, use certbot with the manual plugin as the authenticator and the nginx plugin as the installer to grant your wildcard cert:

# server with publicly available static ip
certbot run -a manual -i nginx -d *.mydomain.com

When prompted, create a TXT record in your DNS like below:

sample_txt_record

Be careful to not put your domain in as the Host in the TXT record

Wait until your TXT record propogates through the Internet’s DNS servers. You can use a site like https://mxtoolbox.com/SuperTool.aspx
When you successfully acquire your wildcard cert, create the Kubernetes TLS secret in your publicly available server, like below:

# server with publicly available static ip

# only sudo has access to the directory we need to get to
sudo -s
cd /etc/letsencrypt/live/mydomain.com/
# create the TLS secret
kubectl create secret tls mydomain-dot-com-wildcard-tls --cert fullchain.pem --key privkey.pem --dry-run -o yaml > mydomain-com-wildcard-tls.yaml
# move the yaml file to someplace you'll be able to scp from on your local machine, likely your user's home directory
mv mydomain-com-wildcard-tls.yaml /home/myuser/

Move your secret to your local machine

# local machine

scp my_user@my_server_ip:/home/myuser/mydomain-com-wildcard-tls.yaml .

You likely want to change the namespace of your secret so that when you apply it it’s available for your service to reference by name. Change metadata.namespace accordingly, for example:

# mydomain-com-wildcard-tls.yaml
...
metadata:
  namespace: my-namespace
...

Apply the secret

kubectl apply mydomain-com-wildcard-tls.yaml

You can now guard your service using your newly created cert. Here’s an example of TLS termination using ingress-nginx
Oh yeah, you probably want to delete your NGINX web server instance so as not keeping your TLS secrets “exposed”….