Sign in Subscribe

Docker and Kubernetes

Tyler Eon

• 4 min read

It’s possible to now have Docker Desktop run a single-node Kubernetes cluster, which is wonderful for anyone that needs to emulate the Kubernetes environment for their development purposes. After all, if you’re building an application that will be deployed via Kubernetes, it is helpful to have your development environment mirror your production environment when testing certain features. However, it’s not all sunshine and roses…

First, Docker Compose

I am a huge fan of Docker Compose. I advocate its use whole-heartedly for local development. My career has primarily been backend development, meaning that I have to deal with at least the application and a database, sometimes a caching service. And that’s just the baseline. There are times when I’ve also needed to run elasticsearch, Kafka, and others.

The first big step toward having a simple and consistent development experience was Docker’s ability to easily “containerize” an application and run it locally. The next big step was Docker Compose and its ability to run multiple related images as a set of services that composed a full application stack.

I won’t go in-depth about Docker Compose, but I will just say that it has sped up the onboarding and development processes for every team I’ve been on that has used it. I can’t imagine life without something like Docker Compose now that I’ve been spoiled by it.

However, Docker Compose is primarily meant as a developer tool, not necessarily to spin up environments that mirror production, meaning there are serious limits on what you can accomplish through its configuration. Moreso with Kubernetes, which has a very deep set of customization options, almost none of which are exposed by Docker Compose.

Since I needed to (a) deploy in a Kubernetes environment and (b) have access to Kubernetes’ Downward API, I began searching for Docker Compose alternaties…

How about Docker Stack

At this point you might stumble across Docker Stack, like I did. Although originally intended to be used with Docker Swarm it can also be used with Kubernetes. If you have the latest Docker Desktop installed then you can enable a single-node Kubernetes cluster just by clicking a checkbox in the settings menu.

In the case of Kubernetes, Docker Stack takes a Compose config and uses it to generate a series of Kubernetes resources. What’s better, you can easily update your running cluster by re-running docker stack deploy. You can also easily tear it all down with docker stack rm. It really makes emulating a Kubernetes deployment as simple as one can imagine. Better yet, it just works.

Unfortunately, it still uses a Docker Compose config, which means you still get all the major restrictions that come with it. There are some additional options that Stack accepts which are normally ignored by Compose, such as the replicas option to deploy multiple pods of a service, but ultimately they are not Kubernetes-specific.

So if Docker Compose and Docker Stack can’t help me, what’s next?

Creating Kubernetes resource YAMLs by hand is always an option, but it’s easy to mess that up, and I didn’t want to try and map my working Compose config to a working set of Kubernetes YAMLs if I didn’t need to.

Enter Kompose

Eventually I gave up trying to “run” my Compose config directly and instead focused on tools that could convert a Compose config into one or more Kubernetes YAMLs. That’s when I found Kompose.

At first glance, Kompose looks like a Kubernetes-specific version of Docker Stack. You can kompose up and kompose down, passing in a Compose config file, and it just works. However, you can also use the kompose convert command to turn that config file into Kubernetes YAMLs. Great!

There were only two minor problems for my particular use-case:

  1. Volumes that bind mount to the local file system instead create a persistent volume that does not point to the local file system.
  2. The “shared” network I had to avoid explicitly exposing ports to some of my services (e.g. my database) did not work as expected, meaning anything without explicit port exposure wasn’t accessible in the expected manner.

To solve the bind mounting issue, I simply deleted the persistent volume claim resource and associated metadata, then I added hostPath configuration to my deployment resource. See the official documentation for details on how to configure that.

Instead of trying to solve the issue of a “shared” network, I simply exposed the port of my database in my Compose file and re-ran kompose convert. That generated a service resource in addition to the deployment resource, which was good enough for my use-case.

After that, I just needed to add the extra metadata to use the Downward API in my application deployment resource.

Final Thoughts

I will still use Docker Compose for a majority of my local development. It’s an extremely quick and simple method for getting a working environment running, in addition to creating local bind mounts so that you can edit code on your host machine and have the changes show up in your running container simultaneously.

However, I can now use Docker Stack and Kompose when I need to test my app in the context of Kubernetes. Specifically,

  • Docker Stack is useful when I need to have my service run in Kubernetes without any Kubernetes-specific configuration options.
  • Kompose is useful when I need to convert my existing Compose config into Kubernetes YAMLs so that I can extend them with specific configuration options.

Irony

I have another post about Using GitLab AutoDevOps to build and deploy my application. If you’ve seen that post, or already know what GitLab AutoDevOps is, you can skip to the next paragraph. In short: AutoDevOps uses a custom GitLab pipeline with a custom Helm Chart to test, package, and deploy an application to Kubernetes. Ultimately, making large changes to how steps in the pipeline are executed typically requires using a custom pipeline step or a custom Helm chart.

Because I did not wish to use (read: was not having success with using) a custom Helm chart, I ended up finding a workaround that didn’t involve using the Kubernetes Downward API. Because all containers have the same hostname as the pods they run on, it’s possible to use the hostname -i (or similar) command to get pod’s IP address. I use that to set an environment variable on the command line when executing the application start-up script.

Although I didn’t ultimately get around to using Downward API in production, I was able to test out the application clustering in Kubernetes that I was hoping to do. So it wasn’t a total loss. And I certainly learned more about Kubernetes resource configuration along the way.