Deploying services with Docker Compose and Cloudflare Tunnels

Over the past few years, I have had the privilege to deploy code into several environments ranging from bare-metal, VPS, PAAS, and tier-one cloud providers. There are two major improvements in the past few years that have dramatically changed how I deploy software. The first major shift is the broad usage of containers. The second is the ability to declaratively define the infrastructure and let an orchestrator handle running the containers.

If I have a large number of containers I am probably gravitating toward Kubernetes. For smaller deployments, I might be using something like Google Cloud Run or a PAAS platform. I recently launched my mastodon server and have found that Docker Compose is a viable option for more than just local development. In this article, I will discuss why I feel Docker Compose is still a solid option for small stacks of containers.

Docker Compose

Compose is a tool from Docker that allows you declaratively specify multiple containers and run them together. Compose is often used for local development but I find it is an excellent tool for deploying and managing services in a variety of small use cases. You can specify your infrastructure in a YAML file and then apply that configuration to any instance of Docker running locally or even remotely.

If you want to see an example of a small application deployment check out my previous article about how I deploy a mastodon server. This Compose configuration runs 7 different containers in an isolated network and exposes the server to the public via a custom domain.

I also utilize Compose to run email backups, Plex, Wireguard, and Netdata.

CI/CD with Docker Compose

To manage my Compose configurations I keep them in source control and utilize Gitlab CI/CD to apply changes to my remote servers. To apply a Compose configuration to a remote machine I utilize a lesser-known feature of Docker called Remote Docker Hosts. This feature allows you to control an instance of Docker running on a remote machine. If you have SSH access to a remote server running Docker you can control that instance remotely. Below is an example of a Gitlab CI/CD job that applies a Compose configuration to a remote server.

deploy-node02:
  image: alpine:latest
  stage: deploy
  variables:
    # This variable sets the `docker` command to connect to an instance on a remote host
    DOCKER_HOST: ssh://[email protected]
  script:
    - cd docker.node02
    # The variables below are "file" based secrets stored in Gitlab CI/CD
    - cat $NODE02_ENV > .env
    - cat $NODE02_ENV_PRODUCTION > .env.production
    - cat $NODE02_ENV_POSTGRES > .env.postgres
    # This `docker compose` command will be run against the `DOCKER_HOST` set above
    - docker compose -f docker-compose.node02.yml up --force-recreate --build -d
    - docker image prune -f

There are two things to note in the configuration above. The first is the presence of the DOCKER_HOST variable. This variable is read by docker and specifies the docker instance to connect to. The second is that any docker command run within this shell will be against the remote docker instance.

Exposing containers securely

A number of the services I run via Compose need to be exposed publicly outside of their isolated networks. I am a big fan of using Cloudflare Tunnels to handle this.

• Cloudflare Tunnels are free. Many cloud vendors charge anywhere from $10-$20 per month for load balancers.

• No port forwarding or firewall configuration is required. The tunnel only allows inbound connections via the Cloudflare network and can only talk to the designated destinations.

• I can easily add Cloudflare Access to add additional authentication capabilities to any application.

• I get specialized routing to my containers depending on the client's location. Cloudflare optimizes the routing across its network automatically.

Consider Docker Compose

I think Compose is still a viable option for small production deployments as well as local development. It has a simple declarative model which is just what you need to manage a few containers in several different environments. Let me know what you think on Twitter or Mastodon.