Home cloud ingress tutorial

Web services running on the same network can be difficult to set up correctly, especially if you want all of them to work on an external network.

In this tutorial, I will guide you through setting up a Docker Swarm to route traffic to the appropriate containers, automatically.

There are three parts to get your cloud ready for web services: a load balancer to spread connections to available servers, an ingress router to direct traffic to containers, and DNS to point local traffic to the right IP addresses. Here’s a quick layout:

We’ll set up a couple things to bootstrap your network of web services. I assume you already have a Docker Swarm (Docker & Swarm) and shared volumes (tutorial) set up on at least one node. I also highly recommend another server as a load balancer for higher availability — a Raspberry Pi is fine — but you can skip it if you’re strapped for cash.

In my setup, I’m using Docker Swarm with two nodes: server A 10.1.0.2 and server B 10.1.0.4. Additionally, I’m using a third server 10.1.0.3 as a load balancer. Throughout this tutorial, you’ll need to replace these IPs with your own.

I have reference material on GitHub and if you have any questions in this tutorial, please open an issue.

“Layer 4” load balancer

To use multiple nodes in a Swarm, you may need high availability. In other words, if you have servers A and B, but B is suddenly unplugged, then your cloud should keep working uninterrupted.

One way to achieve basic availability is to set up a dumb “layer 4” load balancer to direct incoming web traffic to multiple Swarm nodes. I grabbed a spare server and installed nginx with a simple configuration. Nginx automatically detects failed connections and routes traffic away from failing nodes. Ideally, this server’s only job is to load balance connections without interruption.

Let’s get started by installing nginx!

Next, copy this config to nginx’s config directory, usually /etc/nginx/conf.d/. Be sure to change the servers to your own IP addresses. The nginx upstreams defined below are for HTTP connections on port 8080, HTTPS on 4443, and DNS on 53.

Configure your home router to port forward 80 to 8080, 443:4443 and 53:53 to the spare server’s IP. This will route traffic from the internet to your load balancer (and later to your containers).

Finally, reload nginx with sudo nginx -s reload.

This ensures basic stability in the event of node failures by balancing traffic between healthy nodes.

If you’ve already used nginx as an HTTP proxy before, then this config may look familiar. However, this is way simpler than an HTTP proxy. It will forward any incoming packets on these ports to a server, HTTP or otherwise. Dumb forwarding is very useful for other protocols, like DNS, as we’ll see later.

Note

You may have noticed that the HTTP and HTTPS servers are configured to use the proxy protocol. This preserves the original IP addresses of incoming requests when they reach the ingress router. If you use an ingress router that is not configured to use the proxy protocol (different from the one in this tutorial), then you should disable it instead.

Ingress router

Now that all incoming traffic is directed to each Swarm node, we need to forward HTTP requests to the appropriate containers. We’ll create an ingress router with Traefik to direct those requests.

First, you’ll need a Traefik config. Copy the below to traefik.toml, remember to replace the email, domain name, and IP addresses with your own:

(For testing purposes, uncomment the caServer line.)

Next, create a shared volume called proxy_traefik and an overlay network named web. If you need to set up shared volumes, you can follow my tutorial. I’m using the convoy volume driver.

Copy the traefik.toml into your shared volume. Here’s one way you could do that:

Next, copy the below Docker config into docker-compose.yml:

Finally, deploy it with docker stack deploy -c docker-compose.yml proxy.

You now have a working ingress router! Next up, we’ll make it possible for other computers on your network to access web services running inside the Swarm.

Note

If you are using an L4 load balancer and you want local network IP addresses to be correct, change the stack to use these settings:

Don’t forget to switch to mode: global under the deploy settings, there can only be one container bound to a host port on each node.

Without the above port changes, the local network IPs will always be the Swarm gateway IP. For more information, see this Docker issue.

Warning: For users with more than one Docker node

If you’re using more than one Docker node, I highly recommend using a key-value database to store your Traefik config and auto-generated certificates.

Using a shared database makes it possible for Traefik to correctly obtain Let’s Encrypt certificates on any node, even if a different node requested new certificates. For more information, see the Traefik docs to set up high-availability.

DNS

The last step to make your cloud usable at home is to add DNS. DNS makes it easy to access your services running in Docker from outside the private Swarm network.

Add this to the end of the proxy’s docker-compose.yml:

And re-deploy the yaml file: docker stack deploy -c docker-compose.yml proxy

Deploying a “hello world” service

Lookin’ good so far. Now we should spin up a service to verify everything is working the way we want. Copy the below into hello.yml:

Deploy it with docker stack deploy -c hello.yml hello.

After about 20 seconds (Traefik’s Docker polling period), verify it’s all working with curl -H "Host: www.yourdomain.com" --insecure https://10.1.0.2/. (Keep in mind, the SSL cert will be invalid if Traefik has errors retrieving certificates. Check the Traefik logs for more information.)

If your laptop’s DNS is pointing to the load balancer, then you can also visit https://www.yourdomain.com in your browser. I’ve configured my home router’s DNS to point to 10.1.0.3 so everything on the network can resolve yourdomain.com.

You now have a fully-functional, highly available ingress. Now all you need are real services to run on it!

Questions?

If you have any questions please open an issue here.

Originally published at johnstarich.com on March 29, 2019.

Software engineer, hiker. Works on the IBM Cloud Kubernetes Service. Opinions are my own. johnstarich.com

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store