Days before, I migrated docker-compose services to Rancher container cloud. I will demostrate how I deploy Gitea, a popular self-hosted coding hosting software, with full Web UI and native-like SSH functionality.

Preparing Nodes

Some ports are required by etcd nodes.

# firewall-cmd --add-port=2376/tcp --permanent
# firewall-cmd --add-port=2379/tcp --permanent
# firewall-cmd --add-port=2380/tcp --permanent
# firewall-cmd --add-port=8472/udp --permanent
# firewall-cmd --add-port=10250/tcp --permanent

Extra ports are required for control plane.

# firewall-cmd --add-port=2376/tcp --permanent
# firewall-cmd --add-port=6443/tcp --permanent
# firewall-cmd --add-port=8472/udp --permanent
# firewall-cmd --add-port=10250/tcp --permanent

For listening and forwarding ssh stream, we need to open up port 30000-32767 on master nodes in firewall.

# firewall-cmd --add-port=30000-32767/tcp --permanent
# firewall-cmd --add-port=30000-32767/udp --permanent

And finally, don't forget to reload the firewall.

# firewall-cmd --reload

For a detailed ports requirement, you could always refer to official documents.

Deploying Gitea Workload

Gitea is very easy to deploy on any container cloud distribution. With Rancher, I only need to specify the docker image and mount a volume for data persistence. An extra step I did was add a NodePort 30022 → 22 for accessing Gitea via SSH.

When pods are ready, gitea will startup in minutes. Don't forget to add an ingress, for example, git-mirror.apps.central.sakuragawa.cloud here to the workload at port 80.

From now on, the web ui and HTTP clone/push should be ready to work.

Let SSH Works "Natively"

Apparently we are not using port 22 for git server now since it is already taken by the container host. There are in total 2 way which are all focused on forwarding the SSH traffic.

Method 1: SSH Forwarding

A front proxy is required to proxy both web and SSH traffic. If you have ever noticed that cloning a repo is via git clone ssh://git@..., that git user is the way to the goal. The idea is to create a git user on the proxy server to forward the SSH command. More details is written in a previous post. Make sure to replace the container address and port with Rancher's NodePort.

Method 2: Nginx TCP Proxy

From plus release 5 and later, nginx supports TCP proxy^[1]. Since sshd would listen on port 22, we need to add a new IP to the reverse proxy and configure nginx to listen on a seperated IP address in order to forward traffic from port 22.

As an example, I will shortly show how the configure nginx in this post. Add the following code into you nginx.conf:

stream {
    server {
        listen 192.168.1.11:22; #! use your extra IP
        proxy_pass git-ssh;
    }

    upstream git-ssh {
        server master-1:30022;
        server master-2:30022;  #! only if exists
        # add more masters
    }
}

Then reload nginx service, and now all the traffic to 192.168.1.11:22 will be forwarded to master-1:30022 or master-2:30022, which are both mapped to the 22 port of the workload.

Encore: Reusing Rancher Node

Rancher nodes need to be cleaned up before reusing. Here is a simple bash script which needs to be executed as root.

#!/bin/sh
docker rm -f $(docker ps -qa | grep k8s)
docker rm -f $(docker ps -qa | grep rancher)
docker volumes prune
cleanupdirs="/var/lib/etcd /etc/kubernetes /etc/cni /opt/cni /var/lib/cni /var/run/calico"
for dir in $cleanupdirs; do
  echo "Removing $dir"
  rm -rf $dir
done