Days before, I migrated docker-compose
services to Rancher container cloud. I will demostrate how I deploy Gitea, a popular self-hosted coding hosting software, with full Web UI and native-like SSH functionality.
Preparing Nodes
Some ports are required by etcd nodes.
# firewall-cmd --add-port=2376/tcp --permanent
# firewall-cmd --add-port=2379/tcp --permanent
# firewall-cmd --add-port=2380/tcp --permanent
# firewall-cmd --add-port=8472/udp --permanent
# firewall-cmd --add-port=10250/tcp --permanent
Extra ports are required for control plane.
# firewall-cmd --add-port=2376/tcp --permanent
# firewall-cmd --add-port=6443/tcp --permanent
# firewall-cmd --add-port=8472/udp --permanent
# firewall-cmd --add-port=10250/tcp --permanent
For listening and forwarding ssh stream, we need to open up port 30000-32767 on master nodes in firewall.
# firewall-cmd --add-port=30000-32767/tcp --permanent
# firewall-cmd --add-port=30000-32767/udp --permanent
And finally, don't forget to reload the firewall.
# firewall-cmd --reload
For a detailed ports requirement, you could always refer to official documents.
Deploying Gitea Workload
Gitea is very easy to deploy on any container cloud distribution. With Rancher, I only need to specify the docker image and mount a volume for data persistence. An extra step I did was add a NodePort 30022 → 22
for accessing Gitea via SSH.
When pods are ready, gitea will startup in minutes. Don't forget to add an ingress, for example, git-mirror.apps.central.sakuragawa.cloud
here to the workload at port 80.
From now on, the web ui and HTTP clone/push should be ready to work.
Let SSH Works "Natively"
Apparently we are not using port 22 for git server now since it is already taken by the container host. There are in total 2 way which are all focused on forwarding the SSH traffic.
Method 1: SSH Forwarding
A front proxy is required to proxy both web and SSH traffic. If you have ever noticed that cloning a repo is via git clone ssh://git@...
, that git
user is the way to the goal. The idea is to create a git
user on the proxy server to forward the SSH command. More details is written in a previous post. Make sure to replace the container address and port with Rancher's NodePort.
Method 2: Nginx TCP Proxy
From plus release 5 and later, nginx supports TCP proxy[1]. Since sshd
would listen on port 22, we need to add a new IP to the reverse proxy and configure nginx to listen on a seperated IP address in order to forward traffic from port 22.
As an example, I will shortly show how the configure nginx in this post. Add the following code into you nginx.conf
:
stream {
server {
listen 192.168.1.11:22; #! use your extra IP
proxy_pass git-ssh;
}
upstream git-ssh {
server master-1:30022;
server master-2:30022; #! only if exists
# add more masters
}
}
Then reload nginx service, and now all the traffic to 192.168.1.11:22
will be forwarded to master-1:30022
or master-2:30022
, which are both mapped to the 22 port of the workload.
Encore: Reusing Rancher Node
Rancher nodes need to be cleaned up before reusing. Here is a simple bash script which needs to be executed as root.
#!/bin/sh
docker rm -f $(docker ps -qa | grep k8s)
docker rm -f $(docker ps -qa | grep rancher)
docker volumes prune
cleanupdirs="/var/lib/etcd /etc/kubernetes /etc/cni /opt/cni /var/lib/cni /var/run/calico"
for dir in $cleanupdirs; do
echo "Removing $dir"
rm -rf $dir
done