How to correctly configure a k3s cluster to be accessible only via a NIC/bind-address reachable through NetFoundry (Ziti Edge Tunnel)?

How things work
1

Hello everyone,

I am looking to configure a k3s single-node-cluster so that it is accessible only via a specific NIC/bind-address that should be reachable exclusively through NetFoundry (Ziti Edge Tunnel).
At the moment, I can interact with the Kube-API-Server via NetFoundry network (from my PC, in which I have Ziti Desktop) and via shell of a device connected to the same LAN of the k3s-node.
The goal is to ensure that all communications to the cluster occur through NetFoundry, enhancing security and controlling access.

Here are some details about my current setup:

  • I am using k3s on a Linux node.
  • I have configured Ziti Edge Tunnel to create a secure endpoint that is reachable via NetFoundry.
  • I want the k3s server to listen only on this specific network interface.

By default, when k3s is started, it uses 0.0.0.0 as the bind-address, which makes it accessible on all network interfaces. For hardening purposes, I want to restrict this and make it accessible only via 127.0.0.1.

So far, I have attempted to set the --bind-address of the k3s server to 127.0.0.1, but I have encountered some difficulties and am not sure of the best way to properly configure it.

In particular, if i start the k3s server with --bind-address=127.0.0.1, I can interact with the API server from my PC (via NetFoundry) and no more from the shell of a device connected to the same LAN of the k3s-node. And this is exactly what I want.
But this way, some system-level pods don’t properly start (CrashLoopBackOff), producing errors like: " dial tcp 10.43.0.1:443: connect: connection refused"

Specific Questions:

  1. What is the correct procedure to configure k3s to use a specific NIC/bind-address that is reachable only via NetFoundry?
  2. Are there any special configurations or security considerations I should take into account to ensure that all traffic to the cluster goes through NetFoundry?
  3. Do you have any examples or guides that demonstrate how to securely and efficiently integrate k3s with NetFoundry (Ziti Edge Tunnel)?

Any suggestions or configuration examples would be extremely helpful.

Thank you very much!

2

Hi tguare,

Thanks for reaching out to Netfoundry.

We are looking into this and will keep you posted.

Regards,
Prasad Rao
Technical Support Engineer

3

If you use --bind-address 127.0.0.1, shouldn’t you use something like --advertise-address 127.0.0.1. This would advertise to all pods where the API server is. No?

4

It looks like it may be already a default value if you don’t pass any values. k3s/pkg/daemons/control/server.go at 2c398c5d5f7b8f9fa7096bbaa571e6766c95d287 · k3s-io/k3s · GitHub

5

Since 127.0.0.1 is not routerable address outside of pods, it is not going to work to advertise 127.0.0.1. I am not sure what the host is for your k3s, but if you have a firewall at the host level you can block ingress to port 6443.
If it does not have a fw, then you could configure a single IP like 10.10.10.10/32 on the loopback and configure bind/advertise k3s IP as that. That ip will only be routable like 127.0.0.1 on the host.You have to make sure that the IP configuration survives the host reboots.

ss -lt | grep 6443
LISTEN 0      4096     10.10.10.10:6443 

cat /etc/rancher/k3s/config.yaml.d/config.yaml
write-kubeconfig-mode: 777
bind-address: 10.10.10.10
advertise-address: 10.10.10.10
6

Hello @DariuszSki , sorry for the late reply.
I was not aware of the “–advertise-address” flag.
By the way, i tried every combination of bind-address and advertise-address, and the only way I could have it to work properly, is the default configuration (bind-address=0.0.0.0 and advertise-address=127.0.0.1).

About your last comment, I am not sure on how I could achieve that.

If I am not wrong, you are suggesting me to exec something like “sudo ip addr add 10.10.10.10/32 dev lo”, then set the bind-address=10.10.10.10 and advertise-address=10.10.10.10 in /etc/rancher/k3s/config.yaml and start the k3s server with these parameters?

Thank you in advance

7

Yes, that is what I meant. The caveat is though if you use ip address add that this ip will not survive node reboot. You will need to make sure that IP is configured as soon as the network is up during bootup using a script or system service if your node supports it.