Architecture

We use Cilium as the Container Network Interface (CNI) in our Kubernetes cluster with an IPv4/IPv6 dual-stack configuration.

Each pod is assigned an internal IPv4 and IPv6 address upon startup, which can be checked with the following command:

$ kubectl get pods $pod -o jsonpath='{.status.podIPs[*].ip}'
10.53.5.113 fd01:4cf0:8:8081:7::73a2

The pods of different clusters can route to each other internally, but they are not accessible outside of their respective clusters.

Communication within the clusters can occur over both IPv4 and IPv6, and services can be offered via IPv4, IPv6, or both protocols. By default, an IPv4-only service is created.

On the other hand, nodes have an internal IPv4 address and an external IPv6 address.

When a network packet exits the cluster, meaning it is not sent to another pod or service within the clusters, it is NATed with the IP of that node upon leaving the cluster.

This means that IPv6 packets from a pod can route to external services outside the university via the external Node IPv6 address. In contrast, IPv4 packets can only route to services within the university network.

The asymmetry between IPv4 and IPv6 is due to the lack of support for IPv6 in our Cilium Egressgateway and will be resolved once the gateway supports IPv6. Nodes will then also be switched to internal IPv6 addresses, and direct access to external services via IPv6 will no longer be possible.

To enable external IPv4 connectivity for pods, we operate specific nodes with external IPv4 addresses. This allows IPv4 traffic to be routed through these nodes when necessary. Additional information can be found under External Routing.