Architecture
We use Cilium as the Container Network Interface (CNI) in our Kubernetes cluster with an IPv4/IPv6 dual-stack configuration.
Each pod is assigned an internal IPv4 and IPv6 address upon startup, which can be checked with the following command:
$ kubectl get pods $pod -o jsonpath='{.status.podIPs[*].ip}'
10.53.5.113 fd01:4cf0:8:8081:7::73a2
The pods of different clusters can route to each other internally, but they are not accessible outside of their respective clusters.
Communication within the clusters can occur over both IPv4 and IPv6, and services can be offered via IPv4, IPv6, or both protocols. By default, an IPv4-only service is created.
On the other hand, nodes have an internal IPv4 address and an external IPv6 address.
When a network packet exits the cluster, meaning it is not sent to another pod or service within the clusters, it is NATed with the IP of that node upon leaving the cluster.
This means that IPv6 packets from a pod can route to external services outside the university via the external Node IPv6 address. In contrast, IPv4 packets can only route to services within the university network.
The asymmetry between IPv4 and IPv6 is due to the lack of support for IPv6 in our Cilium Egressgateway and will be resolved once the gateway supports IPv6. Nodes will then also be switched to internal IPv6 addresses, and direct access to external services via IPv6 will no longer be possible.
To enable external IPv4 connectivity for pods, we operate specific nodes with external IPv4 addresses. This allows IPv4 traffic to be routed through these nodes when necessary. Additional information can be found under External Routing.
Current Node CIDRs
The following CIDRs are currently assigned to our cluster nodes:
| Cluster | IPv4 | IPv6 |
|---|---|---|
| Dev (MS1) | 10.102.8.0/24 | 2001:4cf0:8:12::/64 |
| Dev (MS2) | 10.102.14.0/24 | 2001:4cf0:9:8::/64 |
| Staging (MS1) | 10.102.6.0/24 | 2001:4cf0:8:13::/64 |
| Staging (MS2) | 10.102.12.0/24 | 2001:4cf0:9:5::/64 |
| Production (MS1) | 10.102.4.0/24 | 2001:4cf0:8:14::/64 |
| Production (MS2) | 10.102.10.0/24 | 2001:4cf0:9:6::/64 |
These CIDRs may change at any time without prior notice. Dedicated node groups for specific services (e.g., JupyterHub) use separate CIDRs that are not listed here.
Do not use these CIDRs as a security mechanism. The cluster is a shared multi-tenant platform — all tenants on the same nodes share the same source IP addresses. Allowlisting these CIDRs in a firewall therefore grants access not only to your workloads but to all other tenants as well.
Instead, adopt a Zero Trust approach and secure your connections at the application level:
- Authenticate every connection using mutual TLS (mTLS) or application-level credentials instead of trusting source IPs.
- Encrypt all communication with properly configured TLS.
- Authorize requests based on verified identities, not network location.Current Node CIDR