NIC Integration

Automatic Maintenance of NIC Entries for your own Services.

Hosts contained in Istio Gateways can be automatically registered as CNAMEs in the NIC for our Istio Ingressgateways through the annotation wwu.io/nic_node.

The value for this annotation is the entry for which the CNAME record should be generated.

Each cluster can only create CNAMEs for its own NIC entries, which are currently:

  • Production:
    • All Clusters: istio.k8s.wwu.de
    • Einsteinstraße: istio.ms1.k8s.wwu.de (In most cases, this is the correct one)
    • Schlossplatz: istio.ms2.k8s.wwu.de
  • Staging:
    • All Clusters: istio.staging.k8s.wwu.de
    • Einsteinstraße: istio.ms1.staging.k8s.wwu.de (In most cases, this is the correct one)
    • Schlossplatz: istio.ms2.staging.k8s.wwu.de

Only hosts that have been approved in the project application are allowed in the Gateways. Allowing e.g. domain.uni-muenster.de in your namespace will not allow you to use sub.domain.uni-muenster.de, or any other subdomain for that matter. These will need to be specified during the application or requested afterwards. If you would like to use subdomains, please tell us so, because we need to get the networking department’s permission for them.

We run a service that periodically looks for these annotations on all existing istio gateways and automatically creates DNS entries on the NIC-entries of our Ingress Gateways. It also automatically removes entries that no longer have a corresponding gateway. This is important when you are using the multi-cluster annotation on a gateway that is only present on one cluster, because the clusters will “fight” each other, continuously creating and removing the entry. In short, if you use the multi-cluster annotation, make sure your service is actually multi-cluster.

This would for example configure example.uni-muenster.de as a CNAME in the NIC for the Istio Ingress gateway on the production cluster in the Einsteinstraße:

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  annotations:
    wwu.io/nic_node: istio.ms1.k8s.wwu.de
  name: example
  namespace: example
spec:
  selector:
    istio: ingressgateway
  servers:
    - hosts:
        - example/example.uni-muenster.de
      port:
        name: http
        number: 80
        protocol: HTTP
      tls:
        httpsRedirect: true
    - hosts:
        - example/example.uni-muenster.de
      port:
        name: https
        number: 443
        protocol: HTTPS
      tls:
        credentialName: example--example-certificate
        minProtocolVersion: TLSV1_3
        mode: SIMPLE

Debugging

The controller that synchronizes information between Kubernetes resources and the NIC provides feedback to users about various problems by sending events on the affected resources. You can see these either in your namespace’s events (e.g. kubectl --context wwukube-staging-ms1 -n my-namespace get events) or directly on the resource (e.g. kubectl --context wwukube-staging-ms1 -n my-namespace describe gateway my-gateway).

Events

Here are some of the most common events:

  • UnknownNode: You are trying to add a CNAME for a host that is not managed by this cluster. This mostly happens if, for example, you try to use istio.ms1.k8s.wwu.de on the staging cluster or istio.staging.k8s.wwu.de on production.
  • HostToNICEntryFailure: Sometimes your host cannot be converted into a proper NIC entry. One of the main reasons for this is that you are trying to use a subdomain that our robot account in the cluster does not have permission to use. You can request that our robot account be granted permission for the subdomain directly from the network department (nic@uni-muenster.de).
  • NICAnnotationOnVirtualService: Previously, the annotations for the NIC entries were on the VirtualService resources in the cluster, but we decided to switch to the Gateway resources. Although it still works to put the annotations on the VirtualService resources for now, we might remove this in the future, and the recommendation is to put them on the Gateway resources.
  • IgnoreHost: We maintain a manual exclusion list for some domains and hosts. If you think your domain or host should not be on that list, please contact us.
  • FailedToAddCNAME: The NIC rejected the changes we tried to apply. This can have several causes; the event description should provide more details. One error we have already seen is inconsistent configuration across clusters. For example, if you set the annotation to istio.ms1.staging.k8s.wwu.de on wwukube-staging-ms1 and to istio.staging.k8s.wwu.de on wwukube-staging-ms2, the controllers on those clusters will try to add your CNAME to different nodes, and one of them will fail to apply the change to the NIC.