Network Segmentation for Multi-Tenant Kubernetes Clusters

Intermediate · 5 min read

The Problem

Imagine a scenario where a cloud service provider needs to host multiple customer applications on the same Kubernetes cluster. Each customer’s application must be isolated from others, both for security and performance reasons. This is a common challenge in multi-tenant environments, where different tenants (customers) share the same infrastructure but need to operate independently. Network segmentation is a critical solution to this problem, ensuring that each tenant’s network traffic is isolated and secure.

Core Concepts

Before diving into the specifics of network segmentation, it’s essential to understand some fundamental concepts:

• Kubernetes Cluster: A set of physical or virtual machines that run containerized applications managed by Kubernetes.
• Namespace: A way to divide cluster resources between multiple users, teams, projects, or customers. Namespaces provide a scope for names, allowing you to create isolated environments within the same cluster.
• Pods and Services: Pods are the smallest deployable units in Kubernetes, and services are an abstraction that defines a logical set of pods and a policy by which to access them.
• Network Policies: Rules that control the flow of traffic into and out of pods and services. They allow you to define inbound and outbound traffic rules based on labels, IP ranges, and ports.

How It Works

Network segmentation in Kubernetes is primarily achieved through the use of Network Policies. These policies are applied at the pod level and can be used to control incoming and outgoing traffic. Here’s how it works:

1. Define Network Policies: You create a NetworkPolicy resource that specifies the rules for incoming and outgoing traffic. These rules can be based on pod labels, IP ranges, and ports.
2. Apply Network Policies: The policies are applied to specific namespaces or pods. When a pod is created, the network policy is enforced, and traffic is allowed or blocked based on the defined rules.
3. Isolation and Security: By carefully defining and applying network policies, you can ensure that traffic from one namespace or pod does not inadvertently reach another, providing a high degree of isolation and security.

Prerequisites

Before you start implementing network segmentation, make sure you have the following prerequisites:

• A running Kubernetes cluster (version 1.8 or later, as network policies were introduced in this version).
• A CNI (Container Network Interface) plugin that supports network policies, such as Calico, Cilium, or Antrea.
• Familiarity with basic Kubernetes concepts, such as pods, services, and namespaces.

Hands-On Walkthrough

In this section, we will walk through the process of setting up network segmentation in a multi-tenant Kubernetes cluster. We will create two namespaces, each with its own set of pods and services, and apply network policies to isolate them.

Step 1: Create Namespaces

apiVersion: v1
kind: Namespace
metadata:
  name: tenant-a
---
apiVersion: v1
kind: Namespace
metadata:
  name: tenant-b

Save this to a file named namespaces.yaml and apply it using kubectl apply -f namespaces.yaml.

Step 2: Deploy Pods and Services

apiVersion: v1
kind: Pod
metadata:
  name: app-pod
  namespace: tenant-a
spec:
  containers:
  - name: app-container
    image: nginx:latest
---
apiVersion: v1
kind: Service
metadata:
  name: app-service
  namespace: tenant-a
spec:
  selector:
    app: app-pod
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80
---
apiVersion: v1
kind: Pod
metadata:
  name: app-pod
  namespace: tenant-b
spec:
  containers:
  - name: app-container
    image: nginx:latest
---
apiVersion: v1
kind: Service
metadata:
  name: app-service
  namespace: tenant-b
spec:
  selector:
    app: app-pod
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80

Save this to a file named pods-and-services.yaml and apply it using kubectl apply -f pods-and-services.yaml.

Step 3: Define and Apply Network Policies

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: tenant-a-policy
  namespace: tenant-a
spec:
  podSelector:
    matchLabels:
      app: app-pod
  ingress:
  - from:
    - podSelector:
        matchLabels:
          app: app-pod
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: tenant-b-policy
  namespace: tenant-b
spec:
  podSelector:
    matchLabels:
      app: app-pod
  ingress:
  - from:
    - podSelector:
        matchLabels:
          app: app-pod

Save this to a file named network-policies.yaml and apply it using kubectl apply -f network-policies.yaml.

Verifying It Works

To verify that the network policies are working as expected, you can use the following commands:

# Check if the pods are running
kubectl get pods -n tenant-a
kubectl get pods -n tenant-b
# Check if the services are available
kubectl get svc -n tenant-a
kubectl get svc -n tenant-b
# Test connectivity from a pod in tenant-a to a pod in tenant-b
kubectl exec -it <pod-name-in-tenant-a> -n tenant-a -- wget -qO- http://<service-name-in-tenant-b>.tenant-b.svc.cluster.local:80
# The above command should fail, indicating that the network policy is blocking cross-namespace traffic

Common Mistakes

Here are some common mistakes to avoid when implementing network segmentation in Kubernetes:

• Not Using a Compatible CNI Plugin: Ensure that your CNI plugin supports network policies. If it doesn’t, the policies won’t be enforced, leading to potential security vulnerabilities.
• Incorrectly Defining Network Policies: Carefully define your network policies to ensure they match the desired traffic rules. Misconfigured policies can either block legitimate traffic or allow unwanted traffic.
• Overlooking Default Policies: By default, all traffic is allowed in a namespace unless explicitly restricted. Always define a default-deny policy to ensure that only explicitly allowed traffic is permitted.

Going Further

Once you have a basic understanding of network segmentation in Kubernetes, you can explore more advanced topics:

• Scaling: As your cluster grows, consider using tools like Calico or Cilium to manage and scale your network policies efficiently.
• Security: Implement additional security measures, such as using TLS for inter-pod communication and integrating with external security solutions.
• Performance Tuning: Optimize your network policies for performance by minimizing the number of rules and using efficient selectors.
• Alternative Approaches: Consider using other approaches for network segmentation, such as using separate clusters for each tenant or using a service mesh like Istio for more granular traffic management.

Cleanup

To clean up the resources created in this tutorial, you can use the following commands:

kubectl delete -f namespaces.yaml
kubectl delete -f pods-and-services.yaml
kubectl delete -f network-policies.yaml

Architecture

References

• Network Policies
• Resource Quotas

---

This article was written with AI assistance.