Hello everyone! Welcome to those exploring the world of Kubernetes networking. π
Have you just installed Cilium, the hottest CNI (Container Network Interface) these days? Or are you considering installing it? After being captivated by Cilium’s powerful eBPF-based performance and features and completing the installation, you might suddenly have this question:
“How exactly do my Pods get their IP addresses? Does Cilium just handle it automatically?”
Yes, that’s right! Cilium handles it automatically. But there are several ways it “handles it automatically”! And Cilium adopts the most common and stable method as its default. Today, we’ll uncover the identity of this default method and delve into the reasons why this particular method was chosen as the default over others.
π΅οΈββοΈ The culprit is… Kubernetes Native IPAM!
If you installed Cilium using Helm or similar tools without any special configuration, there’s a 99% chance your cluster is operating in Kubernetes Native IPAM mode, i.e., ipam: kubernetes mode.
What does this mean? Let’s use a simple analogy.
Imagine a huge ‘new city’ called a Kubernetes cluster. ποΈ
>
New City Planning Director (Kubernetes Control Plane): When planning the new city, it pre-allocates address ranges (PodCIDR) for residents (Pods) in each neighborhood (Node). It decides, for example, “Neighborhood A, use addresses from 10.1.1.0-255,” and “Neighborhood B, use addresses from 10.1.2.0-255.”
>
Neighborhood Manager (Cilium Agent): The manager (Cilium Agent) dispatched to each neighborhood checks the address range assigned to their neighborhood by the central planning office.
>
1. Move-in Application (Pod Creation): When a new resident (Pod) moves into Neighborhood A, the Neighborhood Manager (Cilium) assigns an available address (IP), like ‘10.1.1.10’, to it!
In other words, Cilium does not manage IP ranges independently; instead, it takes the IP ranges (PodCIDR) already allocated by Kubernetes to each Node and distributes IPs to Pods within those ranges. Cilium is essentially adding its spoon to a meal already prepared by Kubernetes, a very smart and efficient approach! π
Shall we check it ourselves?
Let’s directly check which IP ranges (PodCIDR) your cluster’s nodes have been allocated. Enter the command below into your terminal.
kubectl get nodes -o jsonpath='{range .items[*]}{.metadata.name}{"t"}{.spec.podCIDR}{"
"}{end}'Then you will see results similar to the following:
# Example output
master-node 10.244.0.0/24
worker-node-1 10.244.1.0/24
worker-node-2 10.244.2.0/24Do you see it? You can confirm that Kubernetes’ kube-controller-manager has fairly distributed /24 sized IP ranges to each node.
When a Pod is created on worker-node-1, Cilium finds and assigns an available IP within the 10.244.1.0/24 range. All this information is recorded and managed in a CRD (Custom Resource Definition) called CiliumNode.
π€ So why aren’t other methods the default?
Cilium supports various IPAM methods besides the kubernetes mode. Let’s understand why the kubernetes mode was chosen as the default by looking at a prominent competitor, the cluster-pool mode.
β Cluster Pool IPAM (ipam: cluster-pool)
This method is where Cilium itself manages a large IP address pool.
Let’s go back to the new city analogy. ποΈ
>
The New City Planning Director (Kubernetes) no longer allocates address ranges per neighborhood. Instead, a specialized real estate developer named Cilium declares, “I’ll manage all the IPs in this new city! The 10.10.0.0/16 range is all mine!” And then it delegates small portions of the IP range to each neighborhood (Node) as needed.
Advantages:
- Useful when not in a cloud environment or when you don’t want to rely on Kubernetes’ PodCIDR allocation feature.
- Allows for more flexible and granular control over IP ranges.
Disadvantages (and why it’s not the default):
- Requires additional configuration! π Users must explicitly specify the overall cluster IP range (clusterPoolIPv4PodCIDRList) in the Cilium configuration file.
# Helm values.yaml example
ipam:
mode: cluster-pool
operator:
clusterPoolIPv4PodCIDRList: ["10.10.0.0/16"]- The philosophy of the default is “to be the easiest, most intuitive, and work well with zero-config.” The cluster-pool method goes against this philosophy. Utilizing Kubernetes’ default features as much as possible is more ‘Kubernetes-native’ and provides convenience to users, which is why the kubernetes mode was chosen as the default.
βοΈ Cloud-Specific IPAM (AWS ENI, Azure IPAM, etc.)
In cloud environments like AWS, GCP, and Azure, there are also modes that utilize the native networking features provided by the respective cloud to allocate IPs. For example, AWS ENI mode can directly assign secondary IPs of EC2 instances to Pods, achieving tremendous performance.
Reasons why it’s not the default:
- The reason is clear. These methods are platform-dependent on specific cloud providers. π
- Cilium’s default should operate universally in any environment (on-premise, all clouds, local development environments, etc.). Therefore, the kubernetes mode, which offers the best versatility, holds the position of the default.
πΌοΈ Understanding the Big Picture: Why is this important?
While knowing individual IPAM modes is important, understanding the overall context of why Cilium offers these choices is key.
| IPAM Mode | Core Idea | Advantages | Disadvantages/Characteristics |
|---|---|---|---|
| Kubernetes (Default) | “Trust and follow Kubernetes!” | Works immediately after installation, no additional configuration needed, most standard | Relies on Kubernetes’ PodCIDR allocation capability |
| Cluster Pool | “Cilium will manage IPs directly!” | Flexible IP range management, breaks K8s dependency | User must manually configure IP ranges |
| Cloud-Specific | “This method is best for this cloud!” | Maximizes cloud-native performance | Dependent on specific cloud, lacks universality |
Ultimately, Cilium’s default strategy is “simplicity and integration.” It provides a method that naturally integrates as part of the Kubernetes ecosystem by default, while offering other powerful options for advanced users with more specific requirements.
β¨ In Summary
Today, we delved deeply into Kubernetes Native IPAM, the default method Cilium uses to allocate IP addresses to Pods. Now, after installing Cilium, you’ll be able to confidently explain where your Pod IPs come from and why that particular method was chosen as the default.
In most cases, this default configuration works excellently. However, if your cluster environment has special requirements, exploring the other IPAM options we discussed today would be a great next step!
Leave a Reply