The Shocking Reason Why My Istio Is Getting Slower (feat. Sidecar Resources)

Is your service mesh doing well? πŸ€” As microservice (MSA) architectures become more complex and the number of services increases, many of you have probably experienced Istio becoming heavier and slower. Sidecar proxy memory usage skyrockets, and the time it takes for configurations to propagate gets longer and longer.

If you’ve been wondering, ‘Should I increase server specifications?’, hold on a moment! βœ‹ You might have overlooked a very simple setting. Today, we’ll delve deep into Sidecar resources, a key to Istio performance optimization that many people overlook.


🌍 The Trap of Default Settings: “I’ll tell you everything!”

When you install Istio and onboard services into the mesh, the sidecar proxy (Envoy) injected into each workload works very kindly and diligently by default. So diligently, in fact, that it receives routing information, endpoints, and all configurations for all services existing in the mesh from the control plane (Istiod).

Why is this a problem? 🀯

It might be fine with 10 services. But imagine if it grows to 100, or 500. Even if service A only needs to communicate with service B, it still diligently receives and stores information for completely unrelated services C, D, E…Z in memory.

It’s like giving a delivery rider who only delivers in a specific neighborhood in Seoul the entire map and address book of South Korea. πŸ—ΊοΈ

  • Memory & CPU Spike: Unnecessary configuration information drastically increases the sidecar’s resource usage.
  • Slow Configuration Propagation: As the configuration size grows, even a small change takes a long time to propagate to all proxies in the entire mesh.
  • Performance Degradation: Ultimately leads to overall service mesh performance degradation.

πŸ¦Έβ€β™‚οΈ The Savior Appears! Sidecar Resources

To solve these inefficiencies, Sidecar resources exist. Sidecar resources intelligently limit the scope of configurations, ensuring that each sidecar proxy ‘only knows what it needs to know.’

In other words, it’s a switch that transitions from ‘I’ll tell you everything!’ mode to ‘I’ll tell you only what you absolutely need!’ mode. πŸ’‘

The core of the Sidecar resource is the egress field. Through this field, you can explicitly specify the targets with which the workload should communicate externally (outbound).

πŸ› οΈ Shall we check it ourselves?

For example, let’s assume the reviews service in the bookinfo example only communicates with the ratings service.

1. Before Applying Sidecar Resources

If no Sidecar resource is applied, the sidecar of the reviews pod receives configurations for all services in the mesh (e.g., productpage, details). You can check this with the command below. (The number of result lines varies depending on the environment.)

# Check the name of the reviews pod
REVIEWS_POD=$(kubectl get pod -l app=reviews -n bookinfo -o jsonpath='{.items[0].metadata.name}')

# Check the list of clusters (services) known to the reviews pod's sidecar
istioctl proxy-config clusters $REVIEWS_POD -n bookinfo | wc -l
# Raw Data (Example)
# You'll probably see a fairly large number. For example...
52

2. Applying Sidecar Resources

Now, let’s define a Sidecar resource for the reviews workload. This configuration restricts the reviews app to communicate only with the ratings service in the same namespace (bookinfo) and essential components in the istio-system namespace.

# Code
apiVersion: networking.istio.io/v1beta1
kind: Sidecar
metadata:
  name: reviews-sidecar
  namespace: bookinfo
spec:
  # Apply this configuration to pods with the 'app: reviews' label.
  workloadSelector:
    labels:
      app: reviews
  egress:
    # Explicitly specify outbound traffic targets.
    - hosts:
        # 1. Allow only services in the current namespace ("./")
        - "./ratings.bookinfo.svc.cluster.local"
        # 2. Essential configuration for control plane communication (e.g., Istiod)
        - "istio-system/*"

Apply the above YAML file with the kubectl apply -f command.

3. After Applying Sidecar Resources

Again, let’s check the proxy configuration.

# Check the configuration again after a moment
istioctl proxy-config clusters $REVIEWS_POD -n bookinfo | wc -l
# Raw Data (Example)
# You'll see a much smaller number than before. For example...
15

Do you see the result? πŸš€ The amount of configuration that the reviews sidecar needs to manage has dramatically decreased. This directly leads to reduced resource usage and improved performance.


🧐 Common Misconceptions and Incorrect Approaches

There are common misconceptions when understanding Sidecar resources. Incorrect methods can actually worsen the problem, so it’s important to address them.

  • “Can’t I just increase memory/CPU allocation?” (❌)
  • This is not solving the root cause; it’s merely a temporary fix, covering up the problem with money. 🚚 It’s like buying a bigger truck when the delivery route is inefficient. While allocating sidecar resources in the pod definition (Deployment) of the resources field is correct, the proper order is to optimize the configuration scope first, then allocate resources as needed.
  • “Can’t I just disable sidecar injection?” (❌)
  • Controlling injection with namespace labels or pod annotations (sidecar.istio.io/inject: "false") completely excludes that workload from the service mesh. This is closer to “giving up” than “optimization.” We want to improve performance while enjoying the benefits of the mesh, not leave the mesh entirely.
  • “Is this a feature to adjust the configuration update frequency?” (❌)
  • Sidecar resources do not control the update frequency. They merely limit the scope and size of the received configuration.

✨ From a Holistic Perspective

While applying individual Sidecar resources is important, it’s better to approach it from a holistic perspective.

  1. Namespace Isolation: First, set up namespace-level Sidecar resources as a baseline so that each namespace can only see services in other namespaces that are absolutely necessary (e.g., ./*, istio-system/*, and other required namespaces like namespace-b/*).
  2. Workload Specificity: After that, if there are workloads with a narrower communication scope, like the reviews service, it’s effective to apply more granular Sidecar resources using workloadSelector.

This approach makes your service mesh more robust, predictable, and most importantly, faster. If your Istio has somehow slowed down, check your Sidecar resources right now!


Comments

Leave a Reply

Your email address will not be published. Required fields are marked *