Advanced Kubernetes: Ingress Controllers and Service Mesh

Introduction to Kubernetes

Kubernetes has become the de facto standard for container orchestration, enabling developers to deploy and manage containerized applications at scale. While Kubernetes simplifies many aspects of application deployment, managing and configuring networking in a Kubernetes cluster can be a complex task, especially as applications become more distributed and interconnected. In this blog, we will delve into advanced Kubernetes networking concepts, focusing on Kubernetes Ingress controller, Service Mesh technologies, and network policies.

Understanding Ingress Controllers

In Kubernetes, an Ingress is an API object that manages external access to services within a cluster. Kubernetes Ingress controllers are responsible for implementing the rules set by the Ingress resource, allowing external traffic to reach the appropriate services. They act as the entry point for external requests, handling routing, SSL termination, and load balancing.

Nginx Ingress Controller Example

Let’s take a look at an example using the Nginx Ingress Controller. First, make sure you have the Kubernetes Ingress controller deployed in your cluster. You can use the following YAML manifest to deploy it:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-ingress
spec:
  rules:
    - host: myapp.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: myapp-service
                port:
                  number: 80

In this example, we define an Ingress resource that routes traffic from myapp.example.com to the myapp-service service on port 80. Adjust the host, path, and backend details according to your application’s needs.

Service Mesh Technologies

Service meshes provide a dedicated infrastructure layer for handling service-to-service communication, offering features like service discovery, load balancing, and security. Istio is one of the most popular service mesh solutions for kubernetes hpa best practices.

Istio Installation

To install Istio, you can use the following steps:

# Download and extract Istio
curl -L https://istio.io/download | ISTIO_VERSION=1.11.2 sh -
cd istio-1.11.2

# Install Istio to your cluster
istioctl install --set profile=default

# Deploy the Istio components
kubectl apply -f samples/addons

Istio Sidecar Injection

To leverage Istio’s features, you need to inject its sidecar proxy into your application pods. You can achieve this by adding the sidecar.istio.io/inject: "true" annotation to your pod template spec.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
spec:
  template:
    metadata:
      annotations:
        sidecar.istio.io/inject: "true"
    spec:
      containers:
        - name: myapp-container
          image: myapp:latest
          ports:
            - containerPort: 80

This annotation ensures that Istio’s sidecar proxy is injected into the myapp deployment.

Network Policies

Kubernetes Network Policies allow you to control the communication between pods. They define rules that specify how groups of pods are allowed to communicate with each other.

Deny All Traffic Example

Here’s an example of a Network Policy that denies all traffic:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-all
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress

This policy selects all pods (podSelector: {}) and specifies that both Ingress and Egress traffic are denied.

Choosing the Right Kubernetes Networking Approach

Not all Kubernetes apps need a comprehensive networking model. The simplest setup has a Service and Ingress, whereas a large platform-hosted app involves dozens of services that must communicate with each other. At the same time, adding unnecessary components to Kubernetes makes it more complicated than it should be and less intuitive and versatile in its purpose.

Ingress is most needed when an application requires external access. It has the necessary rules to route web traffic based on the host or path. It is essential for the case when there is a need to expose multiple applications to the Internet.

A service mesh is most needed to connect applications; it is most useful when the number of services within a cluster is too significant to allow direct communication. It helps to offload some tasks from the application to the service mesh itself, specifically communicating with other services.

Network Policies are a set of rules that restrict traffic between pods. It is necessary to use them when there is a need to limit the traffic between pods instead of allowing all traffic by default.

The cases described above illustrate the situations when it is needed to use one of the three instruments mentioned above. However, these three components are not mutually exclusive, and there might be a need to use several of them at the same time.

Managing Traffic Between Kubernetes Services

In the distributed application, the services are not so often used singularly. For instance, the frontend may call an API service while the latter interacts with the database. The workers in the background may share data among themselves using other services.

The Kubernetes Services allow the workloads to expose their stable network endpoints. The pods can be rescheduled, shut down, or moved to different nodes. However, the applications can communicate with them constantly using services instead of IP addresses.

It is particularly useful when the applications are scaled in the production environment. Moreover, the developers do not need to update the application code according to the changed services. Instead, Kubernetes manages the abstract layer which is resolved to the actual IPs of pods.

The following example defines a simple internal Service for an API deployment:

apiVersion: v1
kind: Service
metadata:
  name: api-service
  namespace: production
spec:
  selector:
    app: api
  ports:
    - name: http
      port: 80
      targetPort: 8080
  type: ClusterIP

The selector makes the Service available to the pods labeled app: api. The Service exposes port 80 within the cluster to 8080 on the selected pods. Since the Service type is ClusterIP it is not accessible from the Internet.

However, as the application grows it is important to understand which services talk with each other, how the traffic is distributed, and what services are causing problems throughout the application.

Ingress and Internal Service Traffic

Ingress mostly handles traffic entering the cluster. Internally, services are communicating with each other in a different network plane than what Ingress is handling. This is done via Kubernetes Services and other networking primitives.

For example, a user request can come into the Ingress controller, get routed to the application service, and the application can in turn make requests to other services. These requests would not be subject to the same Ingress rules that the user’s request was.

A basic Ingress configuration can route requests for a hostname to the API Service:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: api-ingress
  namespace: production
spec:
  ingressClassName: nginx
  rules:
    - host: api.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: api-service
                port:
                  number: 80

This configuration assumes that an Ingress controller, such as NGINX Ingress Controller, is already installed in the cluster. The controller receives requests for api.example.com and forwards them to the api-service Service.

Keeping this distinction clear can make troubleshooting easier. If an external request never reaches the application, checking the Ingress configuration makes sense. If the application receives the request but cannot reach another service, the issue is more likely to involve service discovery, network rules, or the target workload.

Looking at networking as a sequence of smaller paths is often easier than trying to diagnose the entire cluster at once.

Using Network Policies as the Cluster Grows

When a Kubernetes cluster is small, unrestricted communication between workloads may not seem like a major concern. That changes as more applications are added. Not every service needs access to every other service, and leaving those connections open can make the network harder to control.

Network Policies allow teams to define which traffic should be allowed. A policy might permit an API service to reach a database while blocking unrelated workloads from doing the same.

The following policy allows pods labeled app: api to connect to pods labeled app: database on TCP port 5432:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-api-to-database
  namespace: production
spec:
  podSelector:
    matchLabels:
      app: database
  policyTypes:
    - Ingress
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app: api
      ports:
        - protocol: TCP
          port: 5432

This policy applies to the database pods. It allows incoming traffic only from API pods in the same namespace and only on the PostgreSQL port. Other incoming connections to the database may be blocked, depending on the other policies and the behavior of the cluster’s network plugin.

It is important to introduce these policies carefully. A policy that blocks all traffic may look secure on paper but can also stop required application communication. Teams should first understand the traffic their applications depend on and then add rules around those requirements.

Policies also need to be maintained as applications change. A new service or dependency may require an update, so network rules should be treated as part of the application’s configuration rather than something configured once and forgotten.

Observability in Kubernetes Networking

Networking issues can be difficult to diagnose because the failure may happen somewhere between several different components. An application can be running normally while requests still fail because of routing, DNS, service configuration, or network policies.

Logs and metrics can provide useful clues. Error rates may show that a service is failing to respond, while latency metrics can indicate that a request is taking longer somewhere in the communication path.

Service mesh environments can provide additional visibility into service-to-service traffic. This can help teams understand request flows without adding the same monitoring logic to every application.

The main benefit is having enough information to answer a simple question: where did the request fail? Without that visibility, teams often end up checking several configuration files and workloads before finding the actual issue.

Common Kubernetes Networking Problems

Kubernetes networking problems often come down to a small number of configuration issues. A Service may not select the expected pods, an Ingress rule may point to the wrong service, or a Network Policy may block traffic that the application requires.

DNS and service discovery can cause another class of problems. A workload may be running correctly but still fail to connect because it cannot resolve or reach the expected service.

A temporary debugging pod can be used to test DNS and network connectivity from inside the cluster:

kubectl run network-debug \
  --rm -it \
  --restart=Never \
  --image=busybox:1.36 \
  -n production \
  -- sh

After entering the pod, basic checks can be performed:

nslookup api-service
wget -qO- http://api-service

If DNS resolution fails, the problem can be in cluster DNS services or Service name. If DNS resolution was successful, check target pods, Service ports, and Network Policies.

The procedure for troubleshooting is usually much simpler if this sequence is followed. First, make sure that all pods are healthy, then check Service’s selectors, and finally, check the routing information and Network Policies.

Multiple components’ changes at once may lead to an inability to pinpoint the problem. A check of a single component allows to locate the point of failure with a high degree of precision.

Scaling Kubernetes Networking

The requirements to the network usually scale up when the app grows and starts to accommodate more services. The network setup that was perfectly fine for a handful of services will not scale to thousands of workloads.

At some point, the operations team will need more granular control and visibility over the traffic flows between services. It is possible to achieve through the service mesh or more fine-grained Network Policies.

But not every network issue requires a global platform-level solution. In many cases, it is better to design the network topology in a more structured way than to fight the chaos. The same goes for service-to-service communication: it is better to plan what services should talk to each other and establish the required rules upfront rather than allow free communication between all services and then try to sort out who talks to whom.

Kubernetes Networking Best Practices

This assumes that you have installed an Ingress controller such as NGINX Ingress Controller into the cluster. This controller is receiving the request for api.example.com an routing it to the api-service Service.

Now there is a big difference between the two, and I believe it is important to distinguish the two and understand when to check one over the other. If you are having an issue where an external request is not reaching your application, there is no doubt that you should look at the Ingress first. On the other hand if the request is reaching the application but not the other service it’s best to troubleshoot some service discovery or network policies or the target workloads themselves.

It is easier to approach the problems in networking one step at a time

Conclusion

As your Kubernetes deployment grows in complexity, understanding advanced networking concepts becomes crucial. In this blog, we’ve explored Kubernetes Ingress controller for managing external access, Istio as a powerful service mesh solution, and Network Policies for controlling pod-to-pod communication. Implementing these technologies allows you to enhance security, improve observability, and streamline the management of your Kubernetes infrastructure. Feel free to experiment with the provided examples in your own kubernetes hpa best practices environment to deepen your understanding of these advanced networking concepts. For more details Contact Us.

References:

• Kubernetes Docs