Kubernetes Error Codes: What They Mean and How to Fix Them

5 min readSep 27, 2023

As a DevOps Engineer, you are the gatekeeper of seamless application deployment. In your toolkit, Kubernetes stands as a powerful ally. However, the deployment process is not always a smooth ride, and you may find yourself grappling with a range of errors.

Pre-requisites:

What is Kubernetes?

Kubernetes is like a super-hero for managing containers. Read the below blogs to get some basic insights about Kubernetes if required.

Kubernetes Basics: Your First App Deployment

Day 33/90 with Shubham Londhe

keentolearn.medium.com

Anyways, Let’s talk about different errors that we come across while deploying the application.

Error Codes

⫸ PodPending

Pod stays in a “Pending” state.

The pod is waiting to be scheduled to a node. This can happen for a number of reasons, such as:

There are no available nodes with the required resources(CPU, memory).
The pod is waiting for its images to be pulled.
The pod is waiting for its dependencies to be initialized.
The pod is waiting for a specific node to become available.

Solution: Check the Kubernetes logs for any errors or warnings. If there are no errors, try increasing the number of nodes in your cluster or decreasing the resource requirements of your pod.

kubectl describe pod <pod-name>

Use the kubectl describe pod <pod-name> command to get more information about the pod.

⫸ ImagePullBackOff

Kubernetes is unable to pull the container image for a pod. This can happen for a number of reasons, such as:

The image repository is not accessible or the image does not exist.
The image requires authentication and Kubernetes is not configured with the necessary credentials.
The image is too large to be pulled over the network.
Network connectivity issues.

Solution: Check that the image repository is accessible and that the image exists. Make sure that Kubernetes is configured with the necessary credentials to pull the image.
If the image is too large, you can try splitting it into multiple smaller images or using a different image registry.
Check for network connectivity issues between Kubernetes and the image registry.

kubectl get pods

To check the status of your pods. Look for pods in the “ImagePullBackOff” state.

kubectl describe pod <pod-name>

Use the kubectl describe pod <pod-name> command to get more information about the pod including the event and error messages.

kubectl get secrets

To check for image pull secrets and confirm they are associated with your pod.

⫸ Insufficient CPU/Memory

When you encounter the Insufficient CPU/Memory error in Kubernetes, it means that the pod or container cannot be scheduled because there are not enough CPU or memory resources available in the cluster to meet the specified resource requests. This can happen for a number of reasons, such as:

The pod is over-provisioned.
The cluster is under-resourced.
There are too many pods running on the cluster.
A node is unavailable due to a hardware or software issue.

Solution: Adjust the resource requests and limits in the pod’s YAML file, or scale your cluster by adding more nodes if necessary.
If the pod is over-provisioned, you can reduce the resource requests and limits of the pod.
If the cluster is under-resourced, you can add more nodes to the cluster.

resources:
  requests:
    memory: "256Mi"
    cpu: "0.5"
  limits:
    memory: "512Mi"
    cpu: "1"

Review the resource requests and limits defined in the pod’s YAML configuration

kubectl describe nodes

Pay attention to the Allocatable section for CPU and memory.

⫸ Forbidden

The user does not have permission to perform the requested operation.

This error is often related to Role-Based Access Control (RBAC) misconfigurations or inadequate permissions. This can happen for a number of reasons, such as:

The user is not authorized to access the Kubernetes cluster.
The user does not have the necessary role or permissions to perform the operation.
The resource that the user is trying to access is protected by a role-based access control (RBAC) role or binding.

Solution: Check the user’s permissions and make sure that they are authorized to create and manage pods.
Check the RBAC roles and bindings to make sure that the resource that the user is trying to access is protected by a role-based access control (RBAC) role or binding.
Validate Service account permission, Namespace Permissions
If the user does not have the necessary permissions, you can grant the user the necessary permissions or create a new role with the necessary permissions and assign the role to the user.

kubectl describe <resource-type> <resource-name>

To view the RBAC roles and bindings that are applied to a specific resource.

⫸ NodeNotReady

The node is not ready to run pods. This can happen for a number of reasons, such as:

The node is not running the Kubernetes Kubelet service.
The node is not able to connect to the Kubernetes API server.
The node has insufficient resources to run pods.
The node is experiencing a hardware or software issue.

Solution: Check that the kubelet service is running on the node.
Make sure that the node can connect to the Kubernetes API server.
Verify that the node has sufficient resources to run pods.
If the problem persists, investigate the node for any hardware or software issues.

kubectl describe node <node-name>

To get detailed information about the node’s conditions. Look for conditions like Ready, DiskPressure, OutofMemory, or OutOfDisk that might be causing the node to be not ready.

⫸ Timeout

The pod has not started successfully within the specified timeout period.

Solution: Increase the timeout period or check the pod logs for any errors or warnings.
If the timeout occurred during pod initialization, review the pod’s configuration.

readinessProbe:
  httpGet:
    path: /health
    port: 8080
  initialDelaySeconds: 5
  periodSeconds: 5

Check if the pod has readiness and liveness probes defined and if they are correctly configured. These probes determine when a pod is considered healthy.

In addition to the commands listed above, you can also use the following commands to troubleshoot Kubernetes errors:

kubectl get events --all-namespaces to view all events in the cluster.
kubectl logs <pod-name> -c <container-name> to view the logs for a specific container in a pod.
kubectl describe node <node-name> to view the status of a node.
kubectl describe <resource-type> <resource-name> to view the status of any Kubernetes resource.

Below are some more Kubernetes Error Codes we might encounter:

ImagePullFailed: This error occurs when Kubernetes is unable to pull an image from a registry. This can happen for a number of reasons, such as the image does not exist, the registry is unavailable, or you do not have permission to access the image.
PodCrashExitCode: This error occurs when a pod crashes with a non-zero exit code. This can happen for a number of reasons, such as the pod’s container failed, the pod exceeded its resource limits, or the pod encountered a runtime error.
ContainerCannotRun: This error occurs when Kubernetes is unable to start a container. This can happen for a number of reasons, such as the container image is missing or corrupted, the container requires resources that are not available on the node, or the container is not compatible with the node’s operating system.

These are just a few of the many Kubernetes error codes that you may encounter. Feel free to share your inputs on error codes that you have come across while deployments.

Thank you for reading this article.

Do add some claps, if you liked the article 👏
Follow for more such content ❤

LinkedIn: https://www.linkedin.com/in/tejashree-salvi-003aa2195/