Docker and Kubernetes Guide: From Linux Containers to Resilient Clusters

A Linux container is a group of processes with controlled visibility and resource access, not a miniature virtual machine. Linux namespaces provide separate views of resources: PID namespaces isolate process identifiers, mount namespaces isolate mount trees, and network namespaces isolate interfaces, routes, and ports. User namespaces can map container identities to different host identities. Control groups, or cgroups, account for and constrain resources such as CPU and memory. Containers still share the host kernel; running Linux containers on a non-Linux desktop therefore normally involves a Linux virtual machine.

Try `docker run --rm --memory=256m --cpus=0.5 alpine:3.21 sh -c 'cat /proc/1/status; cat /proc/self/cgroup'`. This starts a disposable container with memory and CPU constraints and displays process and cgroup information. The CPU setting restricts available CPU time; it does not dedicate half a physical core. The memory constraint can trigger an out-of-memory kill when usage exceeds the effective limit. Isolation also depends on capabilities, seccomp, and host security policies. Avoid privileged containers, unnecessary host mounts, and Docker socket access: these can undermine the boundary you intended to establish.

An image is a packaged filesystem plus configuration, including an entry point, environment variables, and metadata. OCI specifications standardize image formats, runtime behavior, and registry distribution so tools can interoperate. Docker Engine typically uses containerd for container lifecycle management and an OCI runtime such as runc to create the isolated process. Kubernetes communicates with container runtimes through the Container Runtime Interface, or CRI; containerd and CRI-O are common implementations. Kubernetes does not require Docker Engine, and an image built with Docker can run on a compatible Kubernetes runtime.

Image layers are immutable filesystem changes. On common Linux configurations, OverlayFS-based storage combines lower image layers with a writable container layer; the precise implementation depends on the runtime and its storage configuration. Modifying an existing file may require copying it into the writable layer, which affects write-heavy workloads. Keep databases and durable uploads outside that layer. For an application build, use a multi-stage Dockerfile: compile in a toolchain image, then copy only runtime artifacts into a smaller final image. Run as a non-root user where possible, exclude irrelevant files with `.dockerignore`, and never copy credentials into a layer and assume deleting them later removes the exposure.

For a small worked example, create a directory containing `index.html` and a Dockerfile with two lines: `FROM nginx:stable-alpine` followed by `COPY index.html /usr/share/nginx/html/index.html`. Build with `docker build -t erudex-web:dev .`, then run `docker run -d --name erudex-web -p 127.0.0.1:8080:80 erudex-web:dev`. Request `curl http://127.0.0.1:8080` and verify the page. The port mapping forwards host port 8080 to container port 80; binding to loopback avoids unintentionally exposing this exercise on every host interface.

Inspect behavior rather than accepting a successful start as proof of correctness. Use `docker logs erudex-web` for output and `docker inspect erudex-web` for configuration and state. Remove the container with `docker rm -f erudex-web`, recreate it, and confirm that the page still comes from the image. For deployment, push the image to a registry reachable by the cluster and record its digest. Tags such as `stable-alpine` or `dev` can move; digest references identify exact image content. A production pipeline should also test the application, scan dependencies, and handle intentional base-image updates rather than leaving a pinned image unmaintained.

Kubernetes stores desired state through its API and uses controllers to reconcile actual state toward it. The API server validates requests, etcd stores cluster state, the scheduler selects nodes for unscheduled Pods, and each node’s kubelet works with its runtime to run containers. A Pod is the basic scheduling unit: its containers share networking and can share declared volumes. A Deployment manages ReplicaSets to maintain replicas and coordinate application updates. This is an eventually convergent control system, not a script that executes your instructions once.

On a configured learning cluster, run `kubectl create deployment web --image=nginx:stable-alpine --replicas=3`, followed by `kubectl expose deployment web --port=80 --target-port=80`. Use `kubectl rollout status deployment/web` and `kubectl get pods -o wide` to inspect progress and placement. Delete one Pod and watch the Deployment’s ReplicaSet create a replacement. Three replicas do not guarantee three failure domains: production manifests should express topology-spread or anti-affinity requirements where appropriate. Replace the demonstration image with your published application digest, then store the desired configuration declaratively. Resource requests, probes, security settings, and rollout parameters should be explicit rather than left to assumptions.

Kubernetes networking gives Pods addresses and defines communication expectations, while CNI plugins implement network attachment and related dataplane behavior. Some implementations use overlays such as VXLAN; others use routed networking or different mechanisms. Do not assume that all cluster traffic is encapsulated. A Service supplies a stable discovery and access abstraction over changing backends. In typical ClusterIP implementations, traffic is forwarded to selected Pod endpoints by kube-proxy or an alternative dataplane. Cluster DNS lets applications discover Services by name. For controlled external HTTP access, deploy an appropriate Ingress or Gateway API controller; creating an API object alone does not install the infrastructure that serves traffic.

Apply NetworkPolicy to restrict communication, but verify that your networking implementation enforces it. For example, permit an API tier to reach a database port while denying unrelated workloads; remember DNS and required egress when designing default-deny policies. For durable data, persistent volumes separate storage lifecycle from individual Pods, while PersistentVolumeClaims request capacity and access characteristics. A StorageClass can select dynamic provisioning through a storage driver, commonly using CSI. Access modes describe mounting capabilities, not application-level write coordination. StatefulSets provide stable identities and storage associations, but they do not implement database replication, backups, or consistency. Test restoration and zone-failure behavior before treating stored data as resilient.

Kubernetes needs different signals for different decisions. A readiness probe indicates whether a container should receive normal Service traffic. A liveness probe can trigger a restart when a container is stuck. A startup probe protects slow initialization by delaying readiness and liveness probing until startup succeeds. Avoid liveness checks that fail simply because a shared database is unavailable: restarting every application instance may amplify the incident. For an HTTP API, a practical starting manifest might request 250m CPU and 256Mi memory, cap memory at 512Mi, and probe separate readiness and liveness endpoints. These are testable hypotheses, not universal sizing recommendations.

Kubernetes autoscaling operates at several levels. A Horizontal Pod Autoscaler changes replica counts from resource or custom metrics; CPU utilization targets depend on configured CPU requests. With a 250m request and a 60% target, the reference consumption is 150m per Pod. If three eligible Pods average 300m, the basic proportional calculation suggests six replicas, before stabilization, tolerance, missing metrics, and other controller rules. Metrics infrastructure must be installed and healthy. More replicas still require schedulable capacity, which node autoscaling can provide when configured. Use load tests to evaluate latency, throughput, pending Pods, and memory pressure together. A PodDisruptionBudget can constrain voluntary disruptions, but cannot prevent a node failure.

GitOps makes version-controlled desired state the input to an automated reconciliation process. Tools such as Argo CD or Flux can detect differences and apply approved configuration to clusters. A useful delivery flow builds and tests an image, publishes it, and proposes a configuration change referencing its digest. Reviewers can inspect the change before reconciliation deploys it. Keep plaintext credentials out of Git; use an appropriate secret-management workflow. Kubernetes Secret values are base64-encoded, not inherently encrypted by that encoding. Configure access controls and encryption at rest according to the environment’s threat model.

Production operations also require evidence and recovery procedures. Collect application logs, workload metrics, and traces; alert on user-visible symptoms as well as infrastructure saturation. For a failing rollout, inspect Deployment status, Pod events, probe failures, image-pull errors, and scheduling constraints before guessing. Test node drains, application termination, backup restoration, and capacity exhaustion in controlled environments. Highly available clusters need a resilient control plane and workloads distributed across failure domains, not merely several worker nodes. Document upgrade compatibility and rehearse recovery. Rolling back an image does not reverse a database migration, so use backward-compatible schema changes when releases must coexist. This operational reasoning connects container mechanics to reliable enterprise DevOps practice.