Complete Kubernetes Cluster Installation

# Complete Kubernetes Cluster Installation

Introduction

Installing a Kubernetes cluster is the first step in adopting container orchestration. This tutorial covers complete Kubernetes cluster installation using kubeadm, including control plane initialization, worker node joining, CNI plugin configuration, and production hardening steps.

Learning Objectives

By the end of this tutorial, you will be able to:

• Install Kubernetes using kubeadm on Ubuntu servers
• Initialize control plane and join worker nodes
• Configure CNI networking (Calico, Flannel, or Cilium)
• Install and configure ingress controller
• Set up monitoring stack (Prometheus, Grafana)
• Apply production security hardening
• Backup and restore etcd

Technical Background

kubeadm Installation Flow

flowchart TD
    Install[Install kubelet, kubeadm, kubectl] --> Init[Init Control Plane]
    Init --> CNI[Install CNI Plugin]
    CNI --> Join[Join Worker Nodes]
    Join --> Ingress[Install Ingress Controller]
    Ingress --> Monitor[Install Monitoring]
    Monitor --> Harden[Production Hardening]
    
    subgraph Steps
    Install
    Init
    CNI
    Join
    Ingress
    Monitor
    Harden
    end

Prerequisites per Node

| Requirement | Specification | |-------------|---------------| | OS | Ubuntu 22.04 LTS | | CPU | 2+ vCPU | | RAM | 4+ GB | | Disk | 50+ GB | | Network | Unique MAC, product_uuid, ports open | | Swap | Disabled |

Required Ports

#### Control Plane Ports

| Port | Protocol | Purpose | |------|----------|---------| | 6443 | TCP | Kubernetes API server | | 2379-2380 | TCP | etcd client/server | | 10250 | TCP | Kubelet API | | 10259 | TCP | kube-scheduler | | 10257 | TCP | kube-controller-manager |

#### Worker Node Ports

| Port | Protocol | Purpose | |------|----------|---------| | 10250 | TCP | Kubelet API | | 30000-32767 | TCP | NodePort services |

Prerequisites

• 3+ Ubuntu 22.04 LTS servers (1 control plane, 2+ workers)
• Root or sudo access on all servers
• Unique hostname per server
• Network connectivity between all nodes
• Swap disabled on all nodes

Hardware and Software Requirements

| Component | Requirement | |-----------|-------------| | OS | Ubuntu 22.04 LTS | | Container Runtime | containerd | | Kubernetes | v1.28+ | | CNI Plugin | Calico, Flannel, or Cilium | | Load Balancer | For HA control plane (optional) |

Step-by-Step Implementation

Phase 1: Prepare All Nodes

#### Disable Swap

# Disable swap immediately
sudo swapoff -a

# Disable swap permanently
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

#### Configure Kernel Parameters

sudo tee /etc/modules-load.d/k8s.conf << 'EOF'
overlay
br_netfilter
EOF

sudo modprobe overlay
sudo modprobe br_netfilter

sudo tee /etc/sysctl.d/k8s.conf << 'EOF'
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF

sudo sysctl --system

#### Install Container Runtime (containerd)

# Install containerd
sudo apt update
sudo apt install -y containerd

# Configure containerd
sudo mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml

# Enable systemd cgroup
sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml

# Restart containerd
sudo systemctl restart containerd
sudo systemctl enable containerd

#### Install Kubernetes Packages

# Add Kubernetes apt repository
sudo apt update
sudo apt install -y apt-transport-https ca-certificates curl

sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.28/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.28/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list

# Install kubelet, kubeadm, kubectl
sudo apt update
sudo apt install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

# Enable kubelet
sudo systemctl enable --now kubelet

Phase 2: Initialize Control Plane

# Initialize control plane
sudo kubeadm init \
  --apiserver-advertise-address=192.168.1.10 \
  --pod-network-cidr=10.244.0.0/16 \
  --control-plane-endpoint=192.168.1.10:6443

# Configure kubectl for current user
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

# Verify installation
kubectl get nodes
kubectl cluster-info

Phase 3: Install CNI Plugin (Calico)

# Install Calico
kubectl apply -f https://raw.githubusercontent.com/projectcalico/calico/v3.26.1/manifests/calico.yaml

# Verify Calico pods
kubectl get pods -n kube-system -l k8s-app=calico-node

# Verify all pods running
kubectl get pods -A

Phase 4: Join Worker Nodes

# On control plane, get join command
sudo kubeadm token create --print-join-command

# Output example:
# kubeadm join 192.168.1.10:6443 --token abcdef.0123456789abcdef \
#   --discovery-token-ca-cert-hash sha256:1234...

# On each worker node, run the join command
sudo kubeadm join 192.168.1.10:6443 --token abcdef.0123456789abcdef \
  --discovery-token-ca-cert-hash sha256:1234...

# On control plane, verify nodes
kubectl get nodes

Phase 5: Remove Taint from Control Plane (Single Node)

# For single-node clusters, allow scheduling on control plane
kubectl taint nodes --all node-role.kubernetes.io/control-plane-

Phase 6: Install Ingress Controller (NGINX)

# Add ingress-nginx Helm repo
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update

# Install ingress-nginx
helm install ingress-nginx ingress-nginx/ingress-nginx \
  --namespace ingress-nginx \
  --create-namespace \
  --set controller.service.type=LoadBalancer

# Verify
kubectl get svc -n ingress-nginx

Phase 7: Install Monitoring Stack

# Add Prometheus Helm repo
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

# Install kube-prometheus-stack
helm install prometheus prometheus-community/kube-prometheus-stack \
  --namespace monitoring \
  --create-namespace

# Verify
kubectl get pods -n monitoring

Phase 8: Production Hardening

#### Configure Audit Logging

# /etc/kubernetes/policies/audit-policy.yaml
apiVersion: audit.k8s.io/v1
kind: Policy
rules:
  - level: RequestResponse
    resources:
    - group: ""
      resources: ["pods", "services", "secrets", "configmaps"]
  - level: Metadata
    resources:
    - group: ""
      resources: ["pods/log", "pods/status"]

# Enable audit logging in kube-apiserver
sudo kubeadm init phase control-plane all --config=config.yaml

#### Configure RBAC

# rbac/read-only-role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: read-only
rules:
- apiGroups: [""]
  resources: ["pods", "services", "configmaps"]
  verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: read-only-binding
subjects:
- kind: User
  name: developer
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: read-only
  apiGroup: rbac.authorization.k8s.io

#### Configure Network Policies

# network-policy.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny
  namespace: default
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress

Phase 9: etcd Backup

# Backup etcd
sudo ETCDCTL_API=3 etcdctl snapshot save /tmp/etcd-backup.db \
  --endpoints=https://127.0.0.1:2379 \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key

# Verify backup
sudo ETCDCTL_API=3 etcdctl snapshot status /tmp/etcd-backup.db --write-out=table

Expected Output

• Kubernetes cluster with control plane and worker nodes
• CNI networking configured (Calico)
• Ingress controller installed
• Monitoring stack operational
• Production hardening applied
• etcd backup procedure documented

Verification Procedures

| Check | Command | Expected Result | |-------|---------|-----------------| | Cluster info | `kubectl cluster-info` | Control plane responding | | Node status | `kubectl get nodes` | All nodes Ready | | Pod status | `kubectl get pods -A` | System pods Running | | DNS resolution | `kubectl run debug --image=busybox -- nslookup kubernetes.default` | DNS resolves | | Ingress | `kubectl get svc -n ingress-nginx` | LoadBalancer IP assigned | | Monitoring | `kubectl get pods -n monitoring` | Prometheus, Grafana running |

Troubleshooting Scenarios

| Issue | Diagnosis | Resolution | |-------|-----------|------------| | kubeadm init fails | Port conflict or swap enabled | Check ports; disable swap | | Worker node NotReady | Kubelet not running or CNI missing | Check kubelet; install CNI | | Pods stuck in ContainerCreating | CNI not configured correctly | Verify CNI plugin; check logs | | DNS not working | CoreDNS crash loop | Check CoreDNS logs; verify RBAC | | Ingress not working | Ingress controller not running | Check ingress-nginx pods | | Node join fails | Wrong token or CA cert hash | Regenerate token; verify hash |

Best Practices

• **Use kubeadm:** Standard tool for Kubernetes installation
• **Disable swap:** Required for kubelet
• **Use containerd:** Recommended container runtime
• **Install CNI:** Required for pod networking
• **Backup etcd:** Regular etcd snapshots for disaster recovery
• **Enable audit logging:** For security compliance
• **Use RBAC:** Least-privilege access control
• **Monitor cluster:** Track node and pod health

Security Considerations

• **API server access:** Restrict to authorized IPs
• **etcd encryption:** Enable encryption at rest for etcd
• **RBAC:** Implement least-privilege access control
• **Network policies:** Control pod-to-pod communication
• **Container images:** Use verified images from trusted registries
• **Secrets management:** Use Kubernetes Secrets or external secret manager

Conclusion

Installing a Kubernetes cluster with kubeadm provides a solid foundation for container orchestration. By following this tutorial, you can install a production-ready cluster with control plane, worker nodes, CNI networking, ingress, monitoring, and security hardening. Regular maintenance, backups, and monitoring ensure cluster reliability.

References

• kubeadm Installation: https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/
• Kubernetes Components: https://kubernetes.io/docs/concepts/overview/components/
• Calico Documentation: https://docs.tigera.io/calico/
• Ingress NGINX: https://kubernetes.github.io/ingress-nginx/
• Prometheus Operator: https://github.com/prometheus-operator/kube-prometheus
• etcd Backup: https://etcd.io/docs/v3.5/ops/backup/