Creating High Available Baremetal Kubernetes cluster with Kubeadm and Keepalived (Simple Guide)

Andrei Kvapil

Oct 18, 2018·3 min read

This guide is a free interpretation of official Creating Highly Available Clusters with kubeadm for Stacked control plane nodes. I don’t like this difficult form which used there, so I wrote this article.

If you have any questions, or something is not clear, please refer to the official documentation or ask the Google. All steps described here in the short and simple form.

Input data

We have 3-nodes cluster:

• node1 (10.9.8.11)
• node2 (10.9.8.12)
• node3 (10.9.8.13)

We will create one fault-resistant cluster-ip for them:

• 10.9.8.10

Then install etcd and kubernetes cluster on them.

LoadBalancer setup

First, we need to install keepalived on all three nodes:

apt-get -y install keepalived

Then write config /etc/keepalived/keepalived.conf:

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 1
    priority 100
    advert_int 1
    nopreempt
    authentication {
        auth_type AH
        auth_pass iech6peeBu6Thoo8xaih
    }
    virtual_ipaddress {
        10.9.8.10
    }
}

Enable and start keepalived on all thee nodes

systemctl start keepalived
systemctl enable keepalived

Now we can check that one of node have 10.9.8.10 address on eth0 interface.

Deploying Kubernetes cluster

UPD: This article was written for v1.12, and despite the fact that it was adapted for v1.13, now creating cluster is much easier, check this more simple guide.

Make sure that you have latest kubernetes packages installed on all nodes:

apt-get -y install kubeadm kubelet kubectl

Also stop keepalived daemon on all nodes except the last one:

systemctl stop keepalived

First node

Now we will generate kubeadm-configs (we need separated config for each master node):

CLUSTER_IP=10.9.8.10
NODES=(node1 node2 node3)
IPS=(10.9.8.11 10.9.8.12 10.9.8.13)
POD_SUBNET="192.168.0.0/16"for i in "${!NODES[@]}"; do
  HOST=${IPS[$i]}
  NAME=${NODES[$i]}
  INITIAL_CLUSTER=$(
    for j in "${!NODES[@]}"; do
      echo "${NODES[$j]}=https://${IPS[$j]}:2380"
    done | xargs | tr ' ' ,
  )cat > kubeadm-config-${NODES[$i]}.yaml <<EOT
apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
kubernetesVersion: stable
apiServer:
  certSANs:
  - "${CLUSTER_IP}"
controlPlaneEndpoint: "${CLUSTER_IP}:6443"
etcd:
  local:
    extraArgs:
      initial-cluster: "${INITIAL_CLUSTER}"
      initial-cluster-state: new
      name: ${NODES[$i]}
      listen-peer-urls: "https://${IPS[$i]}:2380"
      listen-client-urls: "https://127.0.0.1:2379,https://${IPS[$i]}:2379"
      advertise-client-urls: "https://${IPS[$i]}:2379"
      initial-advertise-peer-urls: "https://${IPS[$i]}:2380"
    serverCertSANs:
      - "${NODES[$i]}"
      - "${IPS[$i]}"
    peerCertSANs:
      - "${NODES[$i]}"
      - "${IPS[$i]}"
networking:
    podSubnet: "${POD_SUBNET}"
EOT
done

Init etcd on first node, generate ca certs and admin config

kubeadm="kubeadm --config=kubeadm-config-${HOSTNAME}.yaml"
$kubeadm init phase preflight
$kubeadm init phase certs all
$kubeadm init phase kubelet-start
$kubeadm init phase kubeconfig kubelet
$kubeadm init phase etcd local
$kubeadm init phase kubeconfig admin
systemctl start kubelet

Copy generated certs and kubeadm configs to other control plane nodes

NODES="node2 node3"
CERTS=$(find /etc/kubernetes/pki/ -maxdepth 1 -name '*ca.*' -o -name '*sa.*')
ETCD_CERTS=$(find /etc/kubernetes/pki/etcd/ -maxdepth 1 -name '*ca.*')for NODE in $NODES; do
  ssh $NODE mkdir -p /etc/kubernetes/pki/etcd
  scp $CERTS $NODE:/etc/kubernetes/pki/
  scp $ETCD_CERTS $NODE:/etc/kubernetes/pki/etcd/
  scp /etc/kubernetes/admin.conf $NODE:/etc/kubernetes
  scp kubeadm-config-$NODE.yaml $NODE:
done

Second node

Init etcd on second node

kubeadm="kubeadm --config=kubeadm-config-${HOSTNAME}.yaml"
$kubeadm init phase preflight
$kubeadm init phase certs all
$kubeadm init phase kubelet-start
$kubeadm init phase kubeconfig kubelet
$kubeadm init phase etcd local
systemctl start kubelet

Third node

Init kubernetes master with the last etcd on the last node:

(make sure that you have loadbalanser ip targeted to this node)

kubeadm init --config kubeadm-config-${HOSTNAME}.yaml

First and Second nodes

Now we can deploy kubernetes master on the first two nodes

kubeadm="kubeadm --config=kubeadm-config-${HOSTNAME}.yaml"
$kubeadm init phase kubeconfig all
$kubeadm init phase control-plane all
$kubeadm init phase mark-control-plane

And start keepalived daemon also:

systemctl start keepalived