BlogSetting up One Primary and Two Standby openGauss Databases on Kubernetes

Setting up One Primary and Two Standby openGauss Databases on Kubernetes

Bin Zhou2021-10-09Setting up One Primary and Two Standby openGauss Databases on Kubernetes

Setting up One Primary and Two Standby openGauss Databases on Kubernetes

Initialize the environment as the master or node role.

IP	Hostname	Role
*...*	k8smaster	master
*...*	k8snode01	node

Disable firewalld.

systemctl stop firewalld
systemctl disable firewalld

1. Update Docker.

rpm -qa|grep docker
yum remove docker
curl -fsSL https://get.docker.com/ | sh
systemctl start docker
systemctl enable docker

2. Prepare the Kubernetes source.

vim /etc/yum.repos.d/kubernetes.repo

[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg

yum install -y kubeadm kubectl etcd

3. Check image names required by kubeadm.

[root@ecs-66cc dockerimages]# kubeadm config images list

k8s.gcr.io/kube-apiserver:v1.21.1
k8s.gcr.io/kube-controller-manager:v1.21.1
k8s.gcr.io/kube-scheduler:v1.21.1
k8s.gcr.io/kube-proxy:v1.21.1
k8s.gcr.io/pause:3.4.1
k8s.gcr.io/etcd:3.4.13-0
k8s.gcr.io/coredns/coredns:v1.8.0

4. Install images required for Kubernetes.

docker pull registry.aliyuncs.com/google_containers/kube-apiserver:v1.21.1
docker pull registry.aliyuncs.com/google_containers/kube-controller-manager:v1.21.1
docker pull registry.aliyuncs.com/google_containers/kube-scheduler:v1.21.1
docker pull registry.aliyuncs.com/google_containers/kube-proxy:v1.21.1
docker pull registry.aliyuncs.com/google_containers/pause:3.4.1
docker pull registry.aliyuncs.com/google_containers/etcd:3.4.13-0
docker pull coredns/coredns:1.8.0

5. Modify the Docker tags to match those required by kubeadm.

Download images from Chinese sources.

docker tag registry.aliyuncs.com/google_containers/kube-apiserver:v1.21.1 k8s.gcr.io/kube-apiserver:v1.21.1
docker tag registry.aliyuncs.com/google_containers/kube-controller-manager:v1.21.1 k8s.gcr.io/kube-controller-manager:v1.21.1
docker tag registry.aliyuncs.com/google_containers/kube-scheduler:v1.21.1 k8s.gcr.io/kube-scheduler:v1.21.1
docker tag registry.aliyuncs.com/google_containers/kube-proxy:v1.21.1 k8s.gcr.io/kube-proxy:v1.21.1
docker tag registry.aliyuncs.com/google_containers/pause:3.4.1 k8s.gcr.io/pause:3.4.1
docker tag registry.aliyuncs.com/google_containers/etcd:3.4.13-0 k8s.gcr.io/etcd:3.4.13-0
docker tag docker.io/coredns/coredns:1.8.0 k8s.gcr.io/coredns/coredns:v1.8.0

Delete invalid images.

docker rmi registry.aliyuncs.com/google_containers/kube-apiserver:v1.21.1
docker rmi registry.aliyuncs.com/google_containers/kube-controller-manager:v1.21.1
docker rmi registry.aliyuncs.com/google_containers/kube-scheduler:v1.21.1
docker rmi registry.aliyuncs.com/google_containers/kube-proxy:v1.21.1
docker rmi registry.aliyuncs.com/google_containers/pause:3.4.1
docker rmi registry.aliyuncs.com/google_containers/etcd:3.4.13-0
docker rmi coredns/coredns:1.8.0

6. Write Kubernetes initialization configurations and initialize Kubernetes as master.

kubeadm.yaml

apiVersion: kubeadm.k8s.io/v1beta2
clusterName: kubernetes
kind: ClusterConfiguration
kubernetesVersion: v1.21.1
controllerManager:
  extraArgs:
    horizontal-pod-autoscaler-use-rest-clients: "true"
    horizontal-pod-autoscaler-sync-period: "10s"
    node-monitor-grace-period: "10s"
apiServer:
  extraArgs:
    runtime-config: "api/all=true"

Copy the configuration file to Kubernetes and specify it during initialization.

cp kubeadm.yaml /etc/kubernetes/manifests/
kubeadm init --config kubeadm.yaml

After the operation is successful, retain the following information for later use:

kubeadm join ***.***.***.***:6443 --token ru2883.u4rhwkx5oqrol9at \
        --discovery-token-ca-cert-hash sha256:f2dbe7ce49b322e8145b6e9b4303e56468ad1352daabecb797f7bd161a64e018

Perform initialization.

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Install the network plugin.

kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"

7. Join nodes.

After the node is installed, it does not need to be initialized. Run the kubeadm join command to add the node to the primary node.

kubeadm join ***.***.***.***:6443 --token ru2883.u4rhwkx5oqrol9at \
        --discovery-token-ca-cert-hash sha256:f2dbe7ce49b322e8145b6e9b4303e56468ad1352daabecb797f7bd161a64e018

8. Import images as master and node.

docker load < opengauss.tar.gz

9. Create a service (SVC) as master.

Create an SVC for pods:kubectl create -f opengauss-svc.yaml

The content of the opengauss-svc.yaml file is as follows:

apiVersion: v1
kind: Service
metadata:
  name: opengauss-service-1
spec:
  ports:
  - port: 5432
    protocol: TCP
    targetPort: 5432
    name: gsql
  - port: 5434
    protocol: TCP
    targetPort: 5434
    name: localport
  - port: 2380
    protocol: TCP
    targetPort: 2380
    name: etcd1-service
  - port: 2379
    protocol: TCP
    targetPort: 2379
    name: etcd1-local
  selector:
    app: opengauss-1
  clusterIP: None

---

apiVersion: v1
kind: Service
metadata:
  name: opengauss-service-2
spec:
  ports:
  - port: 5432
    protocol: TCP
    targetPort: 5432
    name: gsql
  - port: 5434
    protocol: TCP
    targetPort: 5434
    name: localport
  - port: 2380
    protocol: TCP
    targetPort: 2380
    name: etcd1-service
  - port: 2379
    protocol: TCP
    targetPort: 2379
    name: etcd1-local
  selector:
    app: opengauss-2
  clusterIP: None

---

apiVersion: v1
kind: Service
metadata:
  name: opengauss-service-3
spec:
  ports:
  - port: 5432
    protocol: TCP
    targetPort: 5432
    name: gsql
  - port: 5434
    protocol: TCP
    targetPort: 5434
    name: localport
  - port: 2380
    protocol: TCP
    targetPort: 2380
    name: etcd1-service
  - port: 2379
    protocol: TCP
    targetPort: 2379
    name: etcd1-local
  selector:
    app: opengauss-3
  clusterIP: None

10. Create a pod as master.

Create the primary and standby pods of openGauss.

kubectl create -f opengauss-pod.yaml

The content of the opengauss-pod.yaml file is as follows:

apiVersion: v1
kind: Pod
metadata:
  name: opengauss-1
  labels:
    app: opengauss-1
spec:
  restartPolicy: Never
  containers:
  - name: opengauss-1
    image: opengauss:1.0.5
    imagePullPolicy: Never
    securityContext:
      runAsUser: 0
    volumeMounts:
    - mountPath: /var/lib/opengauss/data/
      name: openGauss-volume
    ports:
    - containerPort: 5432
      name: opengauss
    env:
    - name: HOST_NAME
      value: opengauss-1
    - name: HOST_IP
      valueFrom:
        fieldRef:
          fieldPath: status.podIP
    - name: PEER_IPS
      value: opengauss-service-2,opengauss-service-3
    - name: PEER_HOST_NAMES
      value: opengauss-2,opengauss-3
    - name: PORT
      value: "5432"
    - name: GS_PASSWORD
      value: "Test@56789"
    - name: SERVER_MODE
      value: primary
    - name: db_config
      value:
  volumes:
  - name: openGauss-volume
    hostPath:
      path: /data/opengauss-1/
      type: DirectoryOrCreate

---

apiVersion: v1
kind: Pod
metadata:
  name: opengauss-2
  labels:
    app: opengauss-2
spec:
  restartPolicy: Never
  containers:
  - name: opengauss-2
    image: opengauss:1.0.5
    imagePullPolicy: Never
    securityContext:
      runAsUser: 0
    volumeMounts:
    - mountPath: /var/lib/opengauss/data/
      name: openGauss-volume
    ports:
    - containerPort: 5432
      name: opengauss
    env:
    - name: HOST_NAME
      value: opengauss-2
    - name: HOST_IP
      valueFrom:
        fieldRef:
          fieldPath: status.podIP
    - name: PEER_IPS
      value: opengauss-service-1,opengauss-service-3
    - name: PEER_HOST_NAMES
      value: opengauss-1,opengauss-3
    - name: PORT
      value: "5432"
    - name: GS_PASSWORD
      value: "Test@56789"
    - name: SERVER_MODE
      value: standby
    - name: db_config
      value:
  volumes:
  - name: openGauss-volume
    hostPath:
      path: /data/opengauss-2/
      type: DirectoryOrCreate

---

apiVersion: v1
kind: Pod
metadata:
  name: opengauss-3
  labels:
    app: opengauss-3
spec:
  restartPolicy: Never
  containers:
  - name: opengauss-3
    image: opengauss:1.0.5
    imagePullPolicy: Never
    securityContext:
      runAsUser: 0
    volumeMounts:
    - mountPath: /var/lib/opengauss/data/
      name: openGauss-volume
    ports:
    - containerPort: 5432
      name: opengauss
    env:
    - name: HOST_NAME
      value: opengauss-3
    - name: HOST_IP
      valueFrom:
        fieldRef:
          fieldPath: status.podIP
    - name: PEER_IPS
      value: opengauss-service-1,opengauss-service-2
    - name: PEER_HOST_NAMES
      value: opengauss-1,opengauss-2
    - name: PORT
      value: "5432"
    - name: GS_PASSWORD
      value: "Test@56789"
    - name: SERVER_MODE
      value: standby
    - name: db_config
      value:
  volumes:
  - name: openGauss-volume
    hostPath:
      path: /data/opengauss-3/
      type: DirectoryOrCreate

11. Test the database as master.

1. Access the primary node of the database.

kubectl exec -it opengauss-1 -- /bin/bash
2. Switch the user.
su omm
3. Access the database.
gsql

12. Common Commands

All commands are executed as master.

View cluster nodes.
kubectl get node
View cluster pods.
kubectl get pod --all-namespaces
Check the cluster service.
kubectl get svc --all-namespaces
Access the container.
kubectl exec -it Container name (pod name for a single container) -n opengauss -- /bin/bash
Run the following command to view pod or SVC details:
kubectl describe pod/svc pod/SVC name -n pod/SVC namespaces
View the log information.
kubectl logs pod Pod name -n Pod namespaces