kubeadm部署k8s集群

规划
三台机器
172.16.10.11 部署单机版k8s,部署主jenkins与一个agent
172.16.10.12 裸部署gitlab
172.16.10.13 充当开发人员的开发机,用于上传代码

部署k8s集群

官方文档:
https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/
kubeadm部署k8s高可用集群的官方文档:
https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/high-availability/

(1)基本配置

注意:
确保CPU至少2核
内存的话至少1.5G,推荐2-3G

file

(2)在所有主机执行如下操作

1、配置静态ip地址

2、 每台主机都关闭NetworkManager

systemctl stop NetworkManager
systemctl disable NetworkManager

3、每台主机均关闭selinux与防火墙

sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/sysconfig/selinux
sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config
setenforce 0
systemctl stop firewalld.service
systemctl disable firewalld.service

4、关闭swap分区

# Kubernetes 1.8开始要求关闭系统的Swap,如果不关闭,默认配置下kubelet将无法启动,所以我们有两种处理方式,采用一种即可
方式一:关闭swap分区
swapoff -a  # 先临时关闭,立即生效
sed -i 's/.*swap.*/#&/' /etc/fstab # 注释掉swap,永久关闭,保证即便重启主机也会生效
 
方式二: kubelet忽略swap
echo 'KUBELET_EXTRA_ARGS="--fail-swap-on=false"' > /etc/sysconfig/kubelet

5、为三台主机配置各自的主机名

hostnamectl set-hostname master
hostnamectl set-hostname node01
hostnamectl set-hostname node02

6、添加解析

cat >> /etc/hosts << EOF
172.16.10.14 master
172.16.10.15 node01
172.16.10.16 node02
EOF

7、每台机器均修改ssh配置

加快远程链接速度,可选,但建议做

sed -ri '/#UseDNS yes/c UseDNS no' /etc/ssh/sshd_config
 ​
systemctl restart sshd

8、做免密登录(只有主节点做,此步为建议步骤,非必须)

[root@master ~]# ssh-keygen

[root@master ~]# ssh-copy-id -i root@master
[root@master ~]# ssh-copy-id -i root@node01
[root@master ~]# ssh-copy-id -i root@node02

9、更新系统软件(排除内核)

yum install epel-release -y && yum update -y --exclud=kernel*

10、安装基础常用软件

yum install wget expect vim net-tools ntp bash-completion ipvsadm ipset jq iptables conntrack sysstat libseccomp -y

# 其他(选做)
yum -y install python-setuptools python-pip gcc gcc-c++ autoconf libjpeg libjpeg-devel libpng libpng-devel freetype freetype-devel libxml2 libxml2-devel \
zlib zlib-devel glibc glibc-devel glib2 glib2-devel bzip2 bzip2-devel zip unzip ncurses ncurses-devel curl curl-devel e2fsprogs \
e2fsprogs-devel krb5-devel libidn libidn-devel openssl openssh openssl-devel nss_ldap openldap openldap-devel openldap-clients \
openldap-servers libxslt-devel libevent-devel ntp libtool-ltdl bison libtool vim-enhanced python wget lsof iptraf strace lrzsz \
kernel-devel kernel-headers pam-devel tcl tk cmake ncurses-devel bison setuptool popt-devel net-snmp screen perl-devel \
pcre-devel net-snmp screen tcpdump rsync sysstat man iptables sudo libconfig git  bind-utils \
tmux elinks numactl iftop bwm-ng net-tools expect

11、更新系统内核(docker 对系统内核要求比较高,最好使用4.4+),非必须操作,推荐做
一般来说,只有从https://www.kernel.org/ 下载并编译安装的内核才是官方内核,可以看出目前的稳定版版本为5.18.10

file

不过,大多数 Linux 发行版提供自行维护的内核,可以通过 yum 或 rpm 等包管理系统升级。
ELRepo是一个为Linux提供驱动程序和内核映像的存储库,这里的升级方案就是采用ELRepo提供的内核通道。
ELRepo官网:http://elrepo.org/tiki/tiki-index.php

# 1、升级系统内核

#查看 yum 中可升级的内核版本
yum list kernel --showduplicates
#如果list中有需要的版本可以直接执行 update 升级,多数是没有的,所以要按以下步骤操作

#导入ELRepo软件仓库的公共秘钥
rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org

#Centos7系统安装ELRepo
yum -y install https://www.elrepo.org/elrepo-release-7.el7.elrepo.noarch.rpm
#Centos8系统安装ELRepo
#yum -y install https://www.elrepo.org/elrepo-release-8.el8.elrepo.noarch.rpm

#查看ELRepo提供的内核版本
yum --disablerepo="*" --enablerepo="elrepo-kernel" list available

#kernel-lt:表示longterm,即长期支持的内核;当前为5.4.
#kernel-ml:表示mainline,即当前主线的内核;当前为5.17.
#安装主线内核
yum --enablerepo=elrepo-kernel install kernel-ml.x86_64 -y

#查看系统可用内核,并设置启动项
sudo awk -F\' '$1=="menuentry " {print i++ " : " $2}' /etc/grub2.cfg

#0 : CentOS Linux (5.17.1-1.el7.elrepo.x86_64) 7 (Core)
#1 : CentOS Linux (3.10.0-1160.53.1.el7.x86_64) 7 (Core)
#2 : CentOS Linux (3.10.0-1160.el7.x86_64) 7 (Core)
#3 : CentOS Linux (0-rescue-20220208145000711038896885545492) 7 (Core)

#指定开机启动内核版本
grub2-set-default 0 # 或者 grub2-set-default 'CentOS Linux (5.17.1-1.el7.elrepo.x86_64) 7 (Core)'

#生成 grub 配置文件
grub2-mkconfig -o /boot/grub2/grub.cfg

#查看当前默认启动的内核
grubby --default-kernel

#重启系统,验证
uname -r

8、配置ntp服务,保证集群服务器时间统一(统一时间非常重要,必须要做)

# 大前提:chrony服务端客户端配置完后,重启chronyd服务即可快速完成时间同步,在这之后就不要再手动去修改时间了,一切让时间服务器自己去同步

# ====================>chrony服务端:master节点<====================

# 1、安装
yum -y install chrony
​
# 2、修改配置文件
mv /etc/chrony.conf /etc/chrony.conf.bak
​
cat > /etc/chrony.conf << EOF
server ntp1.aliyun.com iburst minpoll 4 maxpoll 10
server ntp2.aliyun.com iburst minpoll 4 maxpoll 10
server ntp3.aliyun.com iburst minpoll 4 maxpoll 10
server ntp4.aliyun.com iburst minpoll 4 maxpoll 10
server ntp5.aliyun.com iburst minpoll 4 maxpoll 10
server ntp6.aliyun.com iburst minpoll 4 maxpoll 10
server ntp7.aliyun.com iburst minpoll 4 maxpoll 10
driftfile /var/lib/chrony/drift
makestep 10 3
rtcsync
allow 0.0.0.0/0
local stratum 10
keyfile /etc/chrony.keys
logdir /var/log/chrony
stratumweight 0.05
noclientlog
logchange 0.5

EOF
​
# 4、启动chronyd服务
systemctl restart chronyd.service # 最好重启,这样无论原来是否启动都可以重新加载配置
systemctl enable chronyd.service
systemctl status chronyd.service

====================>chrony客户端:其他节点,完全一样的配置与操作<====================

# 下述步骤一次性粘贴到每个客户端执行即可
# 1、安装chrony
yum -y install chrony
# 2、需改客户端配置文件
mv /etc/chrony.conf /etc/chrony.conf.bak
cat > /etc/chrony.conf << EOF
server master iburst
driftfile /var/lib/chrony/drift
makestep 10 3
rtcsync
local stratum 10
keyfile /etc/chrony.key
logdir /var/log/chrony
stratumweight 0.05
noclientlog
logchange 0.5

EOF
# 3、启动chronyd
systemctl restart chronyd.service
systemctl enable chronyd.service
systemctl status chronyd.service

# 4、验证
chronyc sources -v

7、安装docker

# 1、选做,卸载之前的docker
yum -y remove docker \
docker-client \
docker-client-latest \
docker-common \
docker-latest \
docker-latest-logrotate \
docker-logrotate \
docker-selinux \
docker-engine-selinux \
docker-engine

# 2、安装docker所需安装包
yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum install docker-ce -y

# 3、启动并设置开机启动
systemctl start docker && systemctl enable docker && systemctl status docker

# 4、基本配置
cat > /etc/docker/daemon.json << EOF
{
"exec-opts": ["native.cgroupdriver=systemd"],
"registry-mirrors":["https://reg-mirror.qiniu.com/"],
"live-restore":true
}
EOF
# 5、重启
systemctl restart docker && docker info

8、拉取镜像(再次强调,每台机器都需要操作)
kubeadm部署时会去指定的地址拉取镜像,该地址在墙外无法访问,所以我们从阿里云拉取,并tag为指定的地址即可

#1、=====>编写脚本
cat > dockpullImages1.18.1.sh << EOF
#!/bin/bash
##所需要的镜像名字
#k8s.gcr.io/kube-apiserver:v1.18.1
#k8s.gcr.io/kube-controller-manager:v1.18.1
#k8s.gcr.io/kube-scheduler:v1.18.1
#k8s.gcr.io/kube-proxy:v1.18.1
#k8s.gcr.io/pause:3.2
#k8s.gcr.io/etcd:3.4.3-0
#k8s.gcr.io/coredns:1.6.7
###拉取镜像
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver:v1.18.1
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager:v1.18.1
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler:v1.18.1
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy:v1.18.1
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.2
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/etcd:3.4.3-0
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:1.6.7
###修改tag
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver:v1.18.1 k8s.gcr.io/kube-apiserver:v1.18.1
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager:v1.18.1 k8s.gcr.io/kube-controller-manager:v1.18.1
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler:v1.18.1 k8s.gcr.io/kube-scheduler:v1.18.1
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy:v1.18.1 k8s.gcr.io/kube-proxy:v1.18.1
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.2 k8s.gcr.io/pause:3.2
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/etcd:3.4.3-0 k8s.gcr.io/etcd:3.4.3-0
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:1.6.7 k8s.gcr.io/coredns:1.6.7

EOF

# 2、在所有节点执行该脚本
sh dockpullImages1.18.1.sh

9、安装kubelet、kubeadm 和 kubectl(所有节点执行)
kubelet 运行在 Cluster 所有节点上,负责启动 Pod 和容器。
kubeadm 用于初始化 Cluster。
kubectl 是 Kubernetes 命令行工具。通过 kubectl 可以部署和管理应用,查看各种资源,创建、删除和更新各种组件。

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

sed -ri 's/gpgcheck=1/gpgcheck=0/g' /etc/yum.repos.d/kubernetes.repo 

在所有节点安装

1.安装
yum makecache fast
# yum install -y kubelet kubeadm kubectl ipvsadm  #注意,这样默认是下载最新版本v1.22.2
======================================================================
[root@master ~]# yum install -y kubelet-1.18.1-0.x86_64 kubeadm-1.18.1-0.x86_64 kubectl-1.18.1-0.x86_64 ipvsadm 

2.加载ipvs相关内核模块
yum install -y conntrack-tools ipvsadm ipvsadmin ipset conntrack libseccomp 

如果重新开机,需要重新加载(可以写在 /etc/rc.local 中开机自动加载)
modprobe ip_vs
modprobe ip_vs_rr
modprobe ip_vs_wrr
modprobe ip_vs_sh
#modprobe nf_conntrack_ipv4 # 如果是3.x内核,那么应该加载这一样
modprobe nf_conntrack # 如果是高版本内核比如5.x,那么应该加载这个。在高版本内核已经把nf_conntrack_ipv4替换为nf_conntrack了。

3.编辑文件添加开机启动
cat >> /etc/rc.local << EOF
modprobe ip_vs
modprobe ip_vs_rr
modprobe ip_vs_wrr
modprobe ip_vs_sh
modprobe nf_conntrack
#modprobe nf_conntrack_ipv4

EOF

chmod +x /etc/rc.local

重启服务器   reboot

4.配置:
配置转发相关参数,否则可能会出错
cat <<EOF >  /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
vm.swappiness=0
EOF

5.使配置生效
sysctl --system

6.如果net.bridge.bridge-nf-call-iptables报错,加载br_netfilter模块
# modprobe br_netfilter
# sysctl -p /etc/sysctl.d/k8s.conf

7.查看是否加载成功
[root@master ~]# lsmod | grep ip_vs
ip_vs_sh               16384  0 
ip_vs_wrr              16384  0 
ip_vs_rr               16384  0 
ip_vs                 159744  6 ip_vs_rr,ip_vs_sh,ip_vs_wrr
nf_conntrack          151552  5 xt_conntrack,nf_nat,nf_conntrack_netlink,xt_MASQUERADE,ip_vs
nf_defrag_ipv6         24576  2 nf_conntrack,ip_vs
libcrc32c              16384  4 nf_conntrack,nf_nat,xfs,ip_vs

10、启动kubelet

#1.配置kubelet使用pause镜像
#配置变量:
systemctl start docker && systemctl enable docker
DOCKER_CGROUPS=$(docker info | grep 'Cgroup Driver' | cut -d' ' -f4)
echo $DOCKER_CGROUPS

#这个是使用国内的源。-###注意我们使用谷歌的镜像--操作下面的第3标题
#2.配置kubelet的cgroups
cat >/etc/sysconfig/kubelet<<EOF
KUBELET_EXTRA_ARGS="--cgroup-driver=$DOCKER_CGROUPS --pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google_containers/pause-amd64:3.2"
EOF

#cat >/etc/sysconfig/kubelet<<EOF
#KUBELET_EXTRA_ARGS="--cgroup-driver=cgroupfs --pod-infra-container-image=k8s.gcr.io/pause:3.2"
#EOF

启动

systemctl daemon-reload
systemctl enable kubelet && systemctl restart kubelet

# 注意在这里使用 # systemctl status kubelet,你会发现报错误信息;
# 7月 10 23:28:36 master systemd[1]: Unit kubelet.service entered failed state.
# 7月 10 23:28:36 master systemd[1]: kubelet.service failed.

#运行 # journalctl -xefu kubelet 命令查看systemd日志会发现提示缺少一些问题件
#这个错误在运行kubeadm init 生成CA证书后会被自动解决,此处可先忽略。
#简单地说就是在kubeadm init 之前kubelet会不断重启。

11、初始化master

kubeadm init \
--kubernetes-version=v1.18.1 \
--service-cidr=10.96.0.0/12 \
--pod-network-cidr=10.244.0.0/16 \
--apiserver-advertise-address=172.16.10.14 \
--ignore-preflight-errors=Swap

注意修改apiserver-advertise-address为master节点ip
参数解释:

–kubernetes-version: 用于指定k8s版本;
–apiserver-advertise-address:用于指定kube-apiserver监听的ip地址,就是 master本机IP地址。
–pod-network-cidr:用于指定Pod的网络范围; 10.244.0.0/16
–service-cidr:用于指定SVC的网络范围;
–image-repository: 指定阿里云镜像仓库地址

看到以下信息表示安装成功

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 172.16.10.14:6443 --token n3mvgw.56ul27rjtox7fr3n \
    --discovery-token-ca-cert-hash sha256:b93e84284d278e4056ee5a1b0370a20f49a1878df8a3492f5f855e2d5141e6e7 

成功后注意最后一个命令,这个join命令可以用来添加节点。
注意保持好kubeadm join,后面会用到的。
如果初始化失败,请使用如下代码清除后重新初始化

# kubeadm reset

12、按照提示配置kubectl

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

# 查看
  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

13、配置使用网络插件
要让 Kubernetes Cluster 能够工作,必须安装 Pod 网络,否则 Pod 之间无法通信。
Kubernetes 支持多种网络方案,这里我们先使用 flannel,后面还会讨论 Canal。
flannel.yaml

---
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: psp.flannel.unprivileged
  annotations:
    seccomp.security.alpha.kubernetes.io/allowedProfileNames: docker/default
    seccomp.security.alpha.kubernetes.io/defaultProfileName: docker/default
    apparmor.security.beta.kubernetes.io/allowedProfileNames: runtime/default
    apparmor.security.beta.kubernetes.io/defaultProfileName: runtime/default
spec:
  privileged: false
  volumes:
  - configMap
  - secret
  - emptyDir
  - hostPath
  allowedHostPaths:
  - pathPrefix: "/etc/cni/net.d"
  - pathPrefix: "/etc/kube-flannel"
  - pathPrefix: "/run/flannel"
  readOnlyRootFilesystem: false
  # Users and groups
  runAsUser:
    rule: RunAsAny
  supplementalGroups:
    rule: RunAsAny
  fsGroup:
    rule: RunAsAny
  # Privilege Escalation
  allowPrivilegeEscalation: false
  defaultAllowPrivilegeEscalation: false
  # Capabilities
  allowedCapabilities: ['NET_ADMIN', 'NET_RAW']
  defaultAddCapabilities: []
  requiredDropCapabilities: []
  # Host namespaces
  hostPID: false
  hostIPC: false
  hostNetwork: true
  hostPorts:
  - min: 0
    max: 65535
  # SELinux
  seLinux:
    # SELinux is unused in CaaSP
    rule: 'RunAsAny'
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: flannel
rules:
- apiGroups: ['extensions']
  resources: ['podsecuritypolicies']
  verbs: ['use']
  resourceNames: ['psp.flannel.unprivileged']
- apiGroups:
  - ""
  resources:
  - pods
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - nodes
  verbs:
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - nodes/status
  verbs:
  - patch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: flannel
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: flannel
subjects:
- kind: ServiceAccount
  name: flannel
  namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: flannel
  namespace: kube-system
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: kube-flannel-cfg
  namespace: kube-system
  labels:
    tier: node
    app: flannel
data:
  cni-conf.json: |
    {
      "name": "cbr0",
      "cniVersion": "0.3.1",
      "plugins": [
        {
          "type": "flannel",
          "delegate": {
            "hairpinMode": true,
            "isDefaultGateway": true
          }
        },
        {
          "type": "portmap",
          "capabilities": {
            "portMappings": true
          }
        }
      ]
    }
  net-conf.json: |
    {
      "Network": "10.244.0.0/16",
      "Backend": {
        "Type": "vxlan"
      }
    }
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: kube-flannel-ds
  namespace: kube-system
  labels:
    tier: node
    app: flannel
spec:
  selector:
    matchLabels:
      app: flannel
  template:
    metadata:
      labels:
        tier: node
        app: flannel
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: kubernetes.io/os
                operator: In
                values:
                - linux
      hostNetwork: true
      priorityClassName: system-node-critical
      tolerations:
      - operator: Exists
        effect: NoSchedule
      serviceAccountName: flannel
      initContainers:
      - name: install-cni
        image: registry.cn-hangzhou.aliyuncs.com/alvinos/flanned:v0.13.1-rc1
        command:
        - cp
        args:
        - -f
        - /etc/kube-flannel/cni-conf.json
        - /etc/cni/net.d/10-flannel.conflist
        volumeMounts:
        - name: cni
          mountPath: /etc/cni/net.d
        - name: flannel-cfg
          mountPath: /etc/kube-flannel/
      containers:
      - name: kube-flannel
        image: registry.cn-hangzhou.aliyuncs.com/alvinos/flanned:v0.13.1-rc1
        command:
        - /opt/bin/flanneld
        args:
        - --ip-masq
        - --kube-subnet-mgr
        resources:
          requests:
            cpu: "100m"
            memory: "50Mi"
          limits:
            cpu: "100m"
            memory: "50Mi"
        securityContext:
          privileged: false
          capabilities:
            add: ["NET_ADMIN", "NET_RAW"]
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        volumeMounts:
        - name: run
          mountPath: /run/flannel
        - name: flannel-cfg
          mountPath: /etc/kube-flannel/
      volumes:
      - name: run
        hostPath:
          path: /run/flannel
      - name: cni
        hostPath:
          path: /etc/cni/net.d
      - name: flannel-cfg
        configMap:
          name: kube-flannel-cfg

部署flannel

kubectl apply -f flannel.yaml

查看

[root@master ~]# kubectl get nodes
NAME     STATUS   ROLES    AGE     VERSION
master   Ready    master   6m49s   v1.18.1

[root@master ~]# kubectl  get pods -n kube-system
NAME                             READY   STATUS    RESTARTS   AGE
coredns-66bff467f8-mtrkv         1/1     Running   0          3m57s
coredns-66bff467f8-qhwv4         1/1     Running   0          3m57s
etcd-master                      1/1     Running   0          4m9s
kube-apiserver-master            1/1     Running   0          4m9s
kube-controller-manager-master   1/1     Running   0          4m9s
kube-flannel-ds-b25g6            1/1     Running   0          32s
kube-proxy-gkqjv                 1/1     Running   0          3m57s
kube-scheduler-master            1/1     Running   0          4m9s

14、添加node01与node02
在所有node节点执行,登录到node节点,确保已经安装了docker和kubeadm,kubelet,kubectl

如果报错开启ip转发:
# sysctl -w net.ipv4.ip_forward=1

在所有node节点操作,此命令为初始化master成功后返回的结果
kubeadm join 172.16.10.14:6443 --token n3mvgw.56ul27rjtox7fr3n \
    --discovery-token-ca-cert-hash sha256:b93e84284d278e4056ee5a1b0370a20f49a1878df8a3492f5f855e2d5141e6e7 

# 在master节点查看,最开始两个加入的节点是NoReady,过一会后处于Ready状态
[root@master ~]# kubectl get nodes
NAME     STATUS   ROLES    AGE     VERSION
master   Ready    master   7m45s   v1.18.1
node01   Ready    <none>   25s     v1.18.1
node02   Ready    <none>   22s     v1.18.1

15、kubeadm init创建完集群后,有pod一直是pending状态

kubectl describe pod如果发现问题
3 node(s) had taints that the pod didn't tolerate.
 
kubernetes出于安全考虑默认情况下无法在master节点上部署pod,于是用下面方法去掉master节点的污点:
kubectl taint nodes --all node-role.kubernetes.io/master-

16、了解:

# 1、移除node方法
 kubectl drain node02 --delete-local-data --force --ignore-daemonsets
 kubectl delete nodes node02

# 2、添加已删除节点

前提:token未失效

如果这个时候再想添加进来这个node,需要执行两步操作

第一步:停掉kubelet(需要添加进来的节点操作)

[root@node02 ~]# systemctl stop kubelet

第二步:删除相关文件

[root@node02 ~]# rm -rf /etc/kubernetes/*

第三步:添加节点

因为之前的token还有效,我这里并没有超出token的有效期;直接执行加入集群的命令即可;
kubeadm join 172.16.10.14:6443 --token n3mvgw.56ul27rjtox7fr3n \
    --discovery-token-ca-cert-hash sha256:b93e84284d278e4056ee5a1b0370a20f49a1878df8a3492f5f855e2d5141e6e7 

# 3、忘记token了,怎么办,在master节点
[root@master ~]# kubeadm token list  # 可以拿到token值,然后再用openssl解析出hash值就行
TOKEN                     TTL         EXPIRES                     USAGES                   DESCRIPTION                                                EXTRA GROUPS
n3mvgw.56ul27rjtox7fr3n   23h         2022-07-11T23:32:18+08:00   authentication,signing   The default bootstrap token generated by 'kubeadm init'.   system:bootstrappers:kubeadm:default-node-token
[root@master ~]# openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'
b93e84284d278e4056ee5a1b0370a20f49a1878df8a3492f5f855e2d5141e6e7

然后重新拼接命令kubeadm join就行

kubeadm join 172.16.10.14:6443 --token n3mvgw.56ul27rjtox7fr3n     --discovery-token-ca-cert-hash sha256:b93e84284d278e4056ee5a1b0370a20f49a1878df8a3492f5f855e2d5141e6e7
上一篇
下一篇
Copyright © 2022 Egon的技术星球 egonlin.com 版权所有 帮助IT小伙伴学到真正的技术