k8s kubernetes Lesson 3 k8s by hand
Kubernetes wiki
https://wiki.gslin.org/wiki/Kubernetes
How to Install a Kubernetes Docker Cluster on CentOS 7
https://www.howtoforge.com/tutorial/centos-kubernetes-docker-cluster/
https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/
https://sueboy.blogspot.com/2019/11/kubernetes-nodeport.html
https://rickhw.github.io/2019/03/17/Container/Install-K8s-with-Kubeadm/
=====
https://www.howtoforge.com/tutorial/centos-kubernetes-docker-cluster/
only one master/node k8s
yum update
setenforce 0
sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux
modprobe br_netfilter
echo '1' > /proc/sys/net/bridge/bridge-nf-call-iptables
swapoff -a
vim /etc/fstab >> mark swap
yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
yum install -y docker-ce
cat << eof > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg
https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF
yum install -y kubelet kubeadm kubectl
sudo reboot
systemctl start docker && systemctl enable docker
systemctl start kubelet && systemctl enable kubelet
docker info | grep -i cgroup
sed -i 's/cgroup-driver=systemd/cgroup-driver=cgroupfs/g' /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
systemctl daemon-reload
systemctl restart kubelet
## kubeadm init --apiserver-advertise-address=10.0.15.10 --pod-network-cidr=10.244.0.0/16
kubeadm init --pod-network-cidr=10.244.0.0/16
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
kubectl get nodes
kubectl get pods --all-namespaces
https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#control-plane-node-isolation
kubectl taint nodes --all node-role.kubernetes.io/master-
If use calico
https://docs.projectcalico.org/v3.5/getting-started/kubernetes/
DNS
kubectl -n kube-system edit configmap coredns
# Add log
data:
Corefile: |
.:53 {
log
errors
PS： As 10.96.0.10 is a Cluster IP, It is not ping-able. You can use telnet or curl.
https://github.com/kubernetes/kubernetes/issues/78926#issuecomment-503898794
So use nslookup
#check DNS
注意: 不要使用busybox:latest，最高版本的nslookup工具有bug
apiVersion: v1
kind: Pod
metadata:
name: busybox
namespace: default
spec:
containers:
- name: busybox
image: busybox:1.28.4
command:
- sleep
- "3600"
imagePullPolicy: IfNotPresent
restartPolicy: Always
kubectl exec -ti busybox -- nslookup kubernetes.default
#check DNS log
for p in $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name); do kubectl logs --namespace=kube-system $p; done
PS： If need to kubeadm reset … see bootom
Dashboard
##### dashboard
wget https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/recommended/kubernetes-dashboard.yaml
#modify clusterip -> nodeport
nano kubernetes-dashboard.yaml
# ------------------- Dashboard Service ------------------- #
kind: Service
apiVersion: v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kube-system
spec:
type: NodePort
ports:
- port: 443
targetPort: 8443
nodePort: 30393
selector:
k8s-app: kubernetes-dashboard
kubectl apply -n kube-system -f kubernetes-dashboard.yaml
kubectl -n kube-system get service kubernetes-dashboard
[root@localhost ~]# kubectl -n kube-system get service kubernetes-dashboard
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes-dashboard NodePort 10.110.11.88 443:30393/TCP 103
use 30393 EX: https://192.168.99.118:30393 (192.168.99.118 is virtualbox give ip)
Dashboard ERROR NET::ERR_CERT_INVALID
https://github.com/kubernetes/dashboard/issues/2954
https://github.com/kubernetes/dashboard/wiki/Certificate-management
-subj “/C=/ST=/L=/O=/OU=/CN=kubernetes-dashboard”
mkdir certs ( At user home path )
openssl req -nodes -newkey rsa:2048 -keyout certs/dashboard.key -out certs/dashboard.csr -subj "/C=/ST=/L=/O=/OU=/CN=kubernetes-dashboard"
openssl x509 -req -sha256 -days 365 -in certs/dashboard.csr -signkey certs/dashboard.key -out certs/dashboard.crt
kubectl delete -f kubernetes-dashboard.yaml
kubectl create secret generic kubernetes-dashboard-certs --from-file=certs -n kube-system
kubectl create -f kubernetes-dashboard.yaml
Try again. https://192.168.99.118:30393
Get Dashboard token
https://github.com/kubernetes/dashboard/wiki/Creating-sample-user
nano dashboard-adminuser.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: admin-user
namespace: kube-system
kubectl create -f dashboard-adminuser.yaml
kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}')
If see admin-user token, just copy token pass to web login.
IF Get Error
try to Create ClusterRoleBinding
Create dashboard-ClusterRoleBinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: admin-user
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: admin-user
namespace: kube-system
kubectl create -f dashboard-ClusterRoleBinding.yaml
kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}')
nodePort range 1-65535
nano /etc/kubernetes/manifests/kube-apiserver.yaml
#add command
--service-node-port-range=1-65535
kubeadm reset
sudo kubeadm reset
rm -rf .kube
sudo sysctl net.bridge.bridge-nf-call-iptables=1
sudo kubeadm init --pod-network-cidr=10.244.0.0/16
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
kubectl get nodes
kubectl get pods --all-namespaces
kubectl taint nodes --all node-role.kubernetes.io/master-
kubectl create secret generic kubernetes-dashboard-certs --from-file=certs -n kube-system
kubectl create -f kubernetes-dashboard.yaml
## This login token
kubectl create -f dashboard-adminuser.yaml
kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}')
kubectl create -f dashboard-ClusterRoleBinding.yaml
kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}')
nano /etc/kubernetes/manifests/kube-apiserver.yaml
#add command
--service-node-port-range=1-65535
... follow up step
kubeadm update
kubeadm upgrade plan
kubeadm upgrade apply
kubeadm command
clear && for p in $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name); do kubectl logs --namespace=kube-system $p; done
kubectl edit cm coredns -n kube-system
coreDNS no response
coreDNS error
coreDNS work correct
How to fix? Only way is reboot OS.
1. coreDNS error 2. reboot os 3.get coreDNS work correct
