dwcc2018怎么做網(wǎng)站十大跨界營(yíng)銷案例
寫在前面
- 博文內(nèi)容為K8s 鏡像緩存管理 kube-fledged 認(rèn)知
- 內(nèi)容涉及：
- kube-fledged 簡(jiǎn)單介紹
- 部署以及基本使用
- 理解不足小伙伴幫忙指正
不必太糾結(jié)于當(dāng)下，也不必太憂慮未來，當(dāng)你經(jīng)歷過一些事情的時(shí)候，眼前的風(fēng)景已經(jīng)和從前不一樣了?！迳洗簶?/font>
簡(jiǎn)單介紹
我們知道 k8s 上的容器調(diào)度需要在調(diào)度的節(jié)點(diǎn)行拉取當(dāng)前容器的鏡像，在一些特殊場(chǎng)景中，
- 需要
快速啟動(dòng)和/或擴(kuò)展的應(yīng)用程序。例如，由于數(shù)據(jù)量激增，執(zhí)行實(shí)時(shí)數(shù)據(jù)處理的應(yīng)用程序需要快速擴(kuò)展。 - 鏡像比較龐大，涉及多個(gè)版本，節(jié)點(diǎn)存儲(chǔ)有限，需要?jiǎng)討B(tài)清理不需要的鏡像
無服務(wù)器函數(shù)通常需要在幾分之一秒內(nèi)立即對(duì)傳入事件和啟動(dòng)容器做出反應(yīng)。- 在邊緣設(shè)備上運(yùn)行的
IoT 應(yīng)用程序，需要容忍邊緣設(shè)備和鏡像鏡像倉(cāng)庫(kù)之間的間歇性網(wǎng)絡(luò)連接。 - 如果需要從
專用倉(cāng)庫(kù)中拉取鏡像，并且無法授予每個(gè)人從此鏡像倉(cāng)庫(kù)拉取鏡像的訪問權(quán)限，則可以在群集的節(jié)點(diǎn)上提供鏡像。 - 如果集群管理員或操作員需要對(duì)應(yīng)用程序進(jìn)行升級(jí)，并希望事先驗(yàn)證是否可以成功拉取新鏡像。
kube-fledged 是一個(gè) kubernetes operator，用于直接在 Kubernetes 集群的 worker 節(jié)點(diǎn)上創(chuàng)建和管理容器鏡像緩存。它允許用戶定義鏡像列表以及這些鏡像應(yīng)緩存到哪些工作節(jié)點(diǎn)上（即拉取）。因此，應(yīng)用程序 Pod 幾乎可以立即啟動(dòng)，因?yàn)椴恍枰獜溺R像倉(cāng)庫(kù)中提取鏡像。
kube-fledged 提供了 CRUD API 來管理鏡像緩存的生命周期，并支持多個(gè)可配置的參數(shù)，可以根據(jù)自己的需要自定義功能。
Kubernetes 具有內(nèi)置的鏡像垃圾回收機(jī)制。節(jié)點(diǎn)中的 kubelet 會(huì)定期檢查磁盤使用率是否達(dá)到特定閾值（可通過標(biāo)志進(jìn)行配置）。一旦達(dá)到這個(gè)閾值，kubelet 會(huì)自動(dòng)刪除節(jié)點(diǎn)中所有未使用的鏡像。
需要在建議的解決方案中實(shí)現(xiàn)自動(dòng)和定期刷新機(jī)制。如果鏡像緩存中的鏡像被 kubelet 的 gc 刪除，下一個(gè)刷新周期會(huì)將已刪除的鏡像拉入鏡像緩存中。這可確保鏡像緩存是最新的。
設(shè)計(jì)流程
https://github.com/senthilrch/kube-fledged/blob/master/docs/kubefledged-architecture.png
部署 kube-fledged
Helm 方式部署
──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$mkdir kube-fledged
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$cd kube-fledged
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$export KUBEFLEDGED_NAMESPACE=kube-fledged
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$kubectl create namespace ${KUBEFLEDGED_NAMESPACE}
namespace/kube-fledged created
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$helm repo add kubefledged-charts https://senthilrch.github.io/kubefledged-charts/
"kubefledged-charts" has been added to your repositories
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "kubefledged-charts" chart repository
...Successfully got an update from the "kubescape" chart repository
...Successfully got an update from the "rancher-stable" chart repository
...Successfully got an update from the "skm" chart repository
...Successfully got an update from the "openkruise" chart repository
...Successfully got an update from the "awx-operator" chart repository
...Successfully got an update from the "botkube" chart repository
Update Complete. ?Happy Helming!?
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$helm install --verify kube-fledged kubefledged-charts/kube-fledged -n ${KUBEFLEDGED_NAMESPACE} --wait
實(shí)際部署中發(fā)現(xiàn)，由于網(wǎng)絡(luò)問題，chart 無法下載，所以通過 make deploy-using-yaml 使用 yaml 方式部署
Yaml 文件部署
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$git clone https://github.com/senthilrch/kube-fledged.git
正克隆到 'kube-fledged'...
remote: Enumerating objects: 10613, done.
remote: Counting objects: 100% (1501/1501), done.
remote: Compressing objects: 100% (629/629), done.
remote: Total 10613 (delta 845), reused 1357 (delta 766), pack-reused 9112
接收對(duì)象中: 100% (10613/10613), 34.58 MiB | 7.33 MiB/s, done.
處理 delta 中: 100% (4431/4431), done.
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$ls
kube-fledged
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$cd kube-fledged/
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged]
└─$make deploy-using-yaml
kubectl apply -f deploy/kubefledged-namespace.yaml
第一次部署，發(fā)現(xiàn)鏡像拉不下來
┌──[root@vms100.liruilongs.github.io]-[~]
└─$kubectl get all -n kube-fledged
NAME READY STATUS RESTARTS AGE
pod/kube-fledged-controller-df69f6565-drrqg 0/1 CrashLoopBackOff 35 (5h59m ago) 21h
pod/kube-fledged-webhook-server-7bcd589bc4-b7kg2 0/1 Init:CrashLoopBackOff 35 (5h58m ago) 21h
pod/kubefledged-controller-55f848cc67-7f4rl 1/1 Running 0 21h
pod/kubefledged-webhook-server-597dbf4ff5-l8fbh 0/1 Init:CrashLoopBackOff 34 (6h ago) 21hNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kube-fledged-webhook-server ClusterIP 10.100.194.199 <none> 3443/TCP 21h
service/kubefledged-webhook-server ClusterIP 10.101.191.206 <none> 3443/TCP 21hNAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/kube-fledged-controller 0/1 1 0 21h
deployment.apps/kube-fledged-webhook-server 0/1 1 0 21h
deployment.apps/kubefledged-controller 0/1 1 0 21h
deployment.apps/kubefledged-webhook-server 0/1 1 0 21hNAME DESIRED CURRENT READY AGE
replicaset.apps/kube-fledged-controller-df69f6565 1 1 0 21h
replicaset.apps/kube-fledged-webhook-server-7bcd589bc4 1 1 0 21h
replicaset.apps/kubefledged-controller-55f848cc67 1 1 0 21h
replicaset.apps/kubefledged-webhook-server-597dbf4ff5 1 1 0 21h
┌──[root@vms100.liruilongs.github.io]-[~]
└─$
這里我們找一下要拉取的鏡像
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$cat *.yaml | grep image:- image: senthilrch/kubefledged-controller:v0.10.0- image: senthilrch/kubefledged-webhook-server:v0.10.0- image: senthilrch/kubefledged-webhook-server:v0.10.0
單獨(dú)拉取一些，當(dāng)前使用 ansible 在所有工作節(jié)點(diǎn)批量操作
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible k8s_node -m shell -a "docker pull docker.io/senthilrch/kubefledged-cri-client:v0.10.0" -i host.yaml
其他相關(guān)的鏡像都拉取一下
操作完成之后容器狀態(tài)全部正常
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl -n kube-fledged get all
NAME READY STATUS RESTARTS AGE
pod/kube-fledged-controller-df69f6565-wdb4g 1/1 Running 0 13h
pod/kube-fledged-webhook-server-7bcd589bc4-j8xxp 1/1 Running 0 13h
pod/kubefledged-controller-55f848cc67-klxlm 1/1 Running 0 13h
pod/kubefledged-webhook-server-597dbf4ff5-ktbsh 1/1 Running 0 13hNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kube-fledged-webhook-server ClusterIP 10.100.194.199 <none> 3443/TCP 36h
service/kubefledged-webhook-server ClusterIP 10.101.191.206 <none> 3443/TCP 36hNAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/kube-fledged-controller 1/1 1 1 36h
deployment.apps/kube-fledged-webhook-server 1/1 1 1 36h
deployment.apps/kubefledged-controller 1/1 1 1 36h
deployment.apps/kubefledged-webhook-server 1/1 1 1 36hNAME DESIRED CURRENT READY AGE
replicaset.apps/kube-fledged-controller-df69f6565 1 1 1 36h
replicaset.apps/kube-fledged-webhook-server-7bcd589bc4 1 1 1 36h
replicaset.apps/kubefledged-controller-55f848cc67 1 1 1 36h
replicaset.apps/kubefledged-webhook-server-597dbf4ff5 1 1 1 36h
驗(yàn)證是否安裝成功
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged]
└─$kubectl get pods -n kube-fledged -l app=kubefledged
NAME READY STATUS RESTARTS AGE
kubefledged-controller-55f848cc67-klxlm 1/1 Running 0 16h
kubefledged-webhook-server-597dbf4ff5-ktbsh 1/1 Running 0 16h
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged]
└─$kubectl get imagecaches -n kube-fledged
No resources found in kube-fledged namespace.
使用 kubefledged
創(chuàng)建鏡像緩存對(duì)象
根據(jù) Demo 文件，創(chuàng)建鏡像緩存對(duì)象
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged]
└─$cd deploy/
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$cat kubefledged-imagecache.yaml
---
apiVersion: kubefledged.io/v1alpha2
kind: ImageCache
metadata:# Name of the image cache. A cluster can have multiple image cache objectsname: imagecache1namespace: kube-fledged# The kubernetes namespace to be used for this image cache. You can choose a different namepace as per your preferencelabels:app: kubefledgedkubefledged: imagecache
spec:# The "cacheSpec" field allows a user to define a list of images and onto which worker nodes those images should be cached (i.e. pre-pulled).cacheSpec:# Specifies a list of images (nginx:1.23.1) with no node selector, hence these images will be cached in all the nodes in the cluster- images:- ghcr.io/jitesoft/nginx:1.23.1# Specifies a list of images (cassandra:v7 and etcd:3.5.4-0) with a node selector, hence these images will be cached only on the nodes selected by the node selector- images:- us.gcr.io/k8s-artifacts-prod/cassandra:v7- us.gcr.io/k8s-artifacts-prod/etcd:3.5.4-0nodeSelector:tier: backend# Specifies a list of image pull secrets to pull images from private repositories into the cacheimagePullSecrets:- name: myregistrykey
官方的 Demo 中對(duì)應(yīng)的 鏡像拉取不下來，所以換一下
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$docker pull us.gcr.io/k8s-artifacts-prod/cassandra:v7
Error response from daemon: Get "https://us.gcr.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$
為了測(cè)試選擇器標(biāo)簽的使用,我們找一個(gè)節(jié)點(diǎn)的標(biāo)簽單獨(dú)做鏡像緩存
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$kubectl get nodes --show-labels
同時(shí)我們直接從公有倉(cāng)庫(kù)拉取鏡像，所以不需要 imagePullSecrets 對(duì)象
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$vim kubefledged-imagecache.yaml
修改后的 yaml 文件
- 添加了一個(gè)所有節(jié)點(diǎn)的 liruilong/my-busybox:latest 鏡像緩存
- 添加了一個(gè)
kubernetes.io/hostname: vms105.liruilongs.github.io對(duì)應(yīng)標(biāo)簽選擇器的liruilong/hikvision-sdk-config-ftp:latest鏡像緩存
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$cat kubefledged-imagecache.yaml
---
apiVersion: kubefledged.io/v1alpha2
kind: ImageCache
metadata:# Name of the image cache. A cluster can have multiple image cache objectsname: imagecache1namespace: kube-fledged# The kubernetes namespace to be used for this image cache. You can choose a different namepace as per your preferencelabels:app: kubefledgedkubefledged: imagecache
spec:# The "cacheSpec" field allows a user to define a list of images and onto which worker nodes those images should be cached (i.e. pre-pulled).cacheSpec:# Specifies a list of images (nginx:1.23.1) with no node selector, hence these images will be cached in all the nodes in the cluster- images:- liruilong/my-busybox:latest# Specifies a list of images (cassandra:v7 and etcd:3.5.4-0) with a node selector, hence these images will be cached only on the nodes selected by the node selector- images:- liruilong/hikvision-sdk-config-ftp:latestnodeSelector:kubernetes.io/hostname: vms105.liruilongs.github.io# Specifies a list of image pull secrets to pull images from private repositories into the cache#imagePullSecrets:#- name: myregistrykey
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$
直接創(chuàng)建報(bào)錯(cuò)了
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$kubectl create -f kubefledged-imagecache.yaml
Error from server (InternalError): error when creating "kubefledged-imagecache.yaml": Internal error occurred: failed calling webhook "validate-image-cache.kubefledged.io": failed to call webhook: Post "https://kubefledged-webhook-server.kube-fledged.svc:3443/validate-image-cache?timeout=1s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubefledged.io")
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$kubectl get imagecaches -n kube-fledged
No resources found in kube-fledged namespace.
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$
解決辦法，刪除對(duì)應(yīng)的對(duì)象，重新創(chuàng)建
我在當(dāng)前項(xiàng)目的一個(gè) issues 下面找到了解決辦法 https://github.com/senthilrch/kube-fledged/issues/76
看起來這是因?yàn)?Webhook CA 是硬編碼的，但是當(dāng) webhook 服務(wù)器啟動(dòng)時(shí)，會(huì)生成一個(gè)新的 CA 捆綁包并更新 webhook 配置。當(dāng)發(fā)生另一個(gè)部署時(shí)，將重新應(yīng)用原始 CA 捆綁包，并且 Webhook 請(qǐng)求開始失敗，直到再次重新啟動(dòng) Webhook 組件以修補(bǔ)捆綁包init-server
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged]
└─$make remove-kubefledged-and-operator
# Remove kubefledged
kubectl delete -f deploy/kubefledged-operator/deploy/crds/charts.helm.kubefledged.io_v1alpha2_kubefledged_cr.yaml
error: resource mapping not found for name: "kube-fledged" namespace: "kube-fledged" from "deploy/kubefledged-operator/deploy/crds/charts.helm.kubefledged.io_v1alpha2_kubefledged_cr.yaml": no matches for kind "KubeFledged" in version "charts.helm.kubefledged.io/v1alpha2"
ensure CRDs are installed first
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged]
└─$make deploy-using-yaml
kubectl apply -f deploy/kubefledged-namespace.yaml
namespace/kube-fledged created
kubectl apply -f deploy/kubefledged-crd.yaml
customresourcedefinition.apiextensions.k8s.io/imagecaches.kubefledged.io unchanged
....................
kubectl rollout status deployment kubefledged-webhook-server -n kube-fledged --watch
Waiting for deployment "kubefledged-webhook-server" rollout to finish: 0 of 1 updated replicas are available...
deployment "kubefledged-webhook-server" successfully rolled out
kubectl get pods -n kube-fledged
NAME READY STATUS RESTARTS AGE
kubefledged-controller-55f848cc67-76c4v 1/1 Running 0 112s
kubefledged-webhook-server-597dbf4ff5-56h6z 1/1 Running 0 66s
重新創(chuàng)建緩存對(duì)象，創(chuàng)建成功
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$kubectl create -f kubefledged-imagecache.yaml
imagecache.kubefledged.io/imagecache1 created
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$kubectl get imagecaches -n kube-fledged
NAME AGE
imagecache1 10s
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$
查看當(dāng)前被納管的鏡像緩存
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$kubectl get imagecaches imagecache1 -n kube-fledged -o json
{"apiVersion": "kubefledged.io/v1alpha2","kind": "ImageCache","metadata": {"creationTimestamp": "2024-03-01T15:08:42Z","generation": 83,"labels": {"app": "kubefledged","kubefledged": "imagecache"},"name": "imagecache1","namespace": "kube-fledged","resourceVersion": "20169836","uid": "3a680a57-d8ab-444f-b9c9-4382459c5c72"},"spec": {"cacheSpec": [{"images": ["liruilong/my-busybox:latest"]},{"images": ["liruilong/hikvision-sdk-config-ftp:latest"],"nodeSelector": {"kubernetes.io/hostname": "vms105.liruilongs.github.io"}}]},"status": {"completionTime": "2024-03-02T01:06:47Z","message": "All requested images pulled succesfully to respective nodes","reason": "ImageCacheRefresh","startTime": "2024-03-02T01:05:33Z","status": "Succeeded"}
}
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$
通過 ansible 來驗(yàn)證
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/my-busybox" -i host.yaml
192.168.26.102 | CHANGED | rc=0 >>
liruilong/my-busybox latest 497b83a63aad 11 months ago 1.24MB
192.168.26.101 | CHANGED | rc=0 >>
liruilong/my-busybox latest 497b83a63aad 11 months ago 1.24MB
192.168.26.103 | CHANGED | rc=0 >>
liruilong/my-busybox latest 497b83a63aad 11 months ago 1.24MB
192.168.26.105 | CHANGED | rc=0 >>
liruilong/my-busybox latest 497b83a63aad 11 months ago 1.24MB
192.168.26.100 | CHANGED | rc=0 >>
liruilong/my-busybox latest 497b83a63aad 11 months ago 1.24MB
192.168.26.106 | CHANGED | rc=0 >>
liruilong/my-busybox latest 497b83a63aad 11 months ago 1.24MB
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/hikvision-sdk-config-ftp" -i host.yaml
192.168.26.102 | FAILED | rc=1 >>
non-zero return code
192.168.26.100 | FAILED | rc=1 >>
non-zero return code
192.168.26.103 | FAILED | rc=1 >>
non-zero return code
192.168.26.105 | CHANGED | rc=0 >>
liruilong/hikvision-sdk-config-ftp latest a02cd03b4342 4 months ago 830MB
192.168.26.101 | FAILED | rc=1 >>
non-zero return code
192.168.26.106 | FAILED | rc=1 >>
non-zero return code
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$
開啟自動(dòng)刷新
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl annotate imagecaches imagecache1 -n kube-fledged kubefledged.io/refresh-imagecache=
imagecache.kubefledged.io/imagecache1 annotated
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$
添加鏡像緩存
添加一個(gè)新的鏡像緩存
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl get imagecaches.kubefledged.io -n kube-fledged imagecache1 -o json
{"apiVersion": "kubefledged.io/v1alpha2","kind": "ImageCache","metadata": {"creationTimestamp": "2024-03-01T15:08:42Z","generation": 92,"labels": {"app": "kubefledged","kubefledged": "imagecache"},"name": "imagecache1","namespace": "kube-fledged","resourceVersion": "20175233","uid": "3a680a57-d8ab-444f-b9c9-4382459c5c72"},"spec": {"cacheSpec": [{"images": ["liruilong/my-busybox:latest","liruilong/jdk1.8_191:latest"]},{"images": ["liruilong/hikvision-sdk-config-ftp:latest"],"nodeSelector": {"kubernetes.io/hostname": "vms105.liruilongs.github.io"}}]},"status": {"completionTime": "2024-03-02T01:43:32Z","message": "All requested images pulled succesfully to respective nodes","reason": "ImageCacheUpdate","startTime": "2024-03-02T01:40:34Z","status": "Succeeded"}
}
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$
通過 ansible 確認(rèn)
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/jdk1.8_191" -i host.yaml
192.168.26.101 | FAILED | rc=1 >>
non-zero return code
192.168.26.100 | FAILED | rc=1 >>
non-zero return code
192.168.26.102 | FAILED | rc=1 >>
non-zero return code
192.168.26.103 | FAILED | rc=1 >>
non-zero return code
192.168.26.105 | FAILED | rc=1 >>
non-zero return code
192.168.26.106 | FAILED | rc=1 >>
non-zero return code
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/jdk1.8_191" -i host.yaml
192.168.26.101 | CHANGED | rc=0 >>
liruilong/jdk1.8_191 latest 17dbd4002a8c 5 years ago 170MB
192.168.26.102 | CHANGED | rc=0 >>
liruilong/jdk1.8_191 latest 17dbd4002a8c 5 years ago 170MB
192.168.26.100 | CHANGED | rc=0 >>
liruilong/jdk1.8_191 latest 17dbd4002a8c 5 years ago 170MB
192.168.26.103 | CHANGED | rc=0 >>
liruilong/jdk1.8_191 latest 17dbd4002a8c 5 years ago 170MB
192.168.26.105 | CHANGED | rc=0 >>
liruilong/jdk1.8_191 latest 17dbd4002a8c 5 years ago 170MB
192.168.26.106 | CHANGED | rc=0 >>
liruilong/jdk1.8_191 latest 17dbd4002a8c 5 years ago 170MB
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$
刪除鏡像緩存
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl edit imagecaches imagecache1 -n kube-fledged
imagecache.kubefledged.io/imagecache1 edited
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl get imagecaches.kubefledged.io -n kube-fledged imagecache1 -o json
{"apiVersion": "kubefledged.io/v1alpha2","kind": "ImageCache","metadata": {"creationTimestamp": "2024-03-01T15:08:42Z","generation": 94,"labels": {"app": "kubefledged","kubefledged": "imagecache"},"name": "imagecache1","namespace": "kube-fledged","resourceVersion": "20175766","uid": "3a680a57-d8ab-444f-b9c9-4382459c5c72"},"spec": {"cacheSpec": [{"images": ["liruilong/jdk1.8_191:latest"]},{"images": ["liruilong/hikvision-sdk-config-ftp:latest"],"nodeSelector": {"kubernetes.io/hostname": "vms105.liruilongs.github.io"}}]},"status": {"message": "Image cache is being updated. Please view the status after some time","reason": "ImageCacheUpdate","startTime": "2024-03-02T01:48:03Z","status": "Processing"}
}
通過 Ansible 確認(rèn)，可以看到無論是 mastere 上的節(jié)點(diǎn)還是 work 的節(jié)點(diǎn)，對(duì)應(yīng)的鏡像緩存都被清理
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/my-busybox" -i host.yaml
192.168.26.102 | CHANGED | rc=0 >>
liruilong/my-busybox latest 497b83a63aad 11 months ago 1.24MB
192.168.26.101 | CHANGED | rc=0 >>
liruilong/my-busybox latest 497b83a63aad 11 months ago 1.24MB
192.168.26.105 | FAILED | rc=1 >>
non-zero return code
192.168.26.100 | CHANGED | rc=0 >>
liruilong/my-busybox latest 497b83a63aad 11 months ago 1.24MB
192.168.26.103 | FAILED | rc=1 >>
non-zero return code
192.168.26.106 | FAILED | rc=1 >>
non-zero return code
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/my-busybox" -i host.yaml
192.168.26.105 | FAILED | rc=1 >>
non-zero return code
192.168.26.102 | FAILED | rc=1 >>
non-zero return code
192.168.26.103 | FAILED | rc=1 >>
non-zero return code
192.168.26.101 | FAILED | rc=1 >>
non-zero return code
192.168.26.100 | FAILED | rc=1 >>
non-zero return code
192.168.26.106 | FAILED | rc=1 >>
non-zero return code
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$
這里需要注意如果清除所有的鏡像緩存，那么需要把 images 下的數(shù)組 寫成 “”.
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl edit imagecaches imagecache1 -n kube-fledged
imagecache.kubefledged.io/imagecache1 edited
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/jdk1.8_191" -i host.yaml
192.168.26.102 | FAILED | rc=1 >>
non-zero return code
192.168.26.101 | FAILED | rc=1 >>
non-zero return code
192.168.26.100 | FAILED | rc=1 >>
non-zero return code
192.168.26.105 | FAILED | rc=1 >>
non-zero return code
192.168.26.103 | FAILED | rc=1 >>
non-zero return code
192.168.26.106 | FAILED | rc=1 >>
non-zero return code
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl get imagecaches.kubefledged.io -n kube-fledged imagecache1 -o json
{"apiVersion": "kubefledged.io/v1alpha2","kind": "ImageCache","metadata": {"creationTimestamp": "2024-03-01T15:08:42Z","generation": 98,"labels": {"app": "kubefledged","kubefledged": "imagecache"},"name": "imagecache1","namespace": "kube-fledged","resourceVersion": "20176849","uid": "3a680a57-d8ab-444f-b9c9-4382459c5c72"},"spec": {"cacheSpec": [{"images": [""]},{"images": ["liruilong/hikvision-sdk-config-ftp:latest"],"nodeSelector": {"kubernetes.io/hostname": "vms105.liruilongs.github.io"}}]},"status": {"completionTime": "2024-03-02T01:52:16Z","message": "All cached images succesfully deleted from respective nodes","reason": "ImageCacheUpdate","startTime": "2024-03-02T01:51:47Z","status": "Succeeded"}
}
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$
如果通過下面的方式刪除，直接注釋調(diào)對(duì)應(yīng)的標(biāo)簽
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$cat kubefledged-imagecache.yaml
---
apiVersion: kubefledged.io/v1alpha2
kind: ImageCache
metadata:# Name of the image cache. A cluster can have multiple image cache objectsname: imagecache1namespace: kube-fledged# The kubernetes namespace to be used for this image cache. You can choose a different namepace as per your preferencelabels:app: kubefledgedkubefledged: imagecache
spec:# The "cacheSpec" field allows a user to define a list of images and onto which worker nodes those images should be cached (i.e. pre-pulled).cacheSpec:# Specifies a list of images (nginx:1.23.1) with no node selector, hence these images will be cached in all the nodes in the cluster#- images:#- liruilong/my-busybox:latest# Specifies a list of images (cassandra:v7 and etcd:3.5.4-0) with a node selector, hence these images will be cached only on the nodes selected by the node selector- images:- liruilong/hikvision-sdk-config-ftp:latestnodeSelector:kubernetes.io/hostname: vms105.liruilongs.github.io# Specifies a list of image pull secrets to pull images from private repositories into the cache#imagePullSecrets:#- name: myregistrykey
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$
那么會(huì)報(bào)下面的錯(cuò)
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$kubectl edit imagecaches imagecache1 -n kube-fledged
error: imagecaches.kubefledged.io "imagecache1" could not be patched: admission webhook "validate-image-cache.kubefledged.io" denied the request: Mismatch in no. of image lists
You can run `kubectl replace -f /tmp/kubectl-edit-4113815075.yaml` to try this update again.
博文部分內(nèi)容參考
? 文中涉及參考鏈接內(nèi)容版權(quán)歸原作者所有，如有侵權(quán)請(qǐng)告知,如果你認(rèn)可它不要吝嗇星星哦 😃
https://github.com/senthilrch/kube-fledged
? 2018-2024 liruilonger@gmail.com, All rights reserved. 保持署名-非商用-相同方式共享(CC BY-NC-SA 4.0)