使用k8s的过程中,我们的服务都会打印大量的日志,为了方便查看排查定位问题,大部分时候会将日志打印在标准输出,但是有的时候为了方便日志的切割和收集,会将日志写到容器内的某个目录下日志文件。如果我们的日志会占用大量的磁盘空间,有可能还会外挂存储来将日志持久化。
容器内的业务日志有多种输出方式,其实不同日志输出方式,最终都是存在了pod所在的宿主机上,不同的方式对应了宿主机不同的目录,下面我们来详细说明下docker和containerd这2中运行时不同日志输出所在的宿主机目录。
当你了解了日志输出方式和宿主机目录的一一对应,那么后续遇到节点上磁盘空间被占满的问题,你就可以快速清理对应的日志文件来释放磁盘空间。
这里具体说明都是没有对docker或者containerd的存储目录进行数据盘挂载,如果有进行数据盘挂载,则根据实际挂载目录查找即可。
docker日志存储目录说明
这里举例说明docker和kubelet的默认存储目录是/var/lib/docker和/var/lib/kubelet
日志标准输出
容器的标准输出日志默认是在/var/lib/docker/containers/[container_id]/,标准输出日志默认会自动切割轮转,日志切割配置可以查看/etc/docker/daemon.json文件的log-opts配置
1 2 3 4 5 6 7 8 9 10 11
| [root@VM-0-3-centos 6035c5820cf52c7684057d70b4f0f206deb82bad8b931a2e63473ca0f4206384] 6035c5820cf5 quay.io/prometheus/prometheus "/bin/prometheus --w…" 5 days ago Up 5 days k8s_prometheus_prometheus-tke-cls-b3mg1p92-0_prom-rl9st2ym_87269735-b041-41e9-b85f-aef897ac6200_1 [root@VM-0-3-centos 6035c5820cf52c7684057d70b4f0f206deb82bad8b931a2e63473ca0f4206384] [root@VM-0-3-centos 6035c5820cf52c7684057d70b4f0f206deb82bad8b931a2e63473ca0f4206384] total 141544 -rw-r----- 1 root root 44903243 Oct 29 14:33 6035c5820cf52c7684057d70b4f0f206deb82bad8b931a2e63473ca0f4206384-json.log -rw-r----- 1 root root 100000151 Oct 27 19:49 6035c5820cf52c7684057d70b4f0f206deb82bad8b931a2e63473ca0f4206384-json.log.1 drwx------ 2 root root 4096 Oct 23 21:00 checkpoints -rw------- 1 root root 8224 Oct 23 21:00 config.v2.json -rw-r--r-- 1 root root 2625 Oct 23 21:00 hostconfig.json drwx------ 2 root root 4096 Oct 23 21:00 mounts
|
日志写入容器内日志文件
如果日志写到容器日志文件,但是没有进行持久化挂载,则会存在/var/lib/docker/overlay2/[merge_id]/merged/[log_path],具体merge_id,我们可以inspect容器id查找到,而log_path则是你业务日志在容器内的存储路径,下面我们查找下nginx的日志。
1 2 3 4 5 6 7 8 9 10 11
| [root@node1 ~] "MergedDir": "/var/lib/docker/overlay2/136d009b75834ed0867f8e27107c4a945f6c921a629b929ebdecaf11a13629c0/merged", [root@node1 ~] [root@node1 merged] [root@node1 merged] [root@node1 nginx] total 2597864 lrwxrwxrwx 1 101 101 11 Nov 3 2020 access.log -> /dev/stdout lrwxrwxrwx 1 101 101 11 Nov 3 2020 error.log -> /dev/stderr -rw-r--r-- 1 101 101 2660179942 Oct 29 15:01 nginx_access.log -rw-r--r-- 1 101 101 0 Oct 10 19:58 nginx_error.log
|
有的时候我们只能发现/var/lib/docker/overlay2目录占用了大量的磁盘空间,那么我们怎么找到是哪个容器占用了呢?首先查找/var/lib/docker/overlay2目录下的大文件
1 2 3
| [root@node1 ~]# find /var/lib/docker/overlay2 -size +1G -exec du -sm {} \; | sort -nr 2538 /var/lib/docker/overlay2/136d009b75834ed0867f8e27107c4a945f6c921a629b929ebdecaf11a13629c0/merged/var/log/nginx/nginx_access.log 2538 /var/lib/docker/overlay2/136d009b75834ed0867f8e27107c4a945f6c921a629b929ebdecaf11a13629c0/diff/var/log/nginx/nginx_access.log
|
找到对应的merge_id后,我们在查找到对应的容器id,采用下面命令,当能查看到多行信息,则对应的id是容器id,这里容器id是ba62275eab39
1 2 3 4 5 6 7 8 9 10 11
| [root@node1 ~]# docker ps | awk '{print $1}' | xargs -n 1 -I {} -- sh -c "echo {} && docker inspect {} |grep 136d009b75834ed0867f8e27107c4a945f6c921a629b929ebdecaf11a13629c0 -C 1" CONTAINER faf1856fda4e 1fe6f4025507 ba62275eab39 "Data": { "LowerDir": "/var/lib/docker/overlay2/136d009b75834ed0867f8e27107c4a945f6c921a629b929ebdecaf11a13629c0-init/diff:/var/lib/docker/overlay2/a4cdd5dc7a462904471de2d056570ed838b2317e4b59fd5613992196df9ef17f/diff:/var/lib/docker/overlay2/d514eb8d6219065c3a7332c6a0438e0ca041569da38de6fa7a14b84c3b1e5e4e/diff:/var/lib/docker/overlay2/1e0e34d059ac42479185c87e1ad5922cd77d78ab6a53bdd14b45636570146577/diff:/var/lib/docker/overlay2/aa4d53898ff012bfdcbdb453fcea67a9946cb403b27905b586eaf12977f29384/diff:/var/lib/docker/overlay2/dace2fb0d20cf856aa8d9ae16211d7f0e15ad7e0ad4ac87b278c0a14b134f432/diff:/var/lib/docker/overlay2/c1d91f716865e4367007d3d9fe3d19719f70a7befff1a629a074e79b835052f0/diff:/var/lib/docker/overlay2/fbbfb23d2eccb88bb79657a94f06dbde76249256097cf7bdc231df667c8ab6c9/diff:/var/lib/docker/overlay2/07e7cbb11f955c3edbcc5ac66cf966ef3da16d4c336a2022652c1468177934fb/diff:/var/lib/docker/overlay2/a2858bf53ea2c668ccd89bd9a87217513f8c5ef2c1e70ed35885400b883d7e1f/diff:/var/lib/docker/overlay2/ddea490967639aaaabd3bbaa7872be77d642b4e34203672bbc585d76d074d3dd/diff:/var/lib/docker/overlay2/f4e7cae5b5e70b89da96684df655f8f5a830dc751b68f638c2890b44cc6da891/diff:/var/lib/docker/overlay2/891e829575951f1e7a52fea41a138beea7684ca9e723244b84f87f82954f49ca/diff:/var/lib/docker/overlay2/1a2310107ffb59198ed6ff553f320f08f69d11bd7f7bb384f57761eb88852d9b/diff:/var/lib/docker/overlay2/1a75669e46cf247de83f5e74a1205cd1ecaaa13fb14cc6ef0fe7216f06b66786/diff:/var/lib/docker/overlay2/36fff493df92c24d07d46a50ab63d0132c9c18ab22d48b33ed2a1a85fcd05fd6/diff", "MergedDir": "/var/lib/docker/overlay2/136d009b75834ed0867f8e27107c4a945f6c921a629b929ebdecaf11a13629c0/merged", "UpperDir": "/var/lib/docker/overlay2/136d009b75834ed0867f8e27107c4a945f6c921a629b929ebdecaf11a13629c0/diff", "WorkDir": "/var/lib/docker/overlay2/136d009b75834ed0867f8e27107c4a945f6c921a629b929ebdecaf11a13629c0/work" },
|
找到容器id后,进入容器的var/log/nginx/目录清理日志即可。
上面的方法是查找容器存储下的大文件,然后根据大文件所在目录找到对应的容器,但是有的时候不一定是有单个文件大,而是文件夹比较大,里面文件数量比较多,导致占用了大量的磁盘空间,这个时候该怎么查找呢?这个时候我们就只能一层层找到这个大文件夹,下面我们来说明下。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86
| [root@VM-0-10-centos ~]# du -hm --max-depth=1 /var/lib/docker | sort -rn | head -10 44790 /var/lib/docker 39315 /var/lib/docker/overlay2 5420 /var/lib/docker/containers 55 /var/lib/docker/image 1 /var/lib/docker/volumes 1 /var/lib/docker/trust 1 /var/lib/docker/tmp 1 /var/lib/docker/swarm 1 /var/lib/docker/runtimes 1 /var/lib/docker/plugins [root@VM-0-10-centos ~]# du -hm --max-depth=1 /var/lib/docker/overlay2 | sort -rn | head -10 39315 /var/lib/docker/overlay2 2659 /var/lib/docker/overlay2/611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639 1499 /var/lib/docker/overlay2/5ca45180744594cd01643aada607b351dc9a1f011d2807752d2f55f470892a9b 1314 /var/lib/docker/overlay2/bbc80fd995e6b0aa97bc1326a72a4f4049bae33407a0c34c39561d31f8c17598 1253 /var/lib/docker/overlay2/c84ef307e9147365d7a73d160ce898cc70ef93cf1f69be94d7598dd0dc456ae5 1191 /var/lib/docker/overlay2/d5409735e53b1efc9902b72a3b170d4a523c233b807f2f4954ca3cca1cc42e49 1046 /var/lib/docker/overlay2/9ec2b9732d5b64a9a03223d870b4aec520af2b6845ab3c391c2fa95f1e45fe2e 1046 /var/lib/docker/overlay2/69975a1ecf6f7345acf6e97c946966468c69ae850ebd2a63b63ed8f0cd598412 1009 /var/lib/docker/overlay2/693ad529427a4d135b064e2e428c6b269ac3017e1ce0f61faaa49de9cc7a0d81 733 /var/lib/docker/overlay2/cd41e5558d7f317da34280b64e3f446f225139a0c82f59e047af9be8610a1009 [root@VM-0-10-centos ~]# du -hm --max-depth=1 /var/lib/docker/overlay2/611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639 | sort -rn | head -10 2659 /var/lib/docker/overlay2/611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639 1408 /var/lib/docker/overlay2/611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639/merged 1251 /var/lib/docker/overlay2/611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639/diff 1 /var/lib/docker/overlay2/611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639/work [root@VM-0-10-centos ~]# du -hm --max-depth=1 /var/lib/docker/overlay2/611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639/merged/ | sort -rn | head -10 1408 /var/lib/docker/overlay2/611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639/merged/ 1249 /var/lib/docker/overlay2/611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639/merged/tmp 141 /var/lib/docker/overlay2/611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639/merged/usr 13 /var/lib/docker/overlay2/611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639/merged/home 4 /var/lib/docker/overlay2/611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639/merged/var 3 /var/lib/docker/overlay2/611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639/merged/etc 1 /var/lib/docker/overlay2/611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639/merged/sys 1 /var/lib/docker/overlay2/611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639/merged/run 1 /var/lib/docker/overlay2/611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639/merged/root 1 /var/lib/docker/overlay2/611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639/merged/proc [root@VM-0-10-centos ~]# du -hm --max-depth=1 /var/lib/docker/overlay2/611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639/merged/tmp/ | sort -rn | head -10 1249 /var/lib/docker/overlay2/611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639/merged/tmp/ 861 /var/lib/docker/overlay2/611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639/merged/tmp/nvd-data835905766 388 /var/lib/docker/overlay2/611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639/merged/tmp/ubuntu-cve-tracker540878348 2 /var/lib/docker/overlay2/611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639/merged/tmp/alpine-secdbUpdater788692628 [root@VM-0-10-centos ~]# du -hm --max-depth=1 /var/lib/docker/overlay2/611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639/merged/tmp/nvd-data835905766/ | sort -rn | head -10 861 /var/lib/docker/overlay2/611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639/merged/tmp/nvd-data835905766/ [root@VM-0-10-centos ~]# cd /var/lib/docker/overlay2/611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639/merged/tmp/nvd-data835905766/ [root@VM-0-10-centos /var/lib/docker/overlay2/611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639/merged/tmp/nvd-data835905766]# ls -alhS total 861M -rw-r--r-- 1 10000 10000 104M Dec 21 16:11 2020.json -rw-r--r-- 1 10000 10000 82M Dec 21 16:11 2019.json -rw-r--r-- 1 10000 10000 81M Dec 21 16:11 2021.json -rw-r--r-- 1 10000 10000 74M Dec 21 16:11 2018.json -rw-r--r-- 1 10000 10000 70M Dec 21 16:11 2017.json -rw-r--r-- 1 10000 10000 50M Dec 21 16:11 2016.json -rw-r--r-- 1 10000 10000 44M Dec 21 16:10 2013.json -rw-r--r-- 1 10000 10000 39M Dec 21 16:10 2014.json -rw-r--r-- 1 10000 10000 39M Dec 21 16:10 2012.json -rw-r--r-- 1 10000 10000 39M Dec 21 16:11 2015.json -rw-r--r-- 1 10000 10000 34M Dec 21 16:10 2011.json -rw-r--r-- 1 10000 10000 34M Dec 21 16:10 2010.json -rw-r--r-- 1 10000 10000 32M Dec 21 16:10 2008.json -rw-r--r-- 1 10000 10000 32M Dec 21 16:10 2009.json -rw-r--r-- 1 10000 10000 30M Dec 21 16:10 2006.json -rw-r--r-- 1 10000 10000 28M Dec 21 16:10 2007.json -rw-r--r-- 1 10000 10000 21M Dec 21 16:10 2002.json -rw-r--r-- 1 10000 10000 20M Dec 21 16:10 2005.json -rw-r--r-- 1 10000 10000 13M Dec 21 16:10 2004.json -rw-r--r-- 1 10000 10000 5.8M Dec 21 16:10 2003.json drwx------ 2 10000 10000 4.0K Dec 21 16:11 . drwxrwxrwt 1 root root 4.0K Dec 29 00:20 .. [root@VM-0-10-centos ~]# docker ps | awk '{print $1}' | xargs -n 1 -I {} -- sh -c "echo {} && docker inspect {} |grep 611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639 -C 1" CONTAINER Error: No such object: CONTAINER 9d2cd497b1cb e99ce9b11191 6bdca3204a62 b6dfe23c31e9 "Data": { "LowerDir": "/var/lib/docker/overlay2/611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639-init/diff:/var/lib/docker/overlay2/180f09778c2e17d1ffd73aa5be45244ee414f4423bcc2f81e460c52b9a3ca90b/diff:/var/lib/docker/overlay2/bd60ea9822fbf0169cc9ed116e7c82a9d25268995508c37c012b2940eb7fa014/diff:/var/lib/docker/overlay2/cadc2d6308ef60788eb40b05ad5417f3ea89fb96aaa0028b80ccfe80202da748/diff:/var/lib/docker/overlay2/2dc66700322a704c5fce138e66d27ee38ee324eaa2c0c837e2fad842751cf645/diff:/var/lib/docker/overlay2/f0200441bebd98aaa57af2d9ddbbc4c7e8d9c8b982c14f50226e8841ea143380/diff:/var/lib/docker/overlay2/a9b06a4f8b8e8828ac142f9cf10c5c61d3771474ef0fe083fde2caeb2380d3df/diff:/var/lib/docker/overlay2/265bcad9bbbf55480038501a18dd7e03fda4848268dce29203302672fbecd5b3/diff", "MergedDir": "/var/lib/docker/overlay2/611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639/merged", "UpperDir": "/var/lib/docker/overlay2/611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639/diff", "WorkDir": "/var/lib/docker/overlay2/611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639/work" }, ...... [root@VM-0-10-centos /var/lib/docker/overlay2/611ff12eef289600cb685a8c995b24c42c387e99a6e0f0d7b73d29596a1db639/merged/tmp/nvd-data835905766]# docker ps | grep b6dfe23c31e9 b6dfe23c31e9 goharbor/clair-photon "./docker-entrypoint…" 9 days ago Up 9 days k8s_clair_harbor-1611818628-harbor-clair-7d89b99d5f-bcpgr_harbor_2f758cf3-e945-44c3-8a5b-700bf2187100_0
|
通过上面步骤,我们就找到占用了大量磁盘空间的目录和对应的容器是哪个,后面我们可以进入到对应的容器,自行清理不需要的文件即可
日志写入容器内日志文件并挂载
有的时候,我们会将日志写到日志文件并且进行挂载
挂载到emptyDir
根据容器id,inspect查找empty目录,如果你是将容器目录挂载到emptyDir,对应的宿主机目录是在/var/lib/kubelet/pods下,一般这个目录不会外挂数据盘,如果你的日志占用磁盘空间比较大,不建议挂载emptyDir,因为会将/var/lib/kubelet/pods所在磁盘空间占满,一般是系统盘,从而触发kubelet的驱逐。
1 2 3 4
| [root@VM-0-29-centos ~]# docker inspect 9d75f541f36db3a00514aff399e4455dee8dacbf912fa3b72afa675f505255fc | grep empty "/var/lib/kubelet/pods/24fe94a1-295b-4979-8d3f-735671782172/volumes/kubernetes.io~empty-dir/vol:/data", "Source": "/var/lib/kubelet/pods/24fe94a1-295b-4979-8d3f-735671782172/volumes/kubernetes.io~empty-dir/vol",
|
挂载到pvc
如果容器内文件挂载的是pvc,pvc包含cbs,nfs,或者云上的对象存储,这里最终就是将对应的目录挂载到cbs,cfs,对象存储。这里根据cfs的服务端ip,cbs_id,cos实例名称用df查找目录即可。
1 2 3 4 5 6 7 8 9 10
| [root@node1 ~] /dev/vdb 20G 9.7G 9.9G 50% /var/lib/kubelet/plugins/kubernetes.io/qcloud-cbs/mounts/disk-xxxxx /dev/vdg 9.8G 37M 9.7G 1% /var/lib/kubelet/plugins/kubernetes.io/qcloud-cbs/mounts/disk-xxxxx
[root@VM-0-29-centos ~] cosfs 256T 0 256T 0% /var/lib/kubelet/pods/99e9e539-28c2-4da0-9571-37e6dc57dcf4/volumes/kubernetes.io~csi/nwx-test-1/mount
[root@VM-0-2-centos ~] 10.0.0.11:/ 10G 32M 10G 1% /var/lib/kubelet/pods/d5298dfa-0b52-4d11-a34e-e68c7aa8a29e/volumes/kubernetes.io~nfs/vol
|
containerd日志存储目录说明
这里举例说明containerd和kubelet的默认存储目录是/run/containerd和/var/lib/kubelet
日志标准输出
首先根据容器id用inspect查找具体的logPath,然后对应的目录就是容器标准输出在宿主机存储目录,containerd标准输出的宿主机路径是/var/log/pods/下,因此用elk采集containerd的标准输出日志需要采集宿主机的/var/log/pods/路径。
1 2 3 4 5
| [root@VM-17-16-centos ~]# crictl inspect 0375c0e679d57 | grep logPath "logPath": "/var/log/pods/kube-system_tke-log-agent-k82vc_fd92c0e4-e08d-46d5-9a52-e6e1c4f29f3d/loglistener/0.log" [root@VM-17-16-centos ~]# cd /var/log/pods/kube-system_tke-log-agent-k82vc_fd92c0e4-e08d-46d5-9a52-e6e1c4f29f3d/loglistener [root@VM-17-16-centos loglistener]# ls 0.log
|
日志写入容器内日志文件
查找容器id,然后根据容器id,用df查找到容器的rootfs目录,再进入对应的目录即可。
1 2 3 4 5 6 7 8
| [root@VM-17-16-centos vol]# crictl ps | grep go-test 2380b4524eebf 57b1777ee4b87 About a minute ago Running go-test 0 10389f95f91b6 [root@VM-17-16-centos vol]# df -h | grep 2380b4524eebf overlay 50G 14G 34G 30% /run/containerd/io.containerd.runtime.v2.task/k8s.io/2380b4524eebf3455a79442204bf68efb0325f466429238bc4338a8eccea28c8/rootfs [root@VM-17-16-centos vol]# cd /run/containerd/io.containerd.runtime.v2.task/k8s.io/2380b4524eebf3455a79442204bf68efb0325f466429238bc4338a8eccea28c8/rootfs [root@VM-17-16-centos rootfs]# cd data/ [root@VM-17-16-centos data]# ls go.log
|
日志写入容器内日志文件并挂载
挂载到emptyDir
根据容器id,inspect查找empty目录,如果你是将容器目录挂载到emptyDir,对应的宿主机目录是在/var/lib/kubelet下,一般这个目录不会外挂数据盘,如果你的日志占用磁盘空间比较大,不建议挂载emptyDir,因为会将/var/lib/kubelet所在磁盘空间占满,一般是系统盘,从而触发kubelet的驱逐。
1 2 3 4 5 6 7 8 9
| [root@VM-17-16-centos loglistener]# crictl ps | grep go-test 43fd8249aba3b 57b1777ee4b87 2 hours ago Running go-test 0 b4eccdc968348 [root@VM-17-16-centos data]# crictl inspect 43fd8249aba3b | grep empty "hostPath": "/var/lib/kubelet/pods/7d32e2e6-169a-40c3-b36e-4ce22e248776/volumes/kubernetes.io~empty-dir/vol", "host_path": "/var/lib/kubelet/pods/7d32e2e6-169a-40c3-b36e-4ce22e248776/volumes/kubernetes.io~empty-dir/vol" "source": "/var/lib/kubelet/pods/7d32e2e6-169a-40c3-b36e-4ce22e248776/volumes/kubernetes.io~empty-dir/vol", [root@VM-17-16-centos data]# cd /var/lib/kubelet/pods/7d32e2e6-169a-40c3-b36e-4ce22e248776/volumes/kubernetes.io~empty-dir/vol [root@VM-17-16-centos vol]# ls go.log
|
挂载到pvc
如果容器内文件挂载的是pvc,pvc包含cbs,nfs,或者云上的对象存储,这里最终就是将对应的目录挂载到cbs,cfs,对象存储。这里根据cfs的服务端ip,cbs_id,cos实例名称用df查找目录即可。和docker的方式是一致的。