I am not sure. The volume is mentioned there, but I don't know what to look
for:
Feb 20 04:47:56 node1 kubelet[6615]: E0220 04:47:56.351626 6615
summary.go:92] Failed to get system container stats for
"/system.slice/docker.service": failed to get cgroup stats for
"/system.slice/docker.service": failed to get container info for
"/system.slice/docker.service": unknown container
"/system.slice/docker.service"
Feb 20 04:47:59 node1 dockerd-current[4405]: DEBU: 2018/02/20
04:47:59.685252 EVENT AddPod
{"metadata":{"creationTimestamp":"2018-02-20T04:47:59Z","name":"busybox","namespace":"default","resourceVersion":"1945","selfLink":"/api/v1/namespaces/default/pods/busybox","uid":"39e0efdf-15f9-11e8-8091-525400a444d1"},"spec":{"containers":[{"image":"busybox","imagePullPolicy":"IfNotPresent","name":"busybox","terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"Fi
Feb 20 04:47:59 node1 dockerd-current[4405]: DEBU: 2018/02/20
04:47:59.702701 EVENT UpdatePod
{"metadata":{"creationTimestamp":"2018-02-20T04:47:59Z","name":"busybox","namespace":"default","resourceVersion":"1945","selfLink":"/api/v1/namespaces/default/pods/busybox","uid":"39e0efdf-15f9-11e8-8091-525400a444d1"},"spec":{"containers":[{"image":"busybox","imagePullPolicy":"IfNotPresent","name":"busybox","terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":
Feb 20 04:47:59 node1 systemd[1]: Created slice libcontainer container
kubepods-besteffort-pod39e0efdf_15f9_11e8_8091_525400a444d1.slice.
Feb 20 04:47:59 node1 dockerd-current[4405]: DEBU: 2018/02/20
04:47:59.715877 EVENT UpdatePod
{"metadata":{"creationTimestamp":"2018-02-20T04:47:59Z","name":"busybox","namespace":"default","resourceVersion":"1947","selfLink":"/api/v1/namespaces/default/pods/busybox","uid":"39e0efdf-15f9-11e8-8091-525400a444d1"},"spec":{"containers":[{"image":"busybox","imagePullPolicy":"IfNotPresent","name":"busybox","terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":
Feb 20 04:47:59 node1 systemd[1]: Starting libcontainer container
kubepods-besteffort-pod39e0efdf_15f9_11e8_8091_525400a444d1.slice.
Feb 20 04:47:59 node1 kubelet[6615]: I0220 04:47:59.821322 6615
reconciler.go:217] operationExecutor.VerifyControllerAttachedVolume started
for volume "pv0003" (UniqueName:
"flexvolume-fnordian/cv/39e0efdf-15f9-11e8-8091-525400a444d1-pv0003") pod
"busybox" (UID: "39e0efdf-15f9-11e8-8091-525400a444d1")
Feb 20 04:47:59 node1 kubelet[6615]: I0220 04:47:59.821451 6615
reconciler.go:217] operationExecutor.VerifyControllerAttachedVolume started
for volume "default-token-rg4gs" (UniqueName:
"kubernetes.io/secret/39e0efdf-15f9-11e8-8091-525400a444d1-default-token-rg4gs")
pod "busybox" (UID: "39e0efdf-15f9-11e8-8091-525400a444d1")
Feb 20 04:47:59 node1 systemd[1]: Started Kubernetes transient mount for
/var/lib/kubelet/pods/39e0efdf-15f9-11e8-8091-525400a444d1/volumes/kubernetes.io~secret/default-token-rg4gs.
Feb 20 04:47:59 node1 systemd[1]: Starting Kubernetes transient mount for
/var/lib/kubelet/pods/39e0efdf-15f9-11e8-8091-525400a444d1/volumes/kubernetes.io~secret/default-token-rg4gs.
Feb 20 04:48:00 node1 kernel: XFS (dm-8): Mounting V5 Filesystem
Feb 20 04:48:00 node1 kernel: XFS (dm-8): Ending clean mount
Feb 20 04:48:00 node1 kernel: XFS (dm-8): Unmounting Filesystem
Feb 20 04:48:00 node1 kernel: XFS (dm-8): Mounting V5 Filesystem
Feb 20 04:48:00 node1 kernel: XFS (dm-8): Ending clean mount
Feb 20 04:48:00 node1 kernel: XFS (dm-8): Unmounting Filesystem
Feb 20 04:48:00 node1 kernel: XFS (dm-8): Mounting V5 Filesystem
Feb 20 04:48:00 node1 kernel: XFS (dm-8): Ending clean mount
Feb 20 04:48:00 node1 systemd[1]: Started docker container
0a0d728960e7aa35e07305a628b2101d58e1a4f733fe21337c202bc6abd7cf03.
Feb 20 04:48:00 node1 systemd[1]: Starting docker container
0a0d728960e7aa35e07305a628b2101d58e1a4f733fe21337c202bc6abd7cf03.
Feb 20 04:48:00 node1 kernel: SELinux: mount invalid. Same superblock,
different security settings for (dev mqueue, type mqueue)
Feb 20 04:48:00 node1 oci-systemd-hook[9731]: systemdhook <debug>:
0a0d728960e7: Skipping as container command is /pause, not init or systemd
Feb 20 04:48:00 node1 oci-umount[9732]: umounthook <debug>: prestart
container_id:0a0d728960e7
rootfs:/var/lib/docker/devicemapper/mnt/fb24bd63ac000f7d56f6f4b8549f09d70b8e1670c3d7d9d5c2c5ff19a0c490fa/rootfs
Feb 20 04:48:00 node1 kernel: weave: port 2(vethwepl0a0d728) entered
blocking state
Feb 20 04:48:00 node1 kernel: weave: port 2(vethwepl0a0d728) entered
disabled state
Feb 20 04:48:00 node1 kernel: device vethwepl0a0d728 entered promiscuous
mode
Feb 20 04:48:00 node1 NetworkManager[2902]: <info> [1519102080.4792]
manager: (vethwepl0a0d728): new Veth device
(/org/freedesktop/NetworkManager/Devices/12)
Feb 20 04:48:00 node1 kernel: IPv6: ADDRCONF(NETDEV_UP): vethwepl0a0d728:
link is not ready
Feb 20 04:48:00 node1 kernel: IPv6: ADDRCONF(NETDEV_CHANGE):
vethwepl0a0d728: link becomes ready
Feb 20 04:48:00 node1 kernel: weave: port 2(vethwepl0a0d728) entered
blocking state
Feb 20 04:48:00 node1 kernel: weave: port 2(vethwepl0a0d728) entered
forwarding state
Feb 20 04:48:00 node1 NetworkManager[2902]: <info> [1519102080.5192]
device (vethwepl0a0d728): link connected
Feb 20 04:48:00 node1 kernel: XFS (dm-9): Mounting V5 Filesystem
Feb 20 04:48:00 node1 kernel: XFS (dm-9): Ending clean mount
Feb 20 04:48:00 node1 kernel: XFS (dm-9): Unmounting Filesystem
Feb 20 04:48:00 node1 kernel: IPv6: eth0: IPv6 duplicate address
fe80::c84:1aff:fef1:17c6 detected!
Feb 20 04:48:00 node1 kernel: XFS (dm-9): Mounting V5 Filesystem
Feb 20 04:48:00 node1 kernel: XFS (dm-9): Ending clean mount
Feb 20 04:48:00 node1 kernel: XFS (dm-9): Unmounting Filesystem
Feb 20 04:48:00 node1 kernel: XFS (dm-9): Mounting V5 Filesystem
Feb 20 04:48:00 node1 kernel: XFS (dm-9): Ending clean mount
Feb 20 04:48:00 node1 systemd[1]: Started docker container
6913e41bde5576132809089358929755f2d41dfc8f1d2b63cf3b881d7467f404.
Feb 20 04:48:00 node1 systemd[1]: Starting docker container
6913e41bde5576132809089358929755f2d41dfc8f1d2b63cf3b881d7467f404.
Feb 20 04:48:00 node1 kernel: SELinux: mount invalid. Same superblock,
different security settings for (dev mqueue, type mqueue)
Feb 20 04:48:00 node1 oci-systemd-hook[9837]: systemdhook <debug>:
6913e41bde55: Skipping as container command is sleep, not init or systemd
Feb 20 04:48:00 node1 oci-umount[9838]: umounthook <debug>: prestart
container_id:6913e41bde55
rootfs:/var/lib/docker/devicemapper/mnt/f9c55af22adc1aa54163654ae30d5156124119c8b34ebb77a4236cceaeff4a53/rootfs
Feb 20 04:48:00 node1 dockerd-current[4405]:
time="2018-02-20T04:48:00.860981273Z" level=warning msg="Unknown
healthcheck type 'NONE' (expected 'CMD')"
Feb 20 04:48:01 node1 dockerd-current[4405]: DEBU: 2018/02/20
04:48:01.128625 EVENT UpdatePod
{"metadata":{"creationTimestamp":"2018-02-20T04:47:59Z","name":"busybox","namespace":"default","resourceVersion":"1950","selfLink":"/api/v1/namespaces/default/pods/busybox","uid":"39e0efdf-15f9-11e8-8091-525400a444d1"},"spec":{"containers":[{"image":"busybox","imagePullPolicy":"IfNotPresent","name":"busybox","terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":
Feb 20 04:48:01 node1 dockerd-current[4405]: INFO: 2018/02/20
04:48:01.128670 adding entry 10.32.0.2 to weave-local-pods of
39e0efdf-15f9-11e8-8091-525400a444d1
Feb 20 04:48:01 node1 dockerd-current[4405]: INFO: 2018/02/20
04:48:01.128687 added entry 10.32.0.2 to weave-local-pods of
39e0efdf-15f9-11e8-8091-525400a444d1
Feb 20 04:48:01 node1 dockerd-current[4405]: INFO: 2018/02/20
04:48:01.130885 adding entry 10.32.0.2 to weave-k?Z;25^M}|1s7P3|H9i;*;MhG
of 39e0efdf-15f9-11e8-8091-525400a444d1
Feb 20 04:48:01 node1 dockerd-current[4405]: INFO: 2018/02/20
04:48:01.130915 added entry 10.32.0.2 to weave-k?Z;25^M}|1s7P3|H9i;*;MhG of
39e0efdf-15f9-11e8-8091-525400a444d1
Feb 20 04:48:01 node1 dockerd-current[4405]: INFO: 2018/02/20
04:48:01.132339 adding entry 10.32.0.2 to weave-E.1.0W^NGSp]0_t5WwH/]gX@L
of 39e0efdf-15f9-11e8-8091-525400a444d1
Feb 20 04:48:01 node1 dockerd-current[4405]: INFO: 2018/02/20
04:48:01.132367 added entry 10.32.0.2 to weave-E.1.0W^NGSp]0_t5WwH/]gX@L of
39e0efdf-15f9-11e8-8091-525400a444d1
Feb 20 04:48:06 node1 kubelet[6615]: E0220 04:48:06.372263 6615
summary.go:92] Failed to get system container stats for
"/system.slice/kubelet.service": failed to get cgroup stats for
"/system.slice/kubelet.service": failed to get container info for
"/system.slice/kubelet.service": unknown container
"/system.slice/kubelet.service"
Feb 20 04:48:06 node1 kubelet[6615]: E0220 04:48:06.373207 6615
summary.go:92] Failed to get system container stats for
"/system.slice/docker.service": failed to get cgroup stats for
"/system.slice/docker.service": failed to get container info for
"/system.slice/docker.service": unknown container
"/system.slice/docker.service"
Feb 20 04:48:09 node1 kubelet[6615]: E0220 04:48:09.979708 6615
fsHandler.go:121] failed to collect filesystem stats - rootDiskErr: <nil>,
rootInodeErr: <nil>, extraDiskErr: du command failed on
/var/lib/docker/containers/27947eb9e964f761f947f51ba0d9d33bfa9228508652ca54e570e2fdd4e12752
with output stdout: , stderr: du: cannot access
‘/var/lib/docker/containers/27947eb9e964f761f947f51ba0d9d33bfa9228508652ca54e570e2fdd4e12752’:
No such file or directory
Feb 20 04:48:09 node1 kubelet[6615]: - exit status 1
Feb 20 04:48:13 node1 dockerd-current[4405]:
time="2018-02-20T04:48:13.018171965Z" level=error msg="Handler for POST
/v1.24/containers/839ea2a858e1e84de0b78a7cdf050e62967b9fd04ededf0ff1d50933efa4a22b/stop
returned error: Container
839ea2a858e1e84de0b78a7cdf050e62967b9fd04ededf0ff1d50933efa4a22b is already
stopped"
Feb 20 04:48:13 node1 systemd-udevd[9861]: inotify_add_watch(7, /dev/dm-10,
10) failed: No such file or directory
Feb 20 04:48:13 node1 systemd-udevd[9861]: inotify_add_watch(7, /dev/dm-10,
10) failed: No such file or directory
Feb 20 04:48:16 node1 kubelet[6615]: E0220 04:48:16.392269 6615
summary.go:92] Failed to get system container stats for
"/system.slice/kubelet.service": failed to get cgroup stats for
"/system.slice/kubelet.service": failed to get container info for
"/system.slice/kubelet.service": unknown container
"/system.slice/kubelet.service"
Am Dienstag, 20. Februar 2018 00:19:36 UTC+7 schrieb Michelle Au:
>
> In kubelet logs, do you see the volume getting mounted?
>
> On Mon, Feb 19, 2018, 08:49 Marcus <marcus...@gmail.com <javascript:>>
> wrote:
>
>> Hi,
>>
>> I've created a PV, PVC and Pod using it as a volume. As provider for the
>> PV, flexVolume with a script to mount a cifs-share is used. The PV is
>> claimed and the pod starts, everything looks fine. On the node, I see the
>> share mounted, as it should.
>>
>> But inside the container, the volume is mounted as tmpfs and it's
>> contents are different files than in the cifs-share.
>>
>> / # ls -l /var/lib/mysql/
>> total 8
>> prwx------ 1 root root 0 Feb 19 16:34
>> 68e2851c2a83fa34d95ae7b37acdde7a0f8415d8e7eed25a826fd53feb365429-stdin
>> prwx------ 1 root root 0 Feb 19 16:34
>> 68e2851c2a83fa34d95ae7b37acdde7a0f8415d8e7eed25a826fd53feb365429-stdout
>> -rw-r--r-- 1 root root 5986 Feb 19 16:33 config.json
>> prwx------ 1 root root 0 Feb 19 16:33 init-stderr
>> prwx------ 1 root root 0 Feb 19 16:33 init-stdin
>> prwx------ 1 root root 0 Feb 19 16:33 init-stdout
>>
>> / # mount | grep var/lib/mysql
>> tmpfs on /var/lib/mysql type tmpfs (rw,seclabel,nosuid,nodev,mode=755)
>>
>> I am using kubelet 1.9.2 and docker 1.12.6 on centos 7.4. How can I fix
>> this?
>>
>> Thanks, Marcus
>>
>> pv.yaml:
>>
>> apiVersion: v1
>> kind: PersistentVolume
>> metadata:
>> name: pv0003
>> spec:
>> capacity:
>> storage: 1Gi
>> volumeMode: Filesystem
>> accessModes:
>> - ReadWriteOnce
>> persistentVolumeReclaimPolicy: Retain
>> flexVolume:
>> driver: "fnordian/cv"
>> readOnly: false
>> options:
>> source: "//192.168.121.82/kubvolumes"
>> mountOptions: "dir_mode=0700,file_mode=0600"
>> cifsuser: "nobody"
>> cifspass: "nobody"
>>
>> pod.yaml:
>>
>> apiVersion: v1
>> kind: Pod
>> metadata:
>> name: busybox
>> namespace: default
>> spec:
>> containers:
>> - image: busybox
>> command:
>> - sleep
>> - "3600"
>> imagePullPolicy: IfNotPresent
>> name: busybox
>> volumeMounts:
>> - name: mysql-pv
>> mountPath: /var/lib/mysql
>> restartPolicy: Always
>>
>> volumes:
>> - name: mysql-pv
>> persistentVolumeClaim:
>> claimName: mysql-pv-claim
>>
>> ---
>> apiVersion: v1
>> kind: PersistentVolumeClaim
>> metadata:
>> name: mysql-pv-claim
>> spec:
>> accessModes:
>> - ReadWriteOnce
>> resources:
>> requests:
>> storage: 100Mi
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Kubernetes user discussion and Q&A" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to kubernetes-use...@googlegroups.com <javascript:>.
>> To post to this group, send email to kubernet...@googlegroups.com
>> <javascript:>.
>> Visit this group at https://groups.google.com/group/kubernetes-users.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
--
You received this message because you are subscribed to the Google Groups
"Kubernetes user discussion and Q&A" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.
