Unfortunately I did not. I dumped the logs and just removed the node in order to quickly restore the current containers on another node.
At the exact time it failed I saw a lot of the following: === thin_pool_watcher.go:72] encountered error refreshing thin pool watcher: error performing thin_ls on metadata device /dev/mapper/docker_vg-docker--pool_tmeta: Error running command `thin_ls --no-headers -m -o DEV, EXCLUSIVE_BYTES /dev/mapper/docker_vg-docker--pool_tmeta`: exit status 127 failed (failure): rpc error: code = 2 desc = shim error: context deadline exceeded#015 Error running exec in container: rpc error: code = 2 desc = shim error: context deadline exceeded === Seems to match https://bugzilla.redhat.com/show_bug.cgi?id=1427212 On Thu, 20 Apr 2017 at 15:41 Tero Ahonen <taho...@redhat.com> wrote: > Hi > > Did u try to ssh to that node and execute sudo docker run to some > container? > > .t > > Sent from my iPhone > > > On 20 Apr 2017, at 8.18, Andrew Lau <and...@andrewklau.com> wrote: > > > > I'm trying to debug a weird scenario where a node has had every pod > crash with the error: > > "rpc error: code = 2 desc = shim error: context deadline exceeded" > > > > The pods stayed in the state Ready 0/1 > > The docker daemon was responding and the kublet and all it's services > were running. The node was reporting with the OK status. > > > > No resource limits were hit with CPU almost idle and memory at 25% > utilisation. > > > > > > > > > > _______________________________________________ > > users mailing list > > us...@lists.openshift.redhat.com > > http://lists.openshift.redhat.com/openshiftmm/listinfo/users >
_______________________________________________ dev mailing list dev@lists.openshift.redhat.com http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
