I've seem this when docker is pulling a image which has a common layer for a long time, looks like docker client doesn't pull images concurrently since there's issues on the graph driver.
A simple workaround is restart docker daemon. *Diego Castro* The CloudFather (11) 3230.5927 +54 (911) 2159.1779 gtalk: *[email protected] <[email protected]>* 2016-02-18 11:34 GMT-03:00 Bendik Paulsrud <[email protected]>: > Hi again, > > We have had a new occurrence of a hanging pod. I have not restarted the > service and here are the logs from the time interval the pod was deployed. > > "oc describe pod <pod>" gives me this: > https://gist.github.com/bendikp/eafce3e25bf6186b49e3 > > "journalctl -u docker -l" gives me this: > https://gist.github.com/bendikp/63891180689cc23d172f > > "journalctl -u origin-node -l" gives me this: > https://gist.github.com/bendikp/30792a6e66b27b203616 > > "docker info" states there is data space available. > > I tried pull the image (kubernetes/redis:v1) afterwards and that worked > fine. > > Regards, > Bendik Paulsrud > > On 16 February 2016 at 15:48, Andy Goldstein <[email protected]> wrote: > >> If this happens again, please capture the logs prior to restarting the >> service. That's really the only way we can try to discern what's happening. >> >> Andy >> >> On Tue, Feb 16, 2016 at 7:30 AM, Bendik Paulsrud < >> [email protected]> wrote: >> >>> Hi, >>> >>> The node has not out of disk, and "docker info" showed there were >>> available space left. >>> >>> Since I don't know when the pending state occured i restarted >>> origin-node and pasted the logs from origin-node here: >>> https://gist.github.com/bendikp/e9b599887edb10af4f7a >>> >>> The issue seems to stop after restarting both origin-node and docker. >>> >>> Version of Docker is v1.8.2 and OpenShift Origin is v1.1.1.1. >>> >>> Regards, >>> Bendik Paulsrud >>> >>> >>> On 16 February 2016 at 13:06, Andy Goldstein <[email protected]> >>> wrote: >>> >>>> Are you out of disk space anywhere? Can you post the node service logs >>>> somewhere for us to review? >>>> >>>> Andy >>>> >>>> >>>> On Tuesday, February 16, 2016, Bendik Paulsrud < >>>> [email protected]> wrote: >>>> >>>>> Hi, >>>>> >>>>> We have encountered a situation where pods keep hanging in Pending >>>>> state on one node in the cluster. When checking the node(docker ps), there >>>>> are only maintainer containers (k8s_POD) running, no application >>>>> containers. >>>>> >>>>> When describing the pod with "oc describe" no error message is >>>>> displayed, only that the maintainer container is assigned to the node and >>>>> has started. There are no error messages in the event log either. >>>>> >>>>> Checking the origin-node log and docker log I can't see any error >>>>> messages. >>>>> >>>>> Have anyone encountered a similar situation and know why this occurs? >>>>> >>>>> Regards, >>>>> Bendik Paulsrud >>>>> >>>>> >>> >> > > _______________________________________________ > users mailing list > [email protected] > http://lists.openshift.redhat.com/openshiftmm/listinfo/users > >
_______________________________________________ users mailing list [email protected] http://lists.openshift.redhat.com/openshiftmm/listinfo/users
