I have tried restarting the physical machine along with restarting openshift and the docker daemon. Originally I was just seeing a message like this in the deployment logs:
I0126 07:43:30.212175 1 deployer.go:196] Deploying keycloak/keycloakservice-5 for the first time (replicas: 1) 2 I0126 07:43:30.233288 1 recreate.go:105] Scaling keycloak/keycloakservice-5 to 1 before performing acceptance check 3 I0126 07:43:32.260466 1 recreate.go:110] Performing acceptance check of keycloak/keycloakservice-5 4 I0126 07:43:32.261555 1 lifecycle.go:391] Waiting 120 seconds for pods owned by deployment "keycloak/keycloakservice-5" to become ready (checking every 1 seconds; 0 pods previously accepted) 5 F0126 07:45:32.261935 1 deployer.go:65] update acceptor rejected keycloak/keycloakservice-5: pods for deployment "keycloak/keycloakservice-5" took longer than 120 seconds to become ready Since restarting, my docker registry has become unresponsive and I see messages in the openshift console that say: *"container 'registry' is unhealthy, it will be killed and re-created*." It just keeps repeating that over and over. Now I see deployment messages like this: I0126 07:53:26.557631 1 deployer.go:196] Deploying keycloak/keycloakservice-6 for the first time (replicas: 1) 2 I0126 07:53:26.572359 1 recreate.go:105] Scaling keycloak/keycloakservice-6 to 1 before performing acceptance check 3 I0126 07:53:28.600383 1 recreate.go:110] Performing acceptance check of keycloak/keycloakservice-6 4 I0126 07:53:28.600501 1 lifecycle.go:391] Waiting 120 seconds for pods owned by deployment "keycloak/keycloakservice-6" to become ready (checking every 1 seconds; 0 pods previously accepted) 5 E0126 07:54:36.144407 1 reflector.go:206] pkg/deploy/strategy/support/lifecycle.go:350: Failed to watch *api.Pod: Get https://23.25.149.227:8443/api/v1/watch/namespaces/keycloak/pods?labelSelector=deployment%3Dkeycloakservice-6%2Cdeploymentconfig%3Dkeycloakservice%2Cname%3Dkeycloakservice&resourceVersion=48423: dial tcp 23.25.149.227:8443: connection refused 6 E0126 07:54:37.145797 1 reflector.go:206] pkg/deploy/strategy/support/lifecycle.go:350: Failed to watch *api.Pod: Get https://23.25.149.227:8443/api/v1/watch/namespaces/keycloak/pods?labelSelector=deployment%3Dkeycloakservice-6%2Cdeploymentconfig%3Dkeycloakservice%2Cname%3Dkeycloakservice&resourceVersion=48423: dial tcp 23.25.149.227:8443: connection refused 7 E0126 07:54:38.146563 1 reflector.go:206] pkg/deploy/strategy/support/lifecycle.go:350: Failed to watch *api.Pod: Get https://23.25.149.227:8443/api/v1/watch/namespaces/keycloak/pods?labelSelector=deployment%3Dkeycloakservice-6%2Cdeploymentconfig%3Dkeycloakservice%2Cname%3Dkeycloakservice&resourceVersion=48423: dial tcp 23.25.149.227:8443: connection refused 8 E0126 07:54:39.147084 1 reflector.go:206] pkg/deploy/strategy/support/lifecycle.go:350: Failed to watch *api.Pod: Get https://23.25.149.227:8443/api/v1/watch/namespaces/keycloak/pods?labelSelector=deployment%3Dkeycloakservice-6%2Cdeploymentconfig%3Dkeycloakservice%2Cname%3Dkeycloakservice&resourceVersion=48423: dial tcp 23.25.149.227:8443: connection refused 9 E0126 07:54:40.147622 1 reflector.go:206] pkg/deploy/strategy/support/lifecycle.go:350: Failed to watch *api.Pod: Get https://23.25.149.227:8443/api/v1/watch/namespaces/keycloak/pods?labelSelector=deployment%3Dkeycloakservice-6%2Cdeploymentconfig%3Dkeycloakservice%2Cname%3Dkeycloakservice&resourceVersion=48423: dial tcp 23.25.149.227:8443: connection refused 10 E0126 07:54:41.148262 1 reflector.go:206] pkg/deploy/strategy/support/lifecycle.go:350: Failed to watch *api.Pod: Get https://23.25.149.227:8443/api/v1/watch/namespaces/keycloak/pods?labelSelector=deployment%3Dkeycloakservice-6%2Cdeploymentconfig%3Dkeycloakservice%2Cname%3Dkeycloakservice&resourceVersion=48423: dial tcp 23.25.149.227:8443: connection refused 11 E0126 07:54:42.148894 1 reflector.go:206] pkg/deploy/strategy/support/lifecycle.go:350: Failed to watch *api.Pod: Get https://23.25.149.227:8443/api/v1/watch/namespaces/keycloak/pods?labelSelector=deployment%3Dkeycloakservice-6%2Cdeploymentconfig%3Dkeycloakservice%2Cname%3Dkeycloakservice&resourceVersion=48423: dial tcp 23.25.149.227:8443: connection refused 12 E0126 07:54:43.149535 1 reflector.go:206] pkg/deploy/strategy/support/lifecycle.go:350: Failed to watch *api.Pod: Get https://23.25.149.227:8443/api/v1/watch/namespaces/keycloak/pods?labelSelector=deployment%3Dkeycloakservice-6%2Cdeploymentconfig%3Dkeycloakservice%2Cname%3Dkeycloakservice&resourceVersion=48423: dial tcp 23.25.149.227:8443: connection refused 13 E0126 07:54:44.150027 1 reflector.go:206] pkg/deploy/strategy/support/lifecycle.go:350: Failed to watch *api.Pod: Get https://23.25.149.227:8443/api/v1/watch/namespaces/keycloak/pods?labelSelector=deployment%3Dkeycloakservice-6%2Cdeploymentconfig%3Dkeycloakservice%2Cname%3Dkeycloakservice&resourceVersion=48423: dial tcp 23.25.149.227:8443: connection refused 14 F0126 07:55:28.600789 1 deployer.go:65] update acceptor rejected keycloak/keycloakservice-6: pods for deployment "keycloak/keycloakservice-6" took longer than 120 seconds to become ready On Tue, Jan 26, 2016 at 9:01 AM, Clayton Coleman <[email protected]> wrote: > What was the status of the pod keycloakservice-6-deploy > > On Tue, Jan 26, 2016 at 9:55 AM, Dean Peterson <[email protected]> > wrote: > > It appears the node name is correct: > > > > [root@openshift openshift]# ./oc get nodes > > NAME LABELS STATUS > AGE > > openshift.master1 kubernetes.io/hostname=openshift.master1 Ready > 4d > > > > [root@openshift openshift]# hostname > > openshift.master1 > > > > I started openshift the same way I have always started it; with > "./openshift > > start" and that creates a master and node on the same machine. > > > > > > On Tue, Jan 26, 2016 at 8:37 AM, Clayton Coleman <[email protected]> > > wrote: > >> > >> Deletions are graceful by default - you can pass --grace-period=0 to > >> force delete the offending pod. > >> > >> This is almost always related to nodes being misconfigured. There was > >> an issue where nodes may have gotten their node names changed to their > >> IP in 1.1 can you verify that the nodes the pods are assigned to match > >> actual node names in the system? > >> > >> > On Jan 26, 2016, at 2:14 AM, Dean Peterson <[email protected]> > >> > wrote: > >> > > >> > If I try to build and deploy a number of pods simultaneously Openshift > >> > seems to go crazy and I am no longer able to get my app up. I > recently > >> > upgraded to v1.1.1. I think there may be serious locking issues with > this > >> > version. After keycloakservice-5-g75fg pod hangs, I try to delete it > >> > because I can't start anything else in Openshift it seems while that > is > >> > stuck terminating. I try to manually delete the pod but it just keeps > >> > saying deleted, deleted deleted over and over but the pod stays. > >> > > >> > keycloakservice-5-g75fg 0/1 Terminating 0 > 14m > >> > keycloakservice-6-deploy 0/1 Error 0 7m > >> > keycloakservice-build-1-build 0/1 Completed 0 1d > >> > keycloakservice-build-10-build 0/1 Completed 0 > 24m > >> > keycloakservice-build-11-build 0/1 Completed 0 > 17m > >> > keycloakservice-build-2-build 0/1 Completed 0 2h > >> > keycloakservice-build-3-build 0/1 Completed 0 2h > >> > wildfly-jdk-8-build-1-build 0/1 Completed 0 1d > >> > [root@openshift openshift]# ./oc delete pod keycloakservice-5-g75fg > >> > pod "keycloakservice-5-g75fg" deleted > >> > [root@openshift openshift]# ./oc delete pod keycloakservice-5-g75fg > >> > pod "keycloakservice-5-g75fg" deleted > >> > [root@openshift openshift]# ./oc delete pod keycloakservice-5-g75fg > >> > pod "keycloakservice-5-g75fg" deleted > >> > [root@openshift openshift]# ./oc delete pod keycloakservice-5-g75fg > >> > pod "keycloakservice-5-g75fg" deleted > >> > [root@openshift openshift]# ./oc delete pod keycloakservice-5-g75fg > >> > pod "keycloakservice-5-g75fg" deleted > >> > [root@openshift openshift]# ./oc delete pod keycloakservice-5-g75fg > >> > pod "keycloakservice-5-g75fg" deleted > >> > [root@openshift openshift]# ./oc delete pod keycloakservice-5-g75fg > >> > pod "keycloakservice-5-g75fg" deleted > >> > [root@openshift openshift]# ./oc delete pod keycloakservice-5-g75fg > >> > pod "keycloakservice-5-g75fg" deleted > >> > [root@openshift openshift]# ./oc delete pod keycloakservice-5-g75fg > >> > pod "keycloakservice-5-g75fg" deleted > >> > [root@openshift openshift]# ./oc delete pod keycloakservice-5-g75fg > >> > pod "keycloakservice-5-g75fg" deleted > >> > [root@openshift openshift]# ./oc delete pod keycloakservice-5-g75fg > >> > pod "keycloakservice-5-g75fg" deleted > >> > > >> > _______________________________________________ > >> > users mailing list > >> > [email protected] > >> > http://lists.openshift.redhat.com/openshiftmm/listinfo/users > > > > >
_______________________________________________ users mailing list [email protected] http://lists.openshift.redhat.com/openshiftmm/listinfo/users
