Yeah, these look like errors : *Dec 03 10:28:44 node01-2018.drivenow.com.au <http://node01-2018.drivenow.com.au> origin-master-controllers[89591]: E1203 10:28:44.150240 89591 runtime.go:66] Observed a panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference)* Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /builddir/build/BUILD/origin-3.7.0/_output/local/go/src/ github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:72 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /builddir/build/BUILD/origin-3.7.0/_output/local/go/src/ github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:65 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /builddir/build/BUILD/origin-3.7.0/_output/local/go/src/ github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:51 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /usr/lib/golang/src/runtime/asm_amd64.s:514 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /usr/lib/golang/src/runtime/panic.go:489 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /usr/lib/golang/src/runtime/panic.go:63 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /usr/lib/golang/src/runtime/signal_unix.go:290 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /builddir/build/BUILD/origin-3.7.0/_output/local/go/src/ github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/controller/daemon/daemoncontroller.go:155 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /builddir/build/BUILD/origin-3.7.0/_output/local/go/src/ github.com/openshift/origin/vendor/k8s.io/client-go/tools/cache/controller.go:192 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: <autogenerated>:57 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /builddir/build/BUILD/origin-3.7.0/_output/local/go/src/ github.com/openshift/origin/vendor/k8s.io/client-go/tools/cache/shared_informer.go:547 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /usr/lib/golang/src/runtime/asm_amd64.s:2197 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: I1203 10:28:44.230853 89591 controller_utils.go:1032] Caches are synced for RC controller Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: W1203 10:28:44.300741 89591 shared_informer.go:298] resyncPeriod 30000000000 is smaller than resyncCheckPeriod 600000000000 and the informer has already started. Changing it to 600000000000 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: I1203 10:28:44.300811 89591 controllermanager.go:466] Started "statefulset" *Dec 03 10:28:44 node01-2018.drivenow.com.au <http://node01-2018.drivenow.com.au> origin-master-controllers[89591]: E1203 10:28:44.300820 89591 runtime.go:66] Observed a panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference)* Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /builddir/build/BUILD/origin-3.7.0/_output/local/go/src/ github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:72 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /builddir/build/BUILD/origin-3.7.0/_output/local/go/src/ github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:65 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /builddir/build/BUILD/origin-3.7.0/_output/local/go/src/ github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:51 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /usr/lib/golang/src/runtime/asm_amd64.s:514 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /usr/lib/golang/src/runtime/panic.go:489 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /usr/lib/golang/src/runtime/panic.go:63 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /usr/lib/golang/src/runtime/signal_unix.go:290 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /builddir/build/BUILD/origin-3.7.0/_output/local/go/src/ github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/controller/statefulset/stateful_set.go:365 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /builddir/build/BUILD/origin-3.7.0/_output/local/go/src/ github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/controller/statefulset/stateful_set.go:174 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /builddir/build/BUILD/origin-3.7.0/_output/local/go/src/ github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/controller/statefulset/stateful_set.go:111 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /builddir/build/BUILD/origin-3.7.0/_output/local/go/src/ github.com/openshift/origin/vendor/k8s.io/client-go/tools/cache/controller.go:192 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: <autogenerated>:57 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /builddir/build/BUILD/origin-3.7.0/_output/local/go/src/ github.com/openshift/origin/vendor/k8s.io/client-go/tools/cache/shared_informer.go:547 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /usr/lib/golang/src/runtime/asm_amd64.s:2197
Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: I1203 10:28:44.750941 89591 controllermanager.go:456] Starting "deployment" Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: I1203 10:28:44.750997 89591 attach_detach_controller.go:242] Starting attach detach controller Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: I1203 10:28:44.751014 89591 controller_utils.go:1025] Waiting for caches to sync for attach detach controller *Dec 03 10:28:44 node01-2018.drivenow.com.au <http://node01-2018.drivenow.com.au> origin-master-controllers[89591]: E1203 10:28:44.900441 89591 runtime.go:66] Observed a panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference)* Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /builddir/build/BUILD/origin-3.7.0/_output/local/go/src/ github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:72 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /builddir/build/BUILD/origin-3.7.0/_output/local/go/src/ github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:65 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /builddir/build/BUILD/origin-3.7.0/_output/local/go/src/ github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:51 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /usr/lib/golang/src/runtime/asm_amd64.s:514 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /usr/lib/golang/src/runtime/panic.go:489 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /usr/lib/golang/src/runtime/panic.go:63 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /usr/lib/golang/src/runtime/signal_unix.go:290 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /builddir/build/BUILD/origin-3.7.0/_output/local/go/src/ github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/controller/deployment/deployment_controller.go:169 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /builddir/build/BUILD/origin-3.7.0/_output/local/go/src/ github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/controller/deployment/deployment_controller.go:121 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /builddir/build/BUILD/origin-3.7.0/_output/local/go/src/ github.com/openshift/origin/vendor/k8s.io/client-go/tools/cache/controller.go:192 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: <autogenerated>:57 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /builddir/build/BUILD/origin-3.7.0/_output/local/go/src/ github.com/openshift/origin/vendor/k8s.io/client-go/tools/cache/shared_informer.go:547 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: /usr/lib/golang/src/runtime/asm_amd64.s:2197 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: I1203 10:28:44.900445 89591 controllermanager.go:466] Started "deployment" Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: W1203 10:28:44.900490 89591 controllermanager.go:450] "ttl" is disabled Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: I1203 10:28:44.900454 89591 deployment_controller.go:152] Starting deployment controller Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: I1203 10:28:44.900765 89591 controller_utils.go:1025] Waiting for caches to sync for deployment controller Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: E1203 10:28:44.907405 89591 actual_state_of_world.go:478] Failed to set statusUpdateNeeded to needed true because nodeName="node03-2018" does not exist Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: E1203 10:28:44.907431 89591 actual_state_of_world.go:492] Failed to update statusUpdateNeeded field in actual state of world: Failed to set statusUpdateNeeded to needed true because nodeName="node03-2018" does not exist Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: E1203 10:28:44.907467 89591 actual_state_of_world.go:478] Failed to set statusUpdateNeeded to needed true because nodeName="node01-2018" does not exist Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: E1203 10:28:44.907478 89591 actual_state_of_world.go:492] Failed to update statusUpdateNeeded field in actual state of world: Failed to set statusUpdateNeeded to needed true because nodeName="node01-2018" does not exist Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: E1203 10:28:44.907521 89591 actual_state_of_world.go:478] Failed to set statusUpdateNeeded to needed true because nodeName="node02-2018" does not exist Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: E1203 10:28:44.907535 89591 actual_state_of_world.go:492] Failed to update statusUpdateNeeded field in actual state of world: Failed to set statusUpdateNeeded to needed true because nodeName="node02-2018" does not exist Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: I1203 10:28:44.923823 89591 controller_utils.go:1032] Caches are synced for replica set controller Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: I1203 10:28:44.943026 89591 controller_utils.go:1032] Caches are synced for node controller Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: I1203 10:28:44.943076 89591 taint_controller.go:182] Starting NoExecuteTaintManager Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: I1203 10:28:44.943092 89591 nodecontroller.go:525] NodeController observed a new Node: "node03-2018" Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: I1203 10:28:44.943113 89591 controller_utils.go:274] Recording Registered Node node03-2018 in NodeController event message for node node03-2018 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: I1203 10:28:44.943145 89591 nodecontroller.go:542] Initializing eviction metric for zone: Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: I1203 10:28:44.943159 89591 nodecontroller.go:525] NodeController observed a new Node: "node01-2018" Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: I1203 10:28:44.943167 89591 controller_utils.go:274] Recording Registered Node node01-2018 in NodeController event message for node node01-2018 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: I1203 10:28:44.943177 89591 nodecontroller.go:525] NodeController observed a new Node: "node02-2018" Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: I1203 10:28:44.943185 89591 controller_utils.go:274] Recording Registered Node node02-2018 in NodeController event message for node node02-2018 Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: W1203 10:28:44.943257 89591 nodecontroller.go:877] Missing timestamp for Node node03-2018. Assuming now as a timestamp. Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: I1203 10:28:44.943280 89591 event.go:218] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"node03-2018", UID:"9d690c08-f0ed-11e7-a024-0cc47a39144c", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RegisteredNode' Node node03-2018 event: Registered Node node03-2018 in NodeController Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: I1203 10:28:44.943307 89591 event.go:218] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"node01-2018", UID:"9d638709-f0ed-11e7-a024-0cc47a39144c", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RegisteredNode' Node node01-2018 event: Registered Node node01-2018 in NodeController Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: I1203 10:28:44.943319 89591 event.go:218] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"node02-2018", UID:"9d6692f6-f0ed-11e7-a024-0cc47a39144c", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RegisteredNode' Node node02-2018 event: Registered Node node02-2018 in NodeController Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: W1203 10:28:44.943296 89591 nodecontroller.go:877] Missing timestamp for Node node01-2018. Assuming now as a timestamp. Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: W1203 10:28:44.943364 89591 nodecontroller.go:877] Missing timestamp for Node node02-2018. Assuming now as a timestamp. Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: I1203 10:28:44.943394 89591 nodecontroller.go:793] NodeController detected that zone is now in state Normal. Dec 03 10:28:44 node01-2018.drivenow.com.au origin-master-controllers[89591]: I1203 10:28:44.950122 89591 controller_utils.go:1032] Caches are synced for daemon sets controller Cameron On Mon, 3 Dec 2018 at 02:48 Clayton Coleman <[email protected]> wrote: > Are there errors in the controller logs? > > On Dec 2, 2018, at 2:42 AM, Cameron Braid <[email protected]> wrote: > > Sorry, a typo - its a 3.7 cluster not 3.6 > > ~> oc version > > oc v3.7.2+282e43f > > kubernetes v1.7.6+a08f5eeb62 > > features: Basic-Auth > > > Server > > openshift v3.7.0+7ed6862 > > kubernetes v1.7.6+a08f5eeb62 > > On Sun, 2 Dec 2018 at 18:39 Cameron Braid <[email protected]> wrote: > >> I have a strange issue in my 3.6 cluster. I create a extensions/v1beta1 >> Deployment >> and nothing happens. No pods are created. oc describe shows >> >> oc -n drivenow-staging-x describe deployment strimzi-cluster-operator >> Name: strimzi-cluster-operator >> Namespace: drivenow-staging-x >> CreationTimestamp: Sun, 02 Dec 2018 01:33:10 +1100 >> Labels: app=strimzi >> Annotations: <none> >> Selector: name=strimzi-cluster-operator >> *Replicas: 1 desired | 0 updated | 0 total | 0 available | 0 unavailable* >> StrategyType: Recreate >> MinReadySeconds: 0 >> Pod Template: >> Labels: name=strimzi-cluster-operator >> Service Account: strimzi-cluster-operator >> Containers: >> strimzi-cluster-operator: >> Image: strimzi/cluster-operator:0.6.0 >> Port: <none> >> Limits: >> cpu: 1 >> memory: 256Mi >> Requests: >> cpu: 200m >> memory: 256Mi >> Liveness: http-get http://:8080/healthy delay=10s timeout=1s >> period=30s #success=1 #failure=3 >> Readiness: http-get http://:8080/ready delay=10s timeout=1s >> period=30s #success=1 #failure=3 >> Environment: ... >> Mounts: <none> >> Volumes: <none> >> OldReplicaSets: <none> >> NewReplicaSet: <none> >> Events: <none> >> >> All nodes are ready, schedulable and there doesnt appear to be antyhing >> in the logs. >> >> I've tried restarting origin-node, origin-master-api and >> origin-master-controllers on all nodes and that has no impact. >> >> I'm out of ideas. >> >> Cameron >> > _______________________________________________ > users mailing list > [email protected] > http://lists.openshift.redhat.com/openshiftmm/listinfo/users > >
_______________________________________________ users mailing list [email protected] http://lists.openshift.redhat.com/openshiftmm/listinfo/users
