Hi! I have a cluster that was initially installed with 3.7 (afaik) and upgraded to 3.10.
It has been running fine for a many month now, but yesterday problems
occured.
* ImagePullBackOff for newly scheduled pods
* 403 from docker registry (error: build error: Failed to push
image... HTTP 403 response body: unexpected end of JSON input: "")
I had that error (with origin 1.2 or even earlier) when it could be
fixed by deleting the dockercfg secrets. I did that but that did not
fix the problem.
I tried deleting the builder serviceaccount and the dockerfg does not
get created anymore (the new builder sa and secret builder-token-XXXXX
do get created).
In the master-api-<node> logs I get the following log
1 authentication.go:63] Unable to authenticate the request due to an
error: [invalid bearer token, [invalid bearer token, [Token has been
invalidated, oauthaccesstokens.oauth.openshift.io "XXXXXX" not found]]]
and many events about that problem (in fact, if I filter
oauthaccesstoken.oauth.openshift.io and audits, I hardly get any logs; only
etcd diagnostics).
In the master-controllers-<node> logs I get the following log entries
github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/controller/garbagecollector/graph_builder.go:125:
Failed to list <nil>: Unauthorized
endpoints_controller.go:375] Error syncing endpoints for service
"a_namespace/a_service": Unauthorized
Which most likely means that it just can't speak to the api server.
Anything I can do to fix this problem?
Thank you for your help,
Tobias Florek
signature.asc
Description: signature
_______________________________________________ users mailing list users@lists.openshift.redhat.com http://lists.openshift.redhat.com/openshiftmm/listinfo/users
