We found an old copy of the original working registry. We deleted it and then pushes began working.
On Tue, Sep 19, 2017 at 1:34 PM, Graigry Henrie <[email protected]> wrote: > It does appear to be broken for every project we create on the cluster. > Containers that are from outside sources pull and run fine, but all fail > when we try to push to the registry. > > The ansible playbook is the standard openshift origin one found at > https://github.com/openshift/openshift-ansible. > > I am unable to login to the registry using "docker login -u "$(oc whoami)" > -p "$(oc whoami -t)" 172.30.241.53:5000". It spins a while and then > returns a > "Error response from daemon: Get https://172.30.241.53:5000/v1/users/: > dial tcp 172.30.241.53:5000: i/o timeout". > > I confirmed the IP of the docker-registry with "oc -n default get > svc/docker-registry" as 172.30.241.53, which is correct. > > There are 5 pods in the registry. I checked the logs for the registry and > all he shows are successful health checks as well. > > The strange portion of this is that when I try to push to the registry, I > can't see attempted connections in the docker-registry logs at all. We did > see once a permissions error, but we have looked at our storage and the > permissions appear to be correct. Is it possible that there are some > holdover files from a previous install that are for some reason interfering > with openshift as it tries to write? > > On Tue, Sep 19, 2017 at 3:25 AM, Alexey Gladkov <[email protected]> > wrote: > >> On Mon, Sep 18, 2017 at 11:46:40PM -0400, Ben Parees wrote: >> > On Mon, Sep 18, 2017 at 12:47 PM, Graigry Henrie <[email protected] >> > >> > wrote: >> > >> > > We recently installed openshift origin using an ansible playbook. We >> had >> > > used the same one before without issue, except that now when projects >> > > attempt to push to the internal registry, it results in a 500 error. >> > > >> > > Attached is the log from attempting to build the Django Example >> project. >> > > The build itself proceeds perfectly, but fails with a "500 Internal >> Server >> > > Error" when attempting to push to the internal registry. Checking >> the logs >> > > of the registry itself only shows health checks, indicating that >> container >> > > has come up properly. >> > > >> > > What makes it more confusing is that we have installed with this same >> file >> > > multiple times and sometimes it works and sometime it doesn't. We >> believe >> > > that it is a permissions issue. We have checked and rechecked our >> ansible >> > > hosts file, but we cannot see what the problem is. I have included >> it as >> > > well. >> > > >> > >> > can you try manually pushing to the registry using the service account's >> > token? (ie docker login to the registry w/ the token and then tag an >> > image+push to it). >> > >> > There is a known issue that the registry returns 500 errors when it >> should >> > return 403s, so your theory that it is a permission issue is plausible >> but >> > it is strange that some of your clusters work. When it is not working, >> it >> > sounds like it is broken for all projects in the cluster, not just some >> or >> > fails randomly? >> > >> > Adding our registry gurus to CC. >> >> Can we see this ansible playbook ? >> >> How many registry pods in the cluster (maybe the logs from the requests on >> the other pod) ? >> >> Is there a router in front of him and what's in his logs ? >> >> -- >> Rgrds, agladkov >> >> >
_______________________________________________ users mailing list [email protected] http://lists.openshift.redhat.com/openshiftmm/listinfo/users
