That was the conclusion I'd come to. So from here would you think it worthwhile I raise a ticket via the enterprise path, or is there another means of merging the latest docker registry image?
Cheers On 20 July 2016 at 14:41, Clayton Coleman <[email protected]> wrote: > Origin 1.2 and Enterprise 1.2 are the same Docker distribution version, > v2.2. They also use the same s3 driver, so I wouldn't expect a > difference. I don't think anyone has tested sydney, but I would not be > surprised if the S3 client driver does not fully support that region at the > point in time we snapshotted it. The fact that us-east worked makes me > suspicious. > > On Wed, Jul 20, 2016 at 12:38 AM, Lewis Shobbrook < > [email protected]> wrote: > >> Thanks for your response Clayton, >> >> It's not the v4auth, as I've experimented disabling it and the error is >> the same. >> If the s3 driver, this issue would likely be affecting the 3.2 enterprise >> release too? >> I'm planning on launching an enterprise 3.2 stack to confirm this. >> >> Do you know if the code base is the same for the docker-registry? >> >> Might save some time... >> >> Cheers, >> >> Lew >> >> >> >> >> >> >> On 20 July 2016 at 14:19, Clayton Coleman <[email protected]> wrote: >> >>> Is S3 v4auth supported in the sydney region? That is the first thing >>> that comes to mind. The public docker image for the registry is a bit >>> newer than the registry in Origin, so it's possible as well that the S3 >>> driver was updated between that registry and v1.2.0. No registry changes >>> were made in origin between v1.2.0-rc1 and v1.2.0, so I suspect the change >>> is related to S3 config, not the registry code. >>> >>> On Tue, Jul 19, 2016 at 11:21 PM, Lewis Shobbrook < >>> [email protected]> wrote: >>> >>>> Hi Guys, >>>> >>>> Spent quite a bit more time on this and have been able to determine >>>> that the problem is specifically with the openshift registry image. >>>> Using the docker.io/registry:2 to create a registry on the same nodes, >>>> using the same configuration works. >>>> The service is able to write to the designated S3 folder. >>>> >>>> I've adjusted the registry build to use the latest image >>>> oadm registry --config=/etc/origin/master/admin.kubeconfig >>>> --service-account=registry >>>> --credentials=/etc/origin/master/openshift-registry.kubeconfig >>>> --latest-images=true >>>> Which has pulled a newer image, but alas the error is much the same... >>>> >>>> http.request.method=HEAD http.request.remoteaddr="10.1.1.1:51518" >>>> http.request.uri="/v2/bnz-uat/buzybox/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" >>>> http.request.useragent="docker/1.9.1 go/go1.4.2 >>>> kernel/3.10.0-327.22.2.el7.x86_64 os/linux arch/amd64" >>>> instance.id=745d924f-3a9d-4613-bd1d-dd9c826d52e0 >>>> vars.digest="sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" >>>> vars.name="bnz-uat/buzybox" >>>> time="2016-07-20T02:03:20.167566754Z" level=error msg="response >>>> completed with error" err.code=unknown err.detail="s3aws: RequestError: >>>> send request failed\ncaused by: Get >>>> https://os3master-prod-os-aws-XXX-com-au-docker.s3-ap-southeast-2.amazonaws.com/?max-keys=1&prefix=registry%2Fdocker%2Fregistry%2Fv2%2Fblobs%2Fsha256%2Fa3%2Fa3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4%2Fdata: >>>> dial tcp 54.66.155.60:443: getsockopt: connection refused" >>>> err.message="unknown error" go.version=go1.6.2 http.request.host=" >>>> 172.30.160.215:5000" http.request.id=0cf8999f-21b6-4c8c-9ffb-258e607e4914 >>>> http.request.method=HEAD http.request.remoteaddr="10.1.1.1:51518" >>>> http.request.uri="/v2/bnz-uat/buzybox/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" >>>> http.request.useragent="docker/1.9.1 go/go1.4.2 >>>> kernel/3.10.0-327.22.2.el7.x86_64 os/linux arch/amd64" >>>> http.response.contenttype="application/json; charset=utf-8" >>>> http.response.duration=308.013092ms http.response.status=500 >>>> http.response.written=104 instance.id=745d924f-3a9d-4613-bd1d-dd9c826d52e0 >>>> vars.digest="sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" >>>> vars.name="bnz-uat/buzybox" >>>> >>>> Is there anyone willing to help out here. Banging my head. >>>> >>>> Cheers, >>>> >>>> Lew >>>> >>>> On 13 July 2016 at 00:13, Lewis Shobbrook < >>>> [email protected]> wrote: >>>> >>>>> Hi Guys, >>>>> >>>>> I've spent the past two days looking at the problem of s3 backed >>>>> storage from Sydney based instances. The problem now appears to be more >>>>> generally associated with S3 backed registries in the ap-southeast-2 >>>>> region. >>>>> >>>>> With configurations identical other than keys/bucket/region the >>>>> results are considerably different. >>>>> >>>>> In Sydney I see this... >>>>> >>>>> time="2016-07-12T13:03:33.859868304Z" level=error msg="response >>>>> completed with error" err.code=UNKNOWN err.detail="s3: Get >>>>> https://s3-external-1.amazonaws.com/os3master-prod-os-aws-XXX-com-au-docker-registry/registry/docker/registry/v2/repositories/bnz-uat/busybox/_layers/sha256/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4/link: >>>>> dial tcp 54.66.155.60:443: getsockopt: connection refused" >>>>> err.message="unknown error" go.version=go1.6 http.request.host=" >>>>> 172.30.27.237:5000" http.request.id=f5a2a1cb-be54-4f31-a13d-1294fc74a3f8 >>>>> http.request.method=HEAD http.request.remoteaddr="10.1.0.1:45204" >>>>> http.request.uri="/v2/bnz-uat/busybox/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" >>>>> http.request.useragent="docker/1.9.1 go/go1.4.2 >>>>> kernel/3.10.0-327.22.2.el7.x86_64 os/linux arch/amd64" >>>>> http.response.contenttype="application/json; charset=utf-8" >>>>> http.response.duration=4.863003309s http.response.status=500 >>>>> http.response.written=487 instance.id >>>>> =542dd6da-2901-4cc7-975c-4afd71077e19 >>>>> >>>>> Configuration as follows... >>>>> >>>>> version: 0.1 >>>>> log: >>>>> level: debug >>>>> http: >>>>> addr: :5000 >>>>> storage: >>>>> cache: >>>>> layerinfo: inmemory >>>>> s3: >>>>> accesskey: XXX >>>>> secretkey: XXX >>>>> region: us-east-1 >>>>> bucket: os3master-prod-os-aws-XX-com-au-docker-registry >>>>> encrypt: true >>>>> secure: true >>>>> v4auth: true >>>>> rootdirectory: /registry >>>>> auth: >>>>> openshift: >>>>> realm: openshift >>>>> middleware: >>>>> repository: >>>>> - name: openshift >>>>> >>>>> While the working registry in us-east >>>>> time="2016-07-12T13:45:08.168492162Z" level=info msg="response >>>>> completed" go.version=go1.6 http.request.host="172.30.1.78:5000" >>>>> http.request.id=66047be1-0f4e-4045-b710-c94fdaca10d6 >>>>> http.request.method=GET http.request.remoteaddr="10.1.0.1:56051" >>>>> http.request.uri="/v2/paycorp-pty-ltd/paycorp-service-auth1501/manifests/sha256:78a3af175b9d7c653cd4fdb42a3ccf44095c6e07b912c2719a85d5568179129f" >>>>> http.request.useragent="docker/1.9.1 go/go1.4.2 >>>>> kernel/3.10.0-327.13.1.el7.x86_64 os/linux arch/amd64" >>>>> http.response.contenttype="application/json; charset=utf-8" >>>>> http.response.duration=49.599539ms http.response.status=200 >>>>> http.response.written=66957 instance.id >>>>> =04de8cf0-2b4e-4546-b623-43f05c771805 >>>>> >>>>> version: 0.1 >>>>> log: >>>>> level: debug >>>>> http: >>>>> addr: :5000 >>>>> storage: >>>>> cache: >>>>> layerinfo: inmemory >>>>> s3: >>>>> accesskey: XXX >>>>> secretkey: XXX >>>>> region: us-east-1 >>>>> bucket: os3master-test-openshift-XXX-com-au-docker >>>>> encrypt: true >>>>> secure: true >>>>> v4auth: true >>>>> rootdirectory: /registry >>>>> auth: >>>>> openshift: >>>>> realm: openshift >>>>> middleware: >>>>> repository: >>>>> - name: openshif >>>>> >>>>> Looking inside the containers the only diffrence appears to be the >>>>> image id which is openshift/origin-docker-registry:v1.2.0-rc1 in the >>>>> working region & 1.2.0 in the failing. >>>>> >>>>> I'm hoping someone can assist here. >>>>> >>>>> Cheers, >>>>> >>>>> Lew >>>>> >>>> >>>> >>>> _______________________________________________ >>>> users mailing list >>>> [email protected] >>>> http://lists.openshift.redhat.com/openshiftmm/listinfo/users >>>> >>>> >>> >> >
_______________________________________________ users mailing list [email protected] http://lists.openshift.redhat.com/openshiftmm/listinfo/users
