Origin 1.2 and Enterprise 1.2 are the same Docker distribution version, v2.2. They also use the same s3 driver, so I wouldn't expect a difference. I don't think anyone has tested sydney, but I would not be surprised if the S3 client driver does not fully support that region at the point in time we snapshotted it. The fact that us-east worked makes me suspicious.
On Wed, Jul 20, 2016 at 12:38 AM, Lewis Shobbrook < [email protected]> wrote: > Thanks for your response Clayton, > > It's not the v4auth, as I've experimented disabling it and the error is > the same. > If the s3 driver, this issue would likely be affecting the 3.2 enterprise > release too? > I'm planning on launching an enterprise 3.2 stack to confirm this. > > Do you know if the code base is the same for the docker-registry? > > Might save some time... > > Cheers, > > Lew > > > > > > > On 20 July 2016 at 14:19, Clayton Coleman <[email protected]> wrote: > >> Is S3 v4auth supported in the sydney region? That is the first thing >> that comes to mind. The public docker image for the registry is a bit >> newer than the registry in Origin, so it's possible as well that the S3 >> driver was updated between that registry and v1.2.0. No registry changes >> were made in origin between v1.2.0-rc1 and v1.2.0, so I suspect the change >> is related to S3 config, not the registry code. >> >> On Tue, Jul 19, 2016 at 11:21 PM, Lewis Shobbrook < >> [email protected]> wrote: >> >>> Hi Guys, >>> >>> Spent quite a bit more time on this and have been able to determine that >>> the problem is specifically with the openshift registry image. >>> Using the docker.io/registry:2 to create a registry on the same nodes, >>> using the same configuration works. >>> The service is able to write to the designated S3 folder. >>> >>> I've adjusted the registry build to use the latest image >>> oadm registry --config=/etc/origin/master/admin.kubeconfig >>> --service-account=registry >>> --credentials=/etc/origin/master/openshift-registry.kubeconfig >>> --latest-images=true >>> Which has pulled a newer image, but alas the error is much the same... >>> >>> http.request.method=HEAD http.request.remoteaddr="10.1.1.1:51518" >>> http.request.uri="/v2/bnz-uat/buzybox/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" >>> http.request.useragent="docker/1.9.1 go/go1.4.2 >>> kernel/3.10.0-327.22.2.el7.x86_64 os/linux arch/amd64" >>> instance.id=745d924f-3a9d-4613-bd1d-dd9c826d52e0 >>> vars.digest="sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" >>> vars.name="bnz-uat/buzybox" >>> time="2016-07-20T02:03:20.167566754Z" level=error msg="response >>> completed with error" err.code=unknown err.detail="s3aws: RequestError: >>> send request failed\ncaused by: Get >>> https://os3master-prod-os-aws-XXX-com-au-docker.s3-ap-southeast-2.amazonaws.com/?max-keys=1&prefix=registry%2Fdocker%2Fregistry%2Fv2%2Fblobs%2Fsha256%2Fa3%2Fa3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4%2Fdata: >>> dial tcp 54.66.155.60:443: getsockopt: connection refused" >>> err.message="unknown error" go.version=go1.6.2 http.request.host=" >>> 172.30.160.215:5000" http.request.id=0cf8999f-21b6-4c8c-9ffb-258e607e4914 >>> http.request.method=HEAD http.request.remoteaddr="10.1.1.1:51518" >>> http.request.uri="/v2/bnz-uat/buzybox/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" >>> http.request.useragent="docker/1.9.1 go/go1.4.2 >>> kernel/3.10.0-327.22.2.el7.x86_64 os/linux arch/amd64" >>> http.response.contenttype="application/json; charset=utf-8" >>> http.response.duration=308.013092ms http.response.status=500 >>> http.response.written=104 instance.id=745d924f-3a9d-4613-bd1d-dd9c826d52e0 >>> vars.digest="sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" >>> vars.name="bnz-uat/buzybox" >>> >>> Is there anyone willing to help out here. Banging my head. >>> >>> Cheers, >>> >>> Lew >>> >>> On 13 July 2016 at 00:13, Lewis Shobbrook < >>> [email protected]> wrote: >>> >>>> Hi Guys, >>>> >>>> I've spent the past two days looking at the problem of s3 backed >>>> storage from Sydney based instances. The problem now appears to be more >>>> generally associated with S3 backed registries in the ap-southeast-2 >>>> region. >>>> >>>> With configurations identical other than keys/bucket/region the results >>>> are considerably different. >>>> >>>> In Sydney I see this... >>>> >>>> time="2016-07-12T13:03:33.859868304Z" level=error msg="response >>>> completed with error" err.code=UNKNOWN err.detail="s3: Get >>>> https://s3-external-1.amazonaws.com/os3master-prod-os-aws-XXX-com-au-docker-registry/registry/docker/registry/v2/repositories/bnz-uat/busybox/_layers/sha256/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4/link: >>>> dial tcp 54.66.155.60:443: getsockopt: connection refused" >>>> err.message="unknown error" go.version=go1.6 http.request.host=" >>>> 172.30.27.237:5000" http.request.id=f5a2a1cb-be54-4f31-a13d-1294fc74a3f8 >>>> http.request.method=HEAD http.request.remoteaddr="10.1.0.1:45204" >>>> http.request.uri="/v2/bnz-uat/busybox/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" >>>> http.request.useragent="docker/1.9.1 go/go1.4.2 >>>> kernel/3.10.0-327.22.2.el7.x86_64 os/linux arch/amd64" >>>> http.response.contenttype="application/json; charset=utf-8" >>>> http.response.duration=4.863003309s http.response.status=500 >>>> http.response.written=487 instance.id >>>> =542dd6da-2901-4cc7-975c-4afd71077e19 >>>> >>>> Configuration as follows... >>>> >>>> version: 0.1 >>>> log: >>>> level: debug >>>> http: >>>> addr: :5000 >>>> storage: >>>> cache: >>>> layerinfo: inmemory >>>> s3: >>>> accesskey: XXX >>>> secretkey: XXX >>>> region: us-east-1 >>>> bucket: os3master-prod-os-aws-XX-com-au-docker-registry >>>> encrypt: true >>>> secure: true >>>> v4auth: true >>>> rootdirectory: /registry >>>> auth: >>>> openshift: >>>> realm: openshift >>>> middleware: >>>> repository: >>>> - name: openshift >>>> >>>> While the working registry in us-east >>>> time="2016-07-12T13:45:08.168492162Z" level=info msg="response >>>> completed" go.version=go1.6 http.request.host="172.30.1.78:5000" >>>> http.request.id=66047be1-0f4e-4045-b710-c94fdaca10d6 >>>> http.request.method=GET http.request.remoteaddr="10.1.0.1:56051" >>>> http.request.uri="/v2/paycorp-pty-ltd/paycorp-service-auth1501/manifests/sha256:78a3af175b9d7c653cd4fdb42a3ccf44095c6e07b912c2719a85d5568179129f" >>>> http.request.useragent="docker/1.9.1 go/go1.4.2 >>>> kernel/3.10.0-327.13.1.el7.x86_64 os/linux arch/amd64" >>>> http.response.contenttype="application/json; charset=utf-8" >>>> http.response.duration=49.599539ms http.response.status=200 >>>> http.response.written=66957 instance.id >>>> =04de8cf0-2b4e-4546-b623-43f05c771805 >>>> >>>> version: 0.1 >>>> log: >>>> level: debug >>>> http: >>>> addr: :5000 >>>> storage: >>>> cache: >>>> layerinfo: inmemory >>>> s3: >>>> accesskey: XXX >>>> secretkey: XXX >>>> region: us-east-1 >>>> bucket: os3master-test-openshift-XXX-com-au-docker >>>> encrypt: true >>>> secure: true >>>> v4auth: true >>>> rootdirectory: /registry >>>> auth: >>>> openshift: >>>> realm: openshift >>>> middleware: >>>> repository: >>>> - name: openshif >>>> >>>> Looking inside the containers the only diffrence appears to be the >>>> image id which is openshift/origin-docker-registry:v1.2.0-rc1 in the >>>> working region & 1.2.0 in the failing. >>>> >>>> I'm hoping someone can assist here. >>>> >>>> Cheers, >>>> >>>> Lew >>>> >>> >>> >>> _______________________________________________ >>> users mailing list >>> [email protected] >>> http://lists.openshift.redhat.com/openshiftmm/listinfo/users >>> >>> >> >
_______________________________________________ users mailing list [email protected] http://lists.openshift.redhat.com/openshiftmm/listinfo/users
