[
https://issues.apache.org/jira/browse/JCLOUDS-1516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Gaul resolved JCLOUDS-1516.
----------------------------------
Fix Version/s: 2.4.0
Assignee: Andrew Gaul
Resolution: Fixed
> First putblob should be signed with specific region rather than with default
> region during createcontainer API
> ---------------------------------------------------------------------------------------------------------------
>
> Key: JCLOUDS-1516
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1516
> Project: jclouds
> Issue Type: New Feature
> Components: jclouds-blobstore
> Affects Versions: 2.1.2
> Environment: Linux
> Reporter: Dileep Dixith
> Assignee: Andrew Gaul
> Priority: Major
> Fix For: 2.4.0
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> When container in non default region exists, only first time put operation
> will be applied to find out whether bucket exists and have proper access or
> not. Aws sigv4 signature will be created based on the default region only as
> create bucket method was not honoring the region specified.
> So, Put request was first signed with default region and if the user does not
> have access to default(us-east-1) region,. Then it will be re-directed to
> sa-east-1 region, but the request is signed with us-east-1, the request is
> rejected and throws "The authorization header is malformed".
> Flow in case of user has access to default region:
> 17:22:24.460 [bscThread-02] DEBUG o.j.rest.internal.InvokeHttpMethod - >>
> invoking CreateBucket
> 17:22:24.460 [bscThread-02] DEBUG o.j.rest.internal.InvokeHttpMethod - >>
> invoking CreateBucket
> 17:22:24.461 [bscThread-02] DEBUG jclouds.signature - << PUT
> /
> content-length:105
> content-type:text/xml
> host:test3.s3.amazonaws.com
> x-amz-content-sha256:f5d7dd57e1e23b516fc3543b9f24fc19a8409557905f48c6f412b3a67946ce96
> x-amz-date:20190818T115218Z
> content-length;content-type;host;x-amz-content-sha256;x-amz-date
> f5d7dd57e1e23b516fc3543b9f24fc19a8409557905f48c6f412b3a67946ce96
> 17:22:24.461 [bscThread-02] DEBUG jclouds.signature - << AWS4-HMAC-SHA256
> 20190818T115218Z
> 20190818/us-east-1/s3/aws4_request
> 089a5248f5eff6e8b6378154acdf07bff7d208029c98c67af44c99b4a8f2df39
> 17:22:24.463 [bscThread-02] DEBUG o.j.h.i.JavaUrlHttpCommandExecutorService -
> Sending request -1533211628: PUT https://test3.s3.amazonaws.com/ HTTP/1.1
> 17:22:24.463 [bscThread-02] DEBUG jclouds.wire - >>
> "<CreateBucketConfiguration><LocationConstraint>sa-east-1</LocationConstraint></CreateBucketConfiguration>"
> 17:22:24.464 [bscThread-02] DEBUG jclouds.headers - >> PUT
> https://test3.s3.amazonaws.com/ HTTP/1.1
> 17:22:24.464 [bscThread-02] DEBUG jclouds.headers - >> Content-Type: text/xml
> 17:22:24.464 [bscThread-02] DEBUG jclouds.headers - >> Content-Length: 105
> 17:22:24.464 [bscThread-02] DEBUG jclouds.headers - >> Host:
> test3.s3.amazonaws.com
> 17:22:24.464 [bscThread-02] DEBUG jclouds.headers - >> x-amz-content-sha256:
> f5d7dd57e1e23b516fc3543b9f24fc19a8409557905f48c6f412b3a67946ce96
> 17:22:24.464 [bscThread-02] DEBUG jclouds.headers - >> X-Amz-Date:
> 20190818T115218Z
> 17:22:24.464 [bscThread-02] DEBUG jclouds.headers - >> Authorization:
> AWS4-HMAC-SHA256
> Credential=AKIAIGKQ7V52FQQJFYJQ/20190818/us-east-1/s3/aws4_request,
> SignedHeaders=content-length;content-type;host;x-amz-content-sha256;x-amz-date,
> Signature=637d42fbf6684430ab0f08fd82cbae69f3261859e0031ad40054bccb829473da
> 17:22:24.464 [bscThread-02] DEBUG jclouds.headers - >> Content-Type: text/xml
> 17:22:24.464 [bscThread-02] DEBUG jclouds.headers - >> Content-Length: 105
> 17:22:25.671 [bscThread-02] DEBUG o.j.h.i.JavaUrlHttpCommandExecutorService -
> Receiving response -1533211628: HTTP/1.1 409 Conflict
> 17:22:25.671 [bscThread-02] DEBUG jclouds.headers - << HTTP/1.1 409 Conflict
> 17:22:25.672 [bscThread-02] DEBUG jclouds.headers - << Transfer-Encoding:
> chunked
> 17:22:25.672 [bscThread-02] DEBUG jclouds.headers - << Server: AmazonS3
> 17:22:25.672 [bscThread-02] DEBUG jclouds.headers - << x-amz-request-id:
> 09E5163C51F25F34
> 17:22:25.673 [bscThread-02] DEBUG jclouds.headers - << x-amz-id-2:
> WuN84GYMs47Nn6+48XYDpLZNvp0NPszokKyhxlzZk+ub8RhjbLpkfEI8E2tKWVCFKtJiXrhdpkc=
> 17:22:25.673 [bscThread-02] DEBUG jclouds.headers - << Date: Sun, 18 Aug 2019
> 11:52:11 GMT
> 17:22:25.673 [bscThread-02] DEBUG jclouds.headers - << x-amz-bucket-region:
> sa-east-1
> 17:22:25.673 [bscThread-02] DEBUG jclouds.headers - << Content-Type:
> application/xml
> 17:22:25.673 [bscThread-02] DEBUG jclouds.wire - << "<?xml version="1.0"
> encoding="UTF-8"?>[\n]"
> 17:22:25.673 [bscThread-02] DEBUG jclouds.wire - <<
> "<Error><Code>BucketAlreadyOwnedByYou</Code><Message>Your previous request to
> create the named bucket succeeded and you already own
> it.</Message><BucketName>test3</BucketName><RequestId>09E5163C51F25F34</RequestId><HostId>WuN84GYMs47Nn6+48XYDpLZNvp0NPszokKyhxlzZk+ub8RhjbLpkfEI8E2tKWVCFKtJiXrhdpkc=</HostId></Error>"
> 17:22:25.680 [bscThread-03] DEBUG o.j.rest.internal.InvokeHttpMethod - >>
> invoking BucketExists
> 17:22:25.681 [bscThread-03] DEBUG jclouds.signature - << HEAD
> /
> Flow in case of user has access to default region
> host:test3.s3-sa-east-1.amazonaws.com
> x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
> x-amz-date:20190818T115218Z
>
> Flow in case of user does not have access to default region.
> 17:28:41.464 [bscThread-02] DEBUG o.j.rest.internal.InvokeHttpMethod - >>
> invoking CreateBucket
> 17:28:41.464 [bscThread-02] DEBUG o.j.rest.internal.InvokeHttpMethod - >>
> invoking CreateBucket
> 17:28:41.466 [bscThread-02] DEBUG jclouds.signature - << PUT
> /
> content-length:105
> content-type:text/xml
> host:test-poc-spectrum-scale.s3.amazonaws.com
> x-amz-content-sha256:f5d7dd57e1e23b516fc3543b9f24fc19a8409557905f48c6f412b3a67946ce96
> x-amz-date:20190818T115835Z
> content-length;content-type;host;x-amz-content-sha256;x-amz-date
> f5d7dd57e1e23b516fc3543b9f24fc19a8409557905f48c6f412b3a67946ce96
> 17:28:41.466 [bscThread-02] DEBUG jclouds.signature - << AWS4-HMAC-SHA256
> 20190818T115835Z
> 20190818/us-east-1/s3/aws4_request
> d8c68a44175c3b0b49182ca5658bff291b3f382d7d02b3fe25d6970912df1697
> 17:28:41.467 [bscThread-02] DEBUG o.j.h.i.JavaUrlHttpCommandExecutorService -
> Sending request -541426552: PUT
> https://test-poc-spectrum-scale.s3.amazonaws.com/ HTTP/1.1
> 17:28:41.467 [bscThread-02] DEBUG jclouds.wire - >>
> "<CreateBucketConfiguration><LocationConstraint>sa-east-1</LocationConstraint></CreateBucketConfiguration>"
> 17:28:41.467 [bscThread-02] DEBUG jclouds.headers - >> PUT
> https://test-poc-spectrum-scale.s3.amazonaws.com/ HTTP/1.1
> 17:28:41.467 [bscThread-02] DEBUG jclouds.headers - >> Content-Type: text/xml
> 17:28:41.468 [bscThread-02] DEBUG jclouds.headers - >> Content-Length: 105
> 17:28:41.468 [bscThread-02] DEBUG jclouds.headers - >> Host:
> test-poc-spectrum-scale.s3.amazonaws.com
> 17:28:41.468 [bscThread-02] DEBUG jclouds.headers - >> x-amz-content-sha256:
> f5d7dd57e1e23b516fc3543b9f24fc19a8409557905f48c6f412b3a67946ce96
> 17:28:41.468 [bscThread-02] DEBUG jclouds.headers - >> X-Amz-Date:
> 20190818T115835Z
> 17:28:41.468 [bscThread-02] DEBUG jclouds.headers - >> Authorization:
> AWS4-HMAC-SHA256
> Credential=AKIA25YU33MFMAZO3B7F/20190818/us-east-1/s3/aws4_request,
> SignedHeaders=content-length;content-type;host;x-amz-content-sha256;x-amz-date,
> Signature=8cf0d9f58bd0ba5fc607f5018e701cdf14587741169ba7deaaa7a9cad0548cb6
> 17:28:41.468 [bscThread-02] DEBUG jclouds.headers - >> Content-Type: text/xml
> 17:28:41.468 [bscThread-02] DEBUG jclouds.headers - >> Content-Length: 105
> 17:28:43.016 [bscThread-02] DEBUG o.j.h.i.JavaUrlHttpCommandExecutorService -
> Receiving response -541426552: HTTP/1.1 400 Bad Request
> 17:28:43.017 [bscThread-02] DEBUG jclouds.headers - << HTTP/1.1 400 Bad
> Request
> 17:28:43.017 [bscThread-02] DEBUG jclouds.headers - << Transfer-Encoding:
> chunked
> 17:28:43.017 [bscThread-02] DEBUG jclouds.headers - << Server: AmazonS3
> 17:28:43.017 [bscThread-02] DEBUG jclouds.headers - << Connection: close
> 17:28:43.017 [bscThread-02] DEBUG jclouds.headers - << x-amz-request-id:
> 43F137234826AA08
> 17:28:43.017 [bscThread-02] DEBUG jclouds.headers - << x-amz-id-2:
> nzJhRvv6j87VgSKV/mlGNhAqdietQCH4V5ArVUdIX2j1SnTChg2t1/GH20Zs3iz6Bc0MPzzAzwc=
> 17:28:43.017 [bscThread-02] DEBUG jclouds.headers - << Date: Sun, 18 Aug 2019
> 11:58:29 GMT
> 17:28:43.017 [bscThread-02] DEBUG jclouds.headers - << Content-Type:
> application/xml
> 17:28:43.019 [bscThread-02] DEBUG jclouds.wire - << "<?xml version="1.0"
> encoding="UTF-8"?>[\n]"
> 17:28:43.019 [bscThread-02] DEBUG jclouds.wire - <<
> "<Error><Code>AuthorizationHeaderMalformed</Code><Message>The authorization
> header is malformed; the region 'us-east-1' is wrong; expecting
> 'sa-east-1'</Message><Region>sa-east-1</Region><RequestId>43F137234826AA08</RequestId><HostId>nzJhRvv6j87VgSKV/mlGNhAqdietQCH4V5ArVUdIX2j1SnTChg2t1/GH20Zs3iz6Bc0MPzzAzwc=</HostId></Error>"
> EXC:org.jclouds.aws.AWSResponseException: request PUT
> https://test-poc-spectrum-scale.s3.amazonaws.com/ HTTP/1.1 failed with code
> 400, error: AWSError\{requestId='43F137234826AA08',
> requestToken='nzJhRvv6j87VgSKV/mlGNhAqdietQCH4V5ArVUdIX2j1SnTChg2t1/GH20Zs3iz6Bc0MPzzAzwc=',
> code='AuthorizationHeaderMalformed', message='The authorization header is
> malformed; the region 'us-east-1' is wrong; expecting 'sa-east-1'',
> context='{Region=sa-east-1,
> HostId=nzJhRvv6j87VgSKV/mlGNhAqdietQCH4V5ArVUdIX2j1SnTChg2t1/GH20Zs3iz6Bc0MPzzAzwc=}'}
> I have a fix so that the first put operation honors the custom region
> specified and request will be signed with specific region rather than with
> default region. Tested with default region as well and worked well.
> After my fix, flow with stand-alone program is as below.
> 17:36:09.043 [bscThread-02] DEBUG o.j.rest.internal.InvokeHttpMethod - >>
> invoking CreateBucket
> 17:36:09.043 [bscThread-02] DEBUG o.j.rest.internal.InvokeHttpMethod - >>
> invoking CreateBucket
> 17:36:09.044 [bscThread-02] DEBUG jclouds.signature - << PUT
> /
> content-length:105
> content-type:text/xml
> host:test-poc-spectrum-scale.s3-sa-east-1.amazonaws.com
> x-amz-content-sha256:f5d7dd57e1e23b516fc3543b9f24fc19a8409557905f48c6f412b3a67946ce96
> x-amz-date:20190818T120602Z
> content-length;content-type;host;x-amz-content-sha256;x-amz-date
> f5d7dd57e1e23b516fc3543b9f24fc19a8409557905f48c6f412b3a67946ce96
> 17:36:09.044 [bscThread-02] DEBUG jclouds.signature - << AWS4-HMAC-SHA256
> 20190818T120602Z
> 20190818/sa-east-1/s3/aws4_request
> 2996a3b4d85ab8c68e1378ab68db2f7d80b244969d1792391a49b9b7390bc920
> 17:36:09.046 [bscThread-02] DEBUG o.j.h.i.JavaUrlHttpCommandExecutorService -
> Sending request -719121422: PUT
> https://test-poc-spectrum-scale.s3-sa-east-1.amazonaws.com/ HTTP/1.1
> 17:36:09.046 [bscThread-02] DEBUG jclouds.wire - >>
> "<CreateBucketConfiguration><LocationConstraint>sa-east-1</LocationConstraint></CreateBucketConfiguration>"
> 17:36:09.046 [bscThread-02] DEBUG jclouds.headers - >> PUT
> https://test-poc-spectrum-scale.s3-sa-east-1.amazonaws.com/ HTTP/1.1
> 17:36:09.047 [bscThread-02] DEBUG jclouds.headers - >> Content-Type: text/xml
> 17:36:09.047 [bscThread-02] DEBUG jclouds.headers - >> Content-Length: 105
> 17:36:09.047 [bscThread-02] DEBUG jclouds.headers - >> Host:
> test-poc-spectrum-scale.s3-sa-east-1.amazonaws.com
> 17:36:09.047 [bscThread-02] DEBUG jclouds.headers - >> x-amz-content-sha256:
> f5d7dd57e1e23b516fc3543b9f24fc19a8409557905f48c6f412b3a67946ce96
> 17:36:09.047 [bscThread-02] DEBUG jclouds.headers - >> X-Amz-Date:
> 20190818T120602Z
> 17:36:09.047 [bscThread-02] DEBUG jclouds.headers - >> Authorization:
> AWS4-HMAC-SHA256
> Credential=AKIA25YU33MFMAZO3B7F/20190818/sa-east-1/s3/aws4_request,
> SignedHeaders=content-length;content-type;host;x-amz-content-sha256;x-amz-date,
> Signature=982a3a62e85e03c68f4ac1e0da5cfa753cb81cf750a6fbff157b34681ed54774
> 17:36:09.047 [bscThread-02] DEBUG jclouds.headers - >> Content-Type: text/xml
> 17:36:09.047 [bscThread-02] DEBUG jclouds.headers - >> Content-Length: 105
> 17:36:11.076 [bscThread-02] DEBUG o.j.h.i.JavaUrlHttpCommandExecutorService -
> Receiving response -719121422: HTTP/1.1 409 Conflict
> 17:36:11.076 [bscThread-02] DEBUG jclouds.headers - << HTTP/1.1 409 Conflict
> 17:36:11.076 [bscThread-02] DEBUG jclouds.headers - << Transfer-Encoding:
> chunked
> 17:36:11.077 [bscThread-02] DEBUG jclouds.headers - << Server: AmazonS3
> 17:36:11.077 [bscThread-02] DEBUG jclouds.headers - << x-amz-request-id:
> 5FB8D9C1B41E48EB
> 17:36:11.077 [bscThread-02] DEBUG jclouds.headers - << x-amz-id-2:
> 54oqA5Lc+yl6Y9Ppz6kMd6hZs/iGFGfNFPqrOPX90Q731UH5KkYEYc7RS/4W9btahu0dxQKd3iI=
> 17:36:11.077 [bscThread-02] DEBUG jclouds.headers - << Date: Sun, 18 Aug 2019
> 12:05:57 GMT
> 17:36:11.077 [bscThread-02] DEBUG jclouds.headers - << x-amz-bucket-region:
> sa-east-1
> 17:36:11.077 [bscThread-02] DEBUG jclouds.headers - << Content-Type:
> application/xml
> 17:36:11.077 [bscThread-02] DEBUG jclouds.wire - << "<?xml version="1.0"
> encoding="UTF-8"?>[\n]"
> 17:36:11.077 [bscThread-02] DEBUG jclouds.wire - <<
> "<Error><Code>BucketAlreadyOwnedByYou</Code><Message>Your previous request to
> create the named bucket succeeded and you already own
> it.</Message><BucketName>test-poc-spectrum-scale</BucketName><RequestId>5FB8D9C1B41E48EB</RequestId><HostId>54oqA5Lc+yl6Y9Ppz6kMd6hZs/iGFGfNFPqrOPX90Q731UH5KkYEYc7RS/4W9btahu0dxQKd3iI=</HostId></Error>"
> 17:36:11.083 [bscThread-03] DEBUG o.j.rest.internal.InvokeHttpMethod -
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
