[
https://issues.apache.org/jira/browse/HDDS-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Elek reassigned HDDS-4856:
---------------------------------
Assignee: Marton Elek
> Ruby S3 SDK never get authenticated by Ozone
> --------------------------------------------
>
> Key: HDDS-4856
> URL: https://issues.apache.org/jira/browse/HDDS-4856
> Project: Apache Ozone
> Issue Type: Bug
> Components: S3
> Affects Versions: 1.0.0
> Environment: Secure setup of Ozone 1.0.0
> Reporter: UENISHI Kota
> Assignee: Marton Elek
> Priority: Major
> Attachments: ozone-test.py, ozone-test.rb, ruby-sdk-patch.diff
>
>
> When the very first call by Ruby client against secure setup of Ozone, the
> server returns 400 no matter how valid the request is. See the attached
> ruby-sdk-patch.diff, which adds some tests on S3 auth header
> signature-to-sign generation. It consists of two test additions, the "2" is
> the one generated by boto3, the "3" is generated by aws-ruby-sdk. Both passes
> the additional tests, which are definitely valid.
> However, when real HTTP request is sent by Ruby client, e.g. ozone-test.rb
> attached, it fails with 400. The header was like this (though the host names
> and domains are masked):
> {quote}GET //ozone.example.com:9879/sandbox?list-type=2&max-keys=1 HTTP/1.1
> Content-Type:
> Accept-Encoding:
> User-Agent: aws-sdk-ruby3/3.112.0 ruby/2.7.2 x86_64-linux aws-sdk-s3/1.88.1
> Host: ozone.example.com:9879
> X-Amz-Date: 20210222T110554Z
> X-Amz-Content-Sha256:
> e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
> Authorization: AWS4-HMAC-SHA256
> [email protected]/20210222/foobar/s3/aws4_request,
> SignedHeaders=host;user-agent;x-amz-content-sha256;x-amz-date,
> Signature=0c9469f018f5
> b3fd2cff6f8d4e4963f50aa71c6704def59527634404f5fc98a9
> Content-Length: 0
> Accept: */*{quote}
> On the other hand, request headers made by boto3 was:
> {quote}GET //ozone.example.com:9879/sandbox?list-type=2&encoding-type=url
> HTTP/1.1
> Host: ozone.example.com:9879
> Accept-Encoding: identity
> User-Agent: Boto3/1.17.12 Python/3.9.1 Linux/5.10.14-arch1-1 Botocore/1.20.12
> X-Amz-Date: 20210222T110829Z
> X-Amz-Content-SHA256:
> e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
> Authorization: AWS4-HMAC-SHA256
> [email protected]/20210222/us-east-1/s3/aws4_request,
> SignedHeaders=host;x-amz-content-sha256;x-amz-date,
> Signature=94302f21cccac8832d3e
> 4fe25c5f6d8a0307188fb0e1b1983264339381d21dac{quote}
> The difference of these requests are IMHO, "Content-Type" and
> "Accept-Encoding" are both empty in Ruby SDK. I'm afraid this error stems
> from partly Ruby SDK and partly from [Jetty
> Issue|https://github.com/eclipse/jetty.project/issues/2883]. The former sends
> empty header lines and the latter rejects them.
> And the s3g debug log (only error'ish part) follows:
> {quote}2021-02-22 20:55:54,450 [qtp1637061418-81] DEBUG
> servlet.ServletHandler:
> chain=NoCacheFilter@5e600dd5==org.apache.hadoop.hdds.server.http.NoCacheFilter,inst=true,async=true-
> >safety@63a12c68==org.apache.hadoop.hdds.server.http.HttpServer2$QuotingInputFilter,inst=true,async=true->info-page-redirect@576d5deb==org.apache.hadoop.ozone.s3.RootPageDis
> playFilter,inst=true,async=false->jaxrs@603a422==org.glassfish.jersey.servlet.ServletContainer,jsp=null,order=1,inst=true,async=false
> 2021-02-22 20:55:54,450 [qtp1637061418-81] DEBUG servlet.ServletHandler: call
> filter
> NoCacheFilter@5e600dd5==org.apache.hadoop.hdds.server.http.NoCacheFilter,inst=true,async
> =true
> 2021-02-22 20:55:54,450 [qtp1637061418-81] DEBUG servlet.ServletHandler: call
> filter
> safety@63a12c68==org.apache.hadoop.hdds.server.http.HttpServer2$QuotingInputFilter,inst=
> true,async=true
> 2021-02-22 20:55:54,450 [qtp1637061418-81] DEBUG servlet.ServletHandler: call
> filter
> info-page-redirect@576d5deb==org.apache.hadoop.ozone.s3.RootPageDisplayFilter,inst=true,
> async=false
> 2021-02-22 20:55:54,450 [qtp1637061418-81] DEBUG servlet.ServletHandler: call
> servlet
> jaxrs@603a422==org.glassfish.jersey.servlet.ServletContainer,jsp=null,order=1,inst=true
> ,async=false
> 2021-02-22 20:55:54,451 [qtp1637061418-81] DEBUG server.HttpChannelState:
> sendError HttpChannelState@4893b376{s=HANDLING rs=BLOCKING os=OPEN is=IDLE
> awp=false se=false i=tru
> e al=0}
> 2021-02-22 20:55:54,451 [qtp1637061418-81] DEBUG server.session: Leaving
> scope
> org.eclipse.jetty.server.session.SessionHandler367746789==dftMaxIdleSec=-1
> dispatch=REQUEST, a
> sync=false, session=null, oldsession=null, oldsessionhandler=null
> 2021-02-22 20:55:54,451 [qtp1637061418-81] DEBUG server.Server: handled=true
> async=false committed=true on
> HttpChannelOverHttp@769bb34b{s=HttpChannelState@4893b376{s=HANDLIN
> G rs=BLOCKING os=OPEN is=IDLE awp=false se=true i=true
> al=0},r=1,c=false/false,a=HANDLING,uri=https://ozone.example.com:9879/sandbox?list-type=2&ma
> x-keys=1,age=2}
> 2021-02-22 20:55:54,451 [qtp1637061418-81] DEBUG server.HttpChannelState:
> unhandle HttpChannelState@4893b376{s=HANDLING rs=BLOCKING os=OPEN is=IDLE
> awp=false se=true i=true
> al=0}
> 2021-02-22 20:55:54,451 [qtp1637061418-81] DEBUG server.HttpChannelState:
> nextAction(false) SEND_ERROR HttpChannelState@4893b376{s=HANDLING rs=BLOCKING
> os=OPEN is=IDLE awp=f
> alse se=false i=false al=0}
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
