UENISHI Kota created HDDS-4856:
----------------------------------
Summary: Ruby S3 SDK never get authenticated by Ozone
Key: HDDS-4856
URL: https://issues.apache.org/jira/browse/HDDS-4856
Project: Apache Ozone
Issue Type: Bug
Components: S3
Affects Versions: 1.0.0
Environment: Secure setup of Ozone 1.0.0
Reporter: UENISHI Kota
Attachments: ozone-test.py, ozone-test.rb, ruby-sdk-patch.diff
When the very first call by Ruby client against secure setup of Ozone, the
server returns 400 no matter how valid the request is. See the attached
ruby-sdk-patch.diff, which adds some tests on S3 auth header signature-to-sign
generation. It consists of two test additions, the "2" is the one generated by
boto3, the "3" is generated by aws-ruby-sdk. Both passes the additional tests,
which are definitely valid.
However, when real HTTP request is sent by Ruby client, e.g. ozone-test.rb
attached, it fails with 400. The header was like this (though the host names
and domains are masked):
{quote}GET //ozone.example.com:9879/sandbox?list-type=2&max-keys=1 HTTP/1.1
Content-Type:
Accept-Encoding:
User-Agent: aws-sdk-ruby3/3.112.0 ruby/2.7.2 x86_64-linux aws-sdk-s3/1.88.1
Host: ozone.example.com:9879
X-Amz-Date: 20210222T110554Z
X-Amz-Content-Sha256:
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Authorization: AWS4-HMAC-SHA256
[email protected]/20210222/foobar/s3/aws4_request,
SignedHeaders=host;user-agent;x-amz-content-sha256;x-amz-date,
Signature=0c9469f018f5
b3fd2cff6f8d4e4963f50aa71c6704def59527634404f5fc98a9
Content-Length: 0
Accept: */*{quote}
On the other hand, request headers made by boto3 was:
{quote}GET
//ozone.example.com:9879/storageadmin.sandbox.pfn.io?list-type=2&encoding-type=url
HTTP/1.1
Host: ozone.example.com:9879
Accept-Encoding: identity
User-Agent: Boto3/1.17.12 Python/3.9.1 Linux/5.10.14-arch1-1 Botocore/1.20.12
X-Amz-Date: 20210222T110829Z
X-Amz-Content-SHA256:
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Authorization: AWS4-HMAC-SHA256
[email protected]/20210222/us-east-1/s3/aws4_request,
SignedHeaders=host;x-amz-content-sha256;x-amz-date,
Signature=94302f21cccac8832d3e
4fe25c5f6d8a0307188fb0e1b1983264339381d21dac{quote}
The difference of these requests are IMHO, "Content-Type" and "Accept-Encoding"
are both empty in Ruby SDK. I'm afraid this error stems from partly Ruby SDK
and partly from
[https://github.com/eclipse/jetty.project/issues/2883|http://example.com]. The
former sends empty header lines and the latter rejects them.
And the s3g debug log (only error'ish part) follows:
{quote}2021-02-22 20:55:54,450 [qtp1637061418-81] DEBUG servlet.ServletHandler:
chain=NoCacheFilter@5e600dd5==org.apache.hadoop.hdds.server.http.NoCacheFilter,inst=true,async=true-
>safety@63a12c68==org.apache.hadoop.hdds.server.http.HttpServer2$QuotingInputFilter,inst=true,async=true->info-page-redirect@576d5deb==org.apache.hadoop.ozone.s3.RootPageDis
playFilter,inst=true,async=false->jaxrs@603a422==org.glassfish.jersey.servlet.ServletContainer,jsp=null,order=1,inst=true,async=false
2021-02-22 20:55:54,450 [qtp1637061418-81] DEBUG servlet.ServletHandler: call
filter
NoCacheFilter@5e600dd5==org.apache.hadoop.hdds.server.http.NoCacheFilter,inst=true,async
=true
2021-02-22 20:55:54,450 [qtp1637061418-81] DEBUG servlet.ServletHandler: call
filter
safety@63a12c68==org.apache.hadoop.hdds.server.http.HttpServer2$QuotingInputFilter,inst=
true,async=true
2021-02-22 20:55:54,450 [qtp1637061418-81] DEBUG servlet.ServletHandler: call
filter
info-page-redirect@576d5deb==org.apache.hadoop.ozone.s3.RootPageDisplayFilter,inst=true,
async=false
2021-02-22 20:55:54,450 [qtp1637061418-81] DEBUG servlet.ServletHandler: call
servlet
jaxrs@603a422==org.glassfish.jersey.servlet.ServletContainer,jsp=null,order=1,inst=true
,async=false
2021-02-22 20:55:54,451 [qtp1637061418-81] DEBUG server.HttpChannelState:
sendError HttpChannelState@4893b376{s=HANDLING rs=BLOCKING os=OPEN is=IDLE
awp=false se=false i=tru
e al=0}
2021-02-22 20:55:54,451 [qtp1637061418-81] DEBUG server.session: Leaving scope
org.eclipse.jetty.server.session.SessionHandler367746789==dftMaxIdleSec=-1
dispatch=REQUEST, a
sync=false, session=null, oldsession=null, oldsessionhandler=null
2021-02-22 20:55:54,451 [qtp1637061418-81] DEBUG server.Server: handled=true
async=false committed=true on
HttpChannelOverHttp@769bb34b{s=HttpChannelState@4893b376{s=HANDLIN
G rs=BLOCKING os=OPEN is=IDLE awp=false se=true i=true
al=0},r=1,c=false/false,a=HANDLING,uri=https://ozone.example.com:9879/sandbox?list-type=2&ma
x-keys=1,age=2}
2021-02-22 20:55:54,451 [qtp1637061418-81] DEBUG server.HttpChannelState:
unhandle HttpChannelState@4893b376{s=HANDLING rs=BLOCKING os=OPEN is=IDLE
awp=false se=true i=true
al=0}
2021-02-22 20:55:54,451 [qtp1637061418-81] DEBUG server.HttpChannelState:
nextAction(false) SEND_ERROR HttpChannelState@4893b376{s=HANDLING rs=BLOCKING
os=OPEN is=IDLE awp=f
alse se=false i=false al=0}
{quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
