[
https://issues.apache.org/jira/browse/HDDS-5271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17754920#comment-17754920
]
Tejaskriya Madhan commented on HDDS-5271:
-----------------------------------------
I think this is an outdated issue, I tried reproducing the various cases, all
the keys were created successfully and I did not come across any errors. These
are the commands I executed:
{code:java}
aws s3api put-object --endpoint http://localhost:9878 --bucket buck2 --key y
--body temp1
aws s3api put-object --endpoint http://localhost:9878 --bucket buck2 --key
yplain --body temp1 --content-type text/plain
aws s3api put-object --endpoint http://localhost:9878 --bucket buck2 --key
yplainUTF --body temp1 --content-type "text/plain; charset=UTF-8"
aws s3api put-object --endpoint http://localhost:9878 --bucket buck2 --key
yplainutf --body temp1 --content-type "text/plain; charset=utf-8" {code}
The resulting keys created:
{code:java}
sh-4.2$ ozone sh key list s3v/buck2
[ {
"volumeName" : "s3v",
"bucketName" : "buck2",
"name" : "y",
"dataSize" : 3,
"creationTime" : "2023-08-16T07:47:03.408Z",
"modificationTime" : "2023-08-16T07:47:03.457Z",
"replicationConfig" : {
"replicationFactor" : "ONE",
"requiredNodes" : 1,
"replicationType" : "RATIS"
},
"metadata" : { },
"file" : true
}, {
"volumeName" : "s3v",
"bucketName" : "buck2",
"name" : "yplain",
"dataSize" : 3,
"creationTime" : "2023-08-16T07:50:26.857Z",
"modificationTime" : "2023-08-16T07:50:26.895Z",
"replicationConfig" : {
"replicationFactor" : "ONE",
"requiredNodes" : 1,
"replicationType" : "RATIS"
},
"metadata" : { },
"file" : true
}, {
"volumeName" : "s3v",
"bucketName" : "buck2",
"name" : "yplainUTF",
"dataSize" : 3,
"creationTime" : "2023-08-16T07:51:10.710Z",
"modificationTime" : "2023-08-16T07:51:10.752Z",
"replicationConfig" : {
"replicationFactor" : "ONE",
"requiredNodes" : 1,
"replicationType" : "RATIS"
},
"metadata" : { },
"file" : true
}, {
"volumeName" : "s3v",
"bucketName" : "buck2",
"name" : "yplainutf",
"dataSize" : 3,
"creationTime" : "2023-08-16T07:51:25.240Z",
"modificationTime" : "2023-08-16T07:51:25.281Z",
"replicationConfig" : {
"replicationFactor" : "ONE",
"requiredNodes" : 1,
"replicationType" : "RATIS"
},
"metadata" : { },
"file" : true
} ] {code}
> S3G: Upload with Content-Type: plain/text; charset=utf-8 header will fail
> -------------------------------------------------------------------------
>
> Key: HDDS-5271
> URL: https://issues.apache.org/jira/browse/HDDS-5271
> Project: Apache Ozone
> Issue Type: Bug
> Components: Ozone Datanode, S3
> Affects Versions: 1.1.0
> Reporter: Kohei Sugihara
> Assignee: Tejaskriya Madhan
> Priority: Major
> Attachments: s3g.log, upload.go
>
>
> Upload content via S3 gateway with "Content-Type: plain/text; charset=utf-8"
> header will fail with 500 code. At the same time, "Content-Type: plain/text"
> (without charset) and "Content-Type: plain/text; charset=UTF-8" (in
> uppercase) were accepted.
> We're running the official Ozone v1.1.0 release.
> These lines shown as below are known behavior:
> * 'text/plain' ==> OK
> * 'text/plain; charset=UTF-8' ==> OK
> * 'text/plain; charset=utf-8' ==> 500
> * 'text/html; charset=UTF-8' ==> OK (same as text/plain case)
> * 'text/html; charset=utf-8' ==> 500 (same as text/plain case)
> * 'text/markdown; charset=UTF-8' ==> OK
> * 'text/markdown; charset=utf-8' ==> OK
> I've attached a code snippet (upload.go) and a log from S3 gateway (s3g.log).
> This code can run easily with following command lines.
> {code:none}
> % go run upload.go --contenttype='text/plain; charset=utf-8'
> text/plain; charset=utf-8
> InternalServerError: Internal Server Error
> status code: 500, request id: , host id: exit status 1
> % go run upload.go --contenttype='text/plain; charset=UTF-8'
> text/plain; charset=UTF-8
> &\{http://example.com/<BUCKET>/<KEY> <nil> <nil>}%
> {code}
> HTTP request from upload.go is here:
> {code:none}
> PUT /<BUCKET>/<KEY> HTTP/1.1
> Host: localhost:9999
> User-Agent: aws-sdk-go/1.38.46 (go1.16.4; linux; amd64) S3Manager
> Content-Length: 4
> Authorization: AWS4-HMAC-SHA256
> [email protected]/20210525/dummy/s3/aws4_request,
> SignedHeaders=content-length;content-md5;content-type;host;x-amz-content-sha256;x-amz-date,
> Signature=<HASH>
> Content-Md5: AzvZSxFo1+Tw1kTDyV41vw==
> Content-Type: text/plain; charset=utf-8
> X-Amz-Content-Sha256: <HASH>
> X-Amz-Date: 20210525T080147Z
> Accept-Encoding: gzip
> TEST^C
> {code}
> h2. Investigation
> We thought the issue is similar to HDDS-4856 but this issue will happen when
> empty Content-Type header was given. Our facing problem will appear when
> Content-Type defined together with the charset directive.
> The stack trace of S3 gateway indicates the request proceeded to remote
> invocation and failed with authentication error with
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken).
> There are many parts to preprocess a request before reaching RPC Client, so
> I suspect some Java servlet filters like ozone.s3.HeaderPreprocessor and
> hdds.server.http.NoCacheFilter. But as far as I found out, these are not a
> cause of this problem directly (because these are almost for generating
> response header). I also suspected some components like OzoneClientProducer,
> Signature Processor and HeaderPreprocessor. I think this is not direct cause.
> How do I investigate this problem in more detail?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
