[
https://issues.apache.org/jira/browse/HADOOP-17851?focusedWorklogId=639690&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-639690
]
ASF GitHub Bot logged work on HADOOP-17851:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 19/Aug/21 03:47
Start Date: 19/Aug/21 03:47
Worklog Time Spent: 10m
Work Description: holdenk opened a new pull request #3312:
URL: https://github.com/apache/hadoop/pull/3312
### Description of PR
Add support for user specified content encoding for S3A.
### How was this patch tested?
Run existing unit tests, new integration test, and run S3A integration tests
in us-east-2. Added some small changes to S3 integration tests to skip
encryption tests when key is null or alg is null (I believe this matches the
original intention given the use of `StringUtils.isBlank` for skipping the
test).
### For code changes:
- [ X ] Does the title or this PR starts with the corresponding JIRA issue
id (e.g. 'HADOOP-17799. Your PR title ...')?
- [ X ] Object storage: have the integration tests been executed and the
endpoint declared according to the connector-specific documentation?:
(us-east-2 ran with `holden@a61a514145cb:~/hadoop/hadoop-tools/hadoop-aws$
mvn clean verify -Dnon-auth -Ds3guard` inside of the docker dev instance)
Notable failures, which existed without the change are
`org.apache.hadoop.fs.s3a.ITestS3AMiscOperations` (namely ` checksums of empty
files expected:<etag: "5e3037f6b23642c732f1b705db9dcf17"> but was:<etag:
"e5321d4514ae69ccf36995e8e1ac846f">`) . I can dig into this more if this isn't
expected/flaky.
- [ N/A ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ N/A ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 639690)
Remaining Estimate: 0h
Time Spent: 10m
> Support user specified content encoding for S3A
> -----------------------------------------------
>
> Key: HADOOP-17851
> URL: https://issues.apache.org/jira/browse/HADOOP-17851
> Project: Hadoop Common
> Issue Type: Improvement
> Components: hadoop-thirdparty
> Reporter: Holden Karau
> Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> User-specified object content-encoding (part of the object metadata) is
> important for allowing compressed files to be processed in the AWS ecosystem.
> We should allow the user to specify the content encoding of the files being
> written.
> metadata can not be changed after a file is written without a rewrite.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
