[
https://issues.apache.org/jira/browse/HADOOP-19352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17902899#comment-17902899
]
ASF GitHub Bot commented on HADOOP-19352:
-----------------------------------------
zhouaoe opened a new pull request, #7205:
URL: https://github.com/apache/hadoop/pull/7205
### Description of PR
AliyunOSS is about to adjust its security policy: only V4 signature requests
will be supported in the public cloud. Therefore, support for V4 signatures is
also required in Hadoop, and V4 signatures will be the default.
Change 1: Upgraded the OSS SDK version to 3.18.1 to support V4
authentication, while also fixing some bugs inherent in the SDK.
Change 2: Added configuration options to enable V4 authentication support in
the hadoop-oss-connector.
### How was this patch tested?
Integrated testing was conducted to test V4 authentication. A new test,
ITAliyunOSSSignatureV4, is added, which verifies the authentication method
using V4. Additionally, it confirms that V1 authentication is still used by
default.
```shell
mvn test -Dtest=org.apache.hadoop.fs.aliyun.oss.ITAliyunOSSSignatureV4 -pl
hadoop-tools/hadoop-aliyun
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.fs.aliyun.oss.ITAliyunOSSSignatureV4
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.934
s - in org.apache.hadoop.fs.aliyun.oss.ITAliyunOSSSignatureV4
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 4.655 s
[INFO] Finished at: 2024-12-03T09:01:00Z
[INFO]
------------------------------------------------------------------------
```
Other Aliyun oss integration tests are also executed.
```shell
mvn test -pl hadoop-tools/hadoop-aliyun
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.fs.aliyun.oss.TestAliyunOSSBlockOutputStream
[INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
37.258 s - in org.apache.hadoop.fs.aliyun.oss.TestAliyunOSSBlockOutputStream
[INFO] Running org.apache.hadoop.fs.aliyun.oss.yarn.TestOSS
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.808
s - in org.apache.hadoop.fs.aliyun.oss.yarn.TestOSS
[INFO] Running org.apache.hadoop.fs.aliyun.oss.yarn.TestOSSMiniYarnCluster
[WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
0.128 s - in org.apache.hadoop.fs.aliyun.oss.yarn.TestOSSMiniYarnCluster
[INFO] Running
org.apache.hadoop.fs.aliyun.oss.TestAliyunOSSFileSystemContract
[INFO] Tests run: 53, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
22.578 s - in org.apache.hadoop.fs.aliyun.oss.TestAliyunOSSFileSystemContract
[INFO] Running org.apache.hadoop.fs.aliyun.oss.TestAliyunOSSFileSystemStore
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
23.816 s - in org.apache.hadoop.fs.aliyun.oss.TestAliyunOSSFileSystemStore
[INFO] Running org.apache.hadoop.fs.aliyun.oss.fileContext.TestOSSFileContext
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.262
s - in org.apache.hadoop.fs.aliyun.oss.fileContext.TestOSSFileContext
[INFO] Running
org.apache.hadoop.fs.aliyun.oss.fileContext.TestOSSFileContextCreateMkdir
[INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
2.784 s - in
org.apache.hadoop.fs.aliyun.oss.fileContext.TestOSSFileContextCreateMkdir
[INFO] Running
org.apache.hadoop.fs.aliyun.oss.fileContext.TestOSSFileContextMainOperations
[WARNING] Tests run: 73, Failures: 0, Errors: 0, Skipped: 4, Time elapsed:
16.198 s - in
org.apache.hadoop.fs.aliyun.oss.fileContext.TestOSSFileContextMainOperations
[INFO] Running
org.apache.hadoop.fs.aliyun.oss.fileContext.TestOSSFileContextStatistics
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.682
s - in org.apache.hadoop.fs.aliyun.oss.fileContext.TestOSSFileContextStatistics
[INFO] Running
org.apache.hadoop.fs.aliyun.oss.fileContext.TestOSSFileContextUtil
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.23
s - in org.apache.hadoop.fs.aliyun.oss.fileContext.TestOSSFileContextUtil
[INFO] Running
org.apache.hadoop.fs.aliyun.oss.fileContext.TestOSSFileContextURI
[WARNING] Tests run: 17, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
6.146 s - in org.apache.hadoop.fs.aliyun.oss.fileContext.TestOSSFileContextURI
[INFO] Running
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractDelete
[WARNING] Tests run: 8, Failures: 0, Errors: 0, Skipped: 8, Time elapsed:
0.158 s - in
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractDelete
[INFO] Running
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractOpen
[WARNING] Tests run: 19, Failures: 0, Errors: 0, Skipped: 19, Time elapsed:
0.201 s - in org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractOpen
[INFO] Running
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractSeek
[WARNING] Tests run: 19, Failures: 0, Errors: 0, Skipped: 19, Time elapsed:
0.204 s - in org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractSeek
[INFO] Running
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractGetFileStatus
[WARNING] Tests run: 20, Failures: 0, Errors: 0, Skipped: 20, Time elapsed:
0.197 s - in
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractGetFileStatus
[INFO] Running
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractDistCp
[WARNING] Tests run: 13, Failures: 0, Errors: 0, Skipped: 13, Time elapsed:
0.179 s - in
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractDistCp
[INFO] Running
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractMkdir
[WARNING] Tests run: 8, Failures: 0, Errors: 0, Skipped: 8, Time elapsed:
0.164 s - in org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractMkdir
[INFO] Running
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractRootDir
[WARNING] Tests run: 9, Failures: 0, Errors: 0, Skipped: 9, Time elapsed:
0.164 s - in
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractRootDir
[INFO] Running
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractRename
[WARNING] Tests run: 10, Failures: 0, Errors: 0, Skipped: 10, Time elapsed:
0.167 s - in
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractRename
[INFO] Running
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractCreate
[WARNING] Tests run: 16, Failures: 0, Errors: 0, Skipped: 16, Time elapsed:
0.182 s - in
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractCreate
[INFO] Running
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractGetFileStatusV1List
[WARNING] Tests run: 20, Failures: 0, Errors: 0, Skipped: 20, Time elapsed:
0.199 s - in
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractGetFileStatusV1List
[INFO] Running org.apache.hadoop.fs.aliyun.oss.TestAliyunOSSInputStream
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.352
s - in org.apache.hadoop.fs.aliyun.oss.TestAliyunOSSInputStream
[INFO] Running org.apache.hadoop.fs.aliyun.oss.TestAliyunCredentials
[WARNING] Tests run: 2, Failures: 0, Errors: 0, Skipped: 2, Time elapsed:
0.131 s - in org.apache.hadoop.fs.aliyun.oss.TestAliyunCredentials
[INFO]
[INFO] Results:
[INFO]
[WARNING] Tests run: 328, Failures: 0, Errors: 0, Skipped: 150
[INFO]
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 02:12 min
[INFO] Finished at: 2024-12-03T08:15:52Z
[INFO]
------------------------------------------------------------------------
```
### For code changes:
- [x] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [x] Object storage: have the integration tests been executed and the
endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
> Hadoop OSS Connector adds support for V4 signatures.
> ----------------------------------------------------
>
> Key: HADOOP-19352
> URL: https://issues.apache.org/jira/browse/HADOOP-19352
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs/oss
> Reporter: zhouao
> Priority: Major
> Labels: pull-request-available
>
> AliyunOSS is about to adjust its security policy: only V4 signature requests
> will be supported in the public cloud. Therefore, support for V4 signatures
> is also required in Hadoop, and V4 signatures will be the default.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
