[
https://issues.apache.org/jira/browse/HADOOP-20021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18127362#comment-18127362
]
ASF GitHub Bot commented on HADOOP-20021:
-----------------------------------------
yangdong2398 opened a new pull request, #8823:
URL: https://github.com/apache/hadoop/pull/8823
### Description of PR
JIRA: https://issues.apache.org/jira/browse/HADOOP-20021
Follow-up to HDFS-11161 (hadoop-bos), found while verifying the module with
the `hadoop fs` CLI and the contract suite on JDK 17.
- Upgrade `bce-java-sdk` 0.10.330 -> 0.10.332. 0.10.330 throws
`NullPointerException` on `hadoop fs -ls bos://bucket/` because the list
response has no `owner`; 0.10.332 guards it.
- Hierarchy (namespace) buckets:
- `rename`: follow the FileSystem contract when the destination is an
existing directory, and refuse to rename into its own descendant.
- `create`: throw `ParentNotDirectoryException` when an ancestor is a file.
- Map HTTP 400 `InvalidArgument` to `BosException` instead of
`SessionTokenExpireException`; the latter was retried by the retry policy and
caused ~150s timeouts.
- Packaging: stop bundling guava, httpclient, httpcore and commons-codec in
the shaded jar and use the ones shipped with Hadoop (jar ~14MB -> ~9.4MB).
`httpcore-nio` and `httpasyncclient` are still bundled: the SDK's
`BceHttpClient` needs them at class load time even with async put disabled, and
Hadoop does not ship them.
- Update `LICENSE-binary` (SDK version, `httpcore-nio`, `httpasyncclient`).
### How was this patch tested?
JDK 17, endpoint `http://bj.bcebos.com`, run against real BOS with
credentials from environment variables
(`EnvironmentVariableCredentialsProvider`):
- Contract suite on both a flat bucket and a hierarchy bucket: 140 tests, 0
failures, 0 errors, 31 skipped (append, concat, unbuffer, setTimes, rootdir,
and 2 in create).
- `hadoop fs` CLI on both buckets: `mkdir -p`, `put`, `cat`, `mv`, `ls`
(including bucket root), `rm -r`, `-checksum`.
- `mvn checkstyle:check` (0 violations), `javadoc:javadoc` and
`spotbugs:check` (0 bugs) on `hadoop-bos`.
Limitations: the CLI check used a Maven-resolved classpath, not a built
distribution tarball. Since guava/httpclient now come from Hadoop, hadoop-bos
needs `hadoop-common` (not only `hadoop-client-api`) on the classpath.
### For code changes:
- [x] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [x] Object storage: have the integration tests been executed and the
endpoint declared according to the connector-specific documentation?
- [x] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [x] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
### AI Tooling
- [x] The PR includes the phrase "Contains content generated by Claude Code"
where Claude Code is the name of the AI tool used.
- [x] My use of AI contributions follows the ASF legal policy
https://www.apache.org/legal/generative-tooling.html
Contains content generated by Claude Code.
> hadoop-bos: fix hierarchy bucket rename/create and slim down shaded jar
> -----------------------------------------------------------------------
>
> Key: HADOOP-20021
> URL: https://issues.apache.org/jira/browse/HADOOP-20021
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 3.6.0
> Reporter: yangdongdong
> Priority: Major
>
> SDK 0.10.330 throws a NPE when listing the root directory, with rename and
> create operations in hierarchy mode not conforming to the contract, and shade
> packaging includes Hadoop's existing guava and httpclient.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
