[
https://issues.apache.org/jira/browse/HDFS-17944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18100640#comment-18100640
]
ASF GitHub Bot commented on HDFS-17944:
---------------------------------------
johntomcat7408-cmyk opened a new pull request, #8644:
URL: https://github.com/apache/hadoop/pull/8644
### Description of PR
JIRA: [HDFS-17944](https://issues.apache.org/jira/browse/HDFS-17944)
`dfs.client.read.shortcircuit.streams.cache.size=-1` currently passes
`DfsClientConf` parsing and is rejected only when a new
`ShortCircuitCache` is built. This exposes the internal `maxTotalSize`
parameter in the error, while reuse of an existing `ClientContext` can
avoid that late validation entirely.
This PR:
- validates that the configured streams cache size is non-negative while
building `DfsClientConf`;
- reports the public configuration key in the exception;
- adds client-module tests for the invalid `-1` boundary and the valid `0`
boundary.
Zero remains valid and keeps its existing cache-retention semantics. The
change does not modify public APIs, protocols, dependencies,
`hdfs-default.xml`, or the defensive validation in `ShortCircuitCache`.
Source and attribution: the issue comes from HDFS-17944. The implementation
was independently written against the current `trunk`. Its error style is
consistent with merged [HDFS-17943 / PR
#8603](https://github.com/apache/hadoop/pull/8603),
and the existing zero boundary was confirmed from merged
[HDFS-16653 / PR #5568](https://github.com/apache/hadoop/pull/5568).
Open [PR #7215](https://github.com/apache/hadoop/pull/7215) also touches
`DfsClientConf.java`, but only for lease renewal configuration; no code or
text from that unmerged PR was reused.
### How was this patch tested?
On Windows x86_64 with JDK 21.0.11 and the repository Maven Wrapper:
```powershell
.\mvnw.cmd --batch-mode --no-transfer-progress `
-pl :hadoop-hdfs-client -am "-P=-native-win" `
"-Dtest=TestDfsClientConf" `
"-Dsurefire.failIfNoSpecifiedTests=false" `
"-DskipShade" "-DskipDocs" test
```
Result: `BUILD SUCCESS`; 10 reactor modules succeeded; 2 tests ran with
0 failures, 0 errors, and 0 skips.
```powershell
.\mvnw.cmd --batch-mode --no-transfer-progress `
-pl :hadoop-hdfs -am "-P=-native-win" `
"-Dtest=TestShortCircuitCache#testInvalidConfiguration" `
"-Dsurefire.failIfNoSpecifiedTests=false" `
"-DskipShade" "-DskipDocs" test
```
Result: `BUILD SUCCESS`; 12 reactor modules succeeded; 1 test ran with
0 failures, 0 errors, and 0 skips. Bash tests were skipped because `bats`
was not installed; the targeted Java test completed successfully.
```powershell
.\mvnw.cmd --batch-mode --no-transfer-progress `
-pl :hadoop-hdfs-client -am "-P=-native-win" `
"-DskipTests" "-DskipShade" "-DskipDocs" checkstyle:check
```
Result: exit code 0. The new test has no Checkstyle violations.
`DfsClientConf.java` retains a pre-existing `MethodLength` warning at line
171; the production change is at line 844 and introduces no new violation.
A full repository build was not run.
### For code changes:
- [x] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [x] Object storage: N/A; this HDFS client change does not affect an object
storage connector.
- [x] New dependencies: N/A; this PR adds no dependencies.
- [x] License and notice updates: N/A; this PR does not change distributed
dependencies or licensing content.
### AI Tooling
- [x] Contains content generated by Codex
- [x] My use of AI contributions follows the [ASF legal
policy](https://www.apache.org/legal/generative-tooling.html).
> Negative dfs.client.read.shortcircuit.streams.cache.size reports internal
> maxTotalSize error
> ---------------------------------------------------------------------------------------------
>
> Key: HDFS-17944
> URL: https://issues.apache.org/jira/browse/HDFS-17944
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
> Affects Versions: 3.4.3
> Reporter: jiang he
> Priority: Major
>
> When dfs.client.read.shortcircuit.streams.cache.size is set to a negative
> value, HDFS client reports an internal variable name instead of the
> user-facing configuration key.
> Reproduction:
> Use the following HDFS client-related configuration:
> <property>
> <name>dfs.client.read.shortcircuit</name>
> <value>true</value>
> </property>
> <property>
> <name>dfs.client.read.shortcircuit.streams.cache.size</name>
> <value>-1</value>
> </property>
> <property>
> <name>dfs.domain.socket.path</name>
> <value></value>
> </property>
> Run a simple HDFS client operation.
> Actual result:
> The client fails with:
> java.lang.IllegalArgumentException: maxTotalSize must be greater than zero.
> Expected result:
> The exception should mention the actual configuration key, for example:
> dfs.client.read.shortcircuit.streams.cache.size must be greater than zero.
> Reproduced locally:
> - Negative value dfs.client.read.shortcircuit.streams.cache.size=-1 failed.
> - Normal value dfs.client.read.shortcircuit.streams.cache.size=256 passed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
