[
https://issues.apache.org/jira/browse/HDFS-17946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18094951#comment-18094951
]
ASF GitHub Bot commented on HDFS-17946:
---------------------------------------
LiuZheng-Z opened a new pull request, #8593:
URL: https://github.com/apache/hadoop/pull/8593
### Description of PR
This PR fixes [HDFS-17946](https://issues.apache.org/jira/browse/HDFS-17946).
When several HDFS client integer configuration values are set to out-of-range
values (e.g. `-4294967296`, which exceeds the `int` range of
`-2147483648 ~ 2147483647`), the underlying `Configuration` class throws a
raw
`NumberFormatException` without identifying the offending config key:
```
Failed to initialize filesystem hdfs://127.0.0.1:9000:
java.lang.NumberFormatException: For input string: "-4294967296"
```
This makes it hard for users to figure out which property is misconfigured.
The fix wraps the `Integer.parseInt` / `Long.parseLong` / `Float.parseFloat`
calls in `Configuration.getInt`, `Configuration.getInts`,
`Configuration.getLong` and `Configuration.getFloat` with a `try`/`catch` so
that the thrown `NumberFormatException` includes the config key name and the
invalid value, e.g.:
```
NumberFormatException: Failed to parse value of
dfs.client.block.write.retries,
got "-4294967296", expect a valid integer.
```
### Affected configs (from HDFS-17946)
* `dfs.client.block.write.retries`
* `dfs.client.pipeline.recovery.max-retries`
* `dfs.client.retry.max.attempts`
* `dfs.client.congestion.backoff.mean.time`
### Changes
*
`hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java`
* `getInt(String, int)` – wrap parsing in `try`/`catch`
* `getInts(String)` – wrap parsing in `try`/`catch` (includes array index)
* `getLong(String, long)` – wrap parsing in `try`/`catch`
* `getFloat(String, float)` – wrap parsing in `try`/`catch`
*
`hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java`
* Added `testGetIntOutOfRange`, `testGetIntsOutOfRange`,
`testGetLongOutOfRange`, `testGetFloatOutOfRange` covering the new
behavior and the exact value `-4294967296` reported in the JIRA.
### How was this patch tested?
* Added new JUnit test cases in `TestConfiguration.java` that:
* set the four affected configuration keys to out-of-range / invalid values
* call `getInt` / `getInts` / `getLong` / `getFloat`
* assert that a `NumberFormatException` is thrown
* assert that the exception message contains the config key and the
invalid value
* Existing `TestConfiguration` tests should continue to pass since the
change only affects the error message and only when parsing actually
fails – the happy path (return parsed value / return `defaultValue` when
unset) is unchanged.
* Note: I was unable to run `mvn test` locally because the trunk build
requires Maven >= 3.9.15 and JDK >= 17 (see enforcer rules in
`hadoop-project/pom.xml`). The CI pre-commit build on the PR is expected
to run the full test suite. Happy to address any test failures found by
Jenkins.
> Several HDFS client integer configs report raw NumberFormatException for
> out-of-range values
> --------------------------------------------------------------------------------------------
>
> Key: HDFS-17946
> URL: https://issues.apache.org/jira/browse/HDFS-17946
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
> Reporter: jiang he
> Priority: Trivial
>
> Several HDFS client integer configuration keys fail with a raw
> NumberFormatException when set to an out-of-range negative value.
> Instead of exposing a raw NumberFormatException, HDFS should report a clear
> configuration validation error that identifies the invalid key and expected
> range.
> Examples
> Example 1
> <property>
> <name>dfs.client.block.write.retries</name>
> <value>-4294967296</value>
> </property>
> Observed:
> Failed to initialize filesystem hdfs://127.0.0.1:9000:
> java.lang.NumberFormatException: For input string: "-4294967296"
> Example 2
> <property>
> <name>dfs.client.pipeline.recovery.max-retries</name>
> <value>-4294967296</value>
> </property>
> Observed:
> Failed to initialize filesystem hdfs://127.0.0.1:9000:
> java.lang.NumberFormatException: For input string: "-4294967296"
> Example 3
> <property>
> <name>dfs.client.retry.max.attempts</name>
> <value>-4294967296</value>
> </property>
> Observed:
> Failed to initialize filesystem hdfs://127.0.0.1:9000:
> java.lang.NumberFormatException: For input string: "-4294967296"
> Example 4
> <property>
> <name>dfs.client.congestion.backoff.mean.time</name>
> <value>-4294967296</value>
> </property>
> Observed:
> java.lang.NumberFormatException: For input string: "-4294967296"
> Expected
> HDFS should report clear configuration validation errors, for example:
> Invalid value for dfs.client.block.write.retries: -4294967296.
> The value must be a valid integer within range ...
> rather than a raw NumberFormatException.
> Control test
> The following normal values pass successfully:
> <property>
> <name>dfs.client.block.write.retries</name>
> <value>3</value>
> </property>
> <property>
> <name>dfs.client.pipeline.recovery.max-retries</name>
> <value>5</value>
> </property>
> <property>
> <name>dfs.client.retry.max.attempts</name>
> <value>10</value>
> </property>
> <property>
> <name>dfs.client.congestion.backoff.mean.time</name>
> <value>5000</value>
> </property>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
