[
https://issues.apache.org/jira/browse/HADOOP-18896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18098136#comment-18098136
]
ASF GitHub Bot commented on HADOOP-18896:
-----------------------------------------
anshuksi282-ksolves commented on code in PR #8622:
URL: https://github.com/apache/hadoop/pull/8622#discussion_r3629706509
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSOutputSummer.java:
##########
@@ -52,6 +53,9 @@ abstract public class FSOutputSummer extends OutputStream
implements
protected FSOutputSummer(DataChecksum sum) {
this.sum = sum;
+ Preconditions.checkArgument(
+ sum.getBytesPerChecksum() * BUFFER_NUM_CHUNKS > 0,
Review Comment:
Thanks for the review @pjfanning!
Updated to use `Math.multiplyExact` for both `buf` and `checksum` buffer
size checks, as suggested.
I did try adding a dedicated test for the `checksum` buffer overflow path,
but `getChecksumSize()` returns a fixed value from `DataChecksum.Type` (e.g. 4
for CRC32), so it can never actually be large enough to overflow — that path
isn't reachable in practice, just defensive. So I've kept the existing unit
test covering the `buf` overflow case, and re-ran `TestFSOutputSummer`,
`TestChecksumFileSystem`, and `TestChecksumFs` to confirm no regressions.
> NegativeArraySizeException thrown in FSOutputSummer.java given large
> file.bytes-per-checksum
> --------------------------------------------------------------------------------------------
>
> Key: HADOOP-18896
> URL: https://issues.apache.org/jira/browse/HADOOP-18896
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 3.3.6
> Reporter: rstest
> Priority: Critical
> Labels: pull-request-available
>
> Buffer size of FSOutputSummer equals to `file.bytes-per-checksum` times
> `BUFFER_NUM_CHUNKS`. A large `file.bytes-per-checksum` causes buffer size to
> overflow and crash with NegativeArraySizeException.
> To reproduce:
> 1. set `file.bytes-per-checksum` to 238609295
> 2. `mvn surefire:test
> -Dtest=org.apache.hadoop.hdfs.TestDecommissionWithStriped#testFileSmallerThanOneStripe`
> We created a PR that provides a fix which checks the buffer size is positive
> after multiplying `file.bytes-per-checksum` with `BUFFER_NUM_CHUNKS`
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
