tomicooler commented on code in PR #5895:
URL: https://github.com/apache/hadoop/pull/5895#discussion_r1276203099
##########
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/IFile.java:
##########
@@ -433,8 +434,11 @@ public boolean nextRawKey(DataInputBuffer key) throws
IOException {
}
public void nextRawValue(DataInputBuffer value) throws IOException {
+ long targetSizeLong = currentValueLength + (currentValueLength >> 1);
Review Comment:
This overflows like this. Example:
```
class HelloWorld {
public static void main(String[] args) {
int currentValueLength = Integer.MAX_VALUE - 20000;
final int ARRAY_MAX_SIZE = Integer.MAX_VALUE - 8;
for (int i = 0; i < 10; i++) {
long targetSizeLong = currentValueLength + (currentValueLength
>> 1);
int targetSize = (int) Math.min(targetSizeLong, ARRAY_MAX_SIZE);
System.out.println("targetSizeLong: " + targetSizeLong + "
targetSize: " + targetSize);
currentValueLength = targetSize;
}
}
}
```
```
targetSizeLong: -1073771826 targetSize: -1073771826
targetSizeLong: -1610657739 targetSize: -1610657739
targetSizeLong: 1878980687 targetSize: 1878980687
targetSizeLong: -1476496266 targetSize: -1476496266
targetSizeLong: 2080222897 targetSize: 2080222897
targetSizeLong: -1174632951 targetSize: -1174632951
targetSizeLong: -1761949427 targetSize: -1761949427
targetSizeLong: 1652043155 targetSize: 1652043155
targetSizeLong: -1816902564 targetSize: -1816902564
targetSizeLong: 1569613450 targetSize: 1569613450
```
Fix:
```
long targetSizeLong = currentValueLength + (long)(currentValueLength >> 1);
```
With the fix (casting the second part to `long`):
```
targetSizeLong: 3221195470 targetSize: 2147483639
targetSizeLong: 3221225458 targetSize: 2147483639
targetSizeLong: 3221225458 targetSize: 2147483639
targetSizeLong: 3221225458 targetSize: 2147483639
targetSizeLong: 3221225458 targetSize: 2147483639
targetSizeLong: 3221225458 targetSize: 2147483639
targetSizeLong: 3221225458 targetSize: 2147483639
targetSizeLong: 3221225458 targetSize: 2147483639
targetSizeLong: 3221225458 targetSize: 2147483639
targetSizeLong: 3221225458 targetSize: 2147483639
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
