guiyanakuang commented on code in PR #1431:
URL: https://github.com/apache/orc/pull/1431#discussion_r1129152573
##########
java/core/src/java/org/apache/orc/impl/TreeReaderFactory.java:
##########
@@ -2292,10 +2292,12 @@ private void readDictionaryStream(InStream in) throws
IOException {
int dictionaryBufferSize =
dictionaryOffsets[dictionaryOffsets.length - 1];
dictionaryBuffer = new byte[dictionaryBufferSize];
int pos = 0;
- int chunkSize = in.available();
- byte[] chunkBytes = new byte[chunkSize];
+ //check if dictionary size is smaller than available stream size
+ // to avoid ArrayIndexOutOfBoundsException
+ int readSize = Math.min(in.available(), dictionaryBufferSize);
+ byte[] chunkBytes = new byte[readSize];
while (pos < dictionaryBufferSize) {
- int currentLength = in.read(chunkBytes, 0, chunkSize);
+ int currentLength = in.read(chunkBytes, 0, readSize);
System.arraycopy(chunkBytes, 0, dictionaryBuffer, pos,
currentLength);
pos += currentLength;
Review Comment:
Would this fix be better?
```suggestion
currentLength = Math.min(currentLength, dictionaryBufferSize -
pos);
System.arraycopy(chunkBytes, 0, dictionaryBuffer, pos,
currentLength);
pos += currentLength;
```
##########
java/core/src/java/org/apache/orc/impl/TreeReaderFactory.java:
##########
@@ -2292,10 +2292,11 @@ private void readDictionaryStream(InStream in) throws
IOException {
int dictionaryBufferSize =
dictionaryOffsets[dictionaryOffsets.length - 1];
dictionaryBuffer = new byte[dictionaryBufferSize];
int pos = 0;
- int chunkSize = in.available();
- byte[] chunkBytes = new byte[chunkSize];
+ //check if dictionary size is smaller than available stream size to
avoid ArrayIndexOutOfBoundsException
+ int readSize = Math.min(in.available(), dictionaryBufferSize);
+ byte[] chunkBytes = new byte[readSize];
Review Comment:
@zratkai Thank you for your detailed explanation. I understand your use case.
I think `in.available()` will be recalculated every time a new block is
read, right? So this fix seems not perfect.
For example, at the beginning,
`in.available()` = 4k
`dictionaryBufferSize` = 5k
`readSize` = 4k
Entering the loop,
the first time` currentLength` is read as 4k, and then copied.
The second time `currentLength` may still be 4k, which will still cause an
out of bounds error.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
