mikecdavis-dd opened a new issue #7923:
URL: https://github.com/apache/pinot/issues/7923
After migrating from `0.8.0` to `0.9.2` we started to see the receive the
following exception for some of our high volume REALTIME tables:
```
java.lang.IndexOutOfBoundsException: Index 317 out of bounds for length 151
at jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64)
~[?:?]
at
jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70)
~[?:?]
at jdk.internal.util.Preconditions.checkIndex(Preconditions.java:248)
~[?:?]
at java.util.Objects.checkIndex(Objects.java:372) ~[?:?]
at java.util.ArrayList.get(ArrayList.java:459) ~[?:?]
at
org.apache.pinot.segment.local.realtime.impl.invertedindex.RealtimeInvertedIndexReader.add(RealtimeInvertedIndexReader.java:59)
~[pinot-all-0.9.2-jar-with-dependencies.jar:0.9.2-3e5e5aca07ad796d453add80a2b2b1bf1d654cf1]
at
org.apache.pinot.segment.local.indexsegment.mutable.MutableSegmentImpl.addNewRow(MutableSegmentImpl.java:580)
~[pinot-all-0.9.2-jar-with-dependencies.jar:0.9.2-3e5e5aca07ad796d453add80a2b2b1bf1d654cf1]
at
org.apache.pinot.segment.local.indexsegment.mutable.MutableSegmentImpl.index(MutableSegmentImpl.java:491)
~[pinot-all-0.9.2-jar-with-dependencies.jar:0.9.2-3e5e5aca07ad796d453add80a2b2b1bf1d654cf1]
at
org.apache.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager.processStreamEvents(LLRealtimeSegmentDataManager.java:524)
[pinot-all-0.9.2-jar-with-dependencies.jar:0.9.2-3e5e5aca07ad796d453add80a2b2b1bf1d654cf1]
at
org.apache.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager.consumeLoop(LLRealtimeSegmentDataManager.java:420)
[pinot-all-0.9.2-jar-with-dependencies.jar:0.9.2-3e5e5aca07ad796d453add80a2b2b1bf1d654cf1]
at
org.apache.pinot.core.data.manager.realtime.LLRealtimeSegmentDataManager$PartitionConsumer.run(LLRealtimeSegmentDataManager.java:568)
[pinot-all-0.9.2-jar-with-dependencies.jar:0.9.2-3e5e5aca07ad796d453add80a2b2b1bf1d654cf1]
at java.lang.Thread.run(Thread.java:829) [?:?]
```
Another similar example:
```
java.lang.IndexOutOfBoundsException: Index 3021 out of bounds for length 27
```
Digging into the stack trace a bit this
[method](https://github.com/apache/pinot/blob/f4e55182a8c552c8493d1693868b0c752b1a1d56/pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/invertedindex/RealtimeInvertedIndexReader.java#L47-L61)
appears to operate on the assumption that the `dictId` is either the next
index in the underlying `ArrayList` and should be added, or it's already a
valid index and could be used to `get(dictId)`.
When this assumption is not met then an `IndexOutOfBoundsException` is
thrown and the table stops consuming from that partition.
Note: No table configurations were made between the version upgrades.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
