itschrispeck opened a new issue, #12534: URL: https://github.com/apache/pinot/issues/12534
Within a segment, certain chunks cannot be decompressed. I can time bound a query within a single segment, and the query will succeed. e.g. for a segment with data [chunk1][chunk2][chunk3]: 1. I can write a query that reads chunk1, and it will return valid data 2. I can write a query that reads chunk3, and it will return valid data 3. If the query touches data in the middle, we receive a random IllegalArgument/NegativeArraySize exception. This leads me to believe that this should be unit testable, so I'm working on reproducing this with a sharable dataset. Sample exception: ``` java.lang.IllegalArgumentException: newPosition > limit: (538976288 > 8324206) at java.base/java.nio.Buffer.createPositionException(Buffer.java:352) at java.base/java.nio.Buffer.position(Buffer.java:327) at java.base/java.nio.ByteBuffer.position(ByteBuffer.java:1551) at java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:328) at java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:73) at org.apache.pinot.segment.local.segment.index.readers.forward.VarByteChunkForwardIndexReaderV4$UncompressedReaderContext.readSmallUncompressedValue(VarByteChunkForwardIndexReaderV4.java:429) at org.apache.pinot.segment.local.segment.index.readers.forward.VarByteChunkForwardIndexReaderV4$UncompressedReaderContext.processChunkAndReadFirstValue(VarByteChunkForwardIndexReaderV4.java:413) at org.apache.pinot.segment.local.segment.index.readers.forward.VarByteChunkForwardIndexReaderV4$ReaderContext.decompressAndRead(VarByteChunkForwardIndexReaderV4.java:372) at org.apache.pin ot.segment.local.segment.index.readers.forward.VarByteChunkForwardIndexReaderV4$ReaderContext.getValue(VarByteChunkForwardIndexReaderV4.java:326) at org.apache.pinot.segment.local.segment.index.readers.forward.VarByteChunkForwardIndexReaderV4.getString(VarByteChunkForwardIndexReaderV4.java:116) at org.apache.pinot.segment.local.segment.index.readers.forward.VarByteChunkForwardIndexReaderV4.getString(VarByteChunkForwardIndexReaderV4.java:48) at org.apache.pinot.core.common.DataFetcher$ColumnValueReader.readStringValues(DataFetcher.java:601) at org.apache.pinot.core.common.DataFetcher.fetchStringValues(DataFetcher.java:239) at org.apache.pinot.core.common.DataBlockCache.getStringValuesForSVColumn(DataBlockCache.java:277) at org.apache.pinot.core.operator.docvalsets.ProjectionBlockValSet.getStringValuesSV(ProjectionBlockValSet.java:153) at org.apache.pinot.core.common.RowBasedBlockValueFetcher.createFetcher(RowBasedBlockValueFetcher.java:67) at org.apache.pinot.core.common.RowBasedBloc kValueFetcher.<init>(RowBasedBlockValueFetcher.java:33) at org.apache.pinot.core.operator.query.SelectionOnlyOperator.getNextBlock(SelectionOnlyOperator.java:103) at org.apache.pinot.core.operator.query.SelectionOnlyOperator.getNextBlock(SelectionOnlyOperator.java:41) at org.apache.pinot.core.operator.BaseOperator.nextBlock(BaseOperator.java:43) at org.apache.pinot.core.operator.combine.BaseSingleBlockCombineOperator.processSegments(BaseSingleBlockCombineOperator.java:94) at org.apache.pinot.core.operator.combine.BaseCombineOperator$1.runJob(BaseCombineOperator.java:117) at org.apache.pinot.core.util.trace.TraceRunnable.run(TraceRunnable.java:40) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInt erruptibly(TrustedListenableFutureTask.java:125) at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57) at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) at java.base/java.lang.Thread.run(Thread.java:1583) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
