Hi, I have a few question regarding this Exception: 1. Does the table have a string columns for which the length of the data is exceeding 32k characters? 2. Are you able to query(select *) on the table successfully? 3. Can you share the schema of the table?
Meanwhile i am looking into the possibilities of any other thread clearing the MemoryBlock. Regards Kunal Kapoor On Tue, Oct 16, 2018 at 1:31 PM xm_zzc <[email protected]> wrote: > Hi Babu: > Thanks for your reply. > I set enable.unsafe.in.query.processing=false and > enable.unsafe.columnpage=false , and test failed still. > I think the issue I met is not related to MemoryBlock which is cleaned by > some other thread. As the test steps I mentioned above, I copy the wrong > segment and use SDKReader to read data, it failed too, the error message > is > following: > *java.lang.RuntimeException: java.lang.IllegalArgumentException > at > > org.apache.carbondata.core.datastore.chunk.impl.DimensionRawColumnChunk.convertToDimColDataChunkWithOutCache(DimensionRawColumnChunk.java:120) > at > > org.apache.carbondata.core.scan.result.BlockletScannedResult.fillDataChunks(BlockletScannedResult.java:355) > at > > org.apache.carbondata.core.scan.result.BlockletScannedResult.hasNext(BlockletScannedResult.java:559) > at > > org.apache.carbondata.core.scan.collector.impl.DictionaryBasedResultCollector.collectResultInRow(DictionaryBasedResultCollector.java:137) > at > > org.apache.carbondata.core.scan.processor.DataBlockIterator.next(DataBlockIterator.java:109) > at > > org.apache.carbondata.core.scan.result.iterator.DetailQueryResultIterator.getBatchResult(DetailQueryResultIterator.java:49) > at > > org.apache.carbondata.core.scan.result.iterator.DetailQueryResultIterator.next(DetailQueryResultIterator.java:41) > at > > org.apache.carbondata.core.scan.result.iterator.DetailQueryResultIterator.next(DetailQueryResultIterator.java:1) > at > > org.apache.carbondata.core.scan.result.iterator.ChunkRowIterator.hasNext(ChunkRowIterator.java:58) > at > > org.apache.carbondata.hadoop.CarbonRecordReader.nextKeyValue(CarbonRecordReader.java:104) > at > org.apache.carbondata.sdk.file.CarbonReader.hasNext(CarbonReader.java:71) > at > cn.xm.zzc.carbonsdktest.CarbonSDKTest.main(CarbonSDKTest.java:68) > Caused by: java.lang.IllegalArgumentException > at java.nio.Buffer.position(Buffer.java:244) > at > > org.apache.carbondata.core.datastore.chunk.store.impl.unsafe.UnsafeVariableLengthDimensionDataChunkStore.putArray(UnsafeVariableLengthDimensionDataChunkStore.java:97) > at > > org.apache.carbondata.core.datastore.chunk.impl.VariableLengthDimensionColumnPage.<init>(VariableLengthDimensionColumnPage.java:58) > at > > org.apache.carbondata.core.datastore.chunk.reader.dimension.v3.CompressedDimensionChunkFileBasedReaderV3.decodeDimensionLegacy(CompressedDimensionChunkFileBasedReaderV3.java:325) > at > > org.apache.carbondata.core.datastore.chunk.reader.dimension.v3.CompressedDimensionChunkFileBasedReaderV3.decodeDimension(CompressedDimensionChunkFileBasedReaderV3.java:266) > at > > org.apache.carbondata.core.datastore.chunk.reader.dimension.v3.CompressedDimensionChunkFileBasedReaderV3.decodeColumnPage(CompressedDimensionChunkFileBasedReaderV3.java:224) > at > > org.apache.carbondata.core.datastore.chunk.impl.DimensionRawColumnChunk.convertToDimColDataChunkWithOutCache(DimensionRawColumnChunk.java:118) > ... 11 more* > > when error occurred, the values of some parameters in > UnsafeVariableLengthDimensionDataChunkStore.putArray are as following : > > buffer.limit=192000 > buffer.cap=192000 > startOffset=300289 > numberOfRows=32000 > this.dataPointersOffsets=288000 > > startOffset is bigger than buffer.limit, so error occurred. > > > > > -- > Sent from: > http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/ >
