pphust opened a new issue #9597: "lengths must be >= 0" error when querying segments created by hadoop-based ingestion URL: https://github.com/apache/druid/issues/9597 ### Affected Version 0.14.2 ### Description We have a hadoop-based ingestion task on 0.14.2 druid cluster. Sometimes it seems the task generate broken segment， which throws "lengths must be >= 0" exception during querying. There is something looks very strange. It is not all segments are broken. In general we generate 3 segments for each hour. We get about 72 segments per day totally and only 1 or 2 are broken. We can find out it easily just because the same query only fails on specific interval while it works well on other intervals. **The query json:** { "context": { "useCache": false, "populateCache": false, "minTopNThreshold": 60 }, "queryType": "timeseries", "dataSource": "ord_user", "granularity": "ALL", "filter": { "type": "and", "fields": [{ "type": "in", "dimension": "shopType", "values": ["1", "2"] }] }, "aggregations": [{ "type": "thetaSketch", "name": "sketchUnique", "fieldName": "pin", "isInputThetaSketch": false, "size": 16384 }], "intervals": ["2020-03-31T11:00:00.000+0800/2020-03-31T12:00:00.000+0800"] } **The result with error:** { "error" : "Unknown exception", "errorMessage" : "lengths must be >= 0", "errorClass" : "java.lang.IllegalArgumentException", "host" : "xxxx:8083" } **The exception info from historical log:** 2020-04-01T18:18:13,956 ERROR [processing-15] org.apache.druid.query.ChainedExecutionQueryRunner - Exception with one of the sequences! java.lang.IllegalArgumentException: lengths must be >= 0 at net.jpountz.util.SafeUtils.checkLength(SafeUtils.java:38) ~[lz4-java-1.5.0.jar:?] at net.jpountz.util.ByteBufferUtils.checkRange(ByteBufferUtils.java:11) ~[lz4-java-1.5.0.jar:?] at net.jpountz.lz4.LZ4JNISafeDecompressor.decompress(LZ4JNISafeDecompressor.java:46) ~[lz4-java-1.5.0.jar:?] at org.apache.druid.segment.data.CompressionStrategy$LZ4Decompressor.decompress(CompressionStrategy.java:294) ~[druid-processing-0.14.2-incubating.ja r:0.14.2-incubating] at org.apache.druid.segment.data.DecompressingByteBufferObjectStrategy.fromByteBuffer(DecompressingByteBufferObjectStrategy.java:53) ~[druid-processi ng-0.14.2-incubating.jar:0.14.2-incubating] at org.apache.druid.segment.data.DecompressingByteBufferObjectStrategy.fromByteBuffer(DecompressingByteBufferObjectStrategy.java:28) ~[druid-processi ng-0.14.2-incubating.jar:0.14.2-incubating] at org.apache.druid.segment.data.GenericIndexed$BufferIndexed.bufferedIndexedGet(GenericIndexed.java:444) ~[druid-processing-0.14.2-incubating.jar:0. 14.2-incubating] at org.apache.druid.segment.data.GenericIndexed$2.get(GenericIndexed.java:599) ~[druid-processing-0.14.2-incubating.jar:0.14.2-incubating] at org.apache.druid.segment.data.BlockLayoutColumnarLongsSupplier$1.loadBuffer(BlockLayoutColumnarLongsSupplier.java:84) ~[druid-processing-0.14.2-in cubating.jar:0.14.2-incubating] at org.apache.druid.segment.data.BlockLayoutColumnarLongsSupplier$1.get(BlockLayoutColumnarLongsSupplier.java:73) ~[druid-processing-0.14.2-incubatin g.jar:0.14.2-incubating] at org.apache.druid.segment.column.LongsColumn.getLongSingleValueRow(LongsColumn.java:67) ~[druid-processing-0.14.2-incubating.jar:0.14.2-incubating] at org.apache.druid.segment.QueryableIndexStorageAdapter.getMinTime(QueryableIndexStorageAdapter.java:128) ~[druid-processing-0.14.2-incubating.jar:0 .14.2-incubating] at org.apache.druid.segment.QueryableIndexStorageAdapter.makeCursors(QueryableIndexStorageAdapter.java:211) ~[druid-processing-0.14.2-incubating.jar: 0.14.2-incubating] at org.apache.druid.query.QueryRunnerHelper.makeCursorBasedQuery(QueryRunnerHelper.java:59) ~[druid-processing-0.14.2-incubating.jar:0.14.2-incubatin g] at org.apache.druid.query.timeseries.TimeseriesQueryEngine.generateTimeseriesResult(TimeseriesQueryEngine.java:59) ~[druid-processing-0.14.2-incubati ng.jar:0.14.2-incubating] at org.apache.druid.query.timeseries.TimeseriesQueryEngine.process(TimeseriesQueryEngine.java:50) ~[druid-processing-0.14.2-incubating.jar:0.14.2-inc ubating] at org.apache.druid.query.timeseries.TimeseriesQueryRunnerFactory$TimeseriesQueryRunner.run(TimeseriesQueryRunnerFactory.java:103) ~[druid-processing -0.14.2-incubating.jar:0.14.2-incubating] at org.apache.druid.query.ReferenceCountingSegmentQueryRunner.run(ReferenceCountingSegmentQueryRunner.java:52) ~[druid-processing-0.14.2-incubating.j ar:0.14.2-incubating] at org.apache.druid.query.MetricsEmittingQueryRunner.lambda$run$0(MetricsEmittingQueryRunner.java:97) ~[druid-processing-0.14.2-incubating.jar:0.14.2 -incubating] at org.apache.druid.java.util.common.guava.LazySequence.accumulate(LazySequence.java:40) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating] at org.apache.druid.java.util.common.guava.WrappingSequence$1.get(WrappingSequence.java:50) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating] at org.apache.druid.java.util.common.guava.SequenceWrapper.wrap(SequenceWrapper.java:55) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating] at org.apache.druid.java.util.common.guava.WrappingSequence.accumulate(WrappingSequence.java:45) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating ] at org.apache.druid.java.util.common.guava.LazySequence.accumulate(LazySequence.java:40) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating] at org.apache.druid.java.util.common.guava.WrappingSequence$1.get(WrappingSequence.java:50) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating] at org.apache.druid.java.util.common.guava.SequenceWrapper.wrap(SequenceWrapper.java:55) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating] at org.apache.druid.java.util.common.guava.WrappingSequence.accumulate(WrappingSequence.java:45) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating ] at org.apache.druid.query.spec.SpecificSegmentQueryRunner$1.accumulate(SpecificSegmentQueryRunner.java:82) ~[druid-processing-0.14.2-incubating.jar:0 .14.2-incubating] at org.apache.druid.java.util.common.guava.WrappingSequence$1.get(WrappingSequence.java:50) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating] at org.apache.druid.query.spec.SpecificSegmentQueryRunner.doNamed(SpecificSegmentQueryRunner.java:168) ~[druid-processing-0.14.2-incubating.jar:0.14. 2-incubating] at org.apache.druid.query.spec.SpecificSegmentQueryRunner.access$100(SpecificSegmentQueryRunner.java:45) ~[druid-processing-0.14.2-incubating.jar:0.1 4.2-incubating] at org.apache.druid.query.spec.SpecificSegmentQueryRunner$2.wrap(SpecificSegmentQueryRunner.java:148) ~[druid-processing-0.14.2-incubating.jar:0.14.2 -incubating] at org.apache.druid.java.util.common.guava.WrappingSequence.accumulate(WrappingSequence.java:45) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating ] at org.apache.druid.java.util.common.guava.WrappingSequence$1.get(WrappingSequence.java:50) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating] at org.apache.druid.query.CPUTimeMetricQueryRunner$1.wrap(CPUTimeMetricQueryRunner.java:74) ~[druid-processing-0.14.2-incubating.jar:0.14.2-incubatin g] at org.apache.druid.java.util.common.guava.WrappingSequence.accumulate(WrappingSequence.java:45) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating ] at org.apache.druid.java.util.common.guava.Sequence.toList(Sequence.java:76) ~[druid-core-0.14.2-incubating.jar:0.14.2-incubating] at org.apache.druid.query.ChainedExecutionQueryRunner$1$1.call(ChainedExecutionQueryRunner.java:124) [druid-processing-0.14.2-incubating.jar:0.14.2-i ncubating] at org.apache.druid.query.ChainedExecutionQueryRunner$1$1.call(ChainedExecutionQueryRunner.java:114) [druid-processing-0.14.2-incubating.jar:0.14.2-i ncubating] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_121] at org.apache.druid.query.PrioritizedListenableFutureTask.run(PrioritizedExecutorService.java:247) [druid-processing-0.14.2-incubating.jar:0.14.2-inc ubating] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
