soullkk opened a new issue, #17489:
URL: https://github.com/apache/druid/issues/17489
Indexer errors when using PeriodGranity in the groupBy query
### Affected Version
Apache Druid 28.0.1.
### Description
Please include as much detailed information about the problem as possible.
- Steps to reproduce the problem
1、Create a datasource(like ODAEDATASET__DEFAULT_lkk_person_olap__DEFAULT)
and ingest data with timestamps greater than 2024-11-18T12:00:00 and less than
2024-11-18T13:00:00
2、Use the "/druid/v2" to perform the following query :
```
{
"queryType": "groupBy",
"dataSource": "ODAEDATASET__DEFAULT_lkk_person_olap__DEFAULT",
"granularity": {
"type": "period",
"period": "PT5H",
"timeZone": "Asia/Singapore"
},
"dimensions": [{
"type": "default",
"dimension": "address",
"outputName": "address",
"outputType": "STRING"
}
],
"limitSpec": {
"type": "default",
"limit": 14400,
"columns": []
},
"intervals": ["2024-11-18T12:05:00.000Z/2024-11-18T12:15:00.000Z"],
"context": {
"timeout": 30000,
"maxRowsQueuedForOrdering": 128575
}
}
```
- The error message or stack traces encountered. Providing more context,
such as nearby log messages or even entire logs, can be helpful.
The error stack of the historical node is as follows :
```
2024-11-19 03:21:31,097 ERROR
[processing-4][][org.apache.druid.query.groupby.epinephelinae.GroupByMergingQueryRunnerV2]
Exception with one of the sequences!
java.lang.IllegalArgumentException: The end instant must be greater than the
start instant
at
org.joda.time.base.AbstractInterval.checkInterval(AbstractInterval.java:63)
~[joda-time-2.12.5.jar:2.12.5]
at org.joda.time.base.BaseInterval.<init>(BaseInterval.java:94)
~[joda-time-2.12.5.jar:2.12.5]
at org.joda.time.Interval.<init>(Interval.java:201)
~[joda-time-2.12.5.jar:2.12.5]
at
org.apache.druid.query.vector.VectorCursorGranularizer.create(VectorCursorGranularizer.java:82)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.query.groupby.epinephelinae.vector.VectorGroupByEngine$VectorGroupByEngineIterator.<init>(VectorGroupByEngine.java:286)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.query.groupby.epinephelinae.vector.VectorGroupByEngine$1.make(VectorGroupByEngine.java:192)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.query.groupby.epinephelinae.vector.VectorGroupByEngine$1.make(VectorGroupByEngine.java:144)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.java.util.common.guava.BaseSequence.accumulate(BaseSequence.java:39)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.java.util.common.guava.WrappingSequence$1.get(WrappingSequence.java:50)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.java.util.common.guava.SequenceWrapper.wrap(SequenceWrapper.java:55)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.java.util.common.guava.WrappingSequence.accumulate(WrappingSequence.java:45)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.java.util.common.guava.WrappingSequence$1.get(WrappingSequence.java:50)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.java.util.common.guava.SequenceWrapper.wrap(SequenceWrapper.java:55)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.java.util.common.guava.WrappingSequence.accumulate(WrappingSequence.java:45)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.java.util.common.guava.LazySequence.accumulate(LazySequence.java:40)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.java.util.common.guava.WrappingSequence$1.get(WrappingSequence.java:50)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.java.util.common.guava.SequenceWrapper.wrap(SequenceWrapper.java:55)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.java.util.common.guava.WrappingSequence.accumulate(WrappingSequence.java:45)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.java.util.common.guava.MappedSequence.accumulate(MappedSequence.java:43)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.java.util.common.guava.WrappingSequence$1.get(WrappingSequence.java:50)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.java.util.common.guava.SequenceWrapper.wrap(SequenceWrapper.java:55)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.java.util.common.guava.WrappingSequence.accumulate(WrappingSequence.java:45)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.java.util.common.guava.LazySequence.accumulate(LazySequence.java:40)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.java.util.common.guava.WrappingSequence$1.get(WrappingSequence.java:50)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.java.util.common.guava.SequenceWrapper.wrap(SequenceWrapper.java:55)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.java.util.common.guava.WrappingSequence.accumulate(WrappingSequence.java:45)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.query.spec.SpecificSegmentQueryRunner$1.accumulate(SpecificSegmentQueryRunner.java:98)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.java.util.common.guava.WrappingSequence$1.get(WrappingSequence.java:50)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.query.spec.SpecificSegmentQueryRunner.doNamed(SpecificSegmentQueryRunner.java:185)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.query.spec.SpecificSegmentQueryRunner.access$100(SpecificSegmentQueryRunner.java:44)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.query.spec.SpecificSegmentQueryRunner$2.wrap(SpecificSegmentQueryRunner.java:165)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.java.util.common.guava.WrappingSequence.accumulate(WrappingSequence.java:45)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.java.util.common.guava.WrappingSequence$1.get(WrappingSequence.java:50)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.query.CPUTimeMetricQueryRunner$1.wrap(CPUTimeMetricQueryRunner.java:77)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.java.util.common.guava.WrappingSequence.accumulate(WrappingSequence.java:45)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.query.groupby.epinephelinae.GroupByMergingQueryRunnerV2$1$1$1.call(GroupByMergingQueryRunnerV2.java:252)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
org.apache.druid.query.groupby.epinephelinae.GroupByMergingQueryRunnerV2$1$1$1.call(GroupByMergingQueryRunnerV2.java:239)
~[druid-processing-28.0.1-htrunk15.jar:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[?:1.8.0_432]
at
org.apache.druid.query.PrioritizedListenableFutureTask.run(PrioritizedExecutorService.java:259)
~[druid-processing-28.0.1-htrunk15.jar:?]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
~[?:1.8.0_432]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
~[?:1.8.0_432]
at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_432]
```
- Any debugging that you have already done
If the interval of a segment is like this：2024-11-18T12:10:00.000 ---
2024-11-18T12:15:00.000
The result of granularity.bucketEnd(maxTime) will be less than minTime when
using the query mentioned earlier.
```
@Nullable
public static VectorCursorGranularizer create(
final StorageAdapter storageAdapter,
final VectorCursor cursor,
final Granularity granularity,
final Interval queryInterval
)
{
final DateTime minTime = storageAdapter.getMinTime();
final DateTime maxTime = storageAdapter.getMaxTime();
final Interval storageAdapterInterval = new Interval(minTime,
granularity.bucketEnd(maxTime));
```
PeriodGranularity.truncate(long t):
period = "PT5H"
t = 2024-11-18T12:15:00.000
timeZone = "Asia/Singapore"
origin = -27000000
```
final int hours = period.getHours();
if (hours > 0) {
if (hours > 1 || hasOrigin) {
// align on multiples from origin
long h = chronology.hours().getDifferenceAsLong(t, origin);
h -= h % hours;
long tt = chronology.hours().add(origin, h);
// always round down to the previous period (for timestamps prior to
origin)
if (t < tt && origin > 0) {
t = chronology.hours().add(tt, -hours);
} else if (t > tt && origin < 0) {
t = chronology.minuteOfHour().roundFloor(tt);
t = chronology.minuteOfHour().set(t, 0);
} else {
t = tt;
}
return t;
} else {
return chronology.hourOfDay().roundFloor(t);
}
}
```
The result of "long tt = chronology.hours().add(origin, h)" will be
"2024-11-18T07:30:00.000Z"
"t > tt && origin < 0 " is valid.
so "t = chronology.minuteOfHour().set(t, 0)" will be executed
the return will be t = "2024-11-18T07:00:00.000Z"
so granularity.bucketEnd(maxTime) = "2024-11-18T12:00:00.000Z", it is less
than 2024-11-18T12:10:00.000 (the start time of interval)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
