kgyrtkirk opened a new issue, #16554: URL: https://github.com/apache/druid/issues/16554
Queries like ``` For aggregators which have a non-trivial finalization `AggregatorFactory#getResultType() != AggregatorFactory#getIntermediateType()` (so essentialy those which will most likely need some work to be done). The comparator is derived for the underlying `aggregator` for the `postAggregator` [here](https://github.com/apache/druid/blob/30c59042e0cde3c76392afa3ba73d0830b11b0a5/processing/src/main/java/org/apache/druid/query/aggregation/post/FinalizingFieldAccessPostAggregator.java#L124) - so a comparision of finalized values might be tried with the aggregator's comparator. This could cause an exception in case the postAggregators output is being used to order the rows by it. <details> <summary>This causes a ClassCastException</summary> ``` java.lang.ClassCastException: class java.lang.Float cannot be cast to class org.apache.druid.collections.SerializablePair (java.lang.Float is in module java.base of loader 'bootstrap'; org.apache.druid.collections.SerializablePair is in unnamed module of loader 'app') at java.base/java.util.Collections$ReverseComparator2.compare(Collections.java:5278) at java.base/java.util.Comparators$NullComparator.compare(Comparators.java:83) at org.apache.druid.query.topn.TopNNumericResultBuilder.lambda$0(TopNNumericResultBuilder.java:75) at java.base/java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:675) at java.base/java.util.PriorityQueue.siftUp(PriorityQueue.java:652) at java.base/java.util.PriorityQueue.offer(PriorityQueue.java:345) at java.base/java.util.PriorityQueue.add(PriorityQueue.java:326) at org.apache.druid.query.topn.TopNNumericResultBuilder.addEntry(TopNNumericResultBuilder.java:157) at org.apache.druid.query.topn.TopNNumericResultBuilder.addEntry(TopNNumericResultBuilder.java:1) at org.apache.druid.query.topn.PooledTopNAlgorithm.updateResults(PooledTopNAlgorithm.java:765) at org.apache.druid.query.topn.PooledTopNAlgorithm.updateResults(PooledTopNAlgorithm.java:1) at org.apache.druid.query.topn.BaseTopNAlgorithm.runWithCardinalityKnown(BaseTopNAlgorithm.java:121) at org.apache.druid.query.topn.BaseTopNAlgorithm.run(BaseTopNAlgorithm.java:82) at org.apache.druid.query.topn.TopNMapFn.apply(TopNMapFn.java:70) at org.apache.druid.query.topn.TopNQueryEngine.lambda$0(TopNQueryEngine.java:100) at org.apache.druid.java.util.common.guava.MappingAccumulator.accumulate(MappingAccumulator.java:40) at org.apache.druid.java.util.common.guava.FilteringAccumulator.accumulate(FilteringAccumulator.java:41) at org.apache.druid.java.util.common.guava.MappingAccumulator.accumulate(MappingAccumulator.java:40) at org.apache.druid.java.util.common.guava.BaseSequence.accumulate(BaseSequence.java:44) at org.apache.druid.java.util.common.guava.MappedSequence.accumulate(MappedSequence.java:43) at org.apache.druid.java.util.common.guava.WrappingSequence$1.get(WrappingSequence.java:50) at org.apache.druid.java.util.common.guava.SequenceWrapper.wrap(SequenceWrapper.java:55) at org.apache.druid.java.util.common.guava.WrappingSequence.accumulate(WrappingSequence.java:45) at org.apache.druid.java.util.common.guava.FilteredSequence.accumulate(FilteredSequence.java:45) at org.apache.druid.java.util.common.guava.MappedSequence.accumulate(MappedSequence.java:43) at org.apache.druid.java.util.common.guava.FilteredSequence.accumulate(FilteredSequence.java:45) at org.apache.druid.java.util.common.guava.WrappingSequence$1.get(WrappingSequence.java:50) at org.apache.druid.java.util.common.guava.SequenceWrapper.wrap(SequenceWrapper.java:55) at org.apache.druid.java.util.common.guava.WrappingSequence.accumulate(WrappingSequence.java:45) at org.apache.druid.query.spec.SpecificSegmentQueryRunner$1.accumulate(SpecificSegmentQueryRunner.java:98) at org.apache.druid.java.util.common.guava.WrappingSequence$1.get(WrappingSequence.java:50) at org.apache.druid.query.spec.SpecificSegmentQueryRunner.doNamed(SpecificSegmentQueryRunner.java:185) at org.apache.druid.query.spec.SpecificSegmentQueryRunner.access$1(SpecificSegmentQueryRunner.java:181) at org.apache.druid.query.spec.SpecificSegmentQueryRunner$2.wrap(SpecificSegmentQueryRunner.java:165) at org.apache.druid.java.util.common.guava.WrappingSequence.accumulate(WrappingSequence.java:45) at org.apache.druid.java.util.common.guava.Sequence.toList(Sequence.java:87) at org.apache.druid.query.ChainedExecutionQueryRunner$1$3.call(ChainedExecutionQueryRunner.java:112) at org.apache.druid.query.ChainedExecutionQueryRunner$1$3.call(ChainedExecutionQueryRunner.java:1) at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131) at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:75) at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82) at org.apache.druid.java.util.common.concurrent.DirectExecutorService.execute(DirectExecutorService.java:81) at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:140) at com.google.common.util.concurrent.AbstractListeningExecutorService.submit(AbstractListeningExecutorService.java:79) at org.apache.druid.query.ForwardingQueryProcessingPool.submitRunnerTask(ForwardingQueryProcessingPool.java:45) at org.apache.druid.query.ChainedExecutionQueryRunner$1.lambda$0(ChainedExecutionQueryRunner.java:100) at com.google.common.collect.Iterators$6.transform(Iterators.java:828) at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:52) at com.google.common.collect.Iterators.addAll(Iterators.java:366) at com.google.common.collect.Lists.newArrayList(Lists.java:146) at com.google.common.collect.Lists.newArrayList(Lists.java:132) at org.apache.druid.query.ChainedExecutionQueryRunner$1.make(ChainedExecutionQueryRunner.java:92) at org.apache.druid.java.util.common.guava.BaseSequence.toYielder(BaseSequence.java:66) at org.apache.druid.common.guava.CombiningSequence.toYielder(CombiningSequence.java:78) at org.apache.druid.java.util.common.guava.MappedSequence.toYielder(MappedSequence.java:49) at org.apache.druid.java.util.common.guava.MappedSequence.toYielder(MappedSequence.java:49) at org.apache.druid.common.guava.CombiningSequence.toYielder(CombiningSequence.java:78) at org.apache.druid.java.util.common.guava.MappedSequence.toYielder(MappedSequence.java:49) at org.apache.druid.java.util.common.guava.MappedSequence.toYielder(MappedSequence.java:49) at org.apache.druid.java.util.common.guava.MappedSequence.toYielder(MappedSequence.java:49) at org.apache.druid.java.util.common.guava.LazySequence.toYielder(LazySequence.java:46) at org.apache.druid.java.util.common.guava.WrappingSequence$2.get(WrappingSequence.java:88) at org.apache.druid.java.util.common.guava.WrappingSequence$2.get(WrappingSequence.java:1) at org.apache.druid.java.util.common.guava.SequenceWrapper.wrap(SequenceWrapper.java:55) at org.apache.druid.java.util.common.guava.WrappingSequence.toYielder(WrappingSequence.java:83) at org.apache.druid.java.util.common.guava.MergeSequence.lambda$1(MergeSequence.java:66) at org.apache.druid.java.util.common.guava.BaseSequence.accumulate(BaseSequence.java:44) at org.apache.druid.java.util.common.guava.MergeSequence.toYielder(MergeSequence.java:63) at org.apache.druid.query.RetryQueryRunner$1.toYielder(RetryQueryRunner.java:133) at org.apache.druid.java.util.common.guava.YieldingSequenceBase.accumulate(YieldingSequenceBase.java:35) at org.apache.druid.common.guava.CombiningSequence.accumulate(CombiningSequence.java:62) at org.apache.druid.java.util.common.guava.MappedSequence.accumulate(MappedSequence.java:43) at org.apache.druid.java.util.common.guava.MappedSequence.accumulate(MappedSequence.java:43) at org.apache.druid.java.util.common.guava.MappedSequence.accumulate(MappedSequence.java:43) at org.apache.druid.java.util.common.guava.WrappingSequence$1.get(WrappingSequence.java:50) at org.apache.druid.query.CPUTimeMetricQueryRunner$1.wrap(CPUTimeMetricQueryRunner.java:77) at org.apache.druid.java.util.common.guava.WrappingSequence.accumulate(WrappingSequence.java:45) at org.apache.druid.java.util.common.guava.MappedSequence.accumulate(MappedSequence.java:43) at org.apache.druid.java.util.common.guava.ConcatSequence.accumulate(ConcatSequence.java:42) at org.apache.druid.server.ClientQuerySegmentWalker.materializeResultsAsArray(ClientQuerySegmentWalker.java:810) at org.apache.druid.server.ClientQuerySegmentWalker.toInlineDataSource(ClientQuerySegmentWalker.java:680) at org.apache.druid.server.ClientQuerySegmentWalker.inlineIfNecessary(ClientQuerySegmentWalker.java:440) at org.apache.druid.server.ClientQuerySegmentWalker.getQueryRunnerForIntervals(ClientQuerySegmentWalker.java:219) at org.apache.druid.server.SpecificSegmentsQuerySegmentWalker.getQueryRunnerForIntervals(SpecificSegmentsQuerySegmentWalker.java:214) at org.apache.druid.query.spec.MultipleIntervalSegmentSpec.lookup(MultipleIntervalSegmentSpec.java:57) at org.apache.druid.query.BaseQuery.getRunner(BaseQuery.java:119) at org.apache.druid.query.QueryPlus.run(QueryPlus.java:149) at org.apache.druid.server.QueryLifecycle.execute(QueryLifecycle.java:316) at org.apache.druid.server.QueryLifecycle.runSimple(QueryLifecycle.java:163) at org.apache.druid.sql.calcite.run.NativeQueryMaker.execute(NativeQueryMaker.java:193) at org.apache.druid.sql.calcite.run.NativeQueryMaker.runQuery(NativeQueryMaker.java:151) at org.apache.druid.sql.calcite.rel.DruidRel.runQuery(DruidRel.java:60) at org.apache.druid.sql.calcite.planner.QueryHandler.lambda$8(QueryHandler.java:619) at org.apache.druid.sql.calcite.planner.PlannerResult.run(PlannerResult.java:62) at org.apache.druid.sql.DirectStatement$ResultSet.run(DirectStatement.java:109) at org.apache.druid.sql.DirectStatement.execute(DirectStatement.java:184) at org.apache.druid.sql.calcite.QueryTestRunner$ExecuteQuery.lambda$0(QueryTestRunner.java:308) at org.apache.druid.sql.calcite.util.QueryLogHook.logQueriesFor(QueryLogHook.java:67) at org.apache.druid.sql.calcite.QueryTestRunner$ExecuteQuery.runQuery(QueryTestRunner.java:306) at org.apache.druid.sql.calcite.QueryTestRunner$ExecuteQuery.run(QueryTestRunner.java:286) at org.apache.druid.sql.calcite.QueryTestRunner.run(QueryTestRunner.java:768) at org.apache.druid.sql.calcite.QueryTestBuilder.run(QueryTestBuilder.java:303) at org.apache.druid.sql.calcite.BaseCalciteQueryTest.testQuery(BaseCalciteQueryTest.java:712) at org.apache.druid.sql.calcite.CalciteQueryTest.testOrderByEarliestFloat(CalciteQueryTest.java:1772) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:728) at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60) at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131) at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156) at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147) at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86) at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103) at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93) at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37) at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92) at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:218) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:214) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:139) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) at java.base/java.util.ArrayList.forEach(ArrayList.java:1541) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) at java.base/java.util.ArrayList.forEach(ArrayList.java:1541) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35) at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57) at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:147) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:127) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:90) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:55) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:102) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:54) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:95) at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:91) at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:60) at org.eclipse.jdt.internal.junit5.runner.JUnit5TestReference.run(JUnit5TestReference.java:98) at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:40) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:529) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:756) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:452) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:210) ``` </details> <details> <summary>Testcase for CalciteQueryTest</summary> ``` @Test public void testOrderByEarliestFloatSubQuery() { testBuilder() .sql( "select dim1,e from (SELECT dim1, EARLIEST(f1) e FROM druid.numfoo GROUP BY 1 ORDER BY 2 LIMIT 10) t order by 1 limit 2" ) .run(); } ``` </details> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
