lukoou3 opened a new issue, #15210:
URL: https://github.com/apache/druid/issues/15210
When I upgraded the version of Druid from 0.18.1 to 26.0.0, I found that the
SQL parser can not parse the literal parameter of the aggregation function,
when a subquery statement has a limit
### Affected Version
26.0.0
### Description
Please include as much detailed information about the problem as possible.
- Cluster sizeļ¼local for test in iead
- Configurations in use
- Steps to reproduce the problem
use
org.apache.druid.query.aggregation.datasketches.hll.sql.HllSketchSqlAggregatorTest
test sql
```java
@Test
public void testAggQuery() throws Exception {
cannotVectorize();
String sql = "select\n" +
" dim1,\n" +
" APPROX_COUNT_DISTINCT_DS_HLL (hll, 12, 'HLL_4') cnt\n" +
"from (\n" +
" select\n" +
" dim1,\n" +
" DS_HLL (dim1) hll\n" +
" from druid.foo\n" +
" group by dim1\n" +
" limit 10\n" +
") t\n" +
"group by dim1";
QueryTestBuilder builder = testBuilder().sql(sql).skipVectorize();
builder.run();
QueryTestRunner.QueryResults queryResults = builder.results();
List<Object[]> results = queryResults.results;
for (Object[] result : results) {
System.out.println(Arrays.toString(result));
}
}
```
- The error message or stack traces encountered. Providing more context,
such as nearby log messages or even entire logs, can be helpful.
```
2023-10-19T16:17:44,911 WARN [main]
org.apache.druid.sql.calcite.planner.QueryHandler - Query not supported.
Possible error: Aggregation [APPROX_COUNT_DISTINCT_DS_HLL($1, $2, $3)] is not
supported SQL was: select
dim1,
APPROX_COUNT_DISTINCT_DS_HLL (hll, 12, 'HLL_4') cnt
from (
select
dim1,
DS_HLL (dim1) hll
from druid.foo
group by dim1
limit 10
) t
group by dim1 (org.apache.calcite.plan.RelOptPlanner$CannotPlanException:
There are not enough rules to produce a node with desired properties:
convention=DRUID, sort=[].
Missing conversion is LogicalAggregate[convention: NONE -> DRUID]
There is 1 empty subset: rel#51:Subset#5.DRUID.[], the relevant part of the
original plan is as follows
49:LogicalAggregate(group=[{0}], cnt=[APPROX_COUNT_DISTINCT_DS_HLL($1, $2,
$3)])
47:LogicalProject(subset=[rel#48:Subset#4.NONE.[]], dim1=[$0], hll=[$1],
$f2=[12], $f3=['HLL_4'])
45:LogicalSort(subset=[rel#46:Subset#3.NONE.[]], fetch=[10])
43:LogicalAggregate(subset=[rel#44:Subset#2.NONE.[]], group=[{0}],
hll=[DS_HLL($0)])
41:LogicalProject(subset=[rel#42:Subset#1.NONE.[]], dim1=[$1])
6:LogicalTableScan(subset=[rel#40:Subset#0.NONE.[]],
table=[[druid, foo]])
Root: rel#51:Subset#5.DRUID.[]
Original rel:
LogicalAggregate(subset=[rel#51:Subset#5.DRUID.[]], group=[{0}],
cnt=[APPROX_COUNT_DISTINCT_DS_HLL($1, $2, $3)]): rowcount = 1.0, cumulative
cost = {1.125 rows, 0.0 cpu, 0.0 io}, id = 49
LogicalProject(subset=[rel#48:Subset#4.NONE.[]], dim1=[$0], hll=[$1],
$f2=[12], $f3=['HLL_4']): rowcount = 10.0, cumulative cost = {10.0 rows, 40.0
cpu, 0.0 io}, id = 47
LogicalSort(subset=[rel#46:Subset#3.NONE.[]], fetch=[10]): rowcount =
10.0, cumulative cost = {10.0 rows, 184.2068074395237 cpu, 0.0 io}, id = 45
LogicalAggregate(subset=[rel#44:Subset#2.NONE.[]], group=[{0}],
hll=[DS_HLL($0)]): rowcount = 10.0, cumulative cost = {11.25 rows, 0.0 cpu, 0.0
io}, id = 43
LogicalProject(subset=[rel#42:Subset#1.NONE.[]], dim1=[$1]):
rowcount = 100.0, cumulative cost = {100.0 rows, 100.0 cpu, 0.0 io}, id = 41
LogicalTableScan(subset=[rel#40:Subset#0.NONE.[]], table=[[druid,
foo]]): rowcount = 100.0, cumulative cost = {100.0 rows, 101.0 cpu, 0.0 io}, id
= 6
)
org.apache.druid.sql.calcite.planner.UnsupportedSQLQueryException: Query not
supported. Possible error: Aggregation [APPROX_COUNT_DISTINCT_DS_HLL($1, $2,
$3)] is not supported SQL was: select
dim1,
APPROX_COUNT_DISTINCT_DS_HLL (hll, 12, 'HLL_4') cnt
from (
select
dim1,
DS_HLL (dim1) hll
from druid.foo
group by dim1
limit 10
) t
group by dim1
at
org.apache.druid.sql.calcite.planner.QueryHandler.plan(QueryHandler.java:228)
at
org.apache.druid.sql.calcite.planner.DruidPlanner.plan(DruidPlanner.java:234)
at
org.apache.druid.sql.DirectStatement.createPlan(DirectStatement.java:243)
at org.apache.druid.sql.DirectStatement.plan(DirectStatement.java:221)
at
org.apache.druid.sql.DirectStatement.execute(DirectStatement.java:182)
at
org.apache.druid.sql.calcite.QueryTestRunner$ExecuteQuery.runQuery(QueryTestRunner.java:284)
at
org.apache.druid.sql.calcite.QueryTestRunner$ExecuteQuery.run(QueryTestRunner.java:266)
at
org.apache.druid.sql.calcite.QueryTestRunner.run(QueryTestRunner.java:689)
at
org.apache.druid.sql.calcite.QueryTestBuilder.run(QueryTestBuilder.java:316)
at
org.apache.druid.query.aggregation.datasketches.hll.sql.HllSketchSqlAggregatorTest.testAggQuery(HllSketchSqlAggregatorTest.java:171)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.apache.druid.sql.calcite.util.QueryLogHook$1.evaluate(QueryLogHook.java:96)
at
org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:258)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at
org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
at
com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
at
com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
at
com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
at
com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:232)
at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:55)
```
- Any debugging that you have already done
In this case,
org.apache.druid.query.aggregation.datasketches.hll.sql.HllSketchBaseSqlAggregator#toDruidAggregation
can not parse the literal parameter
The following three SQL statements, the first one cannot be parsed, and the
last two can be parsed normally:
```sql
-- can not parse the literal parameter
select
dim1,
APPROX_COUNT_DISTINCT_DS_HLL (hll, 12, 'HLL_4') cnt
from (
select
dim1,
DS_HLL (dim1) hll
from druid.foo
group by dim1
limit 10
) t
group by dim1
-- normal
select
dim1,
APPROX_COUNT_DISTINCT_DS_HLL (hll, 12, 'HLL_4') cnt
from (
select
dim1,
DS_HLL (dim1) hll
from druid.foo
group by dim1
) t
group by dim1
-- normal
select
dim1,
APPROX_COUNT_DISTINCT_DS_HLL (hll) cnt
from (
select
dim1,
DS_HLL (dim1) hll
from druid.foo
group by dim1
limit 10
) t
````
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
