yashmayya opened a new pull request, #19769:
URL: https://github.com/apache/pinot/pull/19769
This PR depends on #19767, which depends on #19766. It contains their
commits. Only the last commit is new.
## The bug
With `explainAskingServers` (or the broker config
`pinot.query.multistage.explain.include.segment.plan`), the broker gets the
plan of each leaf stage from the servers. Then `PlanNodeToRelConverter`
converts the plan nodes back to Calcite nodes.
`visitAggregate` builds the group key only from
`AggregateNode.getGroupKeys()` and ignores `getGroupingSets()`. So a GROUPING
SETS / ROLLUP / CUBE aggregate that runs as a multi-stage operator in a leaf
stage shows as a plain GROUP BY.
For example, with `is_partitioned_by_group_by_keys`, the ROLLUP aggregate of
this query runs in the leaf stage, above the inner aggregate:
```sql
SELECT Carrier, Origin, MAX(v)
FROM (
SELECT /*+ aggOptions(is_partitioned_by_group_by_keys='true') */ Carrier,
Origin, Dest, AVG(DepDelay) AS v
FROM mytable GROUP BY Carrier, Origin, Dest)
GROUP BY ROLLUP(Carrier, Origin)
```
`EXPLAIN PLAN WITHOUT IMPLEMENTATION` shows this aggregate as
`PinotLogicalAggregate(group=[{0, 1}], groups=[[{0, 1}, {0}, {}]],
agg#0=[MAX($2)], aggType=[LEAF])`. EXPLAIN with the plan from the servers shows
the leaf stage as:
```
PinotLogicalExchange(distribution=[hash[0, 1, 2]])
LogicalAggregate(group=[{0, 1}], agg#0=[MAX($2)])
LogicalProject(Carrier=[$0], Origin=[$2], $f2=[DIVIDE($3, $4)])
LeafStageCombineOperator(table=[mytable])
```
## The fix
In `visitAggregate`, if the node has grouping sets, build the group key with
`RelBuilder.groupKey(ImmutableBitSet, Iterable<ImmutableBitSet>)`. Each
grouping set holds indexes into the group keys, so the converter maps them to
input fields first. Plain GROUP BY does not change.
Only the EXPLAIN output changes. Query execution does not use this
conversion.
## Testing
- `PlanNodeToRelConverterTest.testAggregateGroupingSets` converts a ROLLUP
aggregate whose group keys are not the first input fields. Without this change,
it fails, because the plan has no `groups`.
- I ran EXPLAIN in a temporary integration test with the broker config
above. Before this change, the leaf stage showed a plain GROUP BY. After it,
the leaf stage shows the grouping sets:
```
PinotLogicalExchange(distribution=[hash[0, 1, 2]])
LogicalAggregate(group=[{0, 1}], groups=[[{0, 1}, {0}, {}]],
agg#0=[MAX($2)])
LogicalProject(Carrier=[$0], Origin=[$2], $f2=[DIVIDE($3, $4)])
LeafStageCombineOperator(table=[mytable])
```
- All 2660 `pinot-query-planner` tests pass. `spotless`, `checkstyle` and
`license` are clean.
## Not in this PR
- If the single-stage engine runs the ROLLUP aggregate (for example, ROLLUP
directly on a table), the server plan shows `GroupBy(groupKeys=[[Carrier,
Origin]], ...)`. `GroupByOperator` does not print the grouping sets.
- An aggregate above `UNNEST` in a leaf stage shows as `UnknownAggregate`,
with or without grouping sets. `visitUnnest` keeps only the unnested columns,
so the aggregate cannot find its input fields.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
