yashmayya opened a new pull request, #19767:
URL: https://github.com/apache/pinot/pull/19767
This PR depends on #19766 and contains its commit. Only the last commit is
new.
## The bug
With `explainAskingServers` (or the broker config
`pinot.query.multistage.explain.include.segment.plan`), the broker gets the
plan of each leaf stage from the servers. Then `PlanNodeToRelConverter`
converts the plan nodes back to Calcite nodes.
`visitAggregate` ignores `AggregateNode.getFilterArgs()`, so EXPLAIN does
not show the `FILTER` of an aggregate call. No call uses the filter column, so
`RelBuilder` also removes this column from the input project.
For example, with `is_partitioned_by_group_by_keys`, both aggregates of this
query run in the leaf stage:
```sql
SELECT Carrier, MAX(v) FILTER (WHERE v IN (1, 2, 3))
FROM (
SELECT /*+ aggOptions(is_partitioned_by_group_by_keys='true') */ Carrier,
Origin, AVG(DepDelay) AS v
FROM mytable GROUP BY Carrier, Origin)
GROUP BY Carrier
```
EXPLAIN shows the leaf stage as:
```
PinotLogicalExchange(distribution=[hash[0]])
LogicalAggregate(group=[{0}], agg#0=[MAX($1)])
LogicalProject(Carrier=[$0], $f1=[DIVIDE($2, $3)])
LeafStageCombineOperator(table=[mytable])
```
`SUM(CASE WHEN cond THEN 1 ELSE 0 END)` has the same problem, because
Calcite changes it to `COUNT() FILTER (WHERE cond)`.
## The fix
In `visitAggregate`, add the filter to each aggregate call that has a filter
arg, with `RelBuilder.AggCall.filter()`. The input project then keeps the
filter column.
Only the EXPLAIN output changes. Query execution does not use this
conversion.
## Testing
- `PlanNodeToRelConverterTest.testAggregateFilter` converts an aggregate
with one plain and one filtered `COUNT`. Without this change, it fails, because
the plan has no `FILTER` and the project loses the filter column.
- I ran EXPLAIN in a temporary integration test with the default planner
rules. I used the query above, the `SUM(CASE ...)` form, and a mix of calls
with and without `FILTER`. Before this change, the leaf stages showed no
filter. After it, they show all filters. For the query above:
```
PinotLogicalExchange(distribution=[hash[0]])
LogicalAggregate(group=[{0}], agg#0=[MAX($1) FILTER $2])
LogicalProject(Carrier=[$0], $f1=[DIVIDE($2, $3)],
$f2=[SEARCH(DIVIDE($2, $3), Sarg[1.0E0:DOUBLE, 2.0E0:DOUBLE,
3.0E0:DOUBLE]:DOUBLE)])
LeafStageCombineOperator(table=[mytable])
```
- All 2659 `pinot-query-planner` tests pass. `spotless`, `checkstyle` and
`license` are clean.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
