xiangfu0 commented on PR #19264: URL: https://github.com/apache/pinot/pull/19264#issuecomment-5517608065
Fixed a CI failure (`StreamingGroupByCombineOperatorTest.testGroupingSetsKeyColumnLayout`) in cd720a3 — a real regression, not a flake. **Root cause:** only the standard `GroupByCombineOperator` performs the merge-time derive that turns emitted base groups back into grouping-set records. The **streaming** combine (MSE leaf stages, `streamingGroupByFlushThreshold > 0`) flushes incrementally and can't derive over the fully-merged base table, and the **sorted-aggregate** combine has its own merge that doesn't derive either. In those paths, base-aggregation segments emitted base groups *without* the `$groupingId` column, which the streaming operator passed straight through — producing rows one column short of the schema. **Fix:** gate `isGroupingSetsBaseAggregation()` off when the streaming or sorted-aggregate combine would run, so those queries expand per row and emit the `$groupingId` discriminator directly. (Grouping-set queries always take the unsafe-trim path, so the sorted condition is defensive; the streaming condition is the load-bearing part. `InstancePlanMakerImplV2` sets `streamingGroupByFlushThreshold` on the context before building segment plans, so the segment operator sees it.) Also updated the test to set `streamingGroupByFlushThreshold` on the query context, mirroring the real MSE leaf setup so the segment operators expand as they do in production (they were previously built from a context that never occurs in the real streaming path). Verified: all 9 `StreamingGroupByCombineOperatorTest`, the `DeriveGroupingSetsTest` unit tests, and all 76 `GroupingSetsQueriesTest` integration tests pass against current master; checkstyle/license clean. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
