Jackie-Jiang commented on code in PR #18842:
URL: https://github.com/apache/pinot/pull/18842#discussion_r3531438891
##########
pinot-core/src/main/java/org/apache/pinot/core/query/reduce/GroupByDataTableReducer.java:
##########
@@ -148,6 +148,13 @@ public DataTable mergeDataTablesOnly(String tableName,
DataSchema dataSchema,
Collection<DataTable> dataTables = dataTableMap.values();
// Reuse the regular reduce's merge: builds the IndexedTable of group
keys + intermediate agg state.
IndexedTable indexedTable = getIndexedTable(dataSchema, dataTables,
reducerContext);
+ // Sort/top-K only; keep aggregate values as intermediates (NOT
extractFinalResult). storeFinalResult=true
+ // would mutate _dataSchema.getColumnDataTypes() OBJECT→final-type in
place and replace each row's value
+ // with extractFinalResult(...); _storedColumnDataTypes (populated
lazily on the pre-finalize schema and
+ // never invalidated by finish) still says OBJECT, so the subsequent
buildIntermediateDataTable would
+ // hand a finalized scalar (e.g. Integer for DISTINCTCOUNT) into
serializeIntermediateResult(Set) and
+ // throw ClassCastException.
+ indexedTable.finish(true, false);
Review Comment:
Since we are keeping intermediate results, sort is likely not needed
##########
pinot-core/src/main/java/org/apache/pinot/core/query/reduce/GroupByDataTableReducer.java:
##########
@@ -148,6 +148,13 @@ public DataTable mergeDataTablesOnly(String tableName,
DataSchema dataSchema,
Collection<DataTable> dataTables = dataTableMap.values();
// Reuse the regular reduce's merge: builds the IndexedTable of group
keys + intermediate agg state.
IndexedTable indexedTable = getIndexedTable(dataSchema, dataTables,
reducerContext);
+ // Sort/top-K only; keep aggregate values as intermediates (NOT
extractFinalResult). storeFinalResult=true
Review Comment:
Let's document the general contract, instead of the implementation details.
Here we want to keep the intermediate result
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
