xiangfu0 opened a new pull request, #19990: URL: https://github.com/apache/pinot/pull/19990
Stacked on #19986; will be retargeted to `master` once it merges. ## Summary `ANY_VALUE(expr)` now returns the logical type of `expr` in both engines. The single-stage engine returned STRING for every column, e.g. `"1"` for a BOOLEAN and `"1600000000000"` for a TIMESTAMP. ## How - Adds `ANY_VALUE(expr, 'dataType')`, whose result type is fixed at construction (INT, LONG, FLOAT, DOUBLE, BIG_DECIMAL, BOOLEAN, TIMESTAMP, STRING, BYTES, UUID). Values are read as that type, so segments that store the column as another type stay consistent, and empty input is typed. - Rewrites `ANY_VALUE(expr)` to it with the aggregate rewrite framework: `AggregateFunctionRewriteOptimizer` in the single-stage engine, and a separate `PinotAggregateTypeArgumentRewriteRule` instance (`AggregateAnyValueTypeRewrite`) in the multi-stage engine. JSON maps to STRING in both. `SET inferAnyValueType=false` turns the rewrite off. - The multi-stage rule takes the data type from the type the engine reads the argument as, so a DECIMAL expression gets INT, LONG, DOUBLE or BIG_DECIMAL. Validation rejects a data type that does not match the argument, because the multi-stage result type follows the argument. - `AggregationFunctionUtils.getConvertedFinalResult` handles UUID, and the aggregation-only reducer skips formatting null results (it threw for a null BIG_DECIMAL, TIMESTAMP, BYTES or UUID). ## Compatibility `backward-incompat`: single-stage result types change, and so do unaliased column names (`anyvalue(col)` becomes `anyvalue(col,'INT')`). Older servers reject the 2-argument form, so upgrade servers before brokers, or set `inferAnyValueType=false` until they are upgraded. ## Tests - `AnyValueAggregationFunctionTest`: result types for every data type, empty input, a widened physical type (dictionary and raw), serialized merges, invalid arguments. - `AggregateFunctionRewriteOptimizerTest`, `AnyValueQueriesTest`: rewrite and end-to-end results for every type, group-by, empty input, explicit form, `inferAnyValueType=false`. - `AggregationFunctionUtilsTest`: UUID final result. - `AggregatePlans.json`, `UDFAggregates.json`: multi-stage plans and results for every inferred type, including DECIMAL expressions, FILTER, skip-leaf aggregation, rejected data types and the query option. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
