Jackie-Jiang opened a new pull request, #19983: URL: https://github.com/apache/pinot/pull/19983
## Summary With null handling enabled, the single-stage engine ignores `@ScalarFunction(nullableParameters = true)`. `ScalarTransformFunctionWrapper` invokes the function with the values stored in the null rows (e.g. the default null value of a column), and marks a result row null whenever any argument is null in it. The multi-stage engine and ingestion transforms instead pass `null` and let the function decide. The single-stage support added in #10562 was removed in #11008 together with the `transformToXxValuesSVWithNull` APIs, and was not ported to the new design. When null handling is enabled and the function accepts null arguments, the wrapper now: - Passes `null` for the null rows of the non-literal arguments, written into a copy since the values can be the argument's own buffer. - Takes the null rows of the result from the rows where the function returns `null`, instead of the union of the arguments' null bitmaps. - Caches the evaluation per value block, so reading the null bitmap before or after the values invokes the function once. Whether to invoke with nulls is decided at `init` from the query's null handling, not by the API the caller uses, so it also holds when the function is nested inside another function (the problem #11008 fixed for `CASE`). Functions without `nullableParameters`, and every function when null handling is disabled, behave as before. Related changes: - `prefixesWithPrefix` and `suffixesWithSuffix` accept null arguments but threw an NPE on a null input. They now return `null` and are annotated `@Nullable`, which also fixes the multi-stage engine and ingestion. - A multi-value scalar function returning `null` now yields an empty-array placeholder instead of a `null` inner array (or an NPE for boolean and timestamp arrays). ## Behavior change With null handling enabled in the single-stage engine, functions with `nullableParameters` now decide the null rows of their result, matching the multi-stage engine. For example, `jsonPathString(col, '$.a', 'dflt')` returns `'dflt'` instead of `NULL` for a null `col`, and the sketch builders such as `toHLL(col)` return an empty sketch instead of `NULL`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
