Vamsi-klu opened a new pull request, #18979:
URL: https://github.com/apache/pinot/pull/18979
`json_extract_scalar` was only implemented as a transform function
(`JsonExtractScalarTransformFunction`), so it could not be resolved in the
multi-stage query engine or in ad-hoc scalar contexts — a
`SELECT ... json_extract_scalar(...)` across a JOIN failed with
"Unsupported function: JSONEXTRACTSCALAR".
This registers `jsonExtractScalar` as a `@ScalarFunction` in `pinot-common`'s
`JsonFunctions`, with full result-type parity to the transform:
- Single values: `INT`, `LONG`, `FLOAT`, `DOUBLE`, `BIG_DECIMAL`, `BOOLEAN`,
`TIMESTAMP`, `STRING`, `JSON`, `BYTES`.
- Arrays: `INT`, `LONG`, `FLOAT`, `DOUBLE`, `BIG_DECIMAL`, `STRING` variants.
- Coercion mirrors the transform exactly: `BOOLEAN` follows Pinot's numeric
convention and returns stored `INT` (0/1); `TIMESTAMP` accepts numeric
epoch
millis directly and ISO-8601 strings via `TimestampUtils`; `BIG_DECIMAL` /
`STRING` / `JSON` read through a BigDecimal-preserving parser to avoid
float
precision loss.
- Throw-on-unresolved semantics match the transform: an unresolved
single-value path without a default throws; a multi-value path yields an
empty array; a null element inside a resolved array throws unless a default
is supplied. Malformed JSON is treated as unresolved.
Because `pinot-common` cannot depend on `pinot-core`, a BigDecimal-based long
parser stands in for `NumberUtils.parseJsonLong`, reproducing its
truncate-toward-zero and exponent handling for JSON numeric strings.
The change is add-only against the existing `JsonFunctions` (the `jsonPath*`
methods, including `master`'s `canExtractJsonPath()` fast-path, are
untouched).
The multi-stage negative test asserting `json_extract_scalar` is unsupported
is
removed, since it now resolves. `JsonFunctionsTest` is extended across all
result types (happy / default / throw paths, arrays incl. null elements,
malformed JSON, precision, unsupported type).
Supersedes #16910 (revived and rebased on current `master`, with the parity
and coverage gaps from that PR's review closed).
closes #16486
Co-authored-by: Manik Somayaji <[email protected]>
Generated-by: Claude Code (Opus 4.8)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
