raghavyadav01 opened a new pull request, #19607: URL: https://github.com/apache/pinot/pull/19607
Fixes #19466. A materialized child is synthesized as a dimension of the key's **stored** type, so a key declared `TIMESTAMP`, `BOOLEAN`, `JSON` or `UUID` lands on disk as the `LONG`, `INT`, `STRING` or `BYTES` it happens to be stored as. The declaration is honored for coercion and then discarded, taking with it every operator that depends on knowing which type it actually was. The absent-doc default came from the same place, so a document missing the key read the *stored* type's default rather than the declared one — a different value from what `OpenStructDataSource#getValueFieldSpec` resolves for a key absent from the segment entirely. The same key could read two different defaults depending on whether it was missing from the document or from the segment. ### Change Build the child field spec from the declared spec when the parent declares one — its data type and its default null value, including a custom one. `MutableOpenStructIndex` takes the same rule for its absent-doc default, so a document resolves identically either side of `seal()`. Nothing about the physical layout changes: values are still written as the stored type. Only what the column reports itself to be. An undeclared key is unchanged — still a single-value dimension of the inferred stored type. ``` declared before after TIMESTAMP → LONG → TIMESTAMP BOOLEAN → INT → BOOLEAN UUID → BYTES → UUID STRING, default → "null" → the declared default ``` Single-value either way for now: the write path has no multi-value creator, and a declared multi-value key does not work today regardless — a list value fails inference and falls back to STRING. ### Compatibility Segments written before this change record the stored type for a declared key, so a table with segments from both sides reports two types for that key until the older ones are refreshed. OPEN_STRUCT has not shipped in a release yet, which is why this seems worth doing now rather than leaving the declaration half-honored. ### Tests `OpenStructDeclaredChildSpecTest` — UUID, TIMESTAMP and BOOLEAN keep their type; a custom default is what an absent doc reads and what stats collect; a declared type beats the inferred one; an undeclared key is untouched. Existing 69 OPEN_STRUCT tests in `pinot-segment-local` pass unchanged, including the consuming/sealed parity suite. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
