xiangfu0 opened a new pull request, #20027: URL: https://github.com/apache/pinot/pull/20027
## What Adds the `VARIANT` logical type to `pinot-spi`. VARIANT is stored as variable-width `BYTES` holding a Pinot-owned `PVAR` envelope around the Parquet Variant metadata and value buffers. ## Changes - `FieldSpec.DataType.VARIANT` (stored type `BYTES`, appended last so existing ordinals don't shift), with capability methods `supportsEquality/Hashing/Ordering/MinMax/DirectAggregation/PatternMatching`. `equals`, `hashCode` and `compare` throw for VARIANT so a raw value can never be compared by accident. - `VariantEnvelope`: the versioned `PVAR` framing, validation and encode/decode, frozen with golden-byte tests. - `Schema` accepts VARIANT only for dimension fields and tracks `hasVariantColumn()`. `FieldSpec` compares the default-null sentinel structurally, not through value equality. - `PinotDataType.VARIANT`. - `docs/design/VARIANT.md`: the design for the whole stack (format, null semantics, ownership boundaries, rollout contract). - Two small type-enumeration follow-ups that would otherwise fail as soon as the enum constant exists: `TableIndexingTest` skips VARIANT, and the multi-stage `TypeFactory` maps it to Calcite `SqlTypeName.VARIANT`. ## Depends on Nothing; this is the bottom of the stack. ## About this PR This is **part 1 of 8** splitting #19101 (end-to-end Parquet VARIANT support) into layered PRs. #19101 stays open as the umbrella with the full feature description; [`docs/design/VARIANT.md`](https://github.com/apache/pinot/blob/xiangfu0/variant-split/01-spi-foundation/docs/design/VARIANT.md) (added in 1/8) is the design reference. The base branch is the previous PR in the stack, so the diff below is exactly this layer. Each layer compiles and passes its own tests on top of the one below it, and the top of the stack has the same tree as #19101. > VARIANT is not usable end to end until the whole stack has landed. Raw-value opacity guards arrive in 5/8 (single-stage) and 6/8 (multi-stage), so please don't create VARIANT columns on a build that has only part of the stack. The stack should land within one release. ### Full stack (merge bottom → top) _filled in once all PRs are open_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
