nevzheng opened a new issue, #11949:
URL: https://github.com/apache/gravitino/issues/11949
### Describe the feature
Add a first-class `variant` type to Gravitino's unified type model so
Iceberg V3 `variant` columns are represented natively — as `Types.VariantType`,
not the `ExternalType("variant")` stopgap.
Following from #11927, #11928, #11932, and the discussion in #11929, **this
issue is the canonical thread tracking the implementation of Iceberg V3
`variant` support in Gravitino.** Detailed design and review discussion should
happen here; the prior threads remain as background.
### Motivation
Loading an Iceberg V3 table with a `variant` column through the native
metadata API fails with `UnsupportedOperationException: Unsupported type:
variant` (#11927). The immediate options were:
- **`ExternalType` stopgap (#11928)** — maps `variant` →
`external(variant)`. Loads the table, but `variant` stays an opaque string with
no identity: nothing can branch on it, and external types have caused
downstream problems (`external(serial)` unqueryable via Trino #10957;
`catalogString()` written verbatim into DDL #11805).
- **Native `VariantType` (#11932)** — gives `variant` a real, modeled
identity so each connector maps it deliberately.
Per discussion in #11929 (@jerryshao), we're going with native support and
will land it in a later release rather than shipping the interim `ExternalType`
patch now. #11928 is being closed in favor of #11932.
### Describe the solution
Model `variant` as a parameterless leaf (same shape as `UUIDType`):
- **api** — `Type.Name.VARIANT` + `Types.VariantType`.
- **common** — `JsonUtils` serializes/parses it as the token `"variant"`.
- **catalog-lakehouse-iceberg** — map Iceberg `VariantType` ⇄ Gravitino
`VariantType`.
- **Python client** — mirror `Name.VARIANT`, `Types.VariantType`, and the
serde registry.
- **docs** — unified type reference + per-connector type-mapping tables.
- **connector propagation** — native mapping where the engine has Open
Variant (Iceberg, Paimon, Doris); reject-with-test where it's absent or a
different concept, flippable as engines ship the standard.
No formal design doc — small, well-scoped feature (per @jerryshao in #11929).
Implemented by #11932.
### Additional context
- #11927 — the load bug that surfaced this.
- #11928 — the `ExternalType` stopgap (closing in favor of #11932).
- #11929 — `[DISCUSS]` thread; decision to go native.
- #11932 — the native implementation.
- #10994 — variant on the IRC/JDBC backend (already handled).
- #10957 / #11805 — external-type hazards motivating native modeling.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
