nevzheng opened a new issue, #11954: URL: https://github.com/apache/gravitino/issues/11954
### Describe the feature Allow the Iceberg table **format version** to be selected (`2` or `3`) when creating a table through the Gravitino relational API, with validation and a sensible default. Today `format-version` is an unvalidated pass-through property: the effective default is `2`, there is no first-class/validated way to ask for `3`, and misconfigurations surface as low-level Iceberg errors rather than clear Gravitino ones. Format version **3** is a prerequisite for Iceberg's V3 types (e.g. `variant`; see the [Iceberg table spec](https://iceberg.apache.org/spec/) — "Version 3"). Without a supported way to create v3 tables, those types can't be written through Gravitino. ### Motivation Creating a table that needs v3 fails with a downstream Iceberg error, because the format version defaults to `2`. For example, a `variant` column (a V3 type) rejected at v2: ``` Failed to operate object [t_variant] operation [CREATE] under [db], reason [Invalid schema for v2: - Invalid type for payload: variant is not supported until v3] ``` And because the property is an unvalidated pass-through, an empty or invalid value also leaks a raw Iceberg/parse error instead of a clear message: ``` Failed to operate table(s) [t] operation [CREATE] under schema [db], reason [For input string: ""] ``` Users have no documented, validated way to select v3, and no clear signal about which versions are supported. ### Describe the solution Make `format-version` a first-class, validated Iceberg table property: - Accept only `2` or `3`; reject other values at the Gravitino layer with a clear message, e.g. `Invalid value: '4' for property: 'format-version'`. - Default to `2` when unset or empty (`empty -> 2`, `2 -> 2`, `3 -> 3`), so existing behavior is unchanged and v3 is opt-in. - Document the property and add create examples for both v2 and v3. - Add unit tests and REST/IRC integration tests that create a v3 table (incl. a `variant` column) through the Gravitino API and read it back. Example — creating a v3 table via REST: ```bash curl -X POST .../schemas/db/tables -H 'Content-Type: application/json' -d '{ "name": "t_v3", "columns": [{"name": "id", "type": "integer", "nullable": false}, {"name": "payload", "type": "variant", "nullable": true}], "properties": {"format-version": "3"} }' ``` ### Additional context - Related to #11949 (native `variant` type — the primary V3 type motivating this) and its implementation in #11932. This issue is the general "select/enable format version 3" mechanism; it is not limited to variant. - Iceberg format versions: v3 support lands in Iceberg 1.9+, and Gravitino currently builds against Iceberg 1.11.0. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
