This is an automated email from the ASF dual-hosted git repository.
roryqi pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/gravitino.git
The following commit(s) were added to refs/heads/main by this push:
new 799c50c737 [#11954] feat(iceberg): enable creating format-version 3
(V3) tables via the Gravitino API (#11957)
799c50c737 is described below
commit 799c50c737b2f2367fa7481fedb5e20084e6d26a
Author: Nevin Zheng <[email protected]>
AuthorDate: Thu Jul 9 01:18:38 2026 -0700
[#11954] feat(iceberg): enable creating format-version 3 (V3) tables via
the Gravitino API (#11957)
### What changes were proposed in this pull request?
Enable creating Iceberg **format-version 3 (V3)** tables through the
Gravitino relational API — the format version required for Iceberg's V3
types such as `variant` (added in #11932). This makes `format-version` a
first-class, validated table property; everything else follows from
that:
- **Select the version** — Gravitino accepts creating tables at versions
**1–4** (the range the bundled Iceberg 1.11.0 can write); other values
are rejected at the Gravitino layer with a clear message instead of a
low-level Iceberg error.
- **Sensible default** — unset or empty resolves to `2` (`empty → 2`, `2
→ 2`, `3 → 3`), stamped by Gravitino at create time, so existing
behavior is unchanged and v3 is opt-in.
- **Tests** — unit (validation over `1–4` + create-property resolution)
and REST/IRC integration: create a `variant` column via the Gravitino
API at v3 and read it back.
- **Docs** — catalog property table + two OpenAPI create examples
(`IcebergTableCreate` v2, `IcebergVariantTableCreate` v3).
Per-version support: **v1/v2** are fully supported (v2 is the community
default since Iceberg 1.4.0); **v3** is required for V3 types and its
type coverage is being added incrementally (`variant` now); **v4** is
creatable but not yet a finalized Iceberg spec.
### Why are the changes needed?
Format version 3 is a prerequisite for Iceberg's V3 types (e.g.
`variant`; see the [Iceberg table
spec](https://iceberg.apache.org/spec/) — "Version 3"). Before this,
`format-version` was an unvalidated free-form string with no default or
documentation, so creating a v3 table was not a supported, discoverable
operation — attempts that needed v3 failed with confusing downstream
Iceberg errors, and nothing covered the write path to a REST/IRC
backend.
Fixes #11954. Related to #11949 (the umbrella tracking issue for the
whole variant feature).
### Does this PR introduce _any_ user-facing change?
- You can now create Iceberg tables at **format versions 1–4** through
the Gravitino API (default `2`) — e.g. `"format-version": "3"` for V3
types like `variant`.
- `format-version` is now validated: allowed values `1`/`2`/`3`/`4`,
unset/empty defaults to `2`, other values rejected with a clear error.
- New OpenAPI examples for creating Iceberg tables.
### How was this patch tested?
- Unit tests: property validation (default; valid `""`/`1`/`2`/`3`/`4`;
invalid `0`/`5`/`100`/`-1`/`INT_MAX`/`INT_MIN`/non-numeric) and
create-property mapping (`absent/""/2/3`).
- Integration tests (REST/IRC): variant column created via the Gravitino
API at v3 and read back; variant-without-v3 rejected; unset/`2`/empty
resolve to `2`.
- Verified end-to-end in the Gravitino playground against an Iceberg
REST catalog (read back over the `/iceberg` endpoint); confirmed Iceberg
1.11 creates v1–v4 and rejects v5.
- `./gradlew :docs:build` validates the OpenAPI spec.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.8 <[email protected]>
---
.../catalog/lakehouse/iceberg/IcebergTable.java | 11 ++
.../iceberg/IcebergTablePropertiesMetadata.java | 74 +++++++++++++-
.../lakehouse/iceberg/TestIcebergTable.java | 19 ++++
.../TestIcebergTablePropertiesMetadata.java | 87 ++++++++++++++++
.../integration/test/CatalogIcebergBaseIT.java | 112 +++++++++++++++++++++
docs/lakehouse-iceberg-catalog.md | 26 ++---
docs/open-api/tables.yaml | 52 ++++++++++
7 files changed, 366 insertions(+), 15 deletions(-)
diff --git
a/catalogs/catalog-lakehouse-iceberg/src/main/java/org/apache/gravitino/catalog/lakehouse/iceberg/IcebergTable.java
b/catalogs/catalog-lakehouse-iceberg/src/main/java/org/apache/gravitino/catalog/lakehouse/iceberg/IcebergTable.java
index 3f2f54c1b3..68d6cfc0df 100644
---
a/catalogs/catalog-lakehouse-iceberg/src/main/java/org/apache/gravitino/catalog/lakehouse/iceberg/IcebergTable.java
+++
b/catalogs/catalog-lakehouse-iceberg/src/main/java/org/apache/gravitino/catalog/lakehouse/iceberg/IcebergTable.java
@@ -28,6 +28,7 @@ import java.util.Map;
import lombok.Getter;
import lombok.ToString;
import org.apache.commons.lang3.ArrayUtils;
+import org.apache.commons.lang3.StringUtils;
import org.apache.gravitino.catalog.lakehouse.iceberg.converter.ConvertUtil;
import
org.apache.gravitino.catalog.lakehouse.iceberg.converter.FromIcebergPartitionSpec;
import
org.apache.gravitino.catalog.lakehouse.iceberg.converter.FromIcebergSortOrder;
@@ -77,6 +78,16 @@ public class IcebergTable extends BaseTable {
private IcebergTable() {}
public static Map<String, String> rebuildCreateProperties(Map<String,
String> createProperties) {
+ // Gravitino owns the default Iceberg table format version: when it is not
explicitly set, stamp
+ // ICEBERG_DEFAULT_FORMAT_VERSION rather than relying on the Iceberg
library's version-dependent
+ // default or letting a blank value fail to parse downstream.
+ String formatVersion =
createProperties.get(IcebergTablePropertiesMetadata.FORMAT_VERSION);
+ if (StringUtils.isBlank(formatVersion)) {
+ createProperties.put(
+ IcebergTablePropertiesMetadata.FORMAT_VERSION,
+
String.valueOf(IcebergTablePropertiesMetadata.ICEBERG_DEFAULT_FORMAT_VERSION));
+ }
+
String provider = createProperties.get(PROP_PROVIDER);
if (ICEBERG_PARQUET_FILE_FORMAT.equalsIgnoreCase(provider)) {
createProperties.put(DEFAULT_FILE_FORMAT, ICEBERG_PARQUET_FILE_FORMAT);
diff --git
a/catalogs/catalog-lakehouse-iceberg/src/main/java/org/apache/gravitino/catalog/lakehouse/iceberg/IcebergTablePropertiesMetadata.java
b/catalogs/catalog-lakehouse-iceberg/src/main/java/org/apache/gravitino/catalog/lakehouse/iceberg/IcebergTablePropertiesMetadata.java
index d336ad15e4..b2eff94881 100644
---
a/catalogs/catalog-lakehouse-iceberg/src/main/java/org/apache/gravitino/catalog/lakehouse/iceberg/IcebergTablePropertiesMetadata.java
+++
b/catalogs/catalog-lakehouse-iceberg/src/main/java/org/apache/gravitino/catalog/lakehouse/iceberg/IcebergTablePropertiesMetadata.java
@@ -21,10 +21,14 @@ package org.apache.gravitino.catalog.lakehouse.iceberg;
import static
org.apache.gravitino.connector.PropertyEntry.stringImmutablePropertyEntry;
import static
org.apache.gravitino.connector.PropertyEntry.stringReservedPropertyEntry;
+import com.google.common.base.Preconditions;
import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Maps;
import java.util.List;
import java.util.Map;
+import java.util.Set;
+import org.apache.commons.lang3.StringUtils;
import org.apache.gravitino.connector.BasePropertiesMetadata;
import org.apache.gravitino.connector.PropertyEntry;
import org.apache.iceberg.TableProperties;
@@ -42,6 +46,23 @@ public class IcebergTablePropertiesMetadata extends
BasePropertiesMetadata {
public static final String FORMAT_VERSION = IcebergConstants.FORMAT_VERSION;
public static final String DISTRIBUTION_MODE =
TableProperties.WRITE_DISTRIBUTION_MODE;
+ /**
+ * The default Iceberg table format version Gravitino applies when {@link
#FORMAT_VERSION} is not
+ * explicitly set. Gravitino owns this default rather than deferring to the
Iceberg library's own
+ * version-dependent default, and stamps it onto the table at creation.
+ */
+ public static final int ICEBERG_DEFAULT_FORMAT_VERSION = 2;
+
+ /**
+ * The Iceberg table format versions Gravitino allows creating: {@code
1}–{@code 4}, the range the
+ * bundled Iceberg version (1.11.0) can write. Gravitino is not more
restrictive than Iceberg for
+ * the create passthrough; each version's feature set is defined by the
Iceberg spec (v1/v2/v3 are
+ * adopted, v3 is required for V3 types such as {@code variant}, and v4 is
under active
+ * development and not yet finalized). An unset (empty) value defaults to
{@link
+ * #ICEBERG_DEFAULT_FORMAT_VERSION}. Extend this set as newer Iceberg writer
versions ship.
+ */
+ public static final Set<Integer> SUPPORTED_FORMAT_VERSIONS =
ImmutableSet.of(1, 2, 3, 4);
+
private static final Map<String, PropertyEntry<?>> PROPERTIES_METADATA;
static {
@@ -65,8 +86,7 @@ public class IcebergTablePropertiesMetadata extends
BasePropertiesMetadata {
stringReservedPropertyEntry(
IDENTIFIER_FIELDS, "The identifier field(s) for defining the
table", false),
stringReservedPropertyEntry(DISTRIBUTION_MODE, "Write distribution
mode", false),
- stringImmutablePropertyEntry(
- FORMAT_VERSION, "The Iceberg table format version, ", false,
null, false, false),
+ formatVersionPropertyEntry(),
stringImmutablePropertyEntry(
PROVIDER,
"Iceberg provider for Iceberg table fileFormat, such as
Parquet, Orc, Avro, or Iceberg",
@@ -81,4 +101,54 @@ public class IcebergTablePropertiesMetadata extends
BasePropertiesMetadata {
protected Map<String, PropertyEntry<?>> specificPropertyEntries() {
return PROPERTIES_METADATA;
}
+
+ /**
+ * Builds the property entry for {@link #FORMAT_VERSION}, an immutable
property that accepts an
+ * unset value or one of {@link #SUPPORTED_FORMAT_VERSIONS} and defaults to
{@link
+ * #ICEBERG_DEFAULT_FORMAT_VERSION}.
+ *
+ * @return the {@code format-version} property entry.
+ */
+ private static PropertyEntry<Integer> formatVersionPropertyEntry() {
+ return new PropertyEntry.Builder<Integer>()
+ .withName(FORMAT_VERSION)
+ .withDescription(
+ "The Iceberg table format version. Gravitino supports creating
tables at versions 1 to "
+ + "4 (the range the bundled Iceberg version can write) and
defaults to 2 when unset. "
+ + "Version 3 is required for V3 types such as variant; version
4 is not yet a "
+ + "finalized Iceberg spec.")
+ .withRequired(false)
+ .withImmutable(true)
+ .withJavaType(Integer.class)
+ .withDefaultValue(ICEBERG_DEFAULT_FORMAT_VERSION)
+ .withDecoder(IcebergTablePropertiesMetadata::decodeFormatVersion)
+ .withEncoder(String::valueOf)
+ .withHidden(false)
+ .withReserved(false)
+ .build();
+ }
+
+ /**
+ * Decodes and validates a user-supplied {@link #FORMAT_VERSION} value. An
unset (null or blank)
+ * value is allowed and resolves to {@link #ICEBERG_DEFAULT_FORMAT_VERSION};
otherwise the value
+ * must be an integer in {@link #SUPPORTED_FORMAT_VERSIONS}.
+ *
+ * @param value the raw property value.
+ * @return the parsed format version, or {@link
#ICEBERG_DEFAULT_FORMAT_VERSION} when the value is
+ * unset.
+ * @throws IllegalArgumentException if the value is neither blank nor a
version in {@link
+ * #SUPPORTED_FORMAT_VERSIONS}.
+ */
+ private static Integer decodeFormatVersion(String value) {
+ if (StringUtils.isBlank(value)) {
+ return ICEBERG_DEFAULT_FORMAT_VERSION;
+ }
+ int version = Integer.parseInt(value.trim());
+ Preconditions.checkArgument(
+ SUPPORTED_FORMAT_VERSIONS.contains(version),
+ "Unsupported Iceberg format-version: %s, supported versions are %s",
+ version,
+ SUPPORTED_FORMAT_VERSIONS);
+ return version;
+ }
}
diff --git
a/catalogs/catalog-lakehouse-iceberg/src/test/java/org/apache/gravitino/catalog/lakehouse/iceberg/TestIcebergTable.java
b/catalogs/catalog-lakehouse-iceberg/src/test/java/org/apache/gravitino/catalog/lakehouse/iceberg/TestIcebergTable.java
index 5b1b0a2874..c36f1b7855 100644
---
a/catalogs/catalog-lakehouse-iceberg/src/test/java/org/apache/gravitino/catalog/lakehouse/iceberg/TestIcebergTable.java
+++
b/catalogs/catalog-lakehouse-iceberg/src/test/java/org/apache/gravitino/catalog/lakehouse/iceberg/TestIcebergTable.java
@@ -705,6 +705,25 @@ public class TestIcebergTable {
Assertions.assertEquals(distributionName,
DistributionMode.RANGE.modeName());
}
+ @Test
+ void testRebuildCreatePropertiesResolvesFormatVersion() {
+ // The format-version mapping Gravitino applies at create time:
+ // absent -> 2, empty -> 2, "2" -> 2, "3" -> 3.
+ Assertions.assertEquals("2", rebuiltFormatVersion(null));
+ Assertions.assertEquals("2", rebuiltFormatVersion(""));
+ Assertions.assertEquals("2", rebuiltFormatVersion("2"));
+ Assertions.assertEquals("3", rebuiltFormatVersion("3"));
+ }
+
+ private static String rebuiltFormatVersion(String input) {
+ Map<String, String> properties = new HashMap<>();
+ if (input != null) {
+ properties.put(IcebergTablePropertiesMetadata.FORMAT_VERSION, input);
+ }
+ return IcebergTable.rebuildCreateProperties(properties)
+ .get(IcebergTablePropertiesMetadata.FORMAT_VERSION);
+ }
+
protected static String genRandomName() {
return UUID.randomUUID().toString().replace("-", "");
}
diff --git
a/catalogs/catalog-lakehouse-iceberg/src/test/java/org/apache/gravitino/catalog/lakehouse/iceberg/TestIcebergTablePropertiesMetadata.java
b/catalogs/catalog-lakehouse-iceberg/src/test/java/org/apache/gravitino/catalog/lakehouse/iceberg/TestIcebergTablePropertiesMetadata.java
new file mode 100644
index 0000000000..08ae281b91
--- /dev/null
+++
b/catalogs/catalog-lakehouse-iceberg/src/test/java/org/apache/gravitino/catalog/lakehouse/iceberg/TestIcebergTablePropertiesMetadata.java
@@ -0,0 +1,87 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.catalog.lakehouse.iceberg;
+
+import com.google.common.collect.ImmutableMap;
+import java.util.stream.Stream;
+import org.apache.gravitino.catalog.PropertiesMetadataHelpers;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.MethodSource;
+import org.junit.jupiter.params.provider.ValueSource;
+
+public class TestIcebergTablePropertiesMetadata {
+
+ private IcebergTablePropertiesMetadata metadata;
+
+ @BeforeEach
+ void setUp() {
+ metadata = new IcebergTablePropertiesMetadata();
+ }
+
+ @Test
+ void testFormatVersionDefaultsToIcebergDefault() {
+ // Gravitino owns the default format version (2).
+ Assertions.assertEquals(
+ IcebergTablePropertiesMetadata.ICEBERG_DEFAULT_FORMAT_VERSION,
+
metadata.getDefaultValue(IcebergTablePropertiesMetadata.FORMAT_VERSION));
+ }
+
+ @Test
+ void testEmptyFormatVersionResolvesToDefault() {
+ // An unset (empty) value resolves to the Gravitino default via the
decoder.
+ Assertions.assertEquals(
+ IcebergTablePropertiesMetadata.ICEBERG_DEFAULT_FORMAT_VERSION,
+ metadata.getOrDefault(
+ ImmutableMap.of(IcebergTablePropertiesMetadata.FORMAT_VERSION, ""),
+ IcebergTablePropertiesMetadata.FORMAT_VERSION));
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"", "1", "2", "3", "4"})
+ void testFormatVersionAcceptsValidValues(String value) {
+ // Empty defers to the default; 1-4 are the versions the bundled Iceberg
can write.
+ Assertions.assertDoesNotThrow(() -> validateFormatVersion(value));
+ }
+
+ @ParameterizedTest
+ @MethodSource("invalidFormatVersions")
+ void testFormatVersionRejectsInvalidValues(String value) {
+ Assertions.assertThrows(IllegalArgumentException.class, () ->
validateFormatVersion(value));
+ }
+
+ private static Stream<String> invalidFormatVersions() {
+ // Just outside the range (0, 5) and clearly out of range / non-numeric.
+ return Stream.of(
+ "0",
+ "5",
+ "100",
+ "-1",
+ String.valueOf(Integer.MAX_VALUE),
+ String.valueOf(Integer.MIN_VALUE),
+ "not-a-number");
+ }
+
+ private void validateFormatVersion(String value) {
+ PropertiesMetadataHelpers.validatePropertyForCreate(
+ metadata,
ImmutableMap.of(IcebergTablePropertiesMetadata.FORMAT_VERSION, value));
+ }
+}
diff --git
a/catalogs/catalog-lakehouse-iceberg/src/test/java/org/apache/gravitino/catalog/lakehouse/iceberg/integration/test/CatalogIcebergBaseIT.java
b/catalogs/catalog-lakehouse-iceberg/src/test/java/org/apache/gravitino/catalog/lakehouse/iceberg/integration/test/CatalogIcebergBaseIT.java
index f7b6770ca4..eb568ae126 100644
---
a/catalogs/catalog-lakehouse-iceberg/src/test/java/org/apache/gravitino/catalog/lakehouse/iceberg/integration/test/CatalogIcebergBaseIT.java
+++
b/catalogs/catalog-lakehouse-iceberg/src/test/java/org/apache/gravitino/catalog/lakehouse/iceberg/integration/test/CatalogIcebergBaseIT.java
@@ -51,6 +51,7 @@ import org.apache.gravitino.auth.AuthConstants;
import org.apache.gravitino.catalog.lakehouse.iceberg.IcebergCatalogBackend;
import
org.apache.gravitino.catalog.lakehouse.iceberg.IcebergSchemaPropertiesMetadata;
import org.apache.gravitino.catalog.lakehouse.iceberg.IcebergTable;
+import
org.apache.gravitino.catalog.lakehouse.iceberg.IcebergTablePropertiesMetadata;
import
org.apache.gravitino.catalog.lakehouse.iceberg.ops.IcebergCatalogWrapperHelper;
import org.apache.gravitino.client.GravitinoMetalake;
import org.apache.gravitino.exceptions.NoSuchSchemaException;
@@ -690,6 +691,117 @@ public abstract class CatalogIcebergBaseIT extends BaseIT
{
"Expected a Hive metastore 'Invalid column type' rejection, but got: "
+ exception);
}
+ @Test
+ void testCreateVariantColumnViaGravitinoApi() {
+ // Write path: create a variant column *through the Gravitino relational
API* and have it
+ // written
+ // to the REST (IRC) backend, then load it back. REST-backend only, for
the same reason as
+ // testV3TypeConversionViaIcebergClient: the CI Hive metastore cannot
store V3 column types.
+ Assumptions.assumeTrue(
+ "rest".equalsIgnoreCase(TYPE),
+ "Variant columns require a backend that does not validate against the
Hive metastore");
+
+ NameIdentifier ident = NameIdentifier.of(schemaName, "t_variant_write");
+ Column[] columns =
+ new Column[] {
+ Column.of("id", Types.IntegerType.get(), "id"),
+ Column.of("payload", Types.VariantType.get(), "variant col")
+ };
+ Map<String, String> properties = Maps.newHashMap();
+ // Variant is a V3 type, so the table must be created at format-version 3.
+ properties.put(IcebergTablePropertiesMetadata.FORMAT_VERSION, "3");
+
+ TableCatalog tableCatalog = catalog.asTableCatalog();
+ Table created =
+ tableCatalog.createTable(
+ ident,
+ columns,
+ "variant write",
+ properties,
+ Transforms.EMPTY_TRANSFORM,
+ Distributions.NONE,
+ new SortOrder[0]);
+ Assertions.assertInstanceOf(Types.VariantType.class,
created.columns()[1].dataType());
+
+ // Load it back through the native metadata API to confirm the round-trip
against the backend.
+ Table loaded = tableCatalog.loadTable(ident);
+ Assertions.assertInstanceOf(Types.VariantType.class,
loaded.columns()[1].dataType());
+ Assertions.assertEquals(
+ "3",
loaded.properties().get(IcebergTablePropertiesMetadata.FORMAT_VERSION));
+ }
+
+ @Test
+ void testCreateVariantColumnRequiresFormatVersion3() {
+ Assumptions.assumeTrue(
+ "rest".equalsIgnoreCase(TYPE),
+ "Variant columns require a backend that does not validate against the
Hive metastore");
+
+ NameIdentifier ident = NameIdentifier.of(schemaName,
"t_variant_default_v2");
+ Column[] columns =
+ new Column[] {
+ Column.of("id", Types.IntegerType.get(), "id"),
+ Column.of("payload", Types.VariantType.get(), "variant col")
+ };
+ TableCatalog tableCatalog = catalog.asTableCatalog();
+ // Without format-version=3 the table defaults to v2, which Iceberg
rejects for variant columns.
+ Exception exception =
+ Assertions.assertThrows(
+ Exception.class,
+ () ->
+ tableCatalog.createTable(
+ ident,
+ columns,
+ "variant default v2",
+ Collections.emptyMap(),
+ Transforms.EMPTY_TRANSFORM,
+ Distributions.NONE,
+ new SortOrder[0]));
+ Assertions.assertTrue(
+ ExceptionUtils.getStackTrace(exception).contains("variant is not
supported until v3"),
+ "Expected a 'variant is not supported until v3' rejection, but got: "
+ exception);
+ }
+
+ @Test
+ void testCreateTableDefaultsToFormatVersion2() {
+ // Unset format-version: Gravitino stamps its default (version 2).
+ assertLoadedFormatVersion("t_fmt_default", Maps.newHashMap(), "2");
+ }
+
+ @Test
+ void testCreateTableWithExplicitFormatVersion2() {
+ Map<String, String> properties = Maps.newHashMap();
+ properties.put(IcebergTablePropertiesMetadata.FORMAT_VERSION, "2");
+ assertLoadedFormatVersion("t_fmt_v2", properties, "2");
+ }
+
+ @Test
+ void testCreateTableWithEmptyFormatVersionDefaultsToV2() {
+ // An explicit empty format-version is treated as unset and defers to the
backend default.
+ Map<String, String> properties = Maps.newHashMap();
+ properties.put(IcebergTablePropertiesMetadata.FORMAT_VERSION, "");
+ assertLoadedFormatVersion("t_fmt_empty", properties, "2");
+ }
+
+ private void assertLoadedFormatVersion(
+ String tableName, Map<String, String> createProperties, String
expectedVersion) {
+ NameIdentifier ident = NameIdentifier.of(schemaName, tableName);
+ Column[] columns = new Column[] {Column.of("id", Types.IntegerType.get(),
"id")};
+ TableCatalog tableCatalog = catalog.asTableCatalog();
+ tableCatalog.createTable(
+ ident,
+ columns,
+ "format-version round-trip",
+ createProperties,
+ Transforms.EMPTY_TRANSFORM,
+ Distributions.NONE,
+ new SortOrder[0]);
+ // Read the version back from the backend; loadTable reflects the
actually-stored format
+ // version.
+ Table loaded = tableCatalog.loadTable(ident);
+ Assertions.assertEquals(
+ expectedVersion,
loaded.properties().get(IcebergTablePropertiesMetadata.FORMAT_VERSION));
+ }
+
private NameIdentifier createV3Table(
String tableName, org.apache.iceberg.types.Type icebergType) {
NameIdentifier ident = NameIdentifier.of(schemaName, tableName);
diff --git a/docs/lakehouse-iceberg-catalog.md
b/docs/lakehouse-iceberg-catalog.md
index 5f11e68744..df9a44aaf3 100644
--- a/docs/lakehouse-iceberg-catalog.md
+++ b/docs/lakehouse-iceberg-catalog.md
@@ -451,19 +451,19 @@ Pass [Iceberg table
properties](https://iceberg.apache.org/docs/1.5.2/configurat
**Immutable**: Fields that cannot be modified once set.
:::
-| Configuration item | Description
| Default value | Required | Reserved |
Immutable | Since Version |
-|---------------------------|---------------------------------------------------------------------------------------|---------------|----------|----------|-----------|---------------|
-| `location` | Iceberg location for table storage.
| (none) | No | No |
Yes | 0.2.0 |
-| `provider` | The storage provider for table storage.
| (none) | No | No |
Yes | 0.2.0 |
-| `format` | The format of table storage.
| (none) | No | No |
Yes | 0.2.0 |
-| `format-version` | The format version of table storage.
| (none) | No | No |
Yes | 0.2.0 |
-| `comment` | The table comment; use the `comment` field in
table meta instead. | (none) | No | Yes |
No | 0.2.0 |
-| `creator` | The table creator.
| (none) | No | Yes | No
| 0.2.0 |
-| `current-snapshot-id` | The snapshot represents the current state of the
table. | (none) | No | Yes | No
| 0.2.0 |
-| `cherry-pick-snapshot-id` | Selecting a specific snapshot in a merge
operation. | (none) | No | Yes
| No | 0.2.0 |
-| `sort-order` | Iceberg table sort order; use `SortOrder` in
table meta instead. | (none) | No | Yes
| No | 0.2.0 |
-| `identifier-fields` | The identifier fields for defining the table.
| (none) | No | Yes | No
| 0.2.0 |
-| `write.distribution-mode` | Defines distribution of write data; use
`distribution` in table meta instead. | (none) | No | Yes
| No | 0.2.0 |
+| Configuration item | Description
| Default value | Required | Reserved |
Immutable | Since Version |
+|---------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------|----------|----------|-----------|---------------|
+| `location` | Iceberg location for table storage.
| (none) | No | No | Yes
| 0.2.0 |
+| `provider` | The storage provider for table storage.
| (none) | No | No | Yes
| 0.2.0 |
+| `format` | The format of table storage.
| (none) | No | No | Yes
| 0.2.0 |
+| `format-version` | The Iceberg table format version. Gravitino
supports creating tables at versions `1`–`4` (the range the bundled Iceberg
version can write) and defaults to `2` when unset. Version `3` is required for
V3 types such as `variant`; version `4` is not yet a finalized Iceberg spec. |
`2` | No | No | Yes | 0.2.0 |
+| `comment` | The table comment; use the `comment` field in
table meta instead.
| (none) | No | Yes | No
| 0.2.0 |
+| `creator` | The table creator.
| (none) | No | Yes | No
| 0.2.0 |
+| `current-snapshot-id` | The snapshot represents the current state of the
table.
| (none) | No | Yes | No
| 0.2.0 |
+| `cherry-pick-snapshot-id` | Selecting a specific snapshot in a merge
operation.
| (none) | No | Yes
| No | 0.2.0 |
+| `sort-order` | Iceberg table sort order; use `SortOrder` in
table meta instead.
| (none) | No | Yes | No
| 0.2.0 |
+| `identifier-fields` | The identifier fields for defining the table.
| (none) | No | Yes | No
| 0.2.0 |
+| `write.distribution-mode` | Defines distribution of write data; use
`distribution` in table meta instead.
| (none) | No | Yes
| No | 0.2.0 |
### Table Indexes
diff --git a/docs/open-api/tables.yaml b/docs/open-api/tables.yaml
index 1e199c5dd1..68b977978f 100644
--- a/docs/open-api/tables.yaml
+++ b/docs/open-api/tables.yaml
@@ -55,6 +55,10 @@ paths:
$ref: "#/components/examples/MysqlTableCreate"
PostgresqlTableCreate:
$ref: "#/components/examples/PostgresqlTableCreate"
+ IcebergTableCreate:
+ $ref: "#/components/examples/IcebergTableCreate"
+ IcebergVariantTableCreate:
+ $ref: "#/components/examples/IcebergVariantTableCreate"
responses:
"200":
$ref: "#/components/responses/TableResponse"
@@ -998,6 +1002,54 @@ components:
"properties": { }
}
+ IcebergTableCreate:
+ value: {
+ "name": "my_iceberg_table",
+ "comment": "This is my Iceberg table",
+ "columns": [
+ {
+ "name": "id",
+ "type": "integer",
+ "comment": "id column comment",
+ "nullable": false
+ },
+ {
+ "name": "name",
+ "type": "string",
+ "comment": "name column comment",
+ "nullable": true
+ }
+ ],
+ "properties": {
+ "format-version": "2"
+ },
+ indexes: [ ]
+ }
+
+ IcebergVariantTableCreate:
+ value: {
+ "name": "my_iceberg_variant_table",
+ "comment": "Iceberg table with a variant column, which requires
format-version 3",
+ "columns": [
+ {
+ "name": "id",
+ "type": "integer",
+ "comment": "id column comment",
+ "nullable": false
+ },
+ {
+ "name": "payload",
+ "type": "variant",
+ "comment": "semi-structured payload column",
+ "nullable": true
+ }
+ ],
+ "properties": {
+ "format-version": "3"
+ },
+ indexes: [ ]
+ }
+
TableResponse:
value: {
"code": 0,
