plusplusjiajia opened a new pull request, #8795: URL: https://github.com/apache/paimon/pull/8795
### Purpose Add Iceberg v3 `variant` type support to Paimon's Iceberg compatibility layer, with the row lineage and REST catalog upgrade it requires. The three are interdependent: v3 `variant` metadata without row lineage is spec-invalid (Iceberg 1.10+ won't parse metadata lacking `next-row-id`/`first-row-id`), and row lineage without the REST upgrade still leaves `rest-catalog` storage creating v2 tables that reject v3 features. - **Variant type**: `IcebergDataField` maps Paimon `VARIANT` ↔ Iceberg `"variant"` both ways. Before a snapshot is published, `IcebergCommitCallback` rejects types that cannot produce readable metadata: `variant` when `format-version < 3` (and as a partition key, unconditionally), and nanosecond timestamps on every version (Paimon writes them as Parquet `INT96`, which Iceberg cannot read as `timestamp_ns`). Validation runs before publish and before a tag is written. - **Row lineage**: `IcebergManifestFileMeta`/`IcebergDataFileMeta` gain `first_row_id`; ranges are assigned when writing the manifest list, mirroring Iceberg's `ManifestListWriter`. Tables upgraded from v2 get their surviving manifests assigned on the first v3 commit; regeneration continues past the highest published id so ids are never reused; rewritten/merged manifests keep unchanged rows' `_row_id`. - **REST upgrade**: bump to Iceberg 1.10.1 (1.8.1 can't parse `variant`). Creation passes the configured `format-version`; a populated table whose version differs is dropped and recreated. Fallout: Hadoop 3.3.6 for `FileSystem.openFile`, jetty/commons-lang3 test-classpath alignment, `RemoveSnapshot` → `RemoveSnapshots`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
