plusplusjiajia opened a new pull request, #9199:
URL: https://github.com/apache/paimon/pull/9199
…he commit
### Purpose
Some Paimon schemas cannot be represented in the Iceberg metadata this
layer emits, and today the mismatch surfaces only after the Paimon snapshot is
published: the mirror then publishes metadata Iceberg misreads, or fails on
every retry and leaves the manifests it wrote behind. This PR checks before the
commit and vetoes it there.
Rejected: nanosecond-precision timestamps on any format version (Paimon
writes precision > 6 as Parquet INT96, which Iceberg cannot read back as
timestamp_ns), VARIANT on a v2 table, VARIANT partition keys, and
manifest.legacy-version on v3. The check runs as a pre-commit callback, so the
Paimon snapshot is never published, and the manifests written during
preparation are cleaned up exactly like a preparation failure.
Only the schemas a commit newly publishes are gated, so a veto is never
permanent: already mirrored schemas stay valid, a schema vetoed at its
introduction and since remedied (e.g. the column was dropped) is skipped at
emission — with the snapshot entry falling back to the current schema id so no
pointer dangles — and a transient read failure is rethrown instead of silently
dropping a valid schema.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
