hutiefang76 opened a new pull request, #10329: URL: https://github.com/apache/paimon/pull/10329
### Purpose Newly written Iceberg manifests include the required Avro metadata: `schema`, `schema-id`, `partition-spec`, `partition-spec-id`, `format-version`, and `content`. Readers can resolve the schema and partition spec from the file headers. Previously, only `avro.schema` and `avro.codec` were present. Fixes #8440. Related: #9497. The writer factory exposes an opt-in file-metadata capability, keeping `paimon-core` independent of the Avro implementation. Metadata is set before the Avro header is written. Each manifest writer receives its own `data` or `deletes` content value, and `partition-spec` is serialized as the required field array. This draft retains the existing schema and metrics field IDs. Existing headerless manifests are not rewritten; legacy-manifest migration and field-ID remapping are outside this change. ### Tests - Reproduced the missing headers before the production fix: five of nine checks failed. - JDK 17: 56 `AvroFileFormatTest`, 48 `IcebergCompatibilityTest`, 8 `IcebergManifestBlockSizeTest`, and 3 `IcebergDVCompatibilityTest` tests passed (115 total). - Verified all six metadata entries in the physical headers, including partitioned and unpartitioned V2 data manifests and V3 data/delete manifests. - Read generated V2 data and V3 data/delete manifests through Iceberg without an external partition-spec map. - Checkstyle, Spotless, Maven Enforcer, and `git diff --check` passed for the final change. --- [](https://github.com/EveryInc/compound-engineering-plugin) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
