hutiefang76 commented on issue #8440: URL: https://github.com/apache/paimon/issues/8440#issuecomment-4939464382
I investigated this and have a local proof of concept ready. The approach is: - add an opt-in file-metadata capability to the format writer factory, keeping `paimon-core` decoupled from the concrete Avro implementation; - write the six Iceberg manifest metadata fields before the Avro file is created, with `content` bound per data/delete manifest; - cover both V2 data manifests and V3 data/delete manifests by reading the physical Avro headers. Local verification on JDK 17 currently passes `AvroFileFormatTest`, the V2 `IcebergCompatibilityTest` case, and the V3 deletion-vector compatibility case. If this approach looks good, please assign #8440 to me and I will submit the PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
