aiborodin commented on PR #14092: URL: https://github.com/apache/iceberg/pull/14092#issuecomment-3333045351
@pvary @mxm I added the `ContentFileAvroEncoder`, which serialises content files using internal Iceberg Avro serialisation classes to provide backwards compatibility. This serialisation is the same as in `ManifestWriter` (uses `InternalData`), but strips out all manifest-related properties, which we don't need for content file checkpoint persistence. There's a similarly inspired JSON-based serialiser called `ContentFileParser`, which is also used in Flink, but I decided to still use Avro to keep the checkpoint size minimal. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
