tanmayrauth opened a new pull request, #890: URL: https://github.com/apache/iceberg-go/pull/890
Iceberg Java used added_data_files_count, existing_data_files_count, and deleted_data_files_count as manifest list field names (IDs 504/505/506) until apache/iceberg#5338 renamed them to match the spec in the 1.4 release. Tables written before that fix, or by engines still on older Java versions (Athena, some Trino deployments), embed the legacy names in the writer schema of every manifest list OCF file. When iceberg-go read such a file, hamba/avro could not match writer-schema fields to struct tags and silently left the three count fields at zero. Zero counts caused HasAddedFiles() and HasExistingFiles() to return false, which broke fast-append manifest inheritance and caused Table.Scan() to return no data. Fix: add parallel struct fields with the legacy avro tags to manifestFileV1 and manifestFile. Accessors coalesce — preferring the spec-correct name, falling back to the legacy name. No write-path or schema-registration changes are made. Adds two tests (V1 and V2) that write raw Avro OCF using the pre-1.4 field names and assert the counts decode correctly. Fixes #889 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
