unikdahal opened a new issue, #3344: URL: https://github.com/apache/iceberg-rust/issues/3344
Part of #3343. Execution engines may construct `FileScanTask`s without having the delete-file size available. Parquet delete-file reads currently require `file_size_in_bytes` to locate the footer, which forces callers to stat each delete file before constructing the task. Allow `file_size_in_bytes = 0` to represent an unknown size and resolve it lazily when the delete file is first opened. Expected behavior: - Manifest-planned tasks continue using the recorded size. - Known sizes require no additional metadata request. - Unknown sizes are resolved lazily from storage. - Cached delete files require the lookup only on their first load. - Storage error kind and retryability are preserved while adding delete-file context. - Sizes below the Parquet footer minimum fail explicitly. - Position, equality, and encrypted Parquet delete files continue to work. - Deletion vectors remain unaffected. This is independently useful for execution engines constructing Iceberg scan tasks and provides one of the reader foundations required by #3343. ## Willingness to contribute Yes, I have an implementation and tests ready. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
