anuragmantri opened a new issue, #3127: URL: https://github.com/apache/iceberg-rust/issues/3127
Part of https://github.com/apache/iceberg-rust/issues/3126 ## Is your feature request related to a problem or challenge? FileScanTask carries no sort-order information today. A file's sort_order_id is per-file (DataFile.sort_order_id), not table-wide: a table's default_sort_order() describes what new writes should conform to, not what any specific existing file is actually sorted by. Two files in the same manifest can even carry different sort_order_id values, if the table's sort order changed between when they were written. Anything downstream that wants to know whether a set of files is safely treatable as sorted (the DataFusion side of #<epic number>, or any other future consumer) needs that per-file signal resolved and available on FileScanTask. Today there's no way to get it without re-reading manifest data by hand. ##Describe the solution you'd like Resolve each manifest entry's sort_order_id against the table's known sort orders (TableMetadata.sort_orders), and carry the result on a new FileScanTask.sort_order field. None means no sort order could be established, either because the file has no recorded sort_order_id, or because the id doesn't resolve against the table's known sort orders. This threads through the existing PlanContext -> ManifestFileContext -> ManifestEntryContext pipeline in crates/iceberg/src/scan/, following the same pattern already used for unified_partition_type. No dependency on anything else, including the rest of #<epic number>, so this is ready to implement now. ## Willingness to contribute: I can contribute to this feature independently -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
