parthchandra opened a new issue, #7431: URL: https://github.com/apache/datafusion-comet/issues/7431
### What is the problem the feature request solves? Queries on Iceberg metadata tables (`.partitions`, `.snapshots`, `.all_manifests`, `.entries`, `.files`, `.refs`) fall back to Spark. Table-maintenance jobs and partition-listing queries read these, and the operators above the scan stay in Spark. ### Describe the potential solution Read the common metadata tables natively, starting with `.partitions`, `.snapshots` and `.all_manifests`. Check how much iceberg-rust's metadata table support already covers. ### Additional context The native Iceberg scan already supports the `_file`, `_pos`, `_spec_id` and `_partition` metadata columns. This issue is about the metadata tables, which `CometScanRule.isIcebergMetadataTable` sends to Spark. #2033 made these queries fall back cleanly. #6719 covers data tables whose names end in a metadata-table name. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
