eholmer-pltr opened a new issue, #18465: URL: https://github.com/apache/iceberg/issues/18465
Static metadata tables such as `snapshots`, `history`, and `refs` build their rows from already loaded `TableMetadata`. During planning, however, `StaticDataTask` uses `DataFiles.Builder.withInputFile`, which looks up the metadata file's size. On object stores, this can trigger an unnecessary HEAD request against `metadata.json`. For example, after loading a table, planning a query such as: ```sql SELECT * FROM catalog.db.my_table.snapshots; ``` can stat the metadata file solely to populate the synthetic task's file-size field, even though the rows are already available in memory. Planning these tasks should avoid that storage lookup while preserving useful task-size and row-count estimates. The lookup is still present on `main` in [StaticDataTask](https://github.com/apache/iceberg/blob/6dca1cd50484bd7b55f8483d8ef6e9da50b27f67/core/src/main/java/org/apache/iceberg/StaticDataTask.java#L54-L64). Related fix: #16833. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
