jay-dee7 opened a new pull request, #25640: URL: https://github.com/apache/datafusion/pull/25640
## Which issue does this PR close? - Part of #25499 ## Rationale for this change `datafusion-cli` can't show a parquet file's footer metadata (writer, key-value metadata, footer size) or its page index. Checking bloom filter sizing, page index cost or writer metadata currently means using DuckDB or a custom tool. This PR covers items 2 and 3 of #25499; item 1 is in #25638. ## What changes are included in this PR? - New _**parquet_file_metadata(path)**_ table function, one row per file: `created_by`, `version`, `num_rows`, `num_row_groups`, `key_value_metadata` (a map, so `key_value_metadata['ARROW:schema']` works) and `footer_length`. - New **_parquet_page_index(path)_** table function, one row per data page: the page location from the offset index (`page_ordinal`, `first_row_index`, `offset`, `compressed_page_size`) and the column index `min_value`, `max_value` and `null_count`. Files without a page index return no rows. - The path argument parsing is now shared by all three parquet functions. - Docs for both functions in `docs/source/user-guide/cli/functions.md`. ## What is the testing strategy for this PR? Snapshot tests `test_parquet_file_metadata_works` and `test_parquet_page_index_works` in `datafusion-cli/src/main.rs`, run against `parquet-testing` files. They cover an all-null page (NULL min/max) and UTF8 min/max shown as strings. The page values match the parquet-cli output in `int32_with_null_pages.md`, and `footer_length` matches the file's raw footer bytes. ## Are there any user-facing changes? Yes: two new `datafusion-cli` table functions, documented in the CLI functions guide. `parquet_metadata` output is unchanged and there are no breaking changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
