progval opened a new pull request, #12585: URL: https://github.com/apache/datafusion/pull/12585
## Which issue does this PR close? Closes #12584. ## Rationale for this change This allows both Datafusion developers and users to better measure the impact of their optimizations. For example, I am working on #12547 (caching the page index), and my patch makes these metrics go from: > statistics_eval_time=775.283531ms, row_pushdown_eval_time=191ns, time_elapsed_scanning_until_data=16.463873ms, time_elapsed_processing=401.16043928s, **time_elapsed_opening=473.912060225s**, page_index_eval_time=24.19564ms, **metadata_load_time=410.282903954s**, bloom_filter_eval_time=62.038167937s, time_elapsed_scanning_total=16.516948ms to > row_pushdown_eval_time=191ns, time_elapsed_scanning_until_data=1.143893882s, time_elapsed_processing=34.501308377s, **time_elapsed_opening=67.214927741s**, page_index_eval_time=32.002383ms, **metadata_load_time=3.411246ms**, bloom_filter_eval_time=66.495163165s, statistics_eval_time=1.255058963s, time_elapsed_scanning_total=1.143978104s ## What changes are included in this PR? 1. Add `metadata_load_time` 2. Add`statistics_eval_time` and `bloom_filter_eval_time` 3. Rename `pushdown_eval_time` to `row_pushdown_eval_time`, because statistics and eval time are pushdown filters too ## Are these changes tested? Not really. There is no unit tests, but it works in datafusion-cli and as a library ## Are there any user-facing changes? Renamed a metric, and added two others, visible in `EXPLAIN ANALYZE`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
