GitHub user debajyoti-truefoundry closed a discussion: DataSourceExec metrics
explanation
```json
{
"datafusion.metrics.row_groups_pruned_statistics": "0",
"datafusion.metrics.file_open_errors": "0",
"code.filepath": "src/server/app_state.rs",
"datafusion.metrics.page_index_eval_time": "936.547µs",
"datafusion.metrics.row_pushdown_eval_time": "305.394559ms",
"datafusion.metrics.metadata_load_time": "38.869829ms",
"datafusion.preview": "",
"datafusion.emission_type": "Incremental",
"datafusion.metrics.bytes_scanned": "1209599324",
"datafusion.node": "DataSourceExec: file_groups={8 groups: [[...]]},
projection=[...], file_type=parquet, predicate=, required_guarantees=[]",
"datafusion.metrics.end_timestamp": "2025-06-26 12:56:20.582199498 UTC",
"datafusion.metrics.elapsed_compute": "8ns",
"datafusion.metrics.bloom_filter_eval_time": "3.04375ms",
"datafusion.metrics.statistics_eval_time": "3.874945ms",
"datafusion.metrics.output_rows": "913521",
"thread.name": "cpu-runtime",
"datafusion.metrics.time_elapsed_processing": "2.209661124s",
"datafusion.metrics.pushdown_rows_pruned": "44461381",
"datafusion.metrics.row_groups_matched_statistics": "85",
"datafusion.metrics.pushdown_rows_matched": "46243794",
"datafusion.metrics.file_scan_errors": "0",
"datafusion.metrics.page_index_rows_matched": "45374902",
"datafusion.metrics.start_timestamp": "2025-06-26 12:56:20.172305562 UTC",
"datafusion.metrics.predicate_evaluation_errors": "0",
"datafusion.metrics.time_elapsed_scanning_until_data": "2.030212717s",
"datafusion.metrics.time_elapsed_opening": "16.665201ms",
"datafusion.metrics.num_predicate_creation_errors": "0",
"code.namespace": "server::app_state",
"datafusion.partitioning": "UnknownPartitioning(8)",
"datafusion.boundedness": "Bounded",
"datafusion.metrics.time_elapsed_scanning_total": "2.286310309s",
"datafusion.metrics.page_index_rows_pruned": "0",
"datafusion.metrics.row_groups_pruned_bloom_filter": "0",
"datafusion.metrics.row_groups_matched_bloom_filter": "0",
"code.lineno": 74,
"thread.id": 10,
"busy_ns": 345942296,
"idle_ns": 64056220
}
```
`time_elapsed_opening`
`time_elapsed_scanning_total`
[I can see that the above metrics are documented in this
section.](https://github.com/apache/datafusion/blob/b4053801bf6e9345af1e856404d11a437410a7d8/docs/source/user-guide/explain-usage.md?plain=1#L210)
Could someone please explain the following metrics as well? 🙏
`time_elapsed_processing`
`time_elapsed_scanning_until_data`
GitHub link: https://github.com/apache/datafusion/discussions/16572
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
