alamb commented on PR #21637:
URL: https://github.com/apache/datafusion/pull/21637#issuecomment-4454064121
> The issue was that ParquetFileMetrics::new created a
LazyParquetSummaryCount for page_index_pages_skipped_by_fully_matched for every
opened file.
Wild -- that seems like non trivial overhead
Looking at the code and what you changed, maybe it is because the metric
builder is expensive (it is copying strings)
```
let count = MetricBuilder::new(metrics)
.with_new_label("filename", filename.to_string())
.with_type(MetricType::Summary)
.with_category(MetricCategory::Rows)
.counter("page_index_pages_skipped_by_fully_matched", partition);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
