[
https://issues.apache.org/jira/browse/PARQUET-2374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17838731#comment-17838731
]
Parth Chandra commented on PARQUET-2374:
----------------------------------------
I think it would be great to get low level stats into the query engine.
However, the stats we added in this PR are very useful for end users trying to
optimized their jobs. I have a PR for Spark which can collect these metrics for
Parquet file source and the metrics then show up in the UI. I'll post that once
we have the Parquet release.
> Add metrics support for parquet file reader
> -------------------------------------------
>
> Key: PARQUET-2374
> URL: https://issues.apache.org/jira/browse/PARQUET-2374
> Project: Parquet
> Issue Type: Improvement
> Components: parquet-mr
> Affects Versions: 1.13.1
> Reporter: Parth Chandra
> Assignee: Parth Chandra
> Priority: Major
> Fix For: 1.14.0
>
>
> ParquetFileReader is used by many engines - Hadoop, Spark among them. These
> engines report various metrics to measure performance in different
> environments and it is usually useful to be able to get low level metrics out
> of the file reader and writers.
> It would be very useful to allow a simple interface to report the metrics.
> Callers can then implement the interface to record the metrics in any
> subsystem they choose.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
