[
https://issues.apache.org/jira/browse/SPARK-26225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16741315#comment-16741315
]
Yuanjian Li commented on SPARK-26225:
-------------------------------------
Thanks for your reply Wenchen, as our discussion, the decoding time for file
format should hold on until data source v2 implement done, so I just close
[GitHub Pull Request #23378|https://github.com/apache/spark/pull/23378].
For the `RowDataSourceScanExec`, I give a preview PR here [GitHub Pull Request
#23528|https://github.com/apache/spark/pull/23528], but during the work, I
found it does not take too much time, please take a look whether it's necessary
to add this metric for `RowDataSourceScanExec`. Thanks.
> Scan: track decoding time for row-based data sources
> ----------------------------------------------------
>
> Key: SPARK-26225
> URL: https://issues.apache.org/jira/browse/SPARK-26225
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 2.4.0
> Reporter: Reynold Xin
> Priority: Major
>
> Scan node should report decoding time for each record, if it is not too much
> overhead.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
