[
https://issues.apache.org/jira/browse/SPARK-53891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szehon Ho updated SPARK-53891:
------------------------------
Description:
SPARK-52689 added a DataSourceV2 API that sends operation metrics along with
the commit, via a map of string, long. It would be cleaner to model it as a
proper object so that it is more clear what metrics Spark sends, and to handle
future cases where metrics may not be long values.
Suggestion from [~aokolnychyi]
was:
SPARK-52689 added a DataSourceV2 API that sends operation metrics along with
the commit. It sent it through a map. It would be cleaner to model it as a
proper object so that it is more clear what metrics Spark sends, and to handle
future cases where metrics may not be long values.
Suggestion from [~aokolnychyi]
> Model DSV2 Commit Operation Metrics API
> ---------------------------------------
>
> Key: SPARK-53891
> URL: https://issues.apache.org/jira/browse/SPARK-53891
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 4.1.0
> Reporter: Szehon Ho
> Priority: Minor
>
> SPARK-52689 added a DataSourceV2 API that sends operation metrics along with
> the commit, via a map of string, long. It would be cleaner to model it as a
> proper object so that it is more clear what metrics Spark sends, and to
> handle future cases where metrics may not be long values.
> Suggestion from [~aokolnychyi]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
