[ 
https://issues.apache.org/jira/browse/HIVE-18690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16361760#comment-16361760
 ] 

Sahil Takiar commented on HIVE-18690:
-------------------------------------

I hacked some code together locally and this does work. The question is how to 
do it in the correct manner.

> Integrate with Spark OutputMetrics
> ----------------------------------
>
>                 Key: HIVE-18690
>                 URL: https://issues.apache.org/jira/browse/HIVE-18690
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Spark
>            Reporter: Sahil Takiar
>            Priority: Major
>
> Spark has an {{OutputMetrics}} it uses to expose records / bytes written. We 
> currently don't integrate with it and the Spark UI shows a blank value for 
> output records / bytes. We have our own customer accumulators instead (like 
> {{HIVE_RECORDS_OUT}}).
> Spark exposes the {{OutputMetrics}} object inside individual tasks via the 
> {{TaskContext.get()}} method. We can use this method to access the 
> {{OutputMetrics}} object and update it.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to