Sahil Takiar created HIVE-18690: ----------------------------------- Summary: Integrate with Spark OutputMetrics Key: HIVE-18690 URL: https://issues.apache.org/jira/browse/HIVE-18690 Project: Hive Issue Type: Sub-task Components: Spark Reporter: Sahil Takiar
Spark has an {{OutputMetrics}} it uses to expose records / bytes written. We currently don't integrate with it and the Spark UI shows a blank value for output records / bytes. We have our own customer accumulators instead (like {{HIVE_RECORDS_OUT}}). Spark exposes the {{OutputMetrics}} object inside individual tasks via the {{TaskContext.get()}} method. We can use this method to access the {{OutputMetrics}} object and update it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)