[
https://issues.apache.org/jira/browse/SPARK-12895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Or updated SPARK-12895:
------------------------------
Description:
We need to first do this before we can avoid sending TaskMetrics from the
executors to the driver. After we do this, we can send only accumulator updates
instead of both that AND TaskMetrics.
By the end of this issue TaskMetrics will be a wrapper of accumulators. It will
be only syntactic sugar for setting these accumulators.
But first, we need to express everything in TaskMetrics as accumulators.
was:
We need to first do this before we can avoid sending TaskMetrics from the
executors to the driver. After we do this, we can send only accumulator updates
instead of both that AND TaskMetrics.
But first, we need to express everything in TaskMetrics as accumulators.
> Implement TaskMetrics using accumulators
> ----------------------------------------
>
> Key: SPARK-12895
> URL: https://issues.apache.org/jira/browse/SPARK-12895
> Project: Spark
> Issue Type: Sub-task
> Components: Spark Core
> Reporter: Andrew Or
> Assignee: Andrew Or
>
> We need to first do this before we can avoid sending TaskMetrics from the
> executors to the driver. After we do this, we can send only accumulator
> updates instead of both that AND TaskMetrics.
> By the end of this issue TaskMetrics will be a wrapper of accumulators. It
> will be only syntactic sugar for setting these accumulators.
> But first, we need to express everything in TaskMetrics as accumulators.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
