GitHub user zsxwing opened a pull request:
https://github.com/apache/spark/pull/7448
[SPARK-8857][SPARK-8859][Core]Add an internal flag to Accumulable and send
internal accumulator updates to the driver via heartbeats
This PR includes the following changes:
1. Remove the thread local `Accumulators.localAccums`. Instead, all
Accumulators in the executors will register with its TaskContext.
2. Add an internal flag to Accumulable. For internal Accumulators, their
updates will be sent to the driver via heartbeats.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zsxwing/spark accumulators
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/7448.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #7448
----
commit 71828139a4225c44dc2e76f60cdbb2b82bef0c1c
Author: zsxwing <[email protected]>
Date: 2015-07-16T15:31:02Z
Add an internal flag to Accumulable and send internal accumulator updates
to the driver via heartbeats
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
