Matei Zaharia created SPARK-3628:
------------------------------------
Summary: Don't apply accumulator updates multiple times for tasks
in result stages
Key: SPARK-3628
URL: https://issues.apache.org/jira/browse/SPARK-3628
Project: Spark
Issue Type: Bug
Reporter: Matei Zaharia
Priority: Blocker
In previous versions of Spark, accumulator updates only got applied once for
accumulators that are only used in actions (i.e. result stages), letting you
use them to deterministically compute a result. Unfortunately, this got broken
in some recent refactorings.
This is related to https://issues.apache.org/jira/browse/SPARK-732, but that
issue is about applying the same semantics to intermediate stages too, which is
more work and may not be what we want for debugging.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
