[
https://issues.apache.org/jira/browse/SPARK-6075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Rosen updated SPARK-6075:
------------------------------
Summary: After SPARK-3885, some tasks' accumulator updates may be lost
(was: Flaky AccumulatorSuite.add value to collection accumulators test)
> After SPARK-3885, some tasks' accumulator updates may be lost
> -------------------------------------------------------------
>
> Key: SPARK-6075
> URL: https://issues.apache.org/jira/browse/SPARK-6075
> Project: Spark
> Issue Type: Bug
> Components: Spark Core, Tests
> Affects Versions: 1.4.0
> Reporter: Josh Rosen
> Priority: Critical
> Labels: flaky-test
>
> It looks like some of the AccumulatorSuite tests have started failing
> nondeterministically on Jenkins. The errors seem to be due to lost / missing
> accumulator updates, e.g.
> {code}
> Set(843, 356, 437, [...], 181, 618, 131) did not contain element 901
> {code}
> This could somehow be related to SPARK-3885 /
> https://github.com/apache/spark/pull/4021, a patch to garbage-collect
> accumulators, which was only merged into master.
> https://amplab.cs.berkeley.edu/jenkins/view/Spark/job/Spark-Master-SBT/lastCompletedBuild/AMPLAB_JENKINS_BUILD_PROFILE=hadoop2.0,label=centos/testReport/org.apache.spark/AccumulatorSuite/add_value_to_collection_accumulators/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
