GitHub user cloud-fan opened a pull request:
https://github.com/apache/spark/pull/21229
[SPARK-23697][CORE] LegacyAccumulatorWrapper should define isZero correctly
## What changes were proposed in this pull request?
It's possible that Accumulators of Spark 1.x may no longer work with Spark
2.x. This is because `LegacyAccumulatorWrapper.isZero` may return wrong answer
if `AccumulableParam` doesn't define equals/hashCode.
This PR fixes this by using reference equality check in
`LegacyAccumulatorWrapper.isZero`.
## How was this patch tested?
a new test
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cloud-fan/spark accumulator
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21229.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21229
----
commit a2d81dfa5f5b2cf58413f3e39717bf436c956d70
Author: Wenchen Fan <wenchen@...>
Date: 2018-05-03T13:41:49Z
LegacyAccumulatorWrapper should define isZero correctly
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
