Github user AbdealiJK commented on a diff in the pull request:
https://github.com/apache/spark/pull/22635#discussion_r222899988
--- Diff: python/pyspark/accumulators.py ---
@@ -109,10 +109,14 @@
def _deserialize_accumulator(aid, zero_value, accum_param):
from pyspark.accumulators import _accumulatorRegistry
- accum = Accumulator(aid, zero_value, accum_param)
- accum._deserialized = True
- _accumulatorRegistry[aid] = accum
- return accum
+ # If this certain accumulator was deserialized, don't overwrite it.
+ if aid in _accumulatorRegistry:
--- End diff --
That doesnt seem right because the constructor for `Accumulator` has:
```
...
self._deserialized = False
_accumulatorRegistry[aid] = self
```
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
