GitHub user holdenk opened a pull request:
https://github.com/apache/spark/pull/14467
[SPARK-16861][PYSPARK][CORE][WIP] Refactor PySpark accumulator API on top
of Accumulator V2
## What changes were proposed in this pull request?
Move the internals of the PySpark accumulator API from the old deprecated
API on top of the new accumulator API.
## How was this patch tested?
The existing PySpark accumulator tests (both unit tests and doc tests at
the start of accumulator.py).
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/holdenk/spark
SPARK-16861-refactor-pyspark-accumulator-api
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/14467.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #14467
----
commit 3c1ea65ba4fb51b3c0a610a20f7691b0d0176a84
Author: Holden Karau <[email protected]>
Date: 2016-08-01T19:47:44Z
one attempt
commit 1d538fea23087c31e901b2c3749a193a33480575
Author: Holden Karau <[email protected]>
Date: 2016-08-01T19:47:51Z
Revert "one attempt"
This reverts commit 3c1ea65ba4fb51b3c0a610a20f7691b0d0176a84.
commit 4bc43c04df6c1611cf31b1a76b93c9c041ef2d63
Author: Holden Karau <[email protected]>
Date: 2016-08-01T22:37:30Z
Start switching the Python Accumulator backing to the V2 API
commit 46fa97d315438b7ad603e6a290cf7e5b8844b7a2
Author: Holden Karau <[email protected]>
Date: 2016-08-01T22:50:31Z
Start switching the Python side code to match the JVM side code
commit 736f6ce22fc15925e36dff6ee558cd26c2f53aab
Author: Holden Karau <[email protected]>
Date: 2016-08-02T19:26:46Z
Merge branch 'master' into
SPARK-16775-reduce-internal-warnings-from-deprecated-accumulator-api
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
