GitHub user BryanCutler opened a pull request:
https://github.com/apache/spark/pull/20373
[WIP][SPARK-23159][PYTHON] Update cloudpickle to match 0.4.2
## What changes were proposed in this pull request?
The version of cloudpickle in PySpark was close to version 0.4.0 with some
additional backported fixes and some minor additions for Spark related things.
With version 0.4.2 we can remove Spark related changes and make the version in
Spark exactly match 0.4.2 at
https://github.com/cloudpipe/cloudpickle/tree/v0.4.2
Changes by updating to 0.4.2 include:
* Fix pickling of named tuples
https://github.com/cloudpipe/cloudpickle/pull/113
* Built in type constructors for PyPy compatibility
[here](https://github.com/cloudpipe/cloudpickle/commit/d84980ccaafc7982a50d4e04064011f401f17d1b)
* Fix memoryview support https://github.com/cloudpipe/cloudpickle/pull/122
* Improved compatibility with other cloudpickle versions
https://github.com/cloudpipe/cloudpickle/pull/128
* Several cleanups https://github.com/cloudpipe/cloudpickle/pull/121 and
[here](https://github.com/cloudpipe/cloudpickle/commit/c91aaf110441991307f5097f950764079d0f9652)
## How was this patch tested?
Existing pyspark.tests using python 2.7.14 and 3.5.2
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/BryanCutler/spark
pyspark-update-cloudpickle-42-SPARK-23159
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20373.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20373
----
commit 89f13b857dba53754f6813efae2d0ca4540c48f4
Author: Bryan Cutler <cutlerb@...>
Date: 2018-01-23T23:25:29Z
updated cloudpickle to match 0.4.2
commit c362df87e2d5a5f55d2e4f7d48e24b2d7cfda6f7
Author: Bryan Cutler <cutlerb@...>
Date: 2018-01-23T23:55:25Z
removed unused import
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
