[
https://issues.apache.org/jira/browse/SPARK-23159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336583#comment-16336583
]
Apache Spark commented on SPARK-23159:
--------------------------------------
User 'BryanCutler' has created a pull request for this issue:
https://github.com/apache/spark/pull/20373
> Update Cloudpickle to match version 0.4.2
> -----------------------------------------
>
> Key: SPARK-23159
> URL: https://issues.apache.org/jira/browse/SPARK-23159
> Project: Spark
> Issue Type: Improvement
> Components: PySpark
> Affects Versions: 2.3.0
> Reporter: Bryan Cutler
> Priority: Major
>
> Update PySpark's version of Cloudpickle to match version 0.4.2. The reasons
> for doing this are:
> * Pick up bug fixes, improvements with newer version
> * Match a specific version as close as possible (Spark has additional
> changes that might be necessary) to make future upgrades easier
> There are newer versions of Cloudpickle that can fix bugs with NamedTuple
> pickling (that Spark currently has workarounds for), but these include other
> changes that need some verification before bringing into Spark. Upgrading
> first to 0.4.2 will help make this verification easier.
> Discussion on the mailing list:
> http://apache-spark-developers-list.1001551.n3.nabble.com/Thoughts-on-Cloudpickle-Update-td23188.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
