HyukjinKwon edited a comment on issue #20691: [SPARK-18161] [Python] Update cloudpickle to v0.6.1 URL: https://github.com/apache/spark/pull/20691#issuecomment-455590380 @inpefess, also do you mind if I ask to elaborate how highest protocol solves 4GB problem? I think it's good to leave a link related with that if there is in the PR description. Also, it would be great if leave the link that says Pyrolite supports to read all protocols. I know it supports to read all protocols and writes it as protocol 2. So I guess it's no problem but let's mention that in PR description. This is also to let other reviewers check Pyrolite together since we had to manually fix the bugs in Pyrolite at our code base. For instance, see https://github.com/apache/spark/commit/ecf437a64874a31328f4e28c6b24f37557fbe07d and https://github.com/apache/spark/commit/b7a40f64e6d83bb6704ac3a63c46cc7c0e9f9e23
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
