[
https://issues.apache.org/jira/browse/SPARK-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17447865#comment-17447865
]
Hyukjin Kwon commented on SPARK-22674:
--
no biggie either way. I already creatd a PR against that
[
https://issues.apache.org/jira/browse/SPARK-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17447859#comment-17447859
]
Sergei Lebedev commented on SPARK-22674:
Hi [~hyukjin.kwon], very happy to see you working on
[
https://issues.apache.org/jira/browse/SPARK-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17410280#comment-17410280
]
Hyukjin Kwon commented on SPARK-22674:
--
i plan to fix it in Spark 3.3 fwiw.
> PySpark breaks
[
https://issues.apache.org/jira/browse/SPARK-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17365194#comment-17365194
]
Sarth Frey commented on SPARK-22674:
I can confirm this is affecting PySpark 3.1.1
> PySpark breaks
[
https://issues.apache.org/jira/browse/SPARK-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17210396#comment-17210396
]
Konstantinos Katsiapis commented on SPARK-22674:
Does this issue only affect PySpark2 or
[
https://issues.apache.org/jira/browse/SPARK-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17200747#comment-17200747
]
Apache Spark commented on SPARK-22674:
--
User 'HyukjinKwon' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16683033#comment-16683033
]
Apache Spark commented on SPARK-22674:
--
User 'superbobry' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16456551#comment-16456551
]
Apache Spark commented on SPARK-22674:
--
User 'superbobry' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452462#comment-16452462
]
Apache Spark commented on SPARK-22674:
--
User 'superbobry' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399488#comment-16399488
]
Sergei Lebedev commented on SPARK-22674:
[~jonasamrich] I see your point, but the current
[
https://issues.apache.org/jira/browse/SPARK-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398842#comment-16398842
]
Jonas Amrich commented on SPARK-22674:
--
[~lebedev] I completely agree, removing it is IMHO the
[
https://issues.apache.org/jira/browse/SPARK-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396948#comment-16396948
]
Sergei Lebedev commented on SPARK-22674:
Just out of curiosity: what is the original motivation
[
https://issues.apache.org/jira/browse/SPARK-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278197#comment-16278197
]
Hyukjin Kwon commented on SPARK-22674:
--
Will probably take a closer look for your fix and other
[
https://issues.apache.org/jira/browse/SPARK-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278185#comment-16278185
]
Jonas Amrich commented on SPARK-22674:
--
No worries, I'm glad we're on the same page now. However the
[
https://issues.apache.org/jira/browse/SPARK-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278142#comment-16278142
]
Hyukjin Kwon commented on SPARK-22674:
--
If that deduplication brings performance regression or is
[
https://issues.apache.org/jira/browse/SPARK-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278121#comment-16278121
]
Hyukjin Kwon commented on SPARK-22674:
--
Oh, sorry, I overlooked at {{ that regular pickle won't be
[
https://issues.apache.org/jira/browse/SPARK-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16277577#comment-16277577
]
Hyukjin Kwon commented on SPARK-22674:
--
Basically yes, for now. I think we should avoid having a
[
https://issues.apache.org/jira/browse/SPARK-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16277424#comment-16277424
]
Jonas Amrich commented on SPARK-22674:
--
Sure, you're right that pickle won't unpickle it without
[
https://issues.apache.org/jira/browse/SPARK-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16276390#comment-16276390
]
Hyukjin Kwon commented on SPARK-22674:
--
Not really. Regular pickle can pickle it but is unable to
[
https://issues.apache.org/jira/browse/SPARK-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16276375#comment-16276375
]
Jonas Amrich commented on SPARK-22674:
--
I don't think that this cloudpickle fix would solve the
[
https://issues.apache.org/jira/browse/SPARK-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16276287#comment-16276287
]
Hyukjin Kwon commented on SPARK-22674:
--
I am aware of a fix about this in cloudpickle -
21 matches
Mail list logo
