viirya commented on issue #24521: [SPARK-27629][PySpark] Prevent Unpickler from intervening each unpickling URL: https://github.com/apache/spark/pull/24521#issuecomment-489283045 > Ultimately, this does look like a bug in Pyrolite and this workaround might be fine. Would you mind adding a reference to the JIRA in your comment and in the test added in #24519? I'd like to file an issue to Pyrolite. From my view, memo map should be cleared when op code STOP is hit, because each loading should be independent, so the memo map shouldn't be reuse without clearing. I added few comments to the test and the JIRA.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
