[ https://issues.apache.org/jira/browse/SPARK-7891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Davies Liu resolved SPARK-7891. ------------------------------- Resolution: Duplicate Assignee: Shixiong Zhu Fix Version/s: 2.0.0 > Python class in __main__ may trigger AssertionError > --------------------------------------------------- > > Key: SPARK-7891 > URL: https://issues.apache.org/jira/browse/SPARK-7891 > Project: Spark > Issue Type: Bug > Components: PySpark > Affects Versions: 1.2.0 > Environment: Linux, Python 2.7.3 > pickled by Python pickle Lib > Reporter: flykobe cheng > Assignee: Shixiong Zhu > Priority: Minor > Fix For: 2.0.0 > > Attachments: demo_error.log, demo_pickle_error.py > > > Callback functions for spark transformations and actions will be pickled. > If the callback is instancemethod of __main__ module's class, and the class > has more than one instancemethod which using class properties or > classmethods, the class will be pickled twice, and 'pickle.memoize' twice, > then trigger AssertionError. > Demo code and traceback attached. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org