FYI, in 1.3 we support save/load tree models in Scala and Java. We will add save/load support to Python soon. -Xiangrui
On Mon, Feb 23, 2015 at 2:57 PM, Sebastián Ramírez < [email protected]> wrote: > In your log it says: > > pickle.PicklingError: Can't pickle <type 'thread.lock'>: it's not found as > thread.lock > > As far as I know, you can't pickle Spark models. If you go to the > documentation for Pickle you can see that you can pickle only simple Python > structures and code (written in Python), at least as I understand: > https://docs.python.org/2/library/pickle.html#what-can-be-pickled-and-unpickled > > To save a model you can call: model.toDebugString() > <http://spark.apache.org/docs/latest/api/python/pyspark.mllib.html#pyspark.mllib.tree.DecisionTreeModel.toDebugString> > > That gives you a string in pseudo-code that you can save to a file. Then, > you can parse that pseudo code to write a proper script that runs the > Decision Tree. Actually, that's what I did for a Random Forest (an ensamble > of Decision Trees). > > Hope that helps, > > > *Sebastián Ramírez* > Diseñador de Algoritmos > > <http://www.senseta.com> > ________________ > Tel: (+571) 795 7950 ext: 1012 > Cel: (+57) 300 370 77 10 > Calle 73 No 7 - 06 Piso 4 > Linkedin: co.linkedin.com/in/tiangolo/ > Twitter: @tiangolo <https://twitter.com/tiangolo> > Email: [email protected] > www.senseta.com > > On Mon, Feb 23, 2015 at 4:55 AM, Jaggu <[email protected]> wrote: > >> Hi Team, >> I was trying to save a DecisionTree model from Pyspark using joblib. >> It is giving me the following error http://pastebin.com/82CFhPNn . Any >> clue >> how to resolve the same or save a model. >> >> Best regards >> >> Jagan >> >> >> >> -- >> View this message in context: >> http://apache-spark-user-list.1001560.n3.nabble.com/Pyspark-save-Decison-Tree-Module-with-joblib-pickle-tp21765.html >> Sent from the Apache Spark User List mailing list archive at Nabble.com. >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >> > > *----------------------------------------------------* > *This e-mail transmission, including any attachments, is intended only for > the named recipient(s) and may contain information that is privileged, > confidential and/or exempt from disclosure under applicable law. If you > have received this transmission in error, or are not the named > recipient(s), please notify Senseta immediately by return e-mail and > permanently delete this transmission, including any attachments.* >
