Re: [Scikit-learn-general] Sharing objects between Python 2 and 3

2015-01-24 Thread jni . soma
Hi Gaël, I know virtually nothing about string/unicode handling and compression, but I do know what I want to work... I'm happy to open a PR and add the (failing) tests, but someone with more expertise in those fields would have to actually get this working or at least provide me with

Re: [Scikit-learn-general] Sharing objects between Python 2 and 3

2015-01-24 Thread Joel Nothman
They all sound related to the Py3k handling of Unicode, in which case I'm guessing a search should find cases of this issue elsewhere. I'm glad joblib worked in the end, but maybe it's worth leaving an issue on the joblib project so that it could be appropriately tested or documented. On 23

Re: [Scikit-learn-general] Sharing objects between Python 2 and 3

2015-01-24 Thread Gael Varoquaux
They all sound related to the Py3k handling of Unicode, in which case I'm guessing a search should find cases of this issue elsewhere. I'm glad joblib worked in the end, but maybe it's worth leaving an issue on the joblib project so that it could be appropriately tested or documented. joblib

Re: [Scikit-learn-general] Sharing objects between Python 2 and 3

2015-01-23 Thread jni . soma
Hi everyone, Using joblib with compress=0 worked! Is it a joblib bug that compress=3 is not portable? Joel, here are the tracebacks from standard Python pickles of increasing protocols (0, 1, 2), saved in Python 2 and attempting to load them in Python 3:

[Scikit-learn-general] Sharing objects between Python 2 and 3

2015-01-22 Thread jni . soma
Hi all, I'm working on a project that depends on sklearn. I've been up test coverage (which includes saving a RandomForest, so far using joblib serialization), and now I wanted to make the project Python 3-compatible. However, the final roadblock is the sharing of RF objects: I can't load the

Re: [Scikit-learn-general] Sharing objects between Python 2 and 3

2015-01-22 Thread jni . soma
Hi Sebastian, Thanks for the response, but actually joblib doesn't work either: In [1]: from sklearn.externals import joblib In [2]: rf = joblib.load('rf-1.joblib') --- error                                    

Re: [Scikit-learn-general] Sharing objects between Python 2 and 3

2015-01-22 Thread Sebastian Raschka
Hi, Juan, It's been some time, but I remember that I had similar issues. I think it has to do with the numpy arrays that specifically cause problems in pickle. (http://bugs.python.org/issue6784) You could try to use joblib (which should also be more efficient): from sklearn.externals import

Re: [Scikit-learn-general] Sharing objects between Python 2 and 3

2015-01-22 Thread Juan Nunez-Iglesias
Nope, the Py2 RF was saved with joblib! The SO response might work for standard pickling though, I'll give that a try, thanks! On Fri, Jan 23, 2015 at 11:18 AM, Sebastian Raschka se.rasc...@gmail.com wrote: Sorry, I think my previous message was a little bit ambiguous. What I would try

Re: [Scikit-learn-general] Sharing objects between Python 2 and 3

2015-01-22 Thread Joel Nothman
Could you provide the traceback when using pickle? The joblib error is about zipping, which should not be applicable there... On 23 January 2015 at 13:30, Juan Nunez-Iglesias jni.s...@gmail.com wrote: Nope, the Py2 RF was saved with joblib! The SO response might work for standard pickling

Re: [Scikit-learn-general] Sharing objects between Python 2 and 3

2015-01-22 Thread Juan Nunez-Iglesias
Joel, *sorry*, I should probably have mentioned this earlier: joblib.dump takes a compress kwarg, which I used, probably 3 as recommended by the docstring, so that I wouldn't have a bajillion files representing my RF. So the zipping error makes perfect sense, except that I wouldn't expect

Re: [Scikit-learn-general] Sharing objects between Python 2 and 3

2015-01-22 Thread Sebastian Raschka
Sorry, I think my previous message was a little bit ambiguous. What I would try is: 1) Unpickle the original pickle file in Python 2 2) Pickle it via joblib 3) Load it in Python 3 (I think you only did step 3), right? Sorry for the confusion). I also just saw a related SO post that might be