Re: [Scikit-learn-general] Sharing objects between Python 2 and 3

2015-01-24 Thread jni . soma
Hi Gaël, I know virtually nothing about string/unicode handling and compression, but I do know what I want to work... I'm happy to open a PR and add the (failing) tests, but someone with more expertise in those fields would have to actually get this working or at least provide me with extens

Re: [Scikit-learn-general] Sharing objects between Python 2 and 3

2015-01-24 Thread Gael Varoquaux
> They all sound related to the Py3k handling of Unicode, in which case I'm > guessing a search should find cases of this issue elsewhere. I'm glad joblib > worked in the end, but maybe it's worth leaving an issue on the joblib project > so that it could be appropriately tested or documented. job

Re: [Scikit-learn-general] Sharing objects between Python 2 and 3

2015-01-24 Thread Joel Nothman
They all sound related to the Py3k handling of Unicode, in which case I'm guessing a search should find cases of this issue elsewhere. I'm glad joblib worked in the end, but maybe it's worth leaving an issue on the joblib project so that it could be appropriately tested or documented. On 23 Januar

Re: [Scikit-learn-general] Sharing objects between Python 2 and 3

2015-01-23 Thread jni . soma
Hi everyone, Using joblib with compress=0 worked! Is it a joblib bug that compress=3 is not portable? Joel, here are the tracebacks from standard Python pickles of increasing protocols (0, 1, 2), saved in Python 2 and attempting to load them in Python 3: --

Re: [Scikit-learn-general] Sharing objects between Python 2 and 3

2015-01-22 Thread Juan Nunez-Iglesias
Joel, *sorry*, I should probably have mentioned this earlier: joblib.dump takes a "compress" kwarg, which I used, probably 3 as recommended by the docstring, so that I wouldn't have a bajillion files representing my RF. So the zipping error makes perfect sense, except that I wouldn't expect g

Re: [Scikit-learn-general] Sharing objects between Python 2 and 3

2015-01-22 Thread Joel Nothman
Could you provide the traceback when using pickle? The joblib error is about zipping, which should not be applicable there... On 23 January 2015 at 13:30, Juan Nunez-Iglesias wrote: > Nope, the Py2 RF was saved with joblib! > > The SO response might work for standard pickling though, I'll give t

Re: [Scikit-learn-general] Sharing objects between Python 2 and 3

2015-01-22 Thread Juan Nunez-Iglesias
Nope, the Py2 RF was saved with joblib! The SO response might work for standard pickling though, I'll give that a try, thanks! On Fri, Jan 23, 2015 at 11:18 AM, Sebastian Raschka wrote: > Sorry, I think my previous message was a little bit ambiguous. > What I would try is: > 1) Unpickle the

Re: [Scikit-learn-general] Sharing objects between Python 2 and 3

2015-01-22 Thread Sebastian Raschka
Sorry, I think my previous message was a little bit ambiguous. What I would try is: 1) Unpickle the original pickle file in Python 2 2) Pickle it via joblib 3) Load it in Python 3 (I think you only did step 3), right? Sorry for the confusion). I also just saw a related SO post that might be ver

Re: [Scikit-learn-general] Sharing objects between Python 2 and 3

2015-01-22 Thread jni . soma
Hi Sebastian, Thanks for the response, but actually joblib doesn't work either: In [1]: from sklearn.externals import joblib In [2]: rf = joblib.load('rf-1.joblib') --- error                                     T

Re: [Scikit-learn-general] Sharing objects between Python 2 and 3

2015-01-22 Thread Sebastian Raschka
Hi, Juan, It's been some time, but I remember that I had similar issues. I think it has to do with the numpy arrays that specifically cause problems in pickle. (http://bugs.python.org/issue6784) You could try to use joblib (which should also be more efficient): >>> from sklearn.externals impo

[Scikit-learn-general] Sharing objects between Python 2 and 3

2015-01-22 Thread jni . soma
Hi all, I'm working on a project that depends on sklearn. I've been up test coverage (which includes saving a RandomForest, so far using joblib serialization), and now I wanted to make the project Python 3-compatible. However, the final roadblock is the sharing of RF objects: I can't load the