Re: [Scikit-learn-general] Persisting models

On Mon, Aug 24, 2015 at 06:02:19PM -0400, Andreas Mueller wrote: > I think the real solution is to provide backward-compatible ``__getattr__`` > and > ``__setattr_``. It's a lot of work to support this and do QA. I am not sure we want to add this to our plate. I would personnally rather support

Re: [Scikit-learn-general] Persisting models

If you want an absolutely bullet proof way of doing it. Build and serialize your model during docker build stage. It limits your hosting alternatives, but it is guaranteed to work. On Tue, Aug 25, 2015, 00:19 Stefan van der Walt wrote: > > On 2015-08-24 15:08:57, Andreas Mueller wrote: > > > Ag

Re: [Scikit-learn-general] Persisting models

On 2015-08-24 15:08:57, Andreas Mueller wrote: > > Agreed—this is exactly the type of use case I want to support. > > Pickling won't work here, but using HDF5 like MNE does would > > probably be close to ideal (thanks to Chris Holdgraf for the > > heads-up): > > I'm not sure how this solves th

Re: [Scikit-learn-general] Persisting models

Agreed—this is exactly the type of use case I want to support. Pickling won't work here, but using HDF5 like MNE does would probably be close to ideal (thanks to Chris Holdgraf for the heads-up): I'm not sure how this solves the issue, can you elaborate? You still need to map the old data structu

Re: [Scikit-learn-general] Persisting models

That’s true. Often, I create a separate venv for each project plus manifest. I also push everything to a private git repo (next to a couple of “regular” back up solutions) — I am really paranoid when it comes to back-ups and version control :P. > But if you didn't snapshot all libraries you ar

Re: [Scikit-learn-general] Persisting models

I think the real solution is to provide backward-compatible ``__getattr__`` and ``__setattr_``. Theano seems able to do that (at least that is what I was told). It is unclear weather we want to do this. If we want to do this, we probably only want it post 1.0 On 08/19/2015 02:35 AM, Joel Nothm

Re: [Scikit-learn-general] Persisting models

On 08/19/2015 12:37 AM, Sebastian Raschka wrote: >> if the unpickling failed, >> >what would you do? > One lesson “scientific research” taught me is to store the code and dataset > along with a “make” file under version control (git):). I would just run my > make file to re-construct the mode

Re: [Scikit-learn-general] Persisting models

. 404.495.7220 x 4008 f. 404.795.7221 > Nexidia Corporate | 3565 Piedmont Road, Building Two, Suite 400 | Atlanta, > GA 30305 > > > > > -Original Message- > From: Alexandre Gramfort [mailto:alexandre.gramf...@m4x.org] > Sent: Thursday, August 20, 2015 4:28 AM > T

Re: [Scikit-learn-general] Persisting models

Corporate | 3565 Piedmont Road, Building Two, Suite 400 | Atlanta, GA 30305 -Original Message- From: Alexandre Gramfort [mailto:alexandre.gramf...@m4x.org] Sent: Thursday, August 20, 2015 4:28 AM To: scikit-learn-general Subject: Re: [Scikit-learn-general] Persisting models hi,

Re: [Scikit-learn-general] Persisting models

hi, > Agreed—this is exactly the type of use case I want to support. > Pickling won't work here, but using HDF5 like MNE does would > probably be close to ideal (thanks to Chris Holdgraf for the > heads-up): > > https://github.com/mne-tools/mne-python/blob/master/mne/_hdf5.py For your info Eric L

Re: [Scikit-learn-general] Persisting models

See https://github.com/scikit-learn/scikit-learn/issues/1596 On 19 August 2015 at 16:35, Joel Nothman wrote: > Frequently the suggestion of supporting PMML or similar is raised, but > it's not clear whether such models would be importable in to scikit-learn, > or how to translate scikit-learn tr

Re: [Scikit-learn-general] Persisting models

Frequently the suggestion of supporting PMML or similar is raised, but it's not clear whether such models would be importable in to scikit-learn, or how to translate scikit-learn transformation pipelines into its notation without going mad, etc. Still, even a library of exporters for individual com

Re: [Scikit-learn-general] Persisting models

Oh wow, thanks for the link, I just skimmed over the code, but this is an interesting idea snd looks like the sort of thing that would make my life easier in future. I will dig into that! That’s great, thanks! > On Aug 19, 2015, at 12:58 AM, Stefan van der Walt > wrote: > > On 2015-08-18 21:

Re: [Scikit-learn-general] Persisting models

On 2015-08-18 21:37:41, Sebastian Raschka wrote: > I think for “simple” linear models, it would be not a bad idea > to save the weight coefficients in a log file or so. Here, I > think that your model is really not that dependent on the > changes in the scikit-learn code base (for example, ima

Re: [Scikit-learn-general] Persisting models

> if the unpickling failed, > what would you do? One lesson “scientific research” taught me is to store the code and dataset along with a “make” file under version control (git) :). I would just run my make file to re-construct the model and pickle the objects. > I imagine that the ideal way

Re: [Scikit-learn-general] Persisting models

Hi Sebastian On 2015-08-18 20:47:12, Sebastian Raschka wrote: > Stefan, I have no experience with this problem in particular > since I am not pickling objects that often. However, I deployed > a webapp some time ago on Pythonanywhere > (http://raschkas.pythonanywhere.com >

Re: [Scikit-learn-general] Persisting models

Stefan, I have no experience with this problem in particular since I am not pickling objects that often. However, I deployed a webapp some time ago on Pythonanywhere (http://raschkas.pythonanywhere.com ) and meanwhile they upgraded their scikit-learn module;

Re: [Scikit-learn-general] Persisting models

On 2015-08-18 20:26:15, JAGANADH GOPINADHAN wrote: > Use joblib or pickle to achieve this Any method that relies on pickle is broken when you upgrade scikit-learn or move to a different system. I would like to persist models for as long as possible; thus having to deal with pickling errors,

Re: [Scikit-learn-general] Persisting models

Hi Use joblib or pickle to achieve this -Original Message- From: "Stefan van der Walt" Sent: ‎8/‎19/‎2015 8:40 AM To: "scikit-learn-general" Subject: [Scikit-learn-general] Persisting models Hi all, What, currently, is the recommended way for storing trained model

Re: [Scikit-learn-general] Persisting models

I would still say the best way to go is joblib (because of the NumPy arrays). But I would also be interested in better alternatives (if there are any?). PS: One caveat: joblib didn’t work well for me in the past when I was deploying it on Apache servers for webapps on “cheap”, hosted server rack

[Scikit-learn-general] Persisting models

Hi all, What, currently, is the recommended way for storing trained models to disk for later use? Regards Stéfan -- ___ Scikit-learn-general mailing list Scikit-learn-general@