Any RL package will have be heavily focused on non-iid data (timeseries,
basically) with the additional difficulty of the agent
effecting/interacting with the environment it is operating in. I agree with
you Gael - many packages for "deep learning" also don't handle this type of
data/these models (RNNs specifically) all that well. The models used are
also rarely "off the shelf" and generally incorporate prior knowledge about
the problem structure directly into the model, requiring customizing
connections, losses, and so on. All the things that make it really hard to
just load an object, fit, and predict.
It is a hard problem, and a good RL package would be useful. PyRL (
https://github.com/amarack/python-rl/tree/master/pyrl) and RL-Glue both
seem nice, but interacting with new environments is a classic case of "easy
to do specifically, hard to do generically". The generic case usually blows
up into defining a domain specific language (DSL), which is bad when you
are already dealing with at least one DSL (Theano/Tensorflow) if not more.
On Wed, Mar 2, 2016 at 3:58 PM, Gael Varoquaux <
gael.varoqu...@normalesup.org> wrote:
> Pardon me if I am saying something stupid, but isn't Theano/Tensorflow
> about deep learning and not reinforcement learning. RL can be done with
> deep learning, but it's more than that, and I suspect that it requires a
> different API, in particular with the notion of actions.
>
> G
>
> On Wed, Mar 02, 2016 at 03:29:18PM -0500, Sebastian Raschka wrote:
> > You mean a scikit-like interface to Theano/Tensorflow? That’s actually
> what skflow intends to do.
>
> > > On Mar 2, 2016, at 3:02 PM, Nadim Farhat <nadim.far...@gmail.com>
> wrote:
>
> > > I was just thinking the same but , how about just making pipelines to
> Theano , TensorFlow ?
>
> > > On Wed, Mar 2, 2016 at 3:00 PM Sebastian Raschka <se.rasc...@gmail.com>
> wrote:
> > > I am not a core developer and thus really can’t comment about the
> scope of scikit-learn here :P. But I am a curious about how to implement it
> in scikit-learn efficiently. I think an implementation based on Theano or
> TensorFlow may be a better place for such a module (maybe skflow, which has
> a scikit-like API https://github.com/tensorflow/skflow?)
>
> > > > On Mar 2, 2016, at 2:21 PM, Michał Koziarski <
> michalkoziar...@gmail.com> wrote:
>
> > > > Hello everyone,
>
> > > > As far as I can tell, except PyBrain (which doesn't seem to be
> actively developed) there are no reinforcement learning libraries in
> Python. I was wondering if community would be interested in using one and
> making it a part of scikit-learn. Does it lie within the scope of the
> project?
>
> > > > Very raw idea is, as follows:
> > > > - to design common interface, similar to what is used in other parts
> of scikit-sklearn;
> > > > - to implement established RL algorithms, reliant heavily on
> estimators available in scikit-learn;
> > > > - and to prepare practical examples of what RL can be used for, to
> both supplement documentation and encourage people not yet familiar with RL
> to experiment with it in their own projects.
>
> > > > Once again, I would mostly like to know whether it event lies within
> the scope of the project, or if it just won't be added because of project
> philosophy. Other than that, I would obviously appreciate any feedback.
>
> > > > About me: I am a last master's CS student. My research interests
> involve machine learning in general and reinforcement learning in
> particular; this year I hope to start my PhD on the latter. My master's
> thesis revolves around transfer learning in RL. I have experience with
> programming in industry and on large projects.
>
> > > > Cheers,
> > > > Michał
> > > >
> ------------------------------------------------------------------------------
> > > > Site24x7 APM Insight: Get Deep Visibility into Application
> Performance
> > > > APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> > > > Monitor end-to-end web transactions and take corrective actions now
> > > > Troubleshoot faster and improve end-user experience. Signup Now!
> > > >
> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140_______________________________________________
> > > > Scikit-learn-general mailing list
> > > > Scikit-learn-general@lists.sourceforge.net
> > > > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
> > >
> ------------------------------------------------------------------------------
> > > Site24x7 APM Insight: Get Deep Visibility into Application Performance
> > > APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> > > Monitor end-to-end web transactions and take corrective actions now
> > > Troubleshoot faster and improve end-user experience. Signup Now!
> > > http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
> > > _______________________________________________
> > > Scikit-learn-general mailing list
> > > Scikit-learn-general@lists.sourceforge.net
> > > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
> > >
> ------------------------------------------------------------------------------
> > > Site24x7 APM Insight: Get Deep Visibility into Application Performance
> > > APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> > > Monitor end-to-end web transactions and take corrective actions now
> > > Troubleshoot faster and improve end-user experience. Signup Now!
> > >
> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140_______________________________________________
> > > Scikit-learn-general mailing list
> > > Scikit-learn-general@lists.sourceforge.net
> > > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
> >
> ------------------------------------------------------------------------------
> > Site24x7 APM Insight: Get Deep Visibility into Application Performance
> > APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> > Monitor end-to-end web transactions and take corrective actions now
> > Troubleshoot faster and improve end-user experience. Signup Now!
> > http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
> > _______________________________________________
> > Scikit-learn-general mailing list
> > Scikit-learn-general@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
> --
> Gael Varoquaux
> Researcher, INRIA Parietal
> NeuroSpin/CEA Saclay , Bat 145, 91191 Gif-sur-Yvette France
> Phone: ++ 33-1-69-08-79-68
> http://gael-varoquaux.info http://twitter.com/GaelVaroquaux
>
>
> ------------------------------------------------------------------------------
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
> _______________________________________________
> Scikit-learn-general mailing list
> Scikit-learn-general@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
