Hi Shankar.
Can you explain in how far these two ideas are related to online learning?
I am not familiar with Gittin's index but value iteration and policy
iteration
are dynamic programming algorithms that assume a model is given.
I would implement those by either giving the full transition model to
the algorithm
or by giving functions that evaluate the model to the algorithm.
For Monte Carlo algorithms I could the connection a bit better though it
probably
depends a lot on your application. This could also be implemented with a
call-back.
Are there any reinforcement learning experts in the project that can
comment on that?
In general having reinforcement learning in the scikit seems
interesting, though I am not
sure whether that is in the current scope. On the other hand, if it
helps refine the API
that would be great :)
Cheers,
Andy
On 03/11/2012 08:47 AM, Shankar Satish wrote:
Hello everyone,
I am a prospective GSOC-2012 student. I have some project ideas that i
would like to bounce-off the community:
I would like add online-learning functionality. To do so, we can
implement some reinforcement-learning algorithms. The problem is
described in terms of an "agent" that needs to choose between various
possible actions, and it gets feedback based on the results.
Reinforcement learning techniques are concerned with finding optimal
policies for the agent under various conditions.
Specifically, i have 2 algorithms in mind to start off with:
1. Computation of the Gittin's index, which is an optimal solution to
the multi-armed bandit problem:
http://en.wikipedia.org/wiki/Multi-armed_bandit
2. Value iteration/policy iteration:
http://en.wikipedia.org/wiki/Markov_decision_process#Value_iteration
So, what do you think about these ideas in general? Also, are they
suitable candidates for a GSOC project?
regards
shankar.
------------------------------------------------------------------------------
Virtualization& Cloud Management Using Capacity Planning
Cloud computing makes use of virtualization - but cloud computing
also focuses on allowing computing to be delivered as a service.
http://www.accelacomm.com/jaw/sfnl/114/51521223/
_______________________________________________
Scikit-learn-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
------------------------------------------------------------------------------
Virtualization & Cloud Management Using Capacity Planning
Cloud computing makes use of virtualization - but cloud computing
also focuses on allowing computing to be delivered as a service.
http://www.accelacomm.com/jaw/sfnl/114/51521223/
_______________________________________________
Scikit-learn-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
