Call for papers
NIPS-08 Workshop on Model Uncertainty and Risk in Reinforcement Learning
http://www.cs.uwaterloo.ca/~ppoupart/nips08-workshop.html
Whistler, BC, Canada
December 13, 2008
Important Dates
---------------
* Oct 30: submission deadline
* Nov 4: notification of acceptance
Overview
--------
Reinforcement Learning (RL) problems are typically formulated in terms of
Stochastic Decision Processes (SDPs), or a specialization thereof,
Markovian Decision Processes (MDPs), with the goal of identifying an
optimal control policy. In contrast to planning problems, RL problems are
characterized by the lack of complete information concerning the
transition and reward models of the SDP. Hence, algorithms for solving RL
problems need to estimate properties of the system from finite data.
Naturally, any such estimated quantity has inherent uncertainty. One of
the interesting and challenging aspects of RL is that the algorithms have
partial control over the data sample they observe, allowing them to
actively control the amount of this uncertainty, and potentially trade it
off against performance.
Reinforcement Learning as a field of research, has over the past few years
seen renewed interest in methods that explicitly consider the
uncertainties inherent to the learning process. Indeed, interest in
data-driven models that take uncertainties into account, goes beyond RL to
the fields of Control Theory, Operations Research and Statistics. Within
the RL community, relevant lines of research may be classified into the
following (partially overlapping) sub-fields:
1- Bayesian RL. Bayesian methods attempt to explicitly model uncertainties
using posterior probability distributions, computed using Bayes' rule.
Such Bayesian modeling may be used in estimating the MDP's transition and
reward distributions; or in estimating other quantities that are more
directly related to performance, such as value function and policy
gradient.
2- Risk sensitive and robust dynamic decision making. These methods use
information beyond the expected return, to compute policies that are
robust to inaccuracies in the estimated model. Such quantities include
quantiles, as well as higher order moments of the return random variable.
A closely related family of methods use expectations of non-linear
mappings of the return, as their measures of performance.
3- RL with confidence intervals. This research is concerned with methods
that employ Frequentist measures of model uncertainties, based on
confidence intervals. Much of this research is focused on on-line
algorithms, whose performance is evaluated concurrently with the learning
process.
4- Applications of risk-aware and uncertainty-aware decision-making.
Applications in mission critical tasks, finance, and other risk-sensitive
domains, where uncertainties have to be taken into account, in order to
establish a level of worst-case performance, or to guarantee a minimum
level of performance that may be achieved with high probability.
This workshop is aimed at bringing together researchers working in these
and related fields, allow them to present their current research, and
discuss possible directions for future work. We intend to focus on
possible interactions between the sub-fields listed above, as well as on
interactions with other related fields, which are outside of the current
RL mainstream.
Workshop format
---------------
This is a one-day workshop consisting of:
1- Invited talks
2- Contributed talks
3- Panel discussions
3.1- Models that work and those that don't: participants will discuss
specific applications and theoretical models and share experience regarding
the effectiveness of different approaches.
3.2- Benchmarks and challenges: discussion of some proposals for sample
problems that encompass the core challenges of model uncertainty and
risk sensitive control that could serve as benchmarks and/or challenges.
4- Poster session
Call for Contributions
----------------------
Participants are invited to submit either a technical paper (eight pages
in the conference format) or an extended abstract (up to two pages)
describing research relevant to the workshop. Submissions should be sent
via email to Pascal Poupart at [EMAIL PROTECTED] by Oct 30th
in Postscript, PDF, or MS Word format. Previously published work that is
reworded, summarized or extended may be submitted to the workshop.
However, priority will be given to novel work. If the papers are of
sufficient quantity and quality, we will seek to publish them as an
edited book or journal special issue.
Important Dates
---------------
Oct 30: submission deadline
Nov 4: notification of acceptance
Dec 13: workshop in Whistler
Workshop webpage
----------------
http://www.cs.uwaterloo.ca/~ppoupart/nips08-workshop.html
Organizing Committee
--------------------
1- Yaakov Engel ([EMAIL PROTECTED])
2- Mohammad Ghavamzadeh (INRIA - Team SequeL, [EMAIL PROTECTED]),
3- Shie Mannor (McGill University, [EMAIL PROTECTED])
4- Pascal Poupart (University of Waterloo, [EMAIL PROTECTED])
--
------------------------
Pascal Poupart
Assistant Professor
David R. Cheriton School of Computer Science
University of Waterloo
200 University Avenue West
Waterloo, Ontario
Canada N2L 3G1
------------------------
Web: http://www.cs.uwaterloo.ca/~ppoupart
Email: [EMAIL PROTECTED]
Telephone: 1-519-888-4567x36239
Fax: 1-519-885-1208
------------------------
_______________________________________________
uai mailing list
[email protected]
https://secure.engr.oregonstate.edu/mailman/listinfo/uai
