In email message <[EMAIL PROTECTED]>, Adnan Darwiche wrote: >Colleagues, > > Would someone know of a good reference/book on Hidden Markov Models >and their applications? Adnan, As mentioned already, the widely-cited article by Rabiner and the recent book by Jelinek are good places to start for discussions of HMMs in speech and language modeling. The paper by A. Poritz (ICASSP, 1988), also has many good references on early work on HMMs. As also pointed out, there have been numerous applications outside of speech and language using HMMs in recent years, e.g., - computational biology (e.g., Haussler et al) - computer vision (e.g., Bregler/CVPR97, Brand/NIPS 98, etc) - economics (e.g., Hamilton) - control theory (Elliott, Aggoun and Moore, Springer-Verlag, 1995) - rainfall models (e.g., Hughes, Guttorp and Charles, 1999) - + several others Complementing all of this has been a plethora of work on developing general extensions to the basic HMM, both to the underlying model and to associated learning algorithms, e.g., the work of Jordan and colleagues (factorial HMMs, tree-structured HMMs) and the work of Singer, Tishby, and colleagues (e.g., hierarchical HMMs) (+ lots of other neat extensions, these are just 2 of the more well-known ones). A nice recent review of much of this work is by Bengio at: http://www.icsi.berkeley.edu/~jagota/NCS/vol2.html As is typical in learning research, as well as the "computer science HMM world" there is a "parallel world" of work in statistics, with a fair degree of overlap between the two, but also some fairly different ideas. The recent text by MacDonald and Zucchini (1997) has a good summary of much of the statistical work One of the most useful developments (from my own viewpoint at least) has been the realization that one can treat a HMM as a type of belief network - in fact once one does this one sees immediately that it is in fact a relatively *simple* model, i.e., x_1 -- x_2 --...... -- x_T | | | | | | y_1 y_2 y_T where here the x's are the hidden states and y's are the observed variables, and the directionality of the edges is usually assumed to be from x_t to x_t+1 and from x_t to y_t. This is a very handy way to teach the basic ideas of a HMM to students who already know about belief networks (compared to the description provided in Rabiner say, which lacks a simple picture like this). Strictly speaking the picture above should be called a "first-order HMM", or something along those lines. Much of the work in recent years can this be summarized as: (a) extending the dependence structure of the "first-order HMM" to provide a richer semantics for modeling structure in sequences, and/or (b) developing specific models for specific applications. My impression is that we have been alot more succesful in (a) than in (b), i.e., there are many more neat new algorithms than there are neat new applications. But I suspect that there are lots of untried applications which could benefit from the newer models and algorithms. To get back to your original question, I would recommend you start with a look at Bengio's review paper, which primarily focuses on the developments in models and algorithms. For applications-related work, I don't know of any survey paper which has a comprehensive survey of the many applications, so you will probably have to just look at the ones that seem most relevant to your interests. Hope this info is of some use as well to other UAI-list readers who may be interested in HMMs. A point worth re-emphasizing for this audience is that it is often very convenient to think of HMMs as belief networks with a simple temporal structure, i.e., if you understand belief networks, then HMMs can (almost) be treated as a special case. (see the Smyth, Heckerman, Jordan (1997) paper for more on these lines) Regards Padhraic
