With respect to generative grammars and natural language, Peter Norvig just posted an interesting essay about Noam Chomsky's objections to statistical and probabalistic methods in machine learning as part of the MIT 150th anniversary.
(Peter talks a little about sentences like the "fish" and "buffalo" sentence in passing in addressing how pure Markov models are inadequate for language.) -- Charles http://norvig.com/chomsky.html On Chomsky and the Two Cultures of Statistical Learning At the Brains, Minds, and Machines symposium held during MIT's 150th birthday party, Technology Review reports that Prof. Noam Chomsky derided researchers in machine learning who use purely statistical methods to produce behavior that mimics something in the world, but who don't try to understand the meaning of that behavior. Chomsky compared such researchers to scientists who might study the dance made by a bee returning to the hive, and who could produce a statistically based simulation of such a dance without attempting to understand why the bee behaved that way. "That's a notion of [scientific] success that's very novel. I don't know of anything like it in the history of science," said Chomsky. This essay discusses what Chomsky said, speculates on what he might have meant, and tries to determine the truth and importance of his claims.
