> My question is this; people have been messing around with neural nets > and machine learning for 40 years; what was the breakthrough that made > alphago succeed so spectacularly.
5 or 6 orders more magnitude CPU power (relative to the late 90s) (*). This means you can try out ideas to see if they work, and get the answer back in hours, rather than years. After 10 hrs it was playing with an elo somewhere between 0 and 1000 (Figure 3 in the alpha go zero paper). I.e. idiot level. That is something like 1100 years of effort on 1995 hardware. They put together a large team (by hobbyist computer go standards) of top people, at least two of which had made strong go programs before. I'd name two other things: dropout (and other regularization techniques) allowed deeper networks; the work on image recognition gave you production-ready CNNs, without having to work through all the dead ends yourself. Also better optimization techniques. Taken together maybe algorithmic advances are worth another order of magnitude. Darren *: The source is the intro to my own book ;-) From memory, I made the estimate as the average of top supercomputer 20 years apart, and a typical high-end PC 20 years apart. https://en.wikipedia.org/wiki/History_of_supercomputing#Historical_TOP500_table -- Darren Cook, Software Researcher/Developer My New Book: Practical Machine Learning with H2O: http://shop.oreilly.com/product/0636920053170.do _______________________________________________ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
