http://github.com/intridea/ruby_intrigue/tree/master/crawler/
It was mainly an exercise to see the difference between non-threaded, threads, forks and queued threads, but it may have some nuggets of inspiration for you. On Aug 31, 2010, at 11:10 AM, Ray Champagne wrote: > > Care to share that web crawler code with me? Project starting here > that would really benefit from a jump start with that part already > written. > > On Tuesday, August 31, 2010, Zaphod Beeblebrox > <[email protected]> wrote: >> >> I was at Lone Star Ruby Conference last week and sat in on this seminar. >> This is a really interesting talk on the evolution of software design and >> how wrong assumptions have caused us some issues in the past. >> >> http://confreaks.net/videos/282-lsrc2010-real-software-engineering >> >> LSRC was awesome on my fronts...during the training day, among other things, >> I created a multi-threaded web crawler and a kick ass asteroids clone that >> took about 150 lines of code. Then, on the day I get back, Rails 3 is >> released! How good can it get :) >> >> >> >> > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology-Michael-Dinowitz/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-community/message.cfm/messageid:326357 Subscription: http://www.houseoffusion.com/groups/cf-community/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-community/unsubscribe.cfm
