I've been looking for similar stuff recently. I haven't found much, but this is the list of links I've come across so far:
Harvest Man - http://harvestman.freezope.org/ Mechanize - http://wwwsearch.sourceforge.net/mechanize/ Beautiful Soup - http://www.crummy.com/software/BeautifulSoup/ (Neither Beautiful Soup, nor Mechanize are complete crawlers but probably have a lot of the nuts and bolts) If anyone is aware of a book or other documentation like the OP would like, I would be pleased to see it as well. -- http://mail.python.org/mailman/listinfo/python-list
