What is a safe number to delay between page requests from the same host? I want to crawl as much information as possible in the shortest amount of time, but I also don't want to hurt the server i'm crawling.... What do you guys use? I am using 5 seconds right now.
