Hey all, Just curious if anyone has any ideas on what to test for on this weird issue for me. On my laptop from home, I have a list of about 140K urls to inject, and then I get back roughly 12000 urls to retrieve/parse/index. This takes about an hour. Sometimes less. I have a cable modem at home. Now, when I then push this exact same code to my hosted server (dual proc, 2 gig ram. Basically, a real server) it takes roughly 5-6 hours to do the exact same thing.
Has anyone run into this? Ideas on what I could check to see where the bog down could be at? If it was a dns issue, how would I go about checking look up times? (I have seen this as a possible issue in the forums) Thanks, Scott -- View this message in context: http://www.nabble.com/Crawl-slow-on-one-machine%2C-fast-on-another-tf3358653.html#a9342187 Sent from the Nutch - User mailing list archive at Nabble.com.
