I was able to look into ${APACHE_NUTCH_HOME}/conf/nutch-default.xml and it
listed a very good explanation of each term that I can use to throttle my
crawling. I should be all set for now unless there's something that I'm
seriously not getting.
---------- Forwarded message ----------
From: Yves S. Garret <[email protected]>
Date: Thu, Mar 28, 2013 at 12:55 PM
Subject: How to set politeness in Nutch 2.1?
To: [email protected]
Hi, I got a really embarrassing question. After googling for this answer
for some time, I can't find out how to set the politeness level when I
crawl through a site. I don't want to bombard a site. Any thoughts or
pointers on how to do this?
