Hi Andrés: The fetcher.server.delay property as its description says is the number of seconds the fetcher will delay between successive requests to the same server. So, if you configure the fetcher.server.delay property with 2.5 as value, Nutch will wait for 2.5 seconds to make another request to the same server and not between different servers.
Regards. ----- Mensaje original ----- > De: "Andrés Rincón Pacheco" <[email protected]> > Para: [email protected] > Enviados: Viernes, 20 de Noviembre 2015 18:35:48 > Asunto: [MASSMAIL]fetcher.server.delay configuration not working > > Hi, > > I configured the fetcher.server.delay property with 2.5 as value, but when > nutch is fetching urls, the time fetching between urls is bigger that value > configured. > > I attach some information of execution. > > Date fetching Difference in seconds > 2015-11-11 20:56:49,967 5 > 2015-11-11 20:56:54,746 5 > 2015-11-11 20:56:59,391 4 > 2015-11-11 20:57:04,264 5 > 2015-11-11 20:57:09,212 5 > 2015-11-11 20:57:13,873 5 > 2015-11-11 20:57:18,549 5 > > and some lines of log. > > 2015-11-11 20:56:21,674 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=49, fetchQueues.totalSize=359, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:22,674 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=49, fetchQueues.totalSize=359, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:23,675 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=50, fetchQueues.totalSize=359, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:24,676 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=50, fetchQueues.totalSize=359, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:25,677 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=50, fetchQueues.totalSize=359, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:25,981 INFO fetcher.Fetcher - fetching http://www.abcderer > (queue crawl delay=2500ms) > 2015-11-11 20:56:26,677 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=49, fetchQueues.totalSize=358, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:27,678 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=49, fetchQueues.totalSize=358, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:28,679 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=50, fetchQueues.totalSize=358, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:29,679 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=50, fetchQueues.totalSize=358, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:30,680 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=50, fetchQueues.totalSize=358, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:30,791 INFO fetcher.Fetcher - fetching http://www.abcderer > (queue crawl delay=2500ms) > 2015-11-11 20:56:31,681 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=49, fetchQueues.totalSize=357, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:32,681 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=49, fetchQueues.totalSize=357, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:33,682 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=50, fetchQueues.totalSize=357, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:34,683 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=50, fetchQueues.totalSize=357, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:35,684 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=50, fetchQueues.totalSize=357, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:35,960 INFO fetcher.Fetcher - fetching http://www.abcderer > (queue crawl delay=2500ms) > 2015-11-11 20:56:36,684 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=49, fetchQueues.totalSize=356, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:37,685 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=49, fetchQueues.totalSize=356, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:38,686 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=50, fetchQueues.totalSize=356, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:39,687 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=50, fetchQueues.totalSize=356, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:40,552 INFO fetcher.Fetcher - fetching http://www.abcderer > (queue crawl delay=2500ms) > 2015-11-11 20:56:40,688 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=49, fetchQueues.totalSize=355, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:41,689 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=49, fetchQueues.totalSize=355, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:42,690 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=50, fetchQueues.totalSize=355, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:43,691 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=50, fetchQueues.totalSize=355, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:44,691 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=50, fetchQueues.totalSize=355, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:45,128 INFO fetcher.Fetcher - fetching http://www.abcderer > (queue crawl delay=2500ms) > 2015-11-11 20:56:45,692 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=49, fetchQueues.totalSize=354, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:46,693 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=49, fetchQueues.totalSize=354, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:47,694 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=50, fetchQueues.totalSize=354, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:48,695 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=50, fetchQueues.totalSize=354, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:49,696 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=50, fetchQueues.totalSize=354, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:49,967 INFO fetcher.Fetcher - fetching http://www.abcderer > (queue crawl delay=2500ms) > 2015-11-11 20:56:50,697 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=49, fetchQueues.totalSize=353, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:51,698 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=49, fetchQueues.totalSize=353, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:52,699 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=50, fetchQueues.totalSize=353, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:53,700 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=50, fetchQueues.totalSize=353, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:54,701 INFO fetcher.Fetcher - -activeThreads=50, > spinWaiting=50, fetchQueues.totalSize=353, fetchQueues.getQueueCount=1 > 2015-11-11 20:56:54,746 INFO fetcher.Fetcher - fetching http://www.abcderer > (queue crawl delay=2500ms) > > What is the wrong in the configuration? > > Thanks. > Noviembre 13-14: Final Caribeña 2015 del Concurso de Programación ACM-ICPC https://icpc.baylor.edu/regionals/finder/cf-2015
