yes, i have this pb during my fetching. i tried 3 times and each times the process doesn't wait 5s as defined in nutch-default.xml. Don't you have the pb?
> Emmanuel wrote: >> I decided to use Fetcher2 instead of Fetcher and i noticed that >> Fetcher2 doesn't act >> on a polite way. I mean it doesn't wait fetcher.server.delay before >> doing another >> request on the same server. >> >> In Fetcher2 (on the last version of trunk), someone has defined this >> option: >> // set non-blocking & no-robots mode for HTTP protocol plugins. >> getConf().setBoolean(Protocol.CHECK_BLOCKING, false); >> getConf().setBoolean(Protocol.CHECK_ROBOTS, false); >> >> In this case, the protocol HTTP doesn't wait crawlDelay defore doing >> another request. >> May I know exactly why ? >> Is it normal or a bug ? >> > > Have you actually observed this wrong behavior during fetching? Fetcher2 > performs blocking in a different way than Fetcher - it controls the > blocking itself, instead of delegating it to the protocol plugin. These > two properties are set to false on purpose. > > > -- > Best regards, > Andrzej Bialecki <>< > ___. ___ ___ ___ _ _ __________________________________ > [__ || __|__/|__||\/| Information Retrieval, Semantic Web > ___|||__|| \| || | Embedded Unix, System Integration > http://www.sigram.com Contact: info at sigram dot com > >
