Already applied that patch which is actually 721, I was part of that discussion at the time. The difference now is that I moved on a linux box, and working pseudo-distributed hadoop, also I took a later nutch snapshot.
By the way I could not apply Time-Bomb 770 patch command gives me errors. I applied 769 and tried it with a level at threshold at 5 no real improvement either. 2009/11/27 Julien Nioche <lists.digitalpeb...@gmail.com> > there is a jira + a discussion on the mailing list on this. This is a > synchronisation problem which has already been reported, patched but not > yet > committed. See https://issues.apache.org/jira/browse/NUTCH-719 > > J. > > 2009/11/27 MilleBii <mille...@gmail.com> > > > My fetch run is getting to the end now I have the following logs towards > > the > > end > > > > 2009-11-27 19:07:43,866 INFO fetcher.Fetcher - -activeThreads=100, > > spinWaiting=100, fetchQueues.totalSize=12 > > 2009-11-27 19:07:44,866 INFO fetcher.Fetcher - -activeThreads=100, > > spinWaiting=100, fetchQueues.totalSize=12 > > 2009-11-27 19:07:45,866 INFO fetcher.Fetcher - -activeThreads=100, > > spinWaiting=100, fetchQueues.totalSize=12 > > 2009-11-27 19:07:46,866 INFO fetcher.Fetcher - -activeThreads=100, > > spinWaiting=100, fetchQueues.totalSize=12 > > 2009-11-27 19:07:47,867 INFO fetcher.Fetcher - -activeThreads=100, > > spinWaiting=100, fetchQueues.totalSize=12 > > 2009-11-27 19:07:47,867 WARN fetcher.Fetcher - Aborting with 100 hung > > threads. > > > > It was same on previous run, the fetchqueue is not "empty", what does it > > mean ? Looks like there is 'problem' > > > > > > 2009/11/27 Andrzej Bialecki <a...@getopt.org> > > > > > MilleBii wrote: > > > > > >> You mean map/reduce tasks ??? > > >> > > > > > > Yes. > > > > > > > > > Being in pseudo-distributed / single node I only have two maps during > > the > > >> fetch phase... so it would be back to the URLs distribution. > > >> > > > > > > Well, yes, but my explanation is still valid. Which unfortunately > doesn't > > > change the situation. > > > > > > Next week I will be working on integrating the patches from Julien, and > > if > > > time permits I could perhaps start working on a speed monitoring to > lock > > out > > > slow servers. > > > > > > > > > -- > > > Best regards, > > > Andrzej Bialecki <>< > > > ___. ___ ___ ___ _ _ __________________________________ > > > [__ || __|__/|__||\/| Information Retrieval, Semantic Web > > > ___|||__|| \| || | Embedded Unix, System Integration > > > http://www.sigram.com Contact: info at sigram dot com > > > > > > > > > > > > -- > > -MilleBii- > > > > > > -- > DigitalPebble Ltd > http://www.digitalpebble.com > -- -MilleBii-
