If you fetch too hard, your DNS-server may not be able to keep up.
> Hi Lewis, > > I stopped fetcher and started it on the same segment again. > But before doing that I turned off modem and fetcher started giving > Unknown.Host exception. It was not giving any error, with dsl failure, > i.e. I was not able to connect to any sites. Again this is nutch-1.2. > > Thanks. > Alex. > > > > > > -----Original Message----- > From: lewis john mcgibbney <[email protected]> > To: user <[email protected]> > Sent: Tue, Aug 23, 2011 6:37 am > Subject: Re: fetcher runs without error with no internet connection > > > Hi Alex, > > Did you get anywhere with this? > > What condition led to you seeing unknown host exception? > > Unless segment gets corrupted, I would assume you could fetch again. > Hopefully you can confirm this. > > On Tue, Aug 16, 2011 at 9:23 PM, <[email protected]> wrote: > > Hello, > > > > After running bin/nutch fetch $segment for 2 days, internet connection > > was lost, but nutch did not give any errors. Usually I was seeing > > Unknown host exception before. > > Any ideas what happened and is it OK to stop the fetch and run it again > > on the same (old) segment? This is nutch -1.2 > > > > Thanks. > > Alex.
