First I used that command: bin/nutch crawl /usr/apache-nutch-2.1/urls -depth 3
than I pressed ctrl + c and fetching has ended as usual. After that I used that command: bin/nutch crawl /usr/apache-nutch-2.1/urls -depth 2 -topN 2 But fetching takes long time than usual. So I think it continues to previous fetching is that true? 2013/3/21 Markus Jelsma-2 [via Lucene] < [email protected]> > If Nutch exits with an error then the segment is bad, a failing thread is > not an error that leads to a failed segments. This means the segment is > properly fetched but just that some records failed. Those records will be > eligible for refetch. > > Assuming you use the crawl command, the updatedb command will be > successful so there should be no issue here. What's the problem? > > > -----Original message----- > > > From:kamaci <[hidden > > email]<http://user/SendEmail.jtp?type=node&node=4049571&i=0>> > > > Sent: Wed 20-Mar-2013 23:48 > > To: [hidden email]<http://user/SendEmail.jtp?type=node&node=4049571&i=1> > > Subject: How to Continue to Crawl with Nutch Even An Error Occurs? > > > > When I crawl with Nutch and error occurs (i.e. when one of threads > doesn't > > come within a time) it stops crawling and exits. > > > > Is there any configuration to continue crawling even a such kind of > error > > occurs at Nutch? > > > > > > > > -- > > View this message in context: > http://lucene.472066.n3.nabble.com/How-to-Continue-to-Crawl-with-Nutch-Even-An-Error-Occurs-tp4049567.html > > Sent from the Nutch - User mailing list archive at Nabble.com. > > > > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > > http://lucene.472066.n3.nabble.com/How-to-Continue-to-Crawl-with-Nutch-Even-An-Error-Occurs-tp4049567p4049571.html > To unsubscribe from How to Continue to Crawl with Nutch Even An Error > Occurs?, click > here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4049567&code=ZnVya2Fua2FtYWNpQGdtYWlsLmNvbXw0MDQ5NTY3fDEyODM4MDc0Mg==> > . > NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > -- View this message in context: http://lucene.472066.n3.nabble.com/How-to-Continue-to-Crawl-with-Nutch-Even-An-Error-Occurs-tp4049567p4049577.html Sent from the Nutch - User mailing list archive at Nabble.com.
