Hi, the fault of fetcher should not have something to do with screen. Please checkout the newest nutch version from cvs and try again. Bye Matthias
Nutch Crawler schrieb:
Hello,
After running 'screen', I ran the following script using nutch 0.5:
while [ true ] do nutch generate $db $segdir -topN $segsize seg=`ls -d $segdir/2* | tail -1` nutch fetch $seg nutch updatedb $db $seg nutch analyze $db 2 done
I detached from screen, and let the crawler run.
It ran successfully for ~7 hours.
When I came in the next day to check on the log files,
I found that the crawler was hung for 10 hours.
It was in the middle of fetching pages.
I typed 'screen -r', and hit Control-C. For some reason
the crawler started running again, at the updatedb step.
However, a few hours later the crawl was hung again. This happened several times. How do i fix this?
Thanks, Ralph
------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader's Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click _______________________________________________ Nutch-developers mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/nutch-developers
-- http://www.eventax.com - eventax GmbH http://www.umkreisfinder.de - Die Suchmaschine f�r Lokales und Events
------------------------------------------------------- This SF.Net email is sponsored by: Sybase ASE Linux Express Edition - download now for FREE LinuxWorld Reader's Choice Award Winner for best database on Linux. http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click _______________________________________________ Nutch-developers mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/nutch-developers
