Dear Wiki user, You have subscribed to a wiki page or wiki category on "Nutch Wiki" for change notification.
The following page has been changed by MikeBrzozowski: http://wiki.apache.org/nutch/MonitoringNutchCrawls ------------------------------------------------------------------------------ 2. Run your preferred crawl script with nohup, like this: `nohup <nutch crawl command or script> &` 3. By default, this will output to nohup.out in the working directory. From the same directory, run: `sh monitorCrawl.sh` - (Alternately, you can process hadoop.log in the logs/ directory by changing the three references to `nohup.out` to `hadoop.log`.) + (Alternately, you can process hadoop.log in the logs/ directory by changing the three references to `nohup.out` to `hadoop.log`. Be aware, though, that by default hadoop.log only contains activity from today, so your counts will reset to zero each night.) This will give you minute-by-minute stats on how many pages nutch tried to fetch and how many failed with errors (e.g. 404, server unreachable).
