Hi,I'm using the crawl tool in nutch to crawl web starting from a set of URL seeds. The crawl normally finishes after the specified depth was reached. Is it possible to terminate after a pre-defined number of pages or a text data of a pre-defined size (e.g. 500 MB) has been crawled? Thank you for any hints!
Regards, Olena
