Hello, I run crawl command with -depth 7 -topN -1 on my linux box with 1.5Mps internet, amd 3.1ghz processor, 4GB memory, Fedora Linux 14, nutch 1.2. After 1-2 days nutch takes 98% of cpu. My seed file includes about 3500 domains and I put fetch.external links to false.
Is this normal? If not, what can be done to improve it? Thanks. Alex.
