Dear Friends, Wish you all a happy and prosperous new year 2009!
We are trying to crawl 50,0000 pages. we are surprised to see it is taking 3 full days. Though, we have a server with 4GB RAM and Core 2 DUO processor with 250 GB Hard drive, we had numerous issues like power down, internet connectivity etc..... I am really concerned for the time taking for a single crawl. Please, anyone suggest a best way of crawling faster? Also, request you to suggest any idea to configure the crawling from where it stopped due to any reasons like power cut..internet cutoff etc.... Your advises are highly appreciated! Thank you, Neil
