It is possible to do multiple separate crawls and then merge them together.
However you might run into crawls speed problems if you dont take into account how http connections are managed in nutch. Hth, Paul On Nov 21, 2010 10:14 PM, "Sourabh Kasliwal" <[email protected]> wrote: Is it possible to have multiple crawl database in nutch. I want separate crawl database for each seed url. Is it possible to have the same, if not then is there any better alternative to running many nutch jobs in multiple threads. regards Sourabh
