ATB: Merging two sets of crawled data.

2005-12-06 Thread Aled Jones
Thanks Andrzej, I used the bin/nutch crawl command to create both set of data, so however that arranges the indexes will be how I have it. Both index files of the two directories have the same name files so to get a result set of both I'm guessing I'll need to merge these two sets of index files.

Re: ATB: Merging two sets of crawled data.

2005-12-06 Thread Andrzej Bialecki
Ok, let's assume you have crawlA/ and crawlB/, with the usual structure inside (db/, segments/, index/) # ls crawlA/ crawlB/ # mkdir crawl # cd crawl # cp -r ../crawlA/segments . # cp -r ../crawlB/segments/* segments/ # nutch merge index segments/* # ls index/ segments/ That's it. This assumes

ATB: ATB: Merging two sets of crawled data.

2005-12-06 Thread Aled Jones
Nice one Andrzej, works a treat. -Neges Wreiddiol-/-Original Message- Oddi wrth/From: Andrzej Bialecki [mailto:[EMAIL PROTECTED] Anfonwyd/Sent: 06 December 2005 12:16 At/To: nutch-user@lucene.apache.org Pwnc/Subject: Re: ATB: Merging two sets of crawled data. Ok, let's