Ben Halsted wrote:
When I merge this stuff, do I need to merge the segments/* for each crawl into a single segments directory? Or is there data in the merged index file that will direct the web component to the correct segment?
Put the segments in a single directory. The index only has the segment name, not its full path.
Please keep folks on the list updated as to how this works for you. I have not yet used things in this way with the mapred branch, but it is a common use case. Perhaps we can add an option to the crawl command to "crawl more" that automates this.
Doug ------------------------------------------------------- This SF.Net email is sponsored by the JBoss Inc. Get Certified Today Register for a JBoss Training Course. Free Certification Exam for All Training Attendees Through End of 2005. For more info visit: http://ads.osdn.com/?ad_id=7628&alloc_id=16845&op=click _______________________________________________ Nutch-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-general
