Ben Halsted wrote:
When I merge this stuff, do I need to merge the segments/* for each crawl into a single segments directory? Or is there data in the merged index file that will direct the web component to the correct segment?
Put the segments in a single directory. The index only has the segment name, not its full path.
Please keep folks on the list updated as to how this works for you. I have not yet used things in this way with the mapred branch, but it is a common use case. Perhaps we can add an option to the crawl command to "crawl more" that automates this.
Doug
