Kevin A. Burton wrote:
Is it possible to take an existing index (say 1G) and break it up into a
number of smaller indexes (say 10 100M indexes)...
I don't think theres currently an API for this but its certainly
possible (I think).
Yes, it is theoretically possible but not yet implemented.
An easy way to implement it would be to subclass FilterIndexReader to
return a subset of documents, then use IndexWriter.addIndexes() to write
out each subset as a new index. Subsets could be ranges of document
numbers, and one could use TermPositions.skipTo() to accelerate the
TermPositions subset implementation, but this still wouldn't be quite as
fast as an index splitter that only reads each TermPositions once. If
we added a lower-level index writing API then one could use that to
implement this...
Doug
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
