Chris Hostetter wrote:
you can't ... that's why i said you'd need to rebuild the smaller index completley on a periodic basis (going in the same order as the docs in the
Mmm, the annotations would only be stored in the index. It would be possible to store them elsewhere, so I can investigate that, in which case the rebuild would be possible.
i can also imagine a situation where you break both indexes up into lots of pieces (shards) and use a MultiReader over lots of ParallelReaders ... that way you have much smaller "small" indexes to rebuild when someone annotates an email -- and if hte shards are organized by date, you're less likely to ever need to rebuild many of them since people will tend to
Data will be 'sharded' anyway, by date of some granularity. Looking at the source for MultiReader/MultiSearcher, they are single threaded. Is there a performance trade off between single-thread/many small indexes and single-thread/some large indexes. Can a MultiReader work with one..n reader per thread, something like a thread pool of IndexReaders. I expect it would be faster to run the searches in parallel?
Disclaimer: all of this is purely brainstorming, i've never actually tried anything like this, it may be more trouble then it's worth.
:) Thanks for the sounding board - it's always useful to get new ideas! Antony --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
