On Jul 10, 2013, at 9:40 PM, Ed Kohlwey <[email protected]> wrote: > So, the core of the issue here I think is the partitioning scheme for > parallel search over index shards. > > The current approach really strongly favors hash partitioning and locking on > a per-shard incrementer, regardless of what codec is in use. It would be nice > to use all the many Lucene libraries, particularly the searcher, with > indexing storage schemes that don't conform to this model. I understand how > others are doing things now but I would like to discuss an improvement that > suits more use cases. > > What could we do to change that?
Ed, I think what would have to change is a patch demonstrating what you mean (which I know is a significant undertaking), as the current approach is very tightly woven into almost all of Lucene's core. It's not to say that it can't be done (it's just software), it's just saying that it is likely a pretty significant overhaul and it would take a lot of scrutiny, performance testing, etc. and the committers here may not prioritize it given their other work. Personally, if you can pass the tests, show that it performs and it makes Lucene more flexible and broadly used across other systems, I would be for it. -Grant --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
