On Tue, Aug 9, 2016 at 7:14 PM, Yonik Seeley <ysee...@gmail.com> wrote:
But other than perhaps changing how long a DBQ takes to execute, it > should be unrelated to the question of if other merges can proceed in > parallel. > > A quick look at the lucene IndexWriter code says, no... Lucene DBQ > processing cannot proceed in parallel. > IndexWriter.mergeInit is synchronized (on IW). The DBQ processing is > called from there and thus anything else that needs the IW monitor > will block. Yeah, the impl Lucene has today is fully sync'd on IW while deletes are being resolved (from Query or Term to docIDs). I don't think there's a fundamental reason that it has to be, i.e. "it's just software" and we could fix this (patches welcome!), but it's hairy. I do think we would need a lock on each segment that's resolving its deletes, or maybe even that could be made concurrent e.g. where a thread can pull out one Query at a time and resolve it. In any case, fixing it will likely be complex! Mike McCandless http://blog.mikemccandless.com
