abernardi597 opened a new pull request, #16778: URL: https://github.com/apache/lucene/pull/16778
Closes #16729. ### Description `HnswConcurrentMergeBuilder` submitted all of its workers through `TaskExecutor#invokeAll` before any of them ran. The intra-merge executor runs a command on the calling thread when it has no spare thread, so when that happened to the first submission, the worker drained the shared batch counter before the submit loop could offer the remaining workers. Those workers then found no work left, and the merge stayed single-threaded for its whole duration even once sibling merges freed threads up. ### Change Adds `TaskExecutor#invokeNoReentry`, which forks as many tasks as the executor will accept and returns immediately, stopping as soon as the executor runs a command on the calling thread rather than letting that consume the caller. `HnswConcurrentMergeBuilder` now runs its first worker on the merge thread and offers the workers it has not placed yet again between batches, so intra-merge threads that become available mid-merge are picked up. `TaskExecutor#invokeAll` is untouched, so search, faceting and the BP reorderers are unaffected. Work distribution is still the shared counter, so dynamic balance across workers is unchanged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
