GitHub user s1monw opened a pull request:
https://github.com/apache/lucene-solr/pull/503
LUCENE-8571: Don't block on FrozenBufferedUpdates#apply during
IW#processEvents
While indexing we try to apply frozen deletes packages concurrently
on indexing threads if necessary. This is done in an opaque way via
IndexWriter#processEvents. Yet, when we commit or refresh we have to
ensure we apply all frozen update packages before we return.
Today we execute the apply method in a blocking fashion which is unnecessary
when we are in a IndexWriter#processEvents context, we block indexing
threads while they could just continue since it's already being applied.
We also might wait in BufferedUpdatesStream when we apply all necessary
updates
were we can continue with other work instead of waiting.
This change also tries to apply the packages that are not currently applied
first in order to not unnecessarily block.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/s1monw/lucene-solr
optimize_apply_frozen_updates
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/lucene-solr/pull/503.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #503
----
commit caeb68509a53766963bbcb1a96ef204ba6cb1492
Author: Simon Willnauer <simonw@...>
Date: 2018-11-21T09:22:41Z
LUCENE-8571: Don't block on FrozenBufferedUpdates#apply during
IW#processEvents
While indexing we try to apply frozen deletes packages concurrently
on indexing threads if necessary. This is done in an opaque way via
IndexWriter#processEvents. Yet, when we commit or refresh we have to
ensure we apply all frozen update packages before we return.
Today we execute the apply method in a blocking fashion which is unncessary
when we are in a IndexWriter#processEvents context, we block indexing
threads while they could just continue since it's already being applied.
We also might wait in BufferedUpdatesStream when we apply all necessary
updates
were we can continue with other work instead of waiting.
This change also tries to apply the packages that are not currently applied
first in order to not unnecessarily block.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
