[
https://issues.apache.org/jira/browse/LUCENE-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12716118#action_12716118
]
Jed Wesley-Smith commented on LUCENE-1609:
------------------------------------------
We get hit by this too. We'd love to see a fix and we'd agree that up-front
initialisation would work for us.
AFAICT there are a number of other potential subtle concurrency issues with
{{TermInfosReader}}:
# lack of {{final}} on fields - a number of fields ({{directory}}, {{segment}},
{{fieldInfos}}, {{origEnum}}, {{enumerators}} etc.) are never written to after
construction and should be declared {{final}} for better publication semantics
# unsafe publication of {{indexDivisor}} and {{totalIndexInterval}} these
fields are not written to under lock and in a worst-case could be unstable
under use.
# {{close()}} calls {{enumerators.set(null)}} which only clears the value for
the calling thread.
Making the {{TermInfosReader}} more immutable would address some of these
issues.
As far as the root problem goes, uncontended synchronisation is generally _very
fast_, but significantly slows down once a lock becomes contended. The kind of
pattern employed here (do something quite expensive but only once) is not an
ideal use of synchronisation as it commonly leads to a contended lock, which
remains a slow lock well after it is required\*. That being said, it isn't easy
to do correctly and performantly under 1.4.
\* An alternative approach is something like this
[LazyReference|http://labs.atlassian.com/source/browse/CONCURRENT/trunk/src/main/java/com/atlassian/util/concurrent/LazyReference.java?r=2242]
class, although this kind of thing really requires Java5 for full value.
> Eliminate synchronization contention on initial index reading in
> TermInfosReader ensureIndexIsRead
> ---------------------------------------------------------------------------------------------------
>
> Key: LUCENE-1609
> URL: https://issues.apache.org/jira/browse/LUCENE-1609
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.9
> Environment: Solr
> Tomcat 5.5
> Ubuntu 2.6.20-17-generic
> Intel(R) Pentium(R) 4 CPU 2.80GHz, 2Gb RAM
> Reporter: Dan Rosher
> Fix For: 2.9
>
> Attachments: LUCENE-1609.patch, LUCENE-1609.patch
>
>
> synchronized method ensureIndexIsRead in TermInfosReader causes contention
> under heavy load
> Simple to reproduce: e.g. Under Solr, with all caches turned off, do a simple
> range search e.g. id:[0 TO 999999] on even a small index (in my case 28K
> docs) and under a load/stress test application, and later, examining the
> Thread dump (kill -3) , many threads are blocked on 'waiting for monitor
> entry' to this method.
> Rather than using Double-Checked Locking which is known to have issues, this
> implementation uses a state pattern, where only one thread can move the
> object from IndexNotRead state to IndexRead, and in doing so alters the
> objects behavior, i.e. once the index is loaded, the index nolonger needs a
> synchronized method.
> In my particular test, this uncreased throughput at least 30 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
