[
https://issues.apache.org/jira/browse/LUCENE-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15417237#comment-15417237
]
Uwe Schindler commented on LUCENE-7409:
---------------------------------------
We added the remaining methods to make all accesses safe. Unfortunately the
optimizations in Hotspot are incredible complex. We made it working inlined in
getBytes(), but single Byte reads are not inlined, causing 60% slowdown.
I then looked at source code of SwitchPoint and the MutableCallSite behind. It
basically only inserts a store-store barrier using a static AtomicInteger (the
well known [~michaelbusch]] trick from Twitter). Based on this information I
implemented the same using standard static calls.
The new variant also removes the wek hashmap as all accesses to ByteBuffers
from the original and all clones go through one single instance of a "guard"
that checks a boolean. On closing the IndexInput the boolean is changed and
then it thros NPE, causing a AlreadyClosedException. As all clones are using
the same "delegator guard" instance, it can be invalidated with a single call.
After invalidating it accesses the AtomicInteger for the store-store barrier,
then yields its threads (to produce a small pause for inflight requests) and
finally unmaps.
Here is the new branch:
https://github.com/apache/lucene-solr/compare/master...rmuir:ace2
(FYI: The old code is here, but its slows:
https://github.com/apache/lucene-solr/compare/master...rmuir:ace)
> Look into making mmapdirectory's unmap safer
> --------------------------------------------
>
> Key: LUCENE-7409
> URL: https://issues.apache.org/jira/browse/LUCENE-7409
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Robert Muir
>
> I have seen a few bugs around this recently: of course its a bug in
> application code but a JVM crash is not good.
> I think we should see if we can prevent the crashes better than the current
> weak map, e.g. make it a safer option.
> I made an ugly prototype here:
> https://github.com/apache/lucene-solr/compare/master...rmuir:ace?expand=1
> It has a test that crashes the JVM without the patch but passes with.
> Hacky patch only implements readBytes() but has no problems with the
> luceneutil benchmark (1M):
> {noformat}
> Report after iter 19:
> Task QPS base StdDev QPS patch StdDev
> Pct diff
> IntNRQ 105.23 (17.6%) 100.42 (10.1%)
> -4.6% ( -27% - 28%)
> Respell 128.35 (13.2%) 125.88 (7.4%)
> -1.9% ( -19% - 21%)
> Fuzzy1 110.14 (17.2%) 108.28 (13.2%)
> -1.7% ( -27% - 34%)
> LowPhrase 337.02 (13.0%) 333.72 (9.3%)
> -1.0% ( -20% - 24%)
> MedPhrase 146.44 (12.9%) 145.55 (8.0%)
> -0.6% ( -19% - 23%)
> MedSpanNear 96.85 (13.1%) 96.57 (7.8%)
> -0.3% ( -18% - 23%)
> HighSpanNear 95.85 (13.9%) 96.33 (8.2%)
> 0.5% ( -18% - 26%)
> HighPhrase 146.84 (13.6%) 148.40 (8.4%)
> 1.1% ( -18% - 26%)
> HighTerm 295.15 (15.8%) 298.77 (9.5%)
> 1.2% ( -20% - 31%)
> LowSpanNear 268.80 (12.4%) 272.16 (7.9%)
> 1.2% ( -16% - 24%)
> Wildcard 284.09 (11.7%) 290.91 (8.9%)
> 2.4% ( -16% - 25%)
> Prefix3 212.50 (15.4%) 217.76 (10.0%)
> 2.5% ( -19% - 32%)
> OrHighLow 358.65 (15.0%) 368.93 (10.7%)
> 2.9% ( -19% - 33%)
> AndHighMed 799.65 (13.2%) 834.74 (7.8%)
> 4.4% ( -14% - 29%)
> MedSloppyPhrase 229.36 (15.9%) 239.95 (9.8%)
> 4.6% ( -18% - 36%)
> Fuzzy2 69.58 (14.6%) 72.82 (14.5%)
> 4.7% ( -21% - 39%)
> AndHighHigh 426.98 (12.8%) 451.77 (7.3%)
> 5.8% ( -12% - 29%)
> MedTerm 1361.11 (14.5%) 1450.90 (9.2%)
> 6.6% ( -14% - 35%)
> PKLookup 266.61 (13.4%) 284.28 (8.4%)
> 6.6% ( -13% - 32%)
> HighSloppyPhrase 251.22 (16.9%) 268.32 (10.7%)
> 6.8% ( -17% - 41%)
> OrHighMed 235.92 (17.2%) 253.12 (12.8%)
> 7.3% ( -19% - 45%)
> OrHighHigh 186.79 (13.5%) 201.15 (9.7%)
> 7.7% ( -13% - 35%)
> LowSloppyPhrase 395.23 (15.9%) 425.93 (9.3%)
> 7.8% ( -15% - 39%)
> AndHighLow 1128.28 (14.9%) 1242.11 (8.2%)
> 10.1% ( -11% - 38%)
> LowTerm 3024.62 (12.9%) 3367.65 (9.7%)
> 11.3% ( -9% - 39%)
> {noformat}
> We should do more testing. Maybe its totally the wrong tradeoff, maybe we
> only need handles for getters and everything inlines correctly, rather than
> needing a ton for every getXYZ() method...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
