[
https://issues.apache.org/jira/browse/LUCENE-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413487#comment-15413487
]
Robert Muir commented on LUCENE-7409:
-------------------------------------
thanks [~mikemccand] ! Making the tests detect these issues will help.
As far as the SwitchPoint.invalidate, we should at least think about batching,
to reduce costs. The API is designed around that. Maybe we could hook into e.g.
deletePendingFiles() which gets called by IndexWriter (possibly with a hook in
StandardDirectoryReader close when IW is not in use). This way, many maps could
be invalidated at once and we aren't spending a ton of time waiting for
safepoints. Maybe there are other approaches, we have to consider complexity
too.
> Look into making mmapdirectory's unmap safer
> --------------------------------------------
>
> Key: LUCENE-7409
> URL: https://issues.apache.org/jira/browse/LUCENE-7409
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Robert Muir
>
> I have seen a few bugs around this recently: of course its a bug in
> application code but a JVM crash is not good.
> I think we should see if we can prevent the crashes better than the current
> weak map, e.g. make it a safer option.
> I made an ugly prototype here:
> https://github.com/apache/lucene-solr/compare/master...rmuir:ace?expand=1
> It has a test that crashes the JVM without the patch but passes with.
> Hacky patch only implements readBytes() but has no problems with the
> luceneutil benchmark (1M):
> {noformat}
> Report after iter 19:
> Task QPS base StdDev QPS patch StdDev
> Pct diff
> IntNRQ 105.23 (17.6%) 100.42 (10.1%)
> -4.6% ( -27% - 28%)
> Respell 128.35 (13.2%) 125.88 (7.4%)
> -1.9% ( -19% - 21%)
> Fuzzy1 110.14 (17.2%) 108.28 (13.2%)
> -1.7% ( -27% - 34%)
> LowPhrase 337.02 (13.0%) 333.72 (9.3%)
> -1.0% ( -20% - 24%)
> MedPhrase 146.44 (12.9%) 145.55 (8.0%)
> -0.6% ( -19% - 23%)
> MedSpanNear 96.85 (13.1%) 96.57 (7.8%)
> -0.3% ( -18% - 23%)
> HighSpanNear 95.85 (13.9%) 96.33 (8.2%)
> 0.5% ( -18% - 26%)
> HighPhrase 146.84 (13.6%) 148.40 (8.4%)
> 1.1% ( -18% - 26%)
> HighTerm 295.15 (15.8%) 298.77 (9.5%)
> 1.2% ( -20% - 31%)
> LowSpanNear 268.80 (12.4%) 272.16 (7.9%)
> 1.2% ( -16% - 24%)
> Wildcard 284.09 (11.7%) 290.91 (8.9%)
> 2.4% ( -16% - 25%)
> Prefix3 212.50 (15.4%) 217.76 (10.0%)
> 2.5% ( -19% - 32%)
> OrHighLow 358.65 (15.0%) 368.93 (10.7%)
> 2.9% ( -19% - 33%)
> AndHighMed 799.65 (13.2%) 834.74 (7.8%)
> 4.4% ( -14% - 29%)
> MedSloppyPhrase 229.36 (15.9%) 239.95 (9.8%)
> 4.6% ( -18% - 36%)
> Fuzzy2 69.58 (14.6%) 72.82 (14.5%)
> 4.7% ( -21% - 39%)
> AndHighHigh 426.98 (12.8%) 451.77 (7.3%)
> 5.8% ( -12% - 29%)
> MedTerm 1361.11 (14.5%) 1450.90 (9.2%)
> 6.6% ( -14% - 35%)
> PKLookup 266.61 (13.4%) 284.28 (8.4%)
> 6.6% ( -13% - 32%)
> HighSloppyPhrase 251.22 (16.9%) 268.32 (10.7%)
> 6.8% ( -17% - 41%)
> OrHighMed 235.92 (17.2%) 253.12 (12.8%)
> 7.3% ( -19% - 45%)
> OrHighHigh 186.79 (13.5%) 201.15 (9.7%)
> 7.7% ( -13% - 35%)
> LowSloppyPhrase 395.23 (15.9%) 425.93 (9.3%)
> 7.8% ( -15% - 39%)
> AndHighLow 1128.28 (14.9%) 1242.11 (8.2%)
> 10.1% ( -11% - 38%)
> LowTerm 3024.62 (12.9%) 3367.65 (9.7%)
> 11.3% ( -9% - 39%)
> {noformat}
> We should do more testing. Maybe its totally the wrong tradeoff, maybe we
> only need handles for getters and everything inlines correctly, rather than
> needing a ton for every getXYZ() method...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
