[
https://issues.apache.org/jira/browse/LUCENE-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15894320#comment-15894320
]
Steve Rowe commented on LUCENE-7410:
------------------------------------
Policeman Jenkins found a reproducing seed for a test method introduced on this
issue [https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/19092/] (at
https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=df6f830 - seed
reproduces with this hash):
{noformat}
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TestSlowCompositeReaderWrapper -Dtests.method=testOrdMapsAreCached
-Dtests.seed=ED06E16D2F7E2C9F -Dtests.multiplier=3 -Dtests.slow=true
-Dtests.locale=en-BS -Dtests.timezone=Asia/Kashgar -Dtests.asserts=true
-Dtests.file.encoding=UTF-8
[junit4] FAILURE 0.01s J2 |
TestSlowCompositeReaderWrapper.testOrdMapsAreCached <<<
[junit4] > Throwable #1: java.lang.AssertionError
[junit4] > at
__randomizedtesting.SeedInfo.seed([ED06E16D2F7E2C9F:B1C7CA871734CF6C]:0)
[junit4] > at
org.apache.solr.index.TestSlowCompositeReaderWrapper.testOrdMapsAreCached(TestSlowCompositeReaderWrapper.java:111)
[junit4] > at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] > at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] > at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] > at
java.base/java.lang.reflect.Method.invoke(Method.java:547)
[junit4] > at java.base/java.lang.Thread.run(Thread.java:844)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {},
docValues:{sorted=DocValuesFormat(name=Lucene70),
sorted_set=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=1289,
maxMBSortInHeap=5.014157490710387, sim=RandomSimilarity(queryNorm=true): {},
locale=en-BS, timezone=Asia/Kashgar
[junit4] 2> NOTE: Linux 4.4.0-53-generic amd64/Oracle Corporation 9-ea
(64-bit)/cpus=12,threads=1,free=204857864,total=518979584
{noformat}
> Make cache keys and closed listeners less trappy
> ------------------------------------------------
>
> Key: LUCENE-7410
> URL: https://issues.apache.org/jira/browse/LUCENE-7410
> Project: Lucene - Core
> Issue Type: Bug
> Reporter: Adrien Grand
> Fix For: master (7.0)
>
> Attachments: LUCENE-7410.patch, LUCENE-7410.patch, LUCENE-7410.patch
>
>
> IndexReader currently exposes getCoreCacheKey(),
> getCombinedCoreAndDeletesKey(), addCoreClosedListener() and
> addReaderClosedListener(). They are typically used to manage resources whose
> lifetime needs to mimic the lifetime of segments/indexes, typically caches.
> I think this is trappy for various reasons:
> h3. Memory leaks
> When maintaining a cache, entries are added to the cache based on the cache
> key and then evicted using the cache key that is given back by the close
> listener, so it is very important that both keys are the same.
> But if a filter reader happens to delegate get*Key() and not
> add*ClosedListener() or vice-versa then there is potential for a memory leak
> since the closed listener will be called on a different key and entries will
> never be evicted from the cache.
> h3. Lifetime expectations
> The expectation of using the core cache key is that it will not change in
> case of deletions, but this is only true on SegmentReader and LeafReader
> impls that delegate to it. Other implementations such as composite readers or
> parallel leaf readers use the same key for "core" and "combined core and
> deletes".
> h3. Throw-away wrappers cause cache trashing
> An application might want to either expose more (with a ParrallelReader or
> MultiReader) or less information (by filtering fields/docs that can be seen)
> depending on the user who is logged in. In that case the application would
> typically maintain a DirectoryReader and then wrap it per request depending
> on the logged user and throw away the wrapper once the request is completed.
> The problem is that these wrappers have their own cache keys and the
> application may build something costly and put it in a cache to throw it away
> a couple milliseconds later. I would rather like for such readers to have a
> way to opt out from caching on order to avoid this performance trap.
> h3. Type safety
> The keys that are exposed are plain java.lang.Object instances, which
> requires caches to look like a {{Map<Object, ?>}} which makes it very easy to
> either try to get, put or remove on the wrong object since any object would
> be accepted.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
