Thanks Uwe for the details. In our solr (8.4)configuration , we have a fieldcache that has the fields used for sorting. It can be observed that the fieldCache is getting cleared sometimes. But, I do not think we have the below mentioned search manager logic implemented in our setup. We have not modified any solr/lucene implementation.
So, without opening or refreshing a searcher, I am not able to understand how the field cache is getting cleared. Can you please help to clarify this. On 2022/06/08 17:46:50 Uwe Schindler wrote: > Hi, > > You do not neessarily need a commit. If you use SearcherManager in > combination with NRTCachingDirectory you can also refresh you searcher > every few seconds, so in-memory cached segments are searched. But in > short: If you do not explicitly ask for a fresh searcher, there won't be > any automatic refreshes and the caches stays as is. > > It is also important for older Lucene versions that still support > FieldCache: Make sure your queries work "per segment" and not globally. > Because wrongly written applications would have a cache entry per index > and not per segment, so on every refresh the whole cache has to be > rebuilt. This is also one reason why DocValues are preferred for sorting. > > https://2012.berlinbuzzwords.de/sessions/your-index-reader-really-atomic-or-maybe-slow.html > > https://www.youtube.com/watch?v=iZZ1AbJ6dik > > Uwe > > Am 08.06.2022 um 19:05 schrieb Poorna Murali: > > Thanks Uwe! New searcher opens when we do a commit.Apart from this, are > > there other scenarios where a searcher would be refreshed? > > > > On 2022/06/08 16:43:07 Uwe Schindler wrote: > >> Hi, > >> > >> They get evicted when the segment of that index is closed. After that > >> theres no reference to them anymore through a > >> WeakHashMap<LeaveReader,Cache> and thecache object gets freed by GC. > >> This happens on refresh of searcher where unused segments are closed and > >> new ones are openend. There is no way to get rid of entries on a live > >> searcher. > >> > >> FieldCache is no longer available since Lucene 6, so which version are > >> you using? Since Lucene 4 it is better to use DocValues fields for > >> sorting or facetting/aggregations. > >> > >> If you are using Solr, theres still a clone of FieldCache as part of > >> Solr's codebase (and is not supported by Lucene anymore), but thats only > >> for legacy indexes where the schema was not updated to use DocValues. In > >> an "ideally configured Solr server", the Admin UI shows no entries below > >> Core's FieldCache stats. If you see entries there go and replace those > >> field's config by adding docvalues=true. > >> > >> Uwe > >> > >> Am 08.06.2022 um 15:26 schrieb Poorna Murali: > >>> Hi, > >>> > >>> I would like to know if there is any automatic eviction policy for the > >>> field cache entries. I understand that it gets invalidated when a new > >>> searcher opens. But, my question is in case if gc runs or if there is > > any > >>> other scenario which could evict the unused entries from fieldcache. > >>> > >>> Please help to clarify the same. > >>> > >>> Thanks > >>> Poorna > >>> > >> -- > >> Uwe Schindler > >> Achterdiek 19, D-28357 Bremen > >> https://www.thetaphi.de > >> eMail: u...@thetaphi.de > >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > >> For additional commands, e-mail: java-user-h...@lucene.apache.org > >> > >> > -- > Uwe Schindler > Achterdiek 19, D-28357 Bremen > https://www.thetaphi.de > eMail: u...@thetaphi.de > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > >
