RE: Solr Cache clear
Hmm. I am doing the same thing. But, somehow in my browser, after I select the core, it does not stay selected to view the stats/cache. Attaching the gif for when I try it. Anyway, that is a different issue from my side. Thanks for your input. -Lewin -Original Message- From: Shawn Heisey Sent: Tuesday, April 9, 2019 1:52 PM To: solr-user@lucene.apache.org Subject: Re: Solr Cache clear On 4/9/2019 12:38 PM, Lewin Joy (TMNA) wrote: > I just tried to go to the location you have specified. I could not see a > "CACHE" . I can see the "Statistics" section. > I am using Solr 7.2 on solrcloud mode. If you are trying to select a *collection* from a dropdown, you will not see this. It will only show up when you select a *core* from the other dropdown. In SolrCloud, collections are made up of one or more shards. Shards are made up of one or more replicas. Every shard replica is a core. Here are some partial screenshots showing what I clicked on to get to the cache stats: https://urldefense.proofpoint.com/v2/url?u=https-3A__www.dropbox.com_s_ked4xe4w45e9qr2_cache-2D1.png-3Fdl-3D0=DwICaQ=DDPRwrN9uYSNUDpKqPeD1g=WMeiuwk_Qf7aOundlWmtZMlairjO8ZQxQpAndx7JD6A=9o8kD9YOypvijGkYVvikg-74p2wDw2kblxyrVUijnaI=Zr1cZF633BlxWx0semmI-DCywSyOteEEM9_eABk-OrM= https://urldefense.proofpoint.com/v2/url?u=https-3A__www.dropbox.com_s_7v630mr2ii0rtee_cache-2D2.png-3Fdl-3D0=DwICaQ=DDPRwrN9uYSNUDpKqPeD1g=WMeiuwk_Qf7aOundlWmtZMlairjO8ZQxQpAndx7JD6A=9o8kD9YOypvijGkYVvikg-74p2wDw2kblxyrVUijnaI=rCCFdR6yk2ElhXMyxX0LJJoPkymG8Y3rrxCQz3gK8fI= https://urldefense.proofpoint.com/v2/url?u=https-3A__www.dropbox.com_s_6czqtxq5qf8vzwz_cache-2D3.png-3Fdl-3D0=DwICaQ=DDPRwrN9uYSNUDpKqPeD1g=WMeiuwk_Qf7aOundlWmtZMlairjO8ZQxQpAndx7JD6A=9o8kD9YOypvijGkYVvikg-74p2wDw2kblxyrVUijnaI=sA5FxVsaP1M8bGkxUnGdVoW7nLjF7rFthVCG6sKY_po= The system where I captured these screenshots was not running in SolrCloud mode, so it did not have the "collections" dropdown that your admin UI will have. When you reload a collection, Solr uses the information in ZooKeeper to locate all of the shard replicas that make up the collection, and reloads all those cores. So a collection reload is basically equivalent to multiple core reloads. Thanks, Shawn
Re: Solr Cache clear
On 4/9/2019 12:38 PM, Lewin Joy (TMNA) wrote: I just tried to go to the location you have specified. I could not see a "CACHE" . I can see the "Statistics" section. I am using Solr 7.2 on solrcloud mode. If you are trying to select a *collection* from a dropdown, you will not see this. It will only show up when you select a *core* from the other dropdown. In SolrCloud, collections are made up of one or more shards. Shards are made up of one or more replicas. Every shard replica is a core. Here are some partial screenshots showing what I clicked on to get to the cache stats: https://www.dropbox.com/s/ked4xe4w45e9qr2/cache-1.png?dl=0 https://www.dropbox.com/s/7v630mr2ii0rtee/cache-2.png?dl=0 https://www.dropbox.com/s/6czqtxq5qf8vzwz/cache-3.png?dl=0 The system where I captured these screenshots was not running in SolrCloud mode, so it did not have the "collections" dropdown that your admin UI will have. When you reload a collection, Solr uses the information in ZooKeeper to locate all of the shard replicas that make up the collection, and reloads all those cores. So a collection reload is basically equivalent to multiple core reloads. Thanks, Shawn
RE: Solr Cache clear
Hi Shawn, We are facing an issue where the caches got corrupted. We are doing a json.facet and pivoting through 3 levels. We are taking allBuckets from the different levels. In json.facet query, while doing the inner facets, we are keeping a limit. We notice that as we change the limit, we are getting a different value for allBuckets. And this got corrected after I explicitly applied one facet value as a filter one time. I am assuming, it cleared the cache for that filter. Now, I have few other facet values having the similar issue. Assuming, that this issue would get resolved if I clear the cache, I am checking these values once I reload the collection. Anyway, If I am able to look at the cache sizes after reload, this gives me more information. I just tried to go to the location you have specified. I could not see a "CACHE" . I can see the "Statistics" section. I am using Solr 7.2 on solrcloud mode. thanks -Lewin -Original Message- From: Shawn Heisey Sent: Tuesday, April 9, 2019 1:01 PM To: solr-user@lucene.apache.org Subject: Re: Solr Cache clear On 4/9/2019 11:51 AM, Lewin Joy (TMNA) wrote: > Hmm. I only tried reloading the collection as a whole. Not the core reload. > Where do I see the cache sizes after reload? If you do not know how to see the cache sizes, then what information are you looking at which has led you to the conclusion that the caches have not been cleared? To get to cache stats: In the admin UI, choose a core from the dropdown. Then click on Plugins/Stats, then CACHE, and choose the cache you want to look at. Thanks, Shawn
Re: Solr Cache clear
On 4/9/2019 11:51 AM, Lewin Joy (TMNA) wrote: Hmm. I only tried reloading the collection as a whole. Not the core reload. Where do I see the cache sizes after reload? If you do not know how to see the cache sizes, then what information are you looking at which has led you to the conclusion that the caches have not been cleared? To get to cache stats: In the admin UI, choose a core from the dropdown. Then click on Plugins/Stats, then CACHE, and choose the cache you want to look at. Thanks, Shawn
Re: Solr Cache clear
I’d like to know this, too. We run benchmarks with log replay, starting with warming queries, then a measurement run. It is a pain to to a rolling restart of the whole cluster before each benchmark run. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Apr 9, 2019, at 10:55 AM, Lewin Joy (TMNA) wrote: > > Thank you for email, Alex. > > I have the autowarmCount set as 0. > So, this shouldn't prepopulate with old cache data. > > -Lewin > > -Original Message- > From: Alexandre Rafalovitch > Sent: Monday, April 8, 2019 6:45 PM > To: solr-user > Subject: Re: Solr Cache clear > > You may have warming queries to prepopulate your cache. Check your > solrconfig.xml. > > Regards, >Alex > > On Mon, Apr 8, 2019, 4:16 PM Lewin Joy (TMNA), wrote: > >> ** PROTECTED 関係者外秘 >> How do I clear the solr caches without restarting Solr cluster? >> Is there a way? >> I tried reloading the collection. But, it did not help. >> >> Thanks, >> Lewin >> >>
RE: Solr Cache clear
Thank you for email, Alex. I have the autowarmCount set as 0. So, this shouldn't prepopulate with old cache data. -Lewin -Original Message- From: Alexandre Rafalovitch Sent: Monday, April 8, 2019 6:45 PM To: solr-user Subject: Re: Solr Cache clear You may have warming queries to prepopulate your cache. Check your solrconfig.xml. Regards, Alex On Mon, Apr 8, 2019, 4:16 PM Lewin Joy (TMNA), wrote: > ** PROTECTED 関係者外秘 > How do I clear the solr caches without restarting Solr cluster? > Is there a way? > I tried reloading the collection. But, it did not help. > > Thanks, > Lewin > >
RE: Solr Cache clear
Hmm. I only tried reloading the collection as a whole. Not the core reload. Where do I see the cache sizes after reload? -Lewin -Original Message- From: Shawn Heisey Sent: Monday, April 8, 2019 5:10 PM To: solr-user@lucene.apache.org Subject: Re: Solr Cache clear On 4/8/2019 2:14 PM, Lewin Joy (TMNA) wrote: > How do I clear the solr caches without restarting Solr cluster? > Is there a way? > I tried reloading the collection. But, it did not help. When I reload a core on a test setup (solr 7.4.0), I see cache sizes reset. What evidence are you seeing that reloading doesn't work? Thanks, Shawn
Re: Solr Cache clear
You may have warming queries to prepopulate your cache. Check your solrconfig.xml. Regards, Alex On Mon, Apr 8, 2019, 4:16 PM Lewin Joy (TMNA), wrote: > ** PROTECTED 関係者外秘 > How do I clear the solr caches without restarting Solr cluster? > Is there a way? > I tried reloading the collection. But, it did not help. > > Thanks, > Lewin > >
Re: Solr Cache clear
On 4/8/2019 2:14 PM, Lewin Joy (TMNA) wrote: How do I clear the solr caches without restarting Solr cluster? Is there a way? I tried reloading the collection. But, it did not help. When I reload a core on a test setup (solr 7.4.0), I see cache sizes reset. What evidence are you seeing that reloading doesn't work? Thanks, Shawn
Re: Solr cache clear
On 11/21/2018 8:36 AM, Rajdeep Sahoo wrote: The problem is that we are using master slave solr configuration and for similar type of query sometime it is taking 512 ms and sometime it is 29 ms . we are observing this issue since the query modification. As part of modification we have reduced a large no of facet.field param. In solrconfig.xml file we have changed the cache size and reloaded the the cores as part of this modification. Reloading cores is not clearing the cache. Is there any reason why we are getting this much of difference in response time for similar type of queries. What *SPECIFIC* evidence do you have that reloading the cores isn't emptying caches? Have you gone into plugins/stats and checked to see what the current cache sizes are? Often when a query fully utilizes Solr's caches, it will return in one or two milliseconds. A 29 millisecond query time probably means that the result did NOT come from Solr's caches, unless the machine is particularly slow. The cache that is likely affecting query performance in your case is not part of Solr at all -- it's the operating system. All modern operating systems use unallocated memory to cache any data on disk that has been read or written. Solr is highly reliant on this OS feature for good performance -- if you do not have extra memory not allocated to programs, Solr performance is probably going to be very bad. https://wiki.apache.org/solr/SolrPerformanceProblems#RAM Thanks, Shawn
Re: Solr cache clear
Hi all, The problem is that we are using master slave solr configuration and for similar type of query sometime it is taking 512 ms and sometime it is 29 ms . we are observing this issue since the query modification. As part of modification we have reduced a large no of facet.field param. In solrconfig.xml file we have changed the cache size and reloaded the the cores as part of this modification. Reloading cores is not clearing the cache. Is there any reason why we are getting this much of difference in response time for similar type of queries. On Nov 21, 2018 4:16 AM, "Edward Ribeiro" wrote: > Disabling or reducing autowarming can help too, in addition to cache size > reduction. > > Edward > > Em ter, 20 de nov de 2018 17:29, Erick Erickson escreveu: > > > Why would you want to? This sounds like an XY problem, there's some > > problem you think would be cured by clearing the cache. What is > > that problem? > > > > Because I doubt this would do anything useful, pretty soon the caches > > would be filled up again and you'd be right back where you started and > > the real solution is to stop doing whatever you're doing that leads to > > whatever the real problem is. Maybe reducing the cache sizes. > > > > Best, > > Erick > > On Tue, Nov 20, 2018 at 9:05 AM Shawn Heisey > wrote: > > > > > > On 11/20/2018 9:25 AM, Rajdeep Sahoo wrote: > > > > Hi all, > > > > Without restarting is it possible to clear the cache? > > > > > > You'll need to clarify what cache you're talking about, but I think for > > > the most part that if you reload the core (or collection if running > > > SolrCloud) that all caches should be rebuilt empty. > > > > > > Thanks, > > > Shawn > > > > > >
Re: Solr cache clear
Disabling or reducing autowarming can help too, in addition to cache size reduction. Edward Em ter, 20 de nov de 2018 17:29, Erick Erickson Why would you want to? This sounds like an XY problem, there's some > problem you think would be cured by clearing the cache. What is > that problem? > > Because I doubt this would do anything useful, pretty soon the caches > would be filled up again and you'd be right back where you started and > the real solution is to stop doing whatever you're doing that leads to > whatever the real problem is. Maybe reducing the cache sizes. > > Best, > Erick > On Tue, Nov 20, 2018 at 9:05 AM Shawn Heisey wrote: > > > > On 11/20/2018 9:25 AM, Rajdeep Sahoo wrote: > > > Hi all, > > > Without restarting is it possible to clear the cache? > > > > You'll need to clarify what cache you're talking about, but I think for > > the most part that if you reload the core (or collection if running > > SolrCloud) that all caches should be rebuilt empty. > > > > Thanks, > > Shawn > > >
Re: Solr cache clear
Why would you want to? This sounds like an XY problem, there's some problem you think would be cured by clearing the cache. What is that problem? Because I doubt this would do anything useful, pretty soon the caches would be filled up again and you'd be right back where you started and the real solution is to stop doing whatever you're doing that leads to whatever the real problem is. Maybe reducing the cache sizes. Best, Erick On Tue, Nov 20, 2018 at 9:05 AM Shawn Heisey wrote: > > On 11/20/2018 9:25 AM, Rajdeep Sahoo wrote: > > Hi all, > > Without restarting is it possible to clear the cache? > > You'll need to clarify what cache you're talking about, but I think for > the most part that if you reload the core (or collection if running > SolrCloud) that all caches should be rebuilt empty. > > Thanks, > Shawn >
Re: Solr cache clear
On 11/20/2018 9:25 AM, Rajdeep Sahoo wrote: Hi all, Without restarting is it possible to clear the cache? You'll need to clarify what cache you're talking about, but I think for the most part that if you reload the core (or collection if running SolrCloud) that all caches should be rebuilt empty. Thanks, Shawn
