Re: documentCache not used in 4.3.1?
To: solr-user@lucene.apache.org Subject: Re: documentCache not used in 4.3.1? It's especially weird that the hit ratio is so high and you're not seeing anything in the cache. Are you perhaps soft committing frequently? Soft commits throw away all the top-level caches including documentCache I think Erick On Fri, Jun 28, 2013 at 7:23 PM, Tim Vaillancourttim@elementspace. **comt...@elementspace.com wrote: Thanks Otis, Yeah I realized after sending my e-mail that doc cache does not warm, however I'm still lost on why there are no other metrics. Thanks! Tim On 28 June 2013 16:22, Otis Gospodneticotis.gospodnetic@** gmail.com otis.gospodne...@gmail.com wrote: Hi Tim, Not sure about the zeros in 4.3.1, but in SPM we see all these numbers are non-0, though I haven't had the chance to confirm with Solr 4.3.1. Note that you can't really autowarm document cache... Otis -- Solr ElasticSearch Support -- http://sematext.com/ Performance Monitoring -- http://sematext.com/spm On Fri, Jun 28, 2013 at 7:14 PM, Tim Vaillancourt t...@elementspace.com wrote: Hey guys, This has to be a stupid question/I must be doing something wrong, but after frequent load testing with documentCache enabled under Solr 4.3.1 with autoWarmCount=150, I'm noticing that my documentCache metrics are always zero for non-cumlative. At first I thought my commit rate is fast enough I just never see the non-cumlative result, but after 100s of samples I still always get zero values. Here is the current output of my documentCache from Solr's admin for 1 core: - documentCache http://localhost:8983/solr/#/**channels_shard1_replica2/** plugins/cache?en http://localhost:8983/solr/#/channels_shard1_replica2/plugins/cache?en try=documentCache - class:org.apache.solr.search.**LRUCache - version:1.0 - description:LRU Cache(maxSize=512, initialSize=512, autowarmCount=150, regenerator=null) - src:$URL: https:/ / svn.apache.org/repos/asf/**lucene/dev/branches/lucene_** solr_4_3/ http://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_4_3/ solr/core/src/java/org/apache/**solr/search/LRUCache.java https://svn.apache.org/repos/**asf/lucene/dev/branches/** lucene_solr_4_3/s https://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_4_3/s olr/core/src/java/org/apache/**solr/search/LRUCache.java $ - stats: - lookups:0 - hits:0 - hitratio:0.00 - inserts:0 - evictions:0 - size:0 - warmupTime:0 - cumulative_lookups:65198986 - cumulative_hits:63075669 - cumulative_hitratio:0.96 - cumulative_inserts:2123317 - cumulative_evictions:1010262 The cumulative values seem to rise, suggesting doc cache is working, but at the same time it seems I never see non-cumlative metrics, most importantly warmupTime. Am I doing something wrong, is this normal/by-design, or is there an issue here? Thanks for helping with my silly question! Have a good weekend, Tim
Re: documentCache not used in 4.3.1?
. If anyone else can reproduce this on 4.3.1 I will feel less crazy :). Cheers, Tim -Original Message- From: Erick Erickson [mailto:erickerickson@gmail.**com erickerick...@gmail.com ] Sent: Saturday, June 29, 2013 10:13 AM To: solr-user@lucene.apache.org Subject: Re: documentCache not used in 4.3.1? It's especially weird that the hit ratio is so high and you're not seeing anything in the cache. Are you perhaps soft committing frequently? Soft commits throw away all the top-level caches including documentCache I think Erick On Fri, Jun 28, 2013 at 7:23 PM, Tim Vaillancourttim@elementspace. **comt...@elementspace.com wrote: Thanks Otis, Yeah I realized after sending my e-mail that doc cache does not warm, however I'm still lost on why there are no other metrics. Thanks! Tim On 28 June 2013 16:22, Otis Gospodneticotis.gospodnetic@** gmail.com otis.gospodne...@gmail.com wrote: Hi Tim, Not sure about the zeros in 4.3.1, but in SPM we see all these numbers are non-0, though I haven't had the chance to confirm with Solr 4.3.1. Note that you can't really autowarm document cache... Otis -- Solr ElasticSearch Support -- http://sematext.com/Performance Monitoring -- http://sematext.com/spm On Fri, Jun 28, 2013 at 7:14 PM, Tim Vaillancourt t...@elementspace.com wrote: Hey guys, This has to be a stupid question/I must be doing something wrong, but after frequent load testing with documentCache enabled under Solr 4.3.1 with autoWarmCount=150, I'm noticing that my documentCache metrics are always zero for non-cumlative. At first I thought my commit rate is fast enough I just never see the non-cumlative result, but after 100s of samples I still always get zero values. Here is the current output of my documentCache from Solr's admin for 1 core: - documentCache http://localhost:8983/solr/#/**channels_shard1_replica2/** plugins/cache?en http://localhost:8983/solr/#/channels_shard1_replica2/plugins/cache?en try=documentCache - class:org.apache.solr.search.**LRUCache - version:1.0 - description:LRU Cache(maxSize=512, initialSize=512, autowarmCount=150, regenerator=null) - src:$URL: https:/ / svn.apache.org/repos/asf/**lucene/dev/branches/lucene_** solr_4_3/ http://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_4_3/ solr/core/src/java/org/apache/**solr/search/LRUCache.java https://svn.apache.org/repos/**asf/lucene/dev/branches/** lucene_solr_4_3/s https://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_4_3/s olr/core/src/java/org/apache/**solr/search/LRUCache.java $ - stats: - lookups:0 - hits:0 - hitratio:0.00 - inserts:0 - evictions:0 - size:0 - warmupTime:0 - cumulative_lookups:65198986 - cumulative_hits:63075669 - cumulative_hitratio:0.96 - cumulative_inserts:2123317 - cumulative_evictions:1010262 The cumulative values seem to rise, suggesting doc cache is working, but at the same time it seems I never see non-cumlative metrics, most importantly warmupTime. Am I doing something wrong, is this normal/by-design, or is there an issue here? Thanks for helping with my silly question! Have a good weekend, Tim
Re: documentCache not used in 4.3.1?
We see similar results, again we softCommit every 1s (trying to get as NRT as we can), and we very rarely get any hits in our caches. As an unscheduled test last week, we did shutdown indexing and noticed about 80% hit rate in caches (and average query time dropped from ~1s to 100ms!) so I think we are in the same position as you. I appreciate with such a frequent soft commit that the caches get invalidated, but I was expecting cache warming to help though it doesn't appear to be. We *don't* currently run a warming query, my impression of NRT was that it was better to not do that as otherwise you spend more time warming the searcher and caches, and by the time you've done all that, the searcher is invalidated anyway! On 30 June 2013 01:58, Tim Vaillancourt t...@elementspace.com wrote: That's a good idea, I'll try that next week. Thanks! Tim On 29/06/13 12:39 PM, Erick Erickson wrote: Tim: Yeah, this doesn't make much sense to me either since, as you say, you should be seeing some metrics upon occasion. But do note that the underlying cache only gets filled when getting documents to return in query results, since there's no autowarming going on it may come and go. But you can test this pretty quickly by lengthening your autocommit interval or just not indexing anything for a while, then run a bunch of queries and look at your cache stats. That'll at least tell you whether it works at all. You'll have to have hard commits turned off (or openSearcher set to 'false') for that check too. Best Erick On Sat, Jun 29, 2013 at 2:48 PM, Vaillancourt, Timtvaillanco...@ea.com* *wrote: Yes, we are softCommit'ing every 1000ms, but that should be enough time to see metrics though, right? For example, I still get non-cumulative metrics from the other caches (which are also throw away). I've also curl/sampled enough that I probably should have seen a value by now. If anyone else can reproduce this on 4.3.1 I will feel less crazy :). Cheers, Tim -Original Message- From: Erick Erickson [mailto:erickerickson@gmail.**comerickerick...@gmail.com ] Sent: Saturday, June 29, 2013 10:13 AM To: solr-user@lucene.apache.org Subject: Re: documentCache not used in 4.3.1? It's especially weird that the hit ratio is so high and you're not seeing anything in the cache. Are you perhaps soft committing frequently? Soft commits throw away all the top-level caches including documentCache I think Erick On Fri, Jun 28, 2013 at 7:23 PM, Tim Vaillancourttim@elementspace.**comt...@elementspace.com wrote: Thanks Otis, Yeah I realized after sending my e-mail that doc cache does not warm, however I'm still lost on why there are no other metrics. Thanks! Tim On 28 June 2013 16:22, Otis Gospodneticotis.gospodnetic@**gmail.comotis.gospodne...@gmail.com wrote: Hi Tim, Not sure about the zeros in 4.3.1, but in SPM we see all these numbers are non-0, though I haven't had the chance to confirm with Solr 4.3.1. Note that you can't really autowarm document cache... Otis -- Solr ElasticSearch Support -- http://sematext.com/ Performance Monitoring -- http://sematext.com/spm On Fri, Jun 28, 2013 at 7:14 PM, Tim Vaillancourt t...@elementspace.com wrote: Hey guys, This has to be a stupid question/I must be doing something wrong, but after frequent load testing with documentCache enabled under Solr 4.3.1 with autoWarmCount=150, I'm noticing that my documentCache metrics are always zero for non-cumlative. At first I thought my commit rate is fast enough I just never see the non-cumlative result, but after 100s of samples I still always get zero values. Here is the current output of my documentCache from Solr's admin for 1 core: - documentCache http://localhost:8983/solr/#/**channels_shard1_replica2/** plugins/cache?enhttp://localhost:8983/solr/#/channels_shard1_replica2/plugins/cache?en try=documentCache - class:org.apache.solr.search.**LRUCache - version:1.0 - description:LRU Cache(maxSize=512, initialSize=512, autowarmCount=150, regenerator=null) - src:$URL: https:/ /svn.apache.org/repos/asf/**lucene/dev/branches/lucene_** solr_4_3/http://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_4_3/ solr/core/src/java/org/apache/**solr/search/LRUCache.java https://svn.apache.org/repos/**asf/lucene/dev/branches/** lucene_solr_4_3/shttps://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_4_3/s olr/core/src/java/org/apache/**solr/search/LRUCache.java $ - stats: - lookups:0 - hits:0 - hitratio:0.00 - inserts:0 - evictions:0 - size:0 - warmupTime:0 - cumulative_lookups:65198986 - cumulative_hits:63075669 - cumulative_hitratio:0.96 - cumulative_inserts:2123317 - cumulative_evictions:1010262
Re: documentCache not used in 4.3.1?
Daniel: Soft commits invalidate the top level caches, which include things like filterCache, queryResultCache etc. Various segment-level caches are NOT invalidated, but you really don't have a lot of control from the Solr level over those anyway. But yeah, the tension between caching a bunch of stuff for query speedups and NRT is still with us. Soft commits are much less expensive than hard commits, but not being able to use the caches as much is the price. You're right that with such frequent autocommits, autowarming probably is not worth the effort. The question I always ask is whether 1 second is really necessary. Or, more accurately, worth the price. Often it's not and lengthening it out significantly may be an option, but that's a discussion for you to have with your product manager G I have seen configurations that have a more frequent hard commit (openSearcher=false) than soft commit. The mantra is soft commits are about visibility, hard commits are about durability. FWIW, Erick On Mon, Jul 1, 2013 at 3:40 AM, Daniel Collins danwcoll...@gmail.comwrote: We see similar results, again we softCommit every 1s (trying to get as NRT as we can), and we very rarely get any hits in our caches. As an unscheduled test last week, we did shutdown indexing and noticed about 80% hit rate in caches (and average query time dropped from ~1s to 100ms!) so I think we are in the same position as you. I appreciate with such a frequent soft commit that the caches get invalidated, but I was expecting cache warming to help though it doesn't appear to be. We *don't* currently run a warming query, my impression of NRT was that it was better to not do that as otherwise you spend more time warming the searcher and caches, and by the time you've done all that, the searcher is invalidated anyway! On 30 June 2013 01:58, Tim Vaillancourt t...@elementspace.com wrote: That's a good idea, I'll try that next week. Thanks! Tim On 29/06/13 12:39 PM, Erick Erickson wrote: Tim: Yeah, this doesn't make much sense to me either since, as you say, you should be seeing some metrics upon occasion. But do note that the underlying cache only gets filled when getting documents to return in query results, since there's no autowarming going on it may come and go. But you can test this pretty quickly by lengthening your autocommit interval or just not indexing anything for a while, then run a bunch of queries and look at your cache stats. That'll at least tell you whether it works at all. You'll have to have hard commits turned off (or openSearcher set to 'false') for that check too. Best Erick On Sat, Jun 29, 2013 at 2:48 PM, Vaillancourt, Timtvaillanco...@ea.com * *wrote: Yes, we are softCommit'ing every 1000ms, but that should be enough time to see metrics though, right? For example, I still get non-cumulative metrics from the other caches (which are also throw away). I've also curl/sampled enough that I probably should have seen a value by now. If anyone else can reproduce this on 4.3.1 I will feel less crazy :). Cheers, Tim -Original Message- From: Erick Erickson [mailto:erickerickson@gmail.**com erickerick...@gmail.com ] Sent: Saturday, June 29, 2013 10:13 AM To: solr-user@lucene.apache.org Subject: Re: documentCache not used in 4.3.1? It's especially weird that the hit ratio is so high and you're not seeing anything in the cache. Are you perhaps soft committing frequently? Soft commits throw away all the top-level caches including documentCache I think Erick On Fri, Jun 28, 2013 at 7:23 PM, Tim Vaillancourttim@elementspace. **comt...@elementspace.com wrote: Thanks Otis, Yeah I realized after sending my e-mail that doc cache does not warm, however I'm still lost on why there are no other metrics. Thanks! Tim On 28 June 2013 16:22, Otis Gospodneticotis.gospodnetic@**gmail.com otis.gospodne...@gmail.com wrote: Hi Tim, Not sure about the zeros in 4.3.1, but in SPM we see all these numbers are non-0, though I haven't had the chance to confirm with Solr 4.3.1. Note that you can't really autowarm document cache... Otis -- Solr ElasticSearch Support -- http://sematext.com/ Performance Monitoring -- http://sematext.com/spm On Fri, Jun 28, 2013 at 7:14 PM, Tim Vaillancourt t...@elementspace.com wrote: Hey guys, This has to be a stupid question/I must be doing something wrong, but after frequent load testing with documentCache enabled under Solr 4.3.1 with autoWarmCount=150, I'm noticing that my documentCache metrics are always zero for non-cumlative. At first I thought my commit rate is fast enough I just never see the non-cumlative result, but after 100s of samples I still always get zero values. Here is the current output of my documentCache from Solr's admin for 1
Re: documentCache not used in 4.3.1?
Regrettably, visibility is key for us :( Documents must be searchable as soon as they have been indexed (or as near as we can make it). Our old search system didn't do relevance sort, it was time-ordered (so it had a much simpler job) but it did have sub-second latency, and that is what is expected for its replacement (I know Solr doesn't like 1s currently, but we live in hope!). Tried explaining that by doing relevance sort we are searching 100% of the collection, instead of the ~10%-20% a time-ordered sort did (it effectively sharded by date and only searched as far back as it needed to fill a page of results), but that tends to get blank looks from business. :) One of life's little challenges. On 1 July 2013 11:10, Erick Erickson erickerick...@gmail.com wrote: Daniel: Soft commits invalidate the top level caches, which include things like filterCache, queryResultCache etc. Various segment-level caches are NOT invalidated, but you really don't have a lot of control from the Solr level over those anyway. But yeah, the tension between caching a bunch of stuff for query speedups and NRT is still with us. Soft commits are much less expensive than hard commits, but not being able to use the caches as much is the price. You're right that with such frequent autocommits, autowarming probably is not worth the effort. The question I always ask is whether 1 second is really necessary. Or, more accurately, worth the price. Often it's not and lengthening it out significantly may be an option, but that's a discussion for you to have with your product manager G I have seen configurations that have a more frequent hard commit (openSearcher=false) than soft commit. The mantra is soft commits are about visibility, hard commits are about durability. FWIW, Erick On Mon, Jul 1, 2013 at 3:40 AM, Daniel Collins danwcoll...@gmail.com wrote: We see similar results, again we softCommit every 1s (trying to get as NRT as we can), and we very rarely get any hits in our caches. As an unscheduled test last week, we did shutdown indexing and noticed about 80% hit rate in caches (and average query time dropped from ~1s to 100ms!) so I think we are in the same position as you. I appreciate with such a frequent soft commit that the caches get invalidated, but I was expecting cache warming to help though it doesn't appear to be. We *don't* currently run a warming query, my impression of NRT was that it was better to not do that as otherwise you spend more time warming the searcher and caches, and by the time you've done all that, the searcher is invalidated anyway! On 30 June 2013 01:58, Tim Vaillancourt t...@elementspace.com wrote: That's a good idea, I'll try that next week. Thanks! Tim On 29/06/13 12:39 PM, Erick Erickson wrote: Tim: Yeah, this doesn't make much sense to me either since, as you say, you should be seeing some metrics upon occasion. But do note that the underlying cache only gets filled when getting documents to return in query results, since there's no autowarming going on it may come and go. But you can test this pretty quickly by lengthening your autocommit interval or just not indexing anything for a while, then run a bunch of queries and look at your cache stats. That'll at least tell you whether it works at all. You'll have to have hard commits turned off (or openSearcher set to 'false') for that check too. Best Erick On Sat, Jun 29, 2013 at 2:48 PM, Vaillancourt, Tim tvaillanco...@ea.com * *wrote: Yes, we are softCommit'ing every 1000ms, but that should be enough time to see metrics though, right? For example, I still get non-cumulative metrics from the other caches (which are also throw away). I've also curl/sampled enough that I probably should have seen a value by now. If anyone else can reproduce this on 4.3.1 I will feel less crazy :). Cheers, Tim -Original Message- From: Erick Erickson [mailto:erickerickson@gmail.**com erickerick...@gmail.com ] Sent: Saturday, June 29, 2013 10:13 AM To: solr-user@lucene.apache.org Subject: Re: documentCache not used in 4.3.1? It's especially weird that the hit ratio is so high and you're not seeing anything in the cache. Are you perhaps soft committing frequently? Soft commits throw away all the top-level caches including documentCache I think Erick On Fri, Jun 28, 2013 at 7:23 PM, Tim Vaillancourttim@elementspace. **comt...@elementspace.com wrote: Thanks Otis, Yeah I realized after sending my e-mail that doc cache does not warm, however I'm still lost on why there are no other metrics. Thanks! Tim On 28 June 2013 16:22, Otis Gospodneticotis.gospodnetic@** gmail.com otis.gospodne...@gmail.com wrote: Hi Tim, Not sure about
Re: documentCache not used in 4.3.1?
It's especially weird that the hit ratio is so high and you're not seeing anything in the cache. Are you perhaps soft committing frequently? Soft commits throw away all the top-level caches including documentCache I think Erick On Fri, Jun 28, 2013 at 7:23 PM, Tim Vaillancourt t...@elementspace.comwrote: Thanks Otis, Yeah I realized after sending my e-mail that doc cache does not warm, however I'm still lost on why there are no other metrics. Thanks! Tim On 28 June 2013 16:22, Otis Gospodnetic otis.gospodne...@gmail.com wrote: Hi Tim, Not sure about the zeros in 4.3.1, but in SPM we see all these numbers are non-0, though I haven't had the chance to confirm with Solr 4.3.1. Note that you can't really autowarm document cache... Otis -- Solr ElasticSearch Support -- http://sematext.com/ Performance Monitoring -- http://sematext.com/spm On Fri, Jun 28, 2013 at 7:14 PM, Tim Vaillancourt t...@elementspace.com wrote: Hey guys, This has to be a stupid question/I must be doing something wrong, but after frequent load testing with documentCache enabled under Solr 4.3.1 with autoWarmCount=150, I'm noticing that my documentCache metrics are always zero for non-cumlative. At first I thought my commit rate is fast enough I just never see the non-cumlative result, but after 100s of samples I still always get zero values. Here is the current output of my documentCache from Solr's admin for 1 core: - documentCache http://localhost:8983/solr/#/channels_shard1_replica2/plugins/cache?entry=documentCache - class:org.apache.solr.search.LRUCache - version:1.0 - description:LRU Cache(maxSize=512, initialSize=512, autowarmCount=150, regenerator=null) - src:$URL: https:/ /svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_4_3/ solr/core/src/java/org/apache/solr/search/LRUCache.java https://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_4_3/solr/core/src/java/org/apache/solr/search/LRUCache.java $ - stats: - lookups:0 - hits:0 - hitratio:0.00 - inserts:0 - evictions:0 - size:0 - warmupTime:0 - cumulative_lookups:65198986 - cumulative_hits:63075669 - cumulative_hitratio:0.96 - cumulative_inserts:2123317 - cumulative_evictions:1010262 The cumulative values seem to rise, suggesting doc cache is working, but at the same time it seems I never see non-cumlative metrics, most importantly warmupTime. Am I doing something wrong, is this normal/by-design, or is there an issue here? Thanks for helping with my silly question! Have a good weekend, Tim
RE: documentCache not used in 4.3.1?
Yes, we are softCommit'ing every 1000ms, but that should be enough time to see metrics though, right? For example, I still get non-cumulative metrics from the other caches (which are also throw away). I've also curl/sampled enough that I probably should have seen a value by now. If anyone else can reproduce this on 4.3.1 I will feel less crazy :). Cheers, Tim -Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Saturday, June 29, 2013 10:13 AM To: solr-user@lucene.apache.org Subject: Re: documentCache not used in 4.3.1? It's especially weird that the hit ratio is so high and you're not seeing anything in the cache. Are you perhaps soft committing frequently? Soft commits throw away all the top-level caches including documentCache I think Erick On Fri, Jun 28, 2013 at 7:23 PM, Tim Vaillancourt t...@elementspace.comwrote: Thanks Otis, Yeah I realized after sending my e-mail that doc cache does not warm, however I'm still lost on why there are no other metrics. Thanks! Tim On 28 June 2013 16:22, Otis Gospodnetic otis.gospodne...@gmail.com wrote: Hi Tim, Not sure about the zeros in 4.3.1, but in SPM we see all these numbers are non-0, though I haven't had the chance to confirm with Solr 4.3.1. Note that you can't really autowarm document cache... Otis -- Solr ElasticSearch Support -- http://sematext.com/ Performance Monitoring -- http://sematext.com/spm On Fri, Jun 28, 2013 at 7:14 PM, Tim Vaillancourt t...@elementspace.com wrote: Hey guys, This has to be a stupid question/I must be doing something wrong, but after frequent load testing with documentCache enabled under Solr 4.3.1 with autoWarmCount=150, I'm noticing that my documentCache metrics are always zero for non-cumlative. At first I thought my commit rate is fast enough I just never see the non-cumlative result, but after 100s of samples I still always get zero values. Here is the current output of my documentCache from Solr's admin for 1 core: - documentCache http://localhost:8983/solr/#/channels_shard1_replica2/plugins/cache?en try=documentCache - class:org.apache.solr.search.LRUCache - version:1.0 - description:LRU Cache(maxSize=512, initialSize=512, autowarmCount=150, regenerator=null) - src:$URL: https:/ /svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_4_3/ solr/core/src/java/org/apache/solr/search/LRUCache.java https://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_4_3/s olr/core/src/java/org/apache/solr/search/LRUCache.java $ - stats: - lookups:0 - hits:0 - hitratio:0.00 - inserts:0 - evictions:0 - size:0 - warmupTime:0 - cumulative_lookups:65198986 - cumulative_hits:63075669 - cumulative_hitratio:0.96 - cumulative_inserts:2123317 - cumulative_evictions:1010262 The cumulative values seem to rise, suggesting doc cache is working, but at the same time it seems I never see non-cumlative metrics, most importantly warmupTime. Am I doing something wrong, is this normal/by-design, or is there an issue here? Thanks for helping with my silly question! Have a good weekend, Tim
Re: documentCache not used in 4.3.1?
Tim: Yeah, this doesn't make much sense to me either since, as you say, you should be seeing some metrics upon occasion. But do note that the underlying cache only gets filled when getting documents to return in query results, since there's no autowarming going on it may come and go. But you can test this pretty quickly by lengthening your autocommit interval or just not indexing anything for a while, then run a bunch of queries and look at your cache stats. That'll at least tell you whether it works at all. You'll have to have hard commits turned off (or openSearcher set to 'false') for that check too. Best Erick On Sat, Jun 29, 2013 at 2:48 PM, Vaillancourt, Tim tvaillanco...@ea.comwrote: Yes, we are softCommit'ing every 1000ms, but that should be enough time to see metrics though, right? For example, I still get non-cumulative metrics from the other caches (which are also throw away). I've also curl/sampled enough that I probably should have seen a value by now. If anyone else can reproduce this on 4.3.1 I will feel less crazy :). Cheers, Tim -Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Saturday, June 29, 2013 10:13 AM To: solr-user@lucene.apache.org Subject: Re: documentCache not used in 4.3.1? It's especially weird that the hit ratio is so high and you're not seeing anything in the cache. Are you perhaps soft committing frequently? Soft commits throw away all the top-level caches including documentCache I think Erick On Fri, Jun 28, 2013 at 7:23 PM, Tim Vaillancourt t...@elementspace.com wrote: Thanks Otis, Yeah I realized after sending my e-mail that doc cache does not warm, however I'm still lost on why there are no other metrics. Thanks! Tim On 28 June 2013 16:22, Otis Gospodnetic otis.gospodne...@gmail.com wrote: Hi Tim, Not sure about the zeros in 4.3.1, but in SPM we see all these numbers are non-0, though I haven't had the chance to confirm with Solr 4.3.1. Note that you can't really autowarm document cache... Otis -- Solr ElasticSearch Support -- http://sematext.com/ Performance Monitoring -- http://sematext.com/spm On Fri, Jun 28, 2013 at 7:14 PM, Tim Vaillancourt t...@elementspace.com wrote: Hey guys, This has to be a stupid question/I must be doing something wrong, but after frequent load testing with documentCache enabled under Solr 4.3.1 with autoWarmCount=150, I'm noticing that my documentCache metrics are always zero for non-cumlative. At first I thought my commit rate is fast enough I just never see the non-cumlative result, but after 100s of samples I still always get zero values. Here is the current output of my documentCache from Solr's admin for 1 core: - documentCache http://localhost:8983/solr/#/channels_shard1_replica2/plugins/cache?en try=documentCache - class:org.apache.solr.search.LRUCache - version:1.0 - description:LRU Cache(maxSize=512, initialSize=512, autowarmCount=150, regenerator=null) - src:$URL: https:/ /svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_4_3/ solr/core/src/java/org/apache/solr/search/LRUCache.java https://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_4_3/s olr/core/src/java/org/apache/solr/search/LRUCache.java $ - stats: - lookups:0 - hits:0 - hitratio:0.00 - inserts:0 - evictions:0 - size:0 - warmupTime:0 - cumulative_lookups:65198986 - cumulative_hits:63075669 - cumulative_hitratio:0.96 - cumulative_inserts:2123317 - cumulative_evictions:1010262 The cumulative values seem to rise, suggesting doc cache is working, but at the same time it seems I never see non-cumlative metrics, most importantly warmupTime. Am I doing something wrong, is this normal/by-design, or is there an issue here? Thanks for helping with my silly question! Have a good weekend, Tim
Re: documentCache not used in 4.3.1?
That's a good idea, I'll try that next week. Thanks! Tim On 29/06/13 12:39 PM, Erick Erickson wrote: Tim: Yeah, this doesn't make much sense to me either since, as you say, you should be seeing some metrics upon occasion. But do note that the underlying cache only gets filled when getting documents to return in query results, since there's no autowarming going on it may come and go. But you can test this pretty quickly by lengthening your autocommit interval or just not indexing anything for a while, then run a bunch of queries and look at your cache stats. That'll at least tell you whether it works at all. You'll have to have hard commits turned off (or openSearcher set to 'false') for that check too. Best Erick On Sat, Jun 29, 2013 at 2:48 PM, Vaillancourt, Timtvaillanco...@ea.comwrote: Yes, we are softCommit'ing every 1000ms, but that should be enough time to see metrics though, right? For example, I still get non-cumulative metrics from the other caches (which are also throw away). I've also curl/sampled enough that I probably should have seen a value by now. If anyone else can reproduce this on 4.3.1 I will feel less crazy :). Cheers, Tim -Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Saturday, June 29, 2013 10:13 AM To: solr-user@lucene.apache.org Subject: Re: documentCache not used in 4.3.1? It's especially weird that the hit ratio is so high and you're not seeing anything in the cache. Are you perhaps soft committing frequently? Soft commits throw away all the top-level caches including documentCache I think Erick On Fri, Jun 28, 2013 at 7:23 PM, Tim Vaillancourtt...@elementspace.com wrote: Thanks Otis, Yeah I realized after sending my e-mail that doc cache does not warm, however I'm still lost on why there are no other metrics. Thanks! Tim On 28 June 2013 16:22, Otis Gospodneticotis.gospodne...@gmail.com wrote: Hi Tim, Not sure about the zeros in 4.3.1, but in SPM we see all these numbers are non-0, though I haven't had the chance to confirm with Solr 4.3.1. Note that you can't really autowarm document cache... Otis -- Solr ElasticSearch Support -- http://sematext.com/ Performance Monitoring -- http://sematext.com/spm On Fri, Jun 28, 2013 at 7:14 PM, Tim Vaillancourt t...@elementspace.com wrote: Hey guys, This has to be a stupid question/I must be doing something wrong, but after frequent load testing with documentCache enabled under Solr 4.3.1 with autoWarmCount=150, I'm noticing that my documentCache metrics are always zero for non-cumlative. At first I thought my commit rate is fast enough I just never see the non-cumlative result, but after 100s of samples I still always get zero values. Here is the current output of my documentCache from Solr's admin for 1 core: - documentCache http://localhost:8983/solr/#/channels_shard1_replica2/plugins/cache?en try=documentCache - class:org.apache.solr.search.LRUCache - version:1.0 - description:LRU Cache(maxSize=512, initialSize=512, autowarmCount=150, regenerator=null) - src:$URL: https:/ /svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_4_3/ solr/core/src/java/org/apache/solr/search/LRUCache.java https://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_4_3/s olr/core/src/java/org/apache/solr/search/LRUCache.java $ - stats: - lookups:0 - hits:0 - hitratio:0.00 - inserts:0 - evictions:0 - size:0 - warmupTime:0 - cumulative_lookups:65198986 - cumulative_hits:63075669 - cumulative_hitratio:0.96 - cumulative_inserts:2123317 - cumulative_evictions:1010262 The cumulative values seem to rise, suggesting doc cache is working, but at the same time it seems I never see non-cumlative metrics, most importantly warmupTime. Am I doing something wrong, is this normal/by-design, or is there an issue here? Thanks for helping with my silly question! Have a good weekend, Tim
Re: documentCache not used in 4.3.1?
To answer some of my own question, Shawn H's great reply on this thread explains why I see no autoWarming on doc cache: http://www.marshut.com/iznwr/soft-commit-and-document-cache.html It is still unclear to me why I see no other metrics, however. Thanks Shawn, Tim On 28 June 2013 16:14, Tim Vaillancourt t...@elementspace.com wrote: Hey guys, This has to be a stupid question/I must be doing something wrong, but after frequent load testing with documentCache enabled under Solr 4.3.1 with autoWarmCount=150, I'm noticing that my documentCache metrics are always zero for non-cumlative. At first I thought my commit rate is fast enough I just never see the non-cumlative result, but after 100s of samples I still always get zero values. Here is the current output of my documentCache from Solr's admin for 1 core: - documentCachehttp://localhost:8983/solr/#/channels_shard1_replica2/plugins/cache?entry=documentCache - class:org.apache.solr.search.LRUCache - version:1.0 - description:LRU Cache(maxSize=512, initialSize=512, autowarmCount=150, regenerator=null) - src:$URL: https:/ /svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_4_3/ solr/core/src/java/org/apache/solr/search/LRUCache.javahttps://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_4_3/solr/core/src/java/org/apache/solr/search/LRUCache.java$ - stats: - lookups:0 - hits:0 - hitratio:0.00 - inserts: 0 - evictions:0 - size:0 - warmupTime:0 - cumulative_lookups: 65198986 - cumulative_hits:63075669 - cumulative_hitratio:0.96 - cumulative_inserts: 2123317 - cumulative_evictions:1010262 The cumulative values seem to rise, suggesting doc cache is working, but at the same time it seems I never see non-cumlative metrics, most importantly warmupTime. Am I doing something wrong, is this normal/by-design, or is there an issue here? Thanks for helping with my silly question! Have a good weekend, Tim
Re: documentCache not used in 4.3.1?
Hi Tim, Not sure about the zeros in 4.3.1, but in SPM we see all these numbers are non-0, though I haven't had the chance to confirm with Solr 4.3.1. Note that you can't really autowarm document cache... Otis -- Solr ElasticSearch Support -- http://sematext.com/ Performance Monitoring -- http://sematext.com/spm On Fri, Jun 28, 2013 at 7:14 PM, Tim Vaillancourt t...@elementspace.com wrote: Hey guys, This has to be a stupid question/I must be doing something wrong, but after frequent load testing with documentCache enabled under Solr 4.3.1 with autoWarmCount=150, I'm noticing that my documentCache metrics are always zero for non-cumlative. At first I thought my commit rate is fast enough I just never see the non-cumlative result, but after 100s of samples I still always get zero values. Here is the current output of my documentCache from Solr's admin for 1 core: - documentCachehttp://localhost:8983/solr/#/channels_shard1_replica2/plugins/cache?entry=documentCache - class:org.apache.solr.search.LRUCache - version:1.0 - description:LRU Cache(maxSize=512, initialSize=512, autowarmCount=150, regenerator=null) - src:$URL: https:/ /svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_4_3/ solr/core/src/java/org/apache/solr/search/LRUCache.javahttps://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_4_3/solr/core/src/java/org/apache/solr/search/LRUCache.java$ - stats: - lookups:0 - hits:0 - hitratio:0.00 - inserts:0 - evictions:0 - size:0 - warmupTime:0 - cumulative_lookups:65198986 - cumulative_hits:63075669 - cumulative_hitratio:0.96 - cumulative_inserts:2123317 - cumulative_evictions:1010262 The cumulative values seem to rise, suggesting doc cache is working, but at the same time it seems I never see non-cumlative metrics, most importantly warmupTime. Am I doing something wrong, is this normal/by-design, or is there an issue here? Thanks for helping with my silly question! Have a good weekend, Tim
Re: documentCache not used in 4.3.1?
Thanks Otis, Yeah I realized after sending my e-mail that doc cache does not warm, however I'm still lost on why there are no other metrics. Thanks! Tim On 28 June 2013 16:22, Otis Gospodnetic otis.gospodne...@gmail.com wrote: Hi Tim, Not sure about the zeros in 4.3.1, but in SPM we see all these numbers are non-0, though I haven't had the chance to confirm with Solr 4.3.1. Note that you can't really autowarm document cache... Otis -- Solr ElasticSearch Support -- http://sematext.com/ Performance Monitoring -- http://sematext.com/spm On Fri, Jun 28, 2013 at 7:14 PM, Tim Vaillancourt t...@elementspace.com wrote: Hey guys, This has to be a stupid question/I must be doing something wrong, but after frequent load testing with documentCache enabled under Solr 4.3.1 with autoWarmCount=150, I'm noticing that my documentCache metrics are always zero for non-cumlative. At first I thought my commit rate is fast enough I just never see the non-cumlative result, but after 100s of samples I still always get zero values. Here is the current output of my documentCache from Solr's admin for 1 core: - documentCache http://localhost:8983/solr/#/channels_shard1_replica2/plugins/cache?entry=documentCache - class:org.apache.solr.search.LRUCache - version:1.0 - description:LRU Cache(maxSize=512, initialSize=512, autowarmCount=150, regenerator=null) - src:$URL: https:/ /svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_4_3/ solr/core/src/java/org/apache/solr/search/LRUCache.java https://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_4_3/solr/core/src/java/org/apache/solr/search/LRUCache.java $ - stats: - lookups:0 - hits:0 - hitratio:0.00 - inserts:0 - evictions:0 - size:0 - warmupTime:0 - cumulative_lookups:65198986 - cumulative_hits:63075669 - cumulative_hitratio:0.96 - cumulative_inserts:2123317 - cumulative_evictions:1010262 The cumulative values seem to rise, suggesting doc cache is working, but at the same time it seems I never see non-cumlative metrics, most importantly warmupTime. Am I doing something wrong, is this normal/by-design, or is there an issue here? Thanks for helping with my silly question! Have a good weekend, Tim
