Re: Embedded Server, Caching, Stats page updates
So you're right i did miss removing the app deployment but removing that still didn't really do that great. The avg request response time is still slower. The bell curve is a lot more streched than it was before but it doesn't seem to give an overall better performance. Thanks for your suggestions, Tony On Wed, May 19, 2010 at 4:37 PM, Chris Hostetter hossman_luc...@fucit.orgwrote: : Switched works for the specific setup i'm using - the server would refer : to itself in the CommonHttpSolrServer request sent, i.e. it would run both : the server and client sides. Removing this and simply using : EmbeddedSolrServer just made the setup a little more sane in that aspect. : Does that make more sense now? not really ... what *exactly* did you change about your setup and your client code? please be specific -- how did you run solr before when you were using CommonsHttpSolrServer? whare are *all* of the steps you did when you switched to EmbeddedSolrServer (specificly: what did the changes to your java client code look like, and what did you hcange about how you run solr) Because if you still have the solr.war running in your servlet container, and all you did is edit your java code to use EmbeddedSolrServer (poiting at the same directory on disk) instead of COmmonsHttpSolrServer, thne you are now running *two* instances of Solr in your VM, both reading from the same indexes. -Hoss
Re: Embedded Server, Caching, Stats page updates
The way you phrased that paragraph makes me think that one of us doesn't understand what exactly you did when you switched ... Switched works for the specific setup i'm using - the server would refer to itself in the CommonHttpSolrServer request sent, i.e. it would run both the server and client sides. Removing this and simply using EmbeddedSolrServer just made the setup a little more sane in that aspect. Does that make more sense now? Now for starters: if the remote server you were running solr on is more powerful then the local machine you are running your java application on, that alone could explain some performance differences (likewise for JVM settings). The machine I'm running it on is exactly the same - the code change was pushed and I had performance before and after. Same load observed (since it's a testing machine i could regulate that). That's why i was so surprised that removing that additional http request didn't cause improvement. Most importantly: when running solr embedded in your application, there is no stats.jsp page for you to look at -- because solr is no longer running in a servlet container. so if you are seeing stats on your solr server that say your caches aren't being hit, the reason is because the server isn't being hit at all. This is nice to know, I didn't look into how the actual page was generated. I expected something like this to be true. Thank you! When running an embedded solr server, the filterCache and queryResultCache will still be used. the settings in the solrconfig.xml you specify when initializing the SolrCore will be honored. you can see use JMX to monitor those cache hit rates (assuming you have JMX enabled for your application, and the appropriate setting is in your solrconfig.xml) I'll look into using JMX, thanks for the suggestion. Tony
Re: Embedded Server, Caching, Stats page updates
: Switched works for the specific setup i'm using - the server would refer : to itself in the CommonHttpSolrServer request sent, i.e. it would run both : the server and client sides. Removing this and simply using : EmbeddedSolrServer just made the setup a little more sane in that aspect. : Does that make more sense now? not really ... what *exactly* did you change about your setup and your client code? please be specific -- how did you run solr before when you were using CommonsHttpSolrServer? whare are *all* of the steps you did when you switched to EmbeddedSolrServer (specificly: what did the changes to your java client code look like, and what did you hcange about how you run solr) Because if you still have the solr.war running in your servlet container, and all you did is edit your java code to use EmbeddedSolrServer (poiting at the same directory on disk) instead of COmmonsHttpSolrServer, thne you are now running *two* instances of Solr in your VM, both reading from the same indexes. -Hoss
Embedded Server, Caching, Stats page updates
I just switched from using CommonHttpSolrServer to EmbeddedSolrServer and the performance surprisingly deteriorated. I was expecting an improvement so in my confusion i went to the stats page and noticed that the caches were no longer getting hit. The embedded server however should still use IndexSearcher from Lucene (which is what the caches are supposed to be related to). Is there some kind of property that needs to be added or adjusted for embedded server to use cache? Should I create my own cache and wipe the rest out entirely? Should I remove the httpcache from the configuration since i'll no longer be accessing the service remotely? How accurate is the stats page and is the error actually coming from it rather than the actual backend? Thank you beforehand, Tony
Re: Embedded Server, Caching, Stats page updates
: I just switched from using CommonHttpSolrServer to EmbeddedSolrServer and : the performance surprisingly deteriorated. I was expecting an improvement so : in my confusion i went to the stats page and noticed that the caches were no : longer getting hit. The embedded server however should still use : IndexSearcher from Lucene (which is what the caches are supposed to be : related to). The way you phrased that paragraph makes me think that one of us doesn't understand what exactly you did when you switched ... When using CommonsHttpSolrServer in some application you write, you are talk to a remote server that is running Solr. when you use EmbeddedSolrServer, you are running solr directly within the application that you are writing. Now for starters: if the remote server you were running solr on is more powerful then the local machine you are running your java application on, that alone could explain some performance differences (likewise for JVM settings). Most importantly: when running solr embedded in your application, there is no stats.jsp page for you to look at -- because solr is no longer running in a servlet container. so if you are seeing stats on your solr server that say your caches aren't being hit, the reason is because the server isn't being hit at all. : Is there some kind of property that needs to be added or adjusted for : embedded server to use cache? Should I create my own cache and wipe the rest When running an embedded solr server, the filterCache and queryResultCache will still be used. the settings in the solrconfig.xml you specify when initializing the SolrCore will be honored. you can see use JMX to monitor those cache hit rates (assuming you have JMX enabled for your application, and the appropriate setting is in your solrconfig.xml) -Hoss
