Justin, in terms of the overhead, have you noticed if Munin puts much of it when used in production? In terms of the solr farm: how big is a shard's index (given you have sharded architecture).
Dmitry On Sun, Dec 11, 2011 at 6:39 PM, Justin Caratzas <justin.carat...@gmail.com>wrote: > At my work, we use Munin and Nagio for monitoring and alerts. Munin is > great because writing a plugin for it so simple, and with Solr's > statistics handler, we can track almost any solr stat we want. It also > comes with included plugins for load, file system stats, processes, > etc. > > http://munin-monitoring.org/ > > Justin > > Paul Libbrecht <p...@hoplahup.net> writes: > > > Allow me to chim in and ask a generic question about monitoring tools > > for people close to developers: are any of the tools mentioned in this > > thread actually able to show graphs of loads, e.g. cache counts or CPU > > load, in parallel to a console log or to an http request log?? > > > > I am working on such a tool currently but I have a bad feeling of > reinventing the wheel. > > > > thanks in advance > > > > Paul > > > > > > > > Le 8 déc. 2011 à 08:53, Dmitry Kan a écrit : > > > >> Otis, Tomás: thanks for the great links! > >> > >> 2011/12/7 Tomás Fernández Löbbe <tomasflo...@gmail.com> > >> > >>> Hi Dimitry, I pointed to the wiki page to enable JMX, then you can use > any > >>> tool that visualizes JMX stuff like Zabbix. See > >>> > >>> > http://www.lucidimagination.com/blog/2011/10/02/monitoring-apache-solr-and-lucidworks-with-zabbix/ > >>> > >>> On Wed, Dec 7, 2011 at 11:49 AM, Dmitry Kan <dmitry....@gmail.com> > wrote: > >>> > >>>> The culprit seems to be the merger (frontend) SOLR. Talking to one > shard > >>>> directly takes substantially less time (1-2 sec). > >>>> > >>>> On Wed, Dec 7, 2011 at 4:10 PM, Dmitry Kan <dmitry....@gmail.com> > wrote: > >>>> > >>>>> Tomás: thanks. The page you gave didn't mention cache specifically, > is > >>>>> there more documentation on this specifically? I have used solrmeter > >>>> tool, > >>>>> it draws the cache diagrams, is there a similar tool, but which would > >>> use > >>>>> jmx directly and present the cache usage in runtime? > >>>>> > >>>>> pravesh: > >>>>> I have increased the size of filterCache, but the search hasn't > become > >>>> any > >>>>> faster, taking almost 9 sec on avg :( > >>>>> > >>>>> name: search > >>>>> class: org.apache.solr.handler.component.SearchHandler > >>>>> version: $Revision: 1052938 $ > >>>>> description: Search using components: > >>>>> > >>>> > >>> > org.apache.solr.handler.component.QueryComponent,org.apache.solr.handler.component.FacetComponent,org.apache.solr.handler.component.MoreLikeThisComponent,org.apache.solr.handler.component.HighlightComponent,org.apache.solr.handler.component.StatsComponent,org.apache.solr.handler.component.DebugComponent, > >>>>> > >>>>> stats: handlerStart : 1323255147351 > >>>>> requests : 100 > >>>>> errors : 3 > >>>>> timeouts : 0 > >>>>> totalTime : 885438 > >>>>> avgTimePerRequest : 8854.38 > >>>>> avgRequestsPerSecond : 0.008789442 > >>>>> > >>>>> the stats (copying fieldValueCache as well here, to show term > >>>> statistics): > >>>>> > >>>>> name: fieldValueCache > >>>>> class: org.apache.solr.search.FastLRUCache > >>>>> version: 1.0 > >>>>> description: Concurrent LRU Cache(maxSize=10000, initialSize=10, > >>>>> minSize=9000, acceptableSize=9500, cleanupThread=false) > >>>>> stats: lookups : 79 > >>>>> hits : 77 > >>>>> hitratio : 0.97 > >>>>> inserts : 1 > >>>>> evictions : 0 > >>>>> size : 1 > >>>>> warmupTime : 0 > >>>>> cumulative_lookups : 79 > >>>>> cumulative_hits : 77 > >>>>> cumulative_hitratio : 0.97 > >>>>> cumulative_inserts : 1 > >>>>> cumulative_evictions : 0 > >>>>> item_shingleContent_trigram : > >>>>> > >>>> > >>> > {field=shingleContent_trigram,memSize=326924381,tindexSize=4765394,time=215426,phase1=213868,nTerms=14827061,bigTerms=35,termInstances=114359167,uses=78} > >>>>> name: filterCache > >>>>> class: org.apache.solr.search.FastLRUCache > >>>>> version: 1.0 > >>>>> description: Concurrent LRU Cache(maxSize=153600, initialSize=4096, > >>>>> minSize=138240, acceptableSize=145920, cleanupThread=false) > >>>>> stats: lookups : 1082854 > >>>>> hits : 940370 > >>>>> hitratio : 0.86 > >>>>> inserts : 142486 > >>>>> evictions : 0 > >>>>> size : 142486 > >>>>> warmupTime : 0 > >>>>> cumulative_lookups : 1082854 > >>>>> cumulative_hits : 940370 > >>>>> cumulative_hitratio : 0.86 > >>>>> cumulative_inserts : 142486 > >>>>> cumulative_evictions : 0 > >>>>> > >>>>> > >>>>> index size: 3,25 GB > >>>>> > >>>>> Does anyone have some pointers to where to look at and optimize for > >>> query > >>>>> time? > >>>>> > >>>>> > >>>>> 2011/12/7 Tomás Fernández Löbbe <tomasflo...@gmail.com> > >>>>> > >>>>>> Hi Dimitry, cache information is exposed via JMX, so you should be > >>> able > >>>> to > >>>>>> monitor that information with any JMX tool. See > >>>>>> http://wiki.apache.org/solr/SolrJmx > >>>>>> > >>>>>> On Wed, Dec 7, 2011 at 6:19 AM, Dmitry Kan <dmitry....@gmail.com> > >>>> wrote: > >>>>>> > >>>>>>> Yes, we do require that much. > >>>>>>> Ok, thanks, I will try increasing the maxsize. > >>>>>>> > >>>>>>> On Wed, Dec 7, 2011 at 10:56 AM, pravesh <suyalprav...@yahoo.com> > >>>>>> wrote: > >>>>>>> > >>>>>>>>>> facet.limit=500000 > >>>>>>>> your facet.limit seems too high. Do you actually require this > >>> much? > >>>>>>>> > >>>>>>>> Since there a lot of evictions from filtercache, so, increase the > >>>>>> maxsize > >>>>>>>> value to your acceptable limit. > >>>>>>>> > >>>>>>>> Regards > >>>>>>>> Pravesh > >>>>>>>> > >>>>>>>> -- > >>>>>>>> View this message in context: > >>>>>>>> > >>>>>>> > >>>>>> > >>>> > >>> > http://lucene.472066.n3.nabble.com/cache-monitoring-tools-tp3566645p3566811.html > >>>>>>>> Sent from the Solr - User mailing list archive at Nabble.com. > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> -- > >>>>>>> Regards, > >>>>>>> > >>>>>>> Dmitry Kan > >>>>>>> > >>>>>> > >>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> Regards, > >>>>> > >>>>> Dmitry Kan > >>>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> Regards, > >>>> > >>>> Dmitry Kan > >>>> > >>> > >> > >> > >> > >> -- > >> Regards, > >> > >> Dmitry Kan > -- Regards, Dmitry Kan
