My guess is that it's trashing on a "cold" open of the index file. I'm
sure the next query of *:*&rows=2 is pretty fast since caches get populated.
I don't know what to say for next steps - lower the jvm memory and/or check
the stats in the admin console -> core selct -> Plugins/Stats -> CACHE.
What are the storage speeds? IMHO you are disk bound.
On Fri, Mar 18, 2022 at 3:42 PM Vincenzo D'Amore <v.dam...@gmail.com> wrote:
> Is it possible that there are too frequent commits? I mean if each commit
> usually invalidates the cache, even the a stupid *:* rows=1 can be
> affected.
> How can I see how frequent commits are? Or when the latest commit has been
> done?
>
> On Fri, Mar 18, 2022 at 8:36 PM Vincenzo D'Amore <v.dam...@gmail.com>
> wrote:
>
> > Ok, everything you said is right, but nevertheless even right now a
> stupid
> > *:* rows=1 runs in almost 2 seconds.
> > The average document size is pretty small, less than roughly 100/200
> > bytes.
> > Does someone know if the average doc size is available in the metrics?
> >
> > {
> > "responseHeader":{
> > "zkConnected":true,
> > "status":0,
> > "QTime":2033,
> > "params":{
> > "q":"*:*",
> > "rows":"1"}},
> >
> > On Fri, Mar 18, 2022 at 7:50 PM matthew sporleder <msporle...@gmail.com>
> > wrote:
> >
> >> You are getting this general advice but, sadly, it depends on your doc
> >> sizes, query complexity, write frequency, and a bunch of other stuff I
> >> don't know about.
> >>
> >> I prefer to run with the *minimum* JVM memory to handle throughput
> >> (without
> >> OOM) and let the OS do caching because I update/write to the index every
> >> few minutes making my *solr* caching pretty worthless.
> >>
> >> tuning solr also includes tuning queries. Start with timing id:123 type
> >> K:V lookups and work your complexity up from there. Use debug=true and
> >> attempt to read it.
> >>
> >> There are many many knobs. You need to set a baseline, then a target,
> >> then
> >> a thesis on how to get there.
> >>
> >>
> >>
> >> On Fri, Mar 18, 2022 at 2:36 PM Vincenzo D'Amore <v.dam...@gmail.com>
> >> wrote:
> >>
> >> > We have modified the kubernetes configuration and restarted SolrCloud
> >> > cluster, now we have 16 cores per Solr instance.
> >> > The performance does not seem to be improved though.
> >> > The load average is 0.43 0.83 1.00, to me it seems an IO bound
> problem.
> >> > Looking at the index I see 162M documents, 234M maxDocs, 71M
> deleted...
> >> > maybe this core needs to be optimized.
> >> > The INDEX.size is 70GB, what do you think if I raise the size
> allocated
> >> > from the JVM to 64GB in order to have the index in memory?
> >> > At last, I'm looking at Solr metric but really not sure how to
> >> understand
> >> > if it is CPU bound or IO bound.
> >> >
> >> > On Fri, Mar 18, 2022 at 6:34 PM Walter Underwood <
> wun...@wunderwood.org
> >> >
> >> > wrote:
> >> >
> >> > > First look at the system metrics. Is it CPU bound or IO bound? Each
> >> > > request is single threaded, so a CPU bound system will have one core
> >> used
> >> > > at roughly 100% for that time. An IO bound system will not be using
> >> much
> >> > > CPU but will have threads in iowait and lots of disk reads.
> >> > >
> >> > > After you know that, then you know what to work on. If it is IO
> bound,
> >> > get
> >> > > enough RAM for the OS, JVM, and index files to all be in memory. If
> >> it is
> >> > > CPU bound, get a faster processor and work on the config to have the
> >> > > request do less work. Sharding can also help.
> >> > >
> >> > > I’m not a fan of always choosing 31 GB for the JVM. Allocate only as
> >> much
> >> > > as is needed. Java will use the whole heap whether it is needed or
> >> not.
> >> > You
> >> > > might only need 8 GB. All of our clusters run with 16 GB. That
> >> includes
> >> > > some machines with 36 cores.
> >> > >
> >> > >
> >> > --
> >> > Vincenzo D'Amore
> >> >
> >>
> >
> >
> > --
> > Vincenzo D'Amore
> >
> >
>
> --
> Vincenzo D'Amore
>
