Ok, everything you said is right, but nevertheless even right now a stupid
*:* rows=1 runs in almost 2 seconds.
The average document size is pretty small, less than roughly 100/200 bytes.
Does someone know if the average doc size is available in the metrics?
{
"responseHeader":{
"zkConnected":true,
"status":0,
"QTime":2033,
"params":{
"q":"*:*",
"rows":"1"}},
On Fri, Mar 18, 2022 at 7:50 PM matthew sporleder <msporle...@gmail.com>
wrote:
> You are getting this general advice but, sadly, it depends on your doc
> sizes, query complexity, write frequency, and a bunch of other stuff I
> don't know about.
>
> I prefer to run with the *minimum* JVM memory to handle throughput (without
> OOM) and let the OS do caching because I update/write to the index every
> few minutes making my *solr* caching pretty worthless.
>
> tuning solr also includes tuning queries. Start with timing id:123 type
> K:V lookups and work your complexity up from there. Use debug=true and
> attempt to read it.
>
> There are many many knobs. You need to set a baseline, then a target, then
> a thesis on how to get there.
>
>
>
> On Fri, Mar 18, 2022 at 2:36 PM Vincenzo D'Amore <v.dam...@gmail.com>
> wrote:
>
> > We have modified the kubernetes configuration and restarted SolrCloud
> > cluster, now we have 16 cores per Solr instance.
> > The performance does not seem to be improved though.
> > The load average is 0.43 0.83 1.00, to me it seems an IO bound problem.
> > Looking at the index I see 162M documents, 234M maxDocs, 71M deleted...
> > maybe this core needs to be optimized.
> > The INDEX.size is 70GB, what do you think if I raise the size allocated
> > from the JVM to 64GB in order to have the index in memory?
> > At last, I'm looking at Solr metric but really not sure how to understand
> > if it is CPU bound or IO bound.
> >
> > On Fri, Mar 18, 2022 at 6:34 PM Walter Underwood <wun...@wunderwood.org>
> > wrote:
> >
> > > First look at the system metrics. Is it CPU bound or IO bound? Each
> > > request is single threaded, so a CPU bound system will have one core
> used
> > > at roughly 100% for that time. An IO bound system will not be using
> much
> > > CPU but will have threads in iowait and lots of disk reads.
> > >
> > > After you know that, then you know what to work on. If it is IO bound,
> > get
> > > enough RAM for the OS, JVM, and index files to all be in memory. If it
> is
> > > CPU bound, get a faster processor and work on the config to have the
> > > request do less work. Sharding can also help.
> > >
> > > I’m not a fan of always choosing 31 GB for the JVM. Allocate only as
> much
> > > as is needed. Java will use the whole heap whether it is needed or not.
> > You
> > > might only need 8 GB. All of our clusters run with 16 GB. That includes
> > > some machines with 36 cores.
> > >
> > >
> > --
> > Vincenzo D'Amore
> >
>
--
Vincenzo D'Amore
