I've also found that: CONTAINER.fs.coreRoot.spins: true
Can be this considered a problem so huge as to affect the overall
performance?
On Fri, Mar 18, 2022 at 6:46 PM Dave <hastings.recurs...@gmail.com> wrote:
> Walter I agree, but with large indexes (850+Gb before merge) I just found
> 31 to be my happy spot. As well as set xms and xmx to the same value, I
> have no proof but it seems to take less processing to keep them the same
> than to keep allocating different memory footprints
>
> > On Mar 18, 2022, at 1:38 PM, Vincenzo D'Amore <v.dam...@gmail.com>
> wrote:
> >
> > I did it right now in prod environment:
> > {
> > "responseHeader":{
> > "zkConnected":true,
> > "status":0,
> > "QTime":1943,
> > "params":{
> > "q":"*:*",
> > "rows":"1"}},
> >
> > then for a while, the QTime is 0. I assume (obviously) that it is cached,
> > but after a while the cache expires....
> >
> >> On Fri, Mar 18, 2022 at 6:22 PM Dave <hastings.recurs...@gmail.com>
> wrote:
> >>
> >> I’ve found that each solr instance will take as many cores as it needs
> per
> >> request. Your 2 sec response sounds like you just started the server and
> >> then did that search. I never trust the first search as nothing has been
> >> put into memory yet. I like to give my jvms 31 gb each and let Linux
> cache
> >> the rest of the files as it sees fit, with swap turned completely off.
> Also
> >> *:* can be heavier than you think if you have every field indexed since
> >> it’s like a punch card like system where all the fields have to match.
> >>
> >>> On Mar 18, 2022, at 12:45 PM, Vincenzo D'Amore <v.dam...@gmail.com>
> >> wrote:
> >>>
> >>> Thanks for your support, just sharing what I found until now.
> >>>
> >>> I'm working with SolrCloud with a 2 node deployment. This deployment
> has
> >>> many indexes but a main one 160GB index that has become very slow.
> >>> Select *:* rows=1 take 2 seconds.
> >>> SolrCloud instances are running in kubernetes and are deployed in a pod
> >>> with 128GB RAM but only 16GB to JVM.
> >>> Looking at Solr Documentation I've found nothing specific about what
> >>> happens to the performance if the number of CPUs is not correctly
> >> detected.
> >>> The only interesting page is the following and it seems to match with
> >> your
> >>> suggestion.
> >>> At the end of paragraph there is a not very clear reference about how
> the
> >>> Concurrent Merge Scheduler behavior can be impacted by the number of
> >>> detected CPUs.
> >>>
> >>>> Similarly, the system property lucene.cms.override_core_count can be
> set
> >>> to the number of CPU cores to override the auto-detected processor
> count.
> >>>
> >>>> Talking Solr to Production > Dynamic Defaults for
> >> ConcurrentMergeScheduler
> >>>>
> >>>
> >>
> https://solr.apache.org/guide/8_3/taking-solr-to-production.html#dynamic-defaults-for-concurrentmergescheduler
> >>>
> >>>
> >>>
> >>>> On Thu, Mar 17, 2022 at 1:22 PM Thomas Matthijs <li...@selckin.be>
> >> wrote:
> >>>>
> >>>> I don't know how it affects solr, but if you're interested in java's
> >>>> support to detect cgroup/container limits on cpu/memory etc, you can
> use
> >>>> these links as starting points to investigate.
> >>>> It affect some jvm configuration, like initial GC selection & settings
> >>>> that can affect performance.
> >>>> It was only backported to java 8 quite recently, so if you're still on
> >>>> that might want to check if you're on the latest version.
> >>>>
> >>>> https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8146115
> >>>> https://bugs.openjdk.java.net/browse/JDK-8264136
> >>>>
> >>>>
> >>>>> On Thu, Mar 17, 2022, at 01:11, Vincenzo D'Amore wrote:
> >>>>> Hi Shawn, thanks for your help.
> >>>>>
> >>>>> Given that I’ll put the question in another way.
> >>>>> If Java don’t correctly detect the number of CPU how the overall
> >>>>> performance can be affected by this?
> >>>>>
> >>>>> Ciao,
> >>>>> Vincenzo
> >>>>>
> >>>>> --
> >>>>> mobile: 3498513251
> >>>>> skype: free.dev
> >>>>>
> >>>>>> On 16 Mar 2022, at 18:56, Shawn Heisey <elyog...@elyograg.org>
> wrote:
> >>>>>>
> >>>>>> On 3/16/22 03:56, Vincenzo D'Amore wrote:
> >>>>>>> just asking how can I rely on the number of processors the solr
> >>>> dashboard
> >>>>>>> shows.
> >>>>>>>
> >>>>>>> Just to give you a context, I have a 2 nodes solrcloud instance
> >>>> running in
> >>>>>>> kubernetes.
> >>>>>>> Looking at solr dashboard (8.3.0) I see there is only 1 cpu
> available
> >>>> per
> >>>>>>> solr instance.
> >>>>>>> but the Solr pods are deployed in two different kube nodes, and
> >>>> entering
> >>>>>>> the pod with the
> >>>>>>> kubectl exec -ti solr-0 -- /bin/bash
> >>>>>>> and running top I see there are 16 cores available for each solr
> >>>> instance.
> >>>>>>
> >>>>>> The dashboard info comes from Java, and Java gets it from the OS.
> How
> >>>> that works with containers is something I don't know much about.
> Here's
> >>>> what Linux says about a server I have which has two six-core Intel
> CPUs
> >>>> with hyperthreading. This is bare metal, not a VM or container:
> >>>>>>
> >>>>>> elyograg@smeagol:~$ grep processor /proc/cpuinfo
> >>>>>> processor : 0
> >>>>>> processor : 1
> >>>>>> processor : 2
> >>>>>> processor : 3
> >>>>>> processor : 4
> >>>>>> processor : 5
> >>>>>> processor : 6
> >>>>>> processor : 7
> >>>>>> processor : 8
> >>>>>> processor : 9
> >>>>>> processor : 10
> >>>>>> processor : 11
> >>>>>> processor : 12
> >>>>>> processor : 13
> >>>>>> processor : 14
> >>>>>> processor : 15
> >>>>>> processor : 16
> >>>>>> processor : 17
> >>>>>> processor : 18
> >>>>>> processor : 19
> >>>>>> processor : 20
> >>>>>> processor : 21
> >>>>>> processor : 22
> >>>>>> processor : 23
> >>>>>>
> >>>>>> If I start Solr on that server, the dashboard reports 24 processors.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Shawn
> >>>>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> Vincenzo D'Amore
> >>
> >
> >
> > --
> > Vincenzo D'Amore
>
--
Vincenzo D'Amore
