Tulsi Das, Thanks for pointing me to the links, but unfortunately these articles/JIRA don't answer my question. Moreover they are over a decade old and much has changed since in the Solr world :) I would still like to know people's experiences handling a large number of Solr cores on a single node and at what point do cracks start to appear with a large number of loaded cores for their hardware configuration. I am aware of the option of marking cores transient and the loadOnStartup option, but these options have their own limitations.
Still looking forward to getting more inputs on this thread! Thanks, Rahul On Thu, Nov 18, 2021 at 9:27 AM Tulsi Das <tulsi.das1...@gmail.com> wrote: > refer this as well : > > https://solr-user.lucene.apache.narkive.com/d7Y5M6Cl/max-no-of-solr-cores-supported-and-how-to-restrict-a-query-to-a-particular-core > > > On Thu, Nov 18, 2021 at 7:54 PM Tulsi Das <tulsi.das1...@gmail.com> wrote: > > > Probably you can take a look at this jira : > > https://issues.apache.org/jira/browse/SOLR-1028 to get an idea. > > > > On Sat, Nov 13, 2021 at 1:29 AM Rahul Goswami <rahul196...@gmail.com> > > wrote: > > > >> To clarify, I mean the number of Solr cores per JVM instance. How many > >> Solr > >> cores on one node is too many? > >> As mentioned, it would be nice to get the numbers for a Solr instance > >> running in standalone mode (aka non-SolrCloud), but if not, would also > be > >> interested to know about people's experience with SolrCloud in this > >> regard. > >> > >> Thanks, > >> Rahul > >> > >> > >> On Fri, Nov 12, 2021 at 2:29 PM Tulsi Das <tulsi.das1...@gmail.com> > >> wrote: > >> > >> > Hi Rahul , > >> > Are you asking about the cores (number of replica's per node) or the > cpu > >> > cores? > >> > > >> > On Sat, Nov 13, 2021, 12:50 AM Walter Underwood < > wun...@wunderwood.org> > >> > wrote: > >> > > >> > > We’ve run on AWS instances with 72 CPUs. They all get used. > >> Throughput is > >> > > linear with the number of CPUs. You need enough free RAM to cache > all > >> of > >> > > the index files in OS file buffers. > >> > > > >> > > The entire point of avoiding locking in the Lucene index is so that > >> > > multiple threads can read it without contention. We made the same > >> > decision > >> > > in the Ultraseek index design 25 years ago. > >> > > > >> > > We don’t do any special JVM tuning. We use the config that Shawn > >> Heisey > >> > > recommended five years ago. We reacently increased the heap from 8 > GB > >> to > >> > 16 > >> > > GB. > >> > > > >> > > GC_TUNE=" \ > >> > > -XX:+UseG1GC \ > >> > > -XX:+ParallelRefProcEnabled \ > >> > > -XX:G1HeapRegionSize=8m \ > >> > > -XX:MaxGCPauseMillis=200 \ > >> > > -XX:+UseLargePages \ > >> > > -XX:+AggressiveOpts \ > >> > > " > >> > > > >> > > wunder > >> > > Walter Underwood > >> > > wun...@wunderwood.org > >> > > http://observer.wunderwood.org/ (my blog) > >> > > > >> > > > On Nov 12, 2021, at 7:41 AM, Deepak Goel <deic...@gmail.com> > wrote: > >> > > > > >> > > > My guess is (please note it is not a benchmark): you would need a > >> lot > >> > of > >> > > > tuning to make Solr use 32 cpu cores per node. After 4 cpu cores, > >> you > >> > > would > >> > > > have to start tuning Solr, JVM, your app (requirement), IOP'S. > >> > > > > >> > > > Deepak > >> > > > "The greatness of a nation can be judged by the way its animals > are > >> > > treated > >> > > > - Mahatma Gandhi" > >> > > > > >> > > > +91 73500 12833 > >> > > > deic...@gmail.com > >> > > > > >> > > > Facebook: https://www.facebook.com/deicool > >> > > > LinkedIn: www.linkedin.com/in/deicool > >> > > > > >> > > > "Plant a Tree, Go Green" > >> > > > > >> > > > Make In India : http://www.makeinindia.com/home > >> > > > > >> > > > > >> > > > On Fri, Nov 12, 2021 at 8:33 PM Rahul Goswami < > >> rahul196...@gmail.com> > >> > > wrote: > >> > > > > >> > > >> Hi, > >> > > >> Does anyone have benchmarks on performance as the number of cores > >> on a > >> > > Solr > >> > > >> node goes up? I am trying to get an idea about how many cores per > >> node > >> > > is > >> > > >> too much. Assume 31 GB heap size, SSD disk and 32 CPU cores. > >> > > >> Preferably non-SolrCloud (aka standalone), but even if you have > >> > insights > >> > > >> from SolrCloud that would be a good start. > >> > > >> I am using Solr 7.7.2. > >> > > >> > >> > > >> Thanks, > >> > > >> Rahul > >> > > >> > >> > > > >> > > > >> > > >> > > >
