And I'd add - are you sending one document at a time or batching them up? See: https://lucidworks.com/2015/10/05/really-batch-updates-solr-2/
Best, Erick On Wed, Jan 10, 2018 at 1:35 PM, Gus Heck <gus.h...@gmail.com> wrote: > Ok then here's a few things to check... > > - Did you sept up an actual multiple node cluster or are you running > this all on one box? > - Are you configuring Jmeter to send with multiple threads? > - Are they all sending to the same node, or are you distributing across > nodes? Is there a load balancer? > - Are you sending from a machine on the same network as the machines in > the Solr cluster? > - If you are sending requests up to the cloud from your local machine, > that is frequently a slow link. > - Also don't forget to check your zookeeper cluster's health... if it's > bogged down that will slow down solr. > > If you have all machines on the same network, many threads, load balancing > and no questionable equipment (or networking limitations put in place by > IT) in the middle, then something (either CPU or network interface) should > be maxed out somewhere on at least one machine, either on the Jmeter side > or Solr side. > > -Gus > > On Wed, Jan 10, 2018 at 3:54 PM, Shashank Pedamallu <spedama...@vmware.com > > > wrote: > > > Hi Gus, > > > > Thank for the reply. I’m sending via jmeter running on my local machine > > to Solr running on a remote vm. > > > > Thanks, > > Shashank > > > > On 1/10/18, 12:34 PM, "Gus Heck" <gus.h...@gmail.com> wrote: > > > > Ingested how? Sounds like your document sending mechanism is maxed, > > not the > > solr cluster... > > > > On Wed, Jan 10, 2018 at 2:58 PM, Shashank Pedamallu < > > spedama...@vmware.com> > > wrote: > > > > > Hi, > > > > > > > > > > > > I’m trying to find the upper thresholds of ingestion and I have > > tried the > > > following. In each of the experiments, I’m ingesting random > > documents with > > > 5 fields. > > > > > > > > > Number of Cores Number of documents ingested per second per core > > > 1 89000 > > > 3 33000 > > > 5 18000 > > > > > > > > > As you can see, the number of documents being ingested per core is > > not > > > scaling horizontally as I'm adding more cores. Rather the total > > number of > > > documents getting ingested for Solr JVM is being topped around 90k > > > documents per second. > > > > > > > > > From the iostats and top commands, I do not see any bottlenecks > with > > the > > > iops or cpu respectively, CPU usaeg is around 65% and a sample of > > iostats > > > is below: > > > > > > avg-cpu: %user %nice %system %iowait %steal %idle > > > > > > 55.32 0.00 2.33 1.64 0.00 40.71 > > > > > > > > > Device: tps kB_read/s kB_wrtn/s kB_read > > kB_wrtn > > > > > > sda5 2523.00 45812.00 298312.00 45812 > > 298312 > > > > > > > > > Can someone please guide me as to how I can debug this further and > > > root-cause the bottleneck for not being able to increase the > > ingestion > > > horizontally. > > > > > > > > > Thanks, > > > > > > Shashank > > > > > > > > > > > -- > > https://urldefense.proofpoint.com/v2/url?u=http-3A__www. > > the111shift.com&d=DwIFaQ&c=uilaK90D4TOVoH58JNXRgQ&r= > > blJD2pBapH3dDkoajIf9mT9SSbbs19wRbChNde1ErNI&m=DT_ > > 33Z3k4h8T1t65CuyH0oMxay15ddkfDYAQefzgpa4&s=6-1wd3YPVRgcvlk3LkK7Wz- > > 3hDFliEGwVGc44HJH1x4&e= > > > > > > > > > -- > http://www.the111shift.com >
