We haven't tried Java 8 or changing garbage collection yet -- I've heard mixed results on GC. We're using SSD storage on Azure, and have a fairly tweaked out config. I was thinking about turning off any kind of analysis in the template to get it to scale...
On Sunday, March 16, 2014 7:37:09 PM UTC-7, Mark Walkom wrote: > > We are getting ~15K/s with 12 data + 3 master nodes, latest version of > java and ES. > > Some things to try would be; > > - Java 8 > - G1GC > - SSD storage > - tweaking various index and pool caches > - optimising your data inputs, mappings etc > > We're trialling the first two of those on our dev cluster, but it doesn't > do much traffic so I cannot empirically comment on it's capabilities at the > levels you're after as yet. > > What does your current setup (ie infrastructure) look like now? > > > Regards, > Mark Walkom > > Infrastructure Engineer > Campaign Monitor > email: [email protected] <javascript:> > web: www.campaignmonitor.com > > > On 17 March 2014 13:17, <[email protected] <javascript:>> wrote: > >> We've gotten a cluster up to 40K/sec using roughly 40 nodes. We're going >> to switch over to using dedicated master nodes as well. This is all in AWS. >> >> -Justin >> >> On Sunday, March 16, 2014 9:14:43 PM UTC-4, Janet Sullivan wrote: >>> >>> I currently have a proof of concept cluster handling about 12000 >>> msgs/sec. If a certain project kicks off, I would need to scale 10+ >>> times. Is anyone successfully running ES at over 150k/sec? What kind of >>> shard layout are you using, how many data nodes, etc? >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "elasticsearch" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/f6c889f0-5288-4aef-b2c8-007b9272b94a%40googlegroups.com<https://groups.google.com/d/msgid/elasticsearch/f6c889f0-5288-4aef-b2c8-007b9272b94a%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1f430f4b-369f-4db7-bb00-669455ca63f3%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
