We are getting ~15K/s with 12 data + 3 master nodes, latest version of java and ES.
Some things to try would be; - Java 8 - G1GC - SSD storage - tweaking various index and pool caches - optimising your data inputs, mappings etc We're trialling the first two of those on our dev cluster, but it doesn't do much traffic so I cannot empirically comment on it's capabilities at the levels you're after as yet. What does your current setup (ie infrastructure) look like now? Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: [email protected] web: www.campaignmonitor.com On 17 March 2014 13:17, <[email protected]> wrote: > We've gotten a cluster up to 40K/sec using roughly 40 nodes. We're going > to switch over to using dedicated master nodes as well. This is all in AWS. > > -Justin > > On Sunday, March 16, 2014 9:14:43 PM UTC-4, Janet Sullivan wrote: >> >> I currently have a proof of concept cluster handling about 12000 >> msgs/sec. If a certain project kicks off, I would need to scale 10+ >> times. Is anyone successfully running ES at over 150k/sec? What kind of >> shard layout are you using, how many data nodes, etc? >> > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/f6c889f0-5288-4aef-b2c8-007b9272b94a%40googlegroups.com<https://groups.google.com/d/msgid/elasticsearch/f6c889f0-5288-4aef-b2c8-007b9272b94a%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624Z4iTt0cSKZG0ton9tZLuhBWP9WDBj6eJ7Xi%2B1jonniDg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
