With 4 cpu's on your ES server your bulk threads will max out at 4, with a 1000 message output batch size that would set your max to essentially 4k msg/s. I would increase the output batch size and add cpu on the ES vm. Also you didn't mention what your input config is in graylog, are you sure it isn't doing any dns lookups for the incoming messages? That will kill throughput in that situation as well.
On Thursday, November 3, 2016 at 6:48:16 AM UTC-4, Jerri Son wrote: > > I´d like to add 2 more Graphs we have from our ES host (upgrade took place > on wednesday and was reverted thursday): > http://imgur.com/euZOPMq -> TCP MIB, where you can clearly see a lot less > established session with the upgraded setup > > and > > http://imgur.com/qPnR69Y -> Traffic took a serious drop which might > indicate that GL 2.1.1 just can´t keep up sending messages in the > same volume that GL 1.3.4 can (maybe). > > -- CONFIDENTIALITY/EMAIL NOTICE: The material in this transmission contains confidential and privileged information intended only for the addressee. If you are not the intended recipient, please be advised that you have received this material in error and that any forwarding, copying, printing, distribution, use or disclosure of the material is strictly prohibited. If you have received this material in error, please (i) do not read it, (ii) reply to the sender that you received the message in error, and (iii) erase or destroy the material. Emails are not secure and can be intercepted, amended, lost or destroyed, or contain viruses. You are deemed to have accepted these risks if you communicate with us by email. Thank you. -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/261f5b00-c8a5-4432-a6da-5f7079ca9dea%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
