Currently, the log volume is about 1200 logs per second. I have an ES cluster setup with two data storage nodes and an ES node that functions as master only.. The master only node is installed on the same box as graylog, graylog web, and mongodb. Adding the additional 2 vCPU's took care of the problem for now but we plan to add logging for another 200+ devices so am not sure where that leaves us. The current master node has 4 vCPU's, which are XEON 2.7Ghz procs and 12 GB of RAM allocated to it. CPU load is -- load average: 1.92, 2.08, 1.96 and memory usage is -- Mem: 12198260k total, 9416592k used, 2781668k free, 258876k buffers. I changed the HEAP size to -Xms4g -Xmx4g from the original setting of 1g.
On Saturday, May 9, 2015 at 5:42:25 AM UTC-5, Matt Maloney wrote: > > How many messages are you ingesting per second? Zi's recommendation is > right, you should consider running ES separately from Graylog. Moreover, > you will want a load balancer in front of the Graylog servers. I imagine > with the firewall logs being added into the mix, things have gotten a lot > chattier hence the load issue. > > On Friday, May 8, 2015 at 3:31:27 AM UTC+2, BKeep wrote: >> >> We just setup our ASA to send logs into graylog and It appears the >> graylog setup is not keeping up with the incoming log data. The server >> currently has 8 GB of RAM and 2vCPU's. Up until the addition of the ASA, >> that setup has been keeping up pretty well. However, it looks like i am >> going to have to make some adjustments. Prior to adding the ASA logs we >> were logging for about 130 servers. My question is what would be the next >> steps? Obviously add some Memory resources and another couple of vCPU's but >> once the ram is added are there any other adjustments to be made on the >> server itself? >> >> Thanks in advance.. >> > -- You received this message because you are subscribed to the Google Groups "graylog2" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
