I would add another vCPU core or two and some more RAM. Also, have you
adjusted your JVM HEAP size for Elasticsearch and Graylog? I have three
nodes setup for our logging cluster. The master node is Graylog, Mongod,
and Elasticsearch setup as master only with a JVM HEAP of 4g for Graylog
and 4g for Elasticsearch. The other two nodes are data storage only.
Each of those has 4vCPU's and 12GB of RAM where Elasticsearch has HEAP
of 6g. That configuration allows us max throughput of about 2500 logs
per second give or take. I should also note these are VMware VM's
running on B200 blade hardware (2.7Ghz Xeon cores, SAN storage, etc.)
On 06/03/2015 03:21 AM, Matt Hines wrote:
Hey Henrik,
I'm not. That was the first thing that I found when looking into the
issue. We have 1 input, a syslog UDP, but the force_rdns is unticked.
Thanks,
On Tuesday, 2 June 2015 18:33:14 UTC+1, Henrik Johansen wrote:
Hello Matt,
Are you by any chance running inputs that have the force_rdns flag
set?
---
HenrikJ
On 2. jun. 2015 kl. 17.37.16 CEST, Matt Hines <[email protected]
<javascript:>> wrote:
Hey guys,
We've just updated our Graylog setup by moving the ES to another
box to give it space to breathe.
But now we're seeing a large increase in msg/s coming into the
graylog-server, at peak times, between 600-1200. This amount is
only going to get bigger as users increase.
My issue is, when the number breaches the 600 msgs/s number,
graylog starts to struggle and starts filling up its process
buffer and the CPU ramps up to 100% and stays there until the
peak period is over and it has cleared the buffer out.
The box is an AWS box, 2CPU, 8GB RAM. I have no idea if this
hardware should be enough for this number of messages? If so, is
there some tweaking I can do to optimise the system?
If not, what would you recommend, I know it's mainly CPU based
but would it be best to cluster a second box, or increase this box?
Also, at what point will I start to lose messages incoming? When
the process buffer and journal are both full?
We're running Graylog v1.0.2
Thanks for any help!
--
You received this message because you are subscribed to the
Google Groups "graylog2" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to [email protected] <javascript:>.
For more options, visit https://groups.google.com/d/optout
<https://groups.google.com/d/optout>.
--
You received this message because you are subscribed to the Google
Groups "graylog2" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to [email protected]
<mailto:[email protected]>.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"graylog2" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
