I have only one GELF input on each node and no extractors. I forgot to mention that logstash is also running on each node, it receives all the message, does some filtering then sends the messages to graylog via gelf. I've just disabled all the logstash filters but still have the performance issue The thing is that everything was working fine with graylog 1.3 and the exact same setup, the performance problems started just after upgrading to 2.0
Thanks Le vendredi 22 avril 2016 16:47:22 UTC+2, Jochen Schalanda a écrit : > > Hi, > > what kind of inputs are you running and are you using any extractors or > the message processing pipeline? > > A full process buffer usually hints to slow/complex message filtering or > extractors. > > Cheers, > Jochen > > On Friday, 22 April 2016 16:29:41 UTC+2, oliv....@gmail.com wrote: >> >> Hi, >> I have a problem with my output rate which is very low. My input rate is >> not very high (200-400 msg/sec) but the output gets stuck at 100msg/sec >> max. As you can see in the following screenshot, the process and output >> buffers are at 100% and the journal is pretty full. >> >> >> Here is my hardware setup : >> >> - 2 graylog servers. Both nodes have 4g of ram and 2vcpu, I've tried >> increasing it to 4vcpu and 8g of ram but ended up with the same problem >> >> - 3 elasticsearch nodes with each 1vcpu and 2 g of ram, >> >> I have never experienced performance issues with graylog 1.x and the same >> input rate >> >> >> In the logs I can see errors like the following >> ERROR [KafkaJournal] Read offset 91635221 before start of log at >> 91753414, starting to read from the beginning of the journal. >> >> >> Thanks ! >> > -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/a1a04789-b326-4a55-9e89-3e8aaf350952%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
