Petar, we are running on bare metal, with a low load. Tested to 10k messages with the http test input, with everything on one (test)server and running well.
I can tell you that in our production systems in our private/local cloud we are encountering severe network/disk related problems with our systems. All of your network is CPU bound. Sometimes there are delays that we can count in seconds. All VM Hosts is running @75% CPU. Must say that we had problems with sending windows eventlogs thru UDP/GELF with nxlog, those were gone when switching tot TCP. Have you already done some graylog2 performance tweaks already? Op woensdag 28 januari 2015 14:41:43 UTC+1 schreef Petar Koraca: > > Thanks Arie. I have already tried that yesterday and did not help. > > I have removed unnecessary TCP input, and I don't have any NettyTransport > exceptions now. > > I still have problem with RecvQ in peaks (as seen in netstat) which should > be related to slow processing. > > Do you have any benchmark data with bare-metal vs VM, and different > processors numbers / ring_size in graylog2.conf ? > > > On Wed, Jan 28, 2015 at 11:49 AM, Arie <[email protected] <javascript:>> > wrote: > >> Maybe this can be helpfull to you: >> >> >> https://access.redhat.com/documentation/en-US/JBoss_Enterprise_Web_Platform/5/html/Administration_And_Configuration_Guide/jgroups-perf-udpbuffer.html >> >> or this for more advanced network tuning: >> https://wwwx.cs.unc.edu/~sparkst/howto/network_tuning.php >> >> hth,, >> >> Arie >> >> >> >> On Tuesday, January 27, 2015 at 6:27:39 PM UTC+1, Petar Koraca wrote: >>> >>> Hello, >>> >>> I have some performance issues with graylog2-server 0.92.4 (cannot >>> process more than 7-8k per second), and I think it may be related to UDP >>> buffers. This is CentOS 6 virtual machine with 16 vCPU. >>> >>> $ netstat -ulptn|grep 12201 >>> tcp 0 0 :::12201 :::* >>> LISTEN 2311/java >>> udp 75960 0 :::12201 :::* >>> 2311/java >>> >>> I've noticed this in my logs: >>> >>> 16:31:06,719 WARN [NettyTransport] receiveBufferSize (SO_RCVBUF) for >>> [id: 0x537c78d9, /0:0:0:0:0:0:0:0:12201] should be 1048576 but is 43690. >>> >>> I've set udp_recvbuffer_sizes=1048576 but no luck. >>> >>> Also, I've set net.core.rmem_max from 124928 to 26214400. >>> >>> Any idea where did this 43690 come from? >>> >>> if you need additional information I am at your disposal. >>> >>> Kind regards, >>> >>> Petar Koraca >>> >>> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "graylog2" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/graylog2/SR9sqDyZqrU/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> [email protected] <javascript:>. >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "graylog2" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
