Hi Arkadiy, not all network devices support UDP packets bigger than 8KiB (8192 bytes) and this seems to be the case on your network somewhere. To circumvent this restriction, Graylog or more specifically the GELF format supports chunking which means splitting a large message into multiple UDP packets of a certain size (see https://www.graylog.org/resources/gelf for details on the GELF format). So in order to solve your problem you could collect the logs on the same machine which produces those raw log lines with a log shipper like nxlog or logstash and send them to Graylog via GELF. Alternatively you'll have to switch from UDP to TCP which doesn't suffer this kind of restriction.
Cheers, Jochen On Thursday, 7 May 2015 13:56:25 UTC+2, Arkadiy Shinkarev wrote: > > Hi! > > I'm trying to send messages to raw udp input in Graylog 1.0.2. > The message size is 1k-20k, but Graylog only shown first 8k of message. > > I have configured "recv_buffer_size: 10485760" for input, also > set net.core.rmem_max = 26214400 in sysctl.conf. > > When I run tcpdump, I see that message len is ok (>8k). > When I run strace -e trace=network i see the following: > [pid 10539] recvfrom(365, > "\24\21\0\0\370\370\352\1778q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., > > 8192, 0, NULL, NULL) = 4372 > [pid 10539] recvfrom(365, > "\24\21\0\0\371\370\352\1779q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., > > 8192, 0, NULL, NULL) = 4372 > [pid 10537] <... recvfrom resumed> > "\24\21\0\0O\372\352\177\235q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., > > 8192, 0, NULL, NULL) = 4372 > [pid 10537] recvfrom(365, > "\24\21\0\0X\372\352\177\236q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., > > 8192, 0, NULL, NULL) = 4372 > [pid 10541] recvfrom(365, > "\24\21\0\0\200\372\352\177\240q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., > > 8192, 0, NULL, NULL) = 4372 > [pid 10541] recvfrom(365, > "\24\21\0\0\201\372\352\177\241q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., > > 8192, 0, NULL, NULL) = 4372 > [pid 10534] recvfrom(365, > "\24\21\0\0\302\372\352\177\330q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., > > 8192, 0, NULL, NULL) = 4372 > [pid 10534] recvfrom(365, > "\24\21\0\0\303\372\352\177\331q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., > > 8192, 0, NULL, NULL) = 4372 > [pid 10540] <... recvfrom resumed> > "\24\21\0\0$\373\352\177-r\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., > 8192, 0, NULL, NULL) = 4372 > [pid 10540] <... recvfrom resumed> > "\24\21\0\0&\373\352\177/r\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., > 8192, 0, NULL, NULL) = 4372 > [pid 10535] recvfrom(362, > "\24\21\0\0b\373\352\177jr\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., > 8192, 0, NULL, NULL) = 4372 > [pid 10535] recvfrom(362, > "\24\21\0\0c\373\352\177kr\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., > 8192, 0, NULL, NULL) = 4372 > [pid 10537] recvfrom(362, > "\24\21\0\0\315\375\352\177_s\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., > > 8192, 0, NULL, NULL) = 4372 > [pid 10537] recvfrom(362, > "\24\21\0\0\316\375\352\177`s\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., > > 8192, 0, NULL, NULL) = 4372 > [pid 10538] recvfrom(362, > "\24\21\0\0\v\376\352\177\224s\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., > > 8192, 0, NULL, NULL) = 4372 > [pid 10538] recvfrom(362, > "\24\21\0\0\f\376\352\177\225s\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., > > 8192, 0, NULL, NULL) = 4372 > [pid 10534] <... recvfrom resumed> > "\24\21\0\0\234\376\352\177\30t\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., > > 8192, 0, NULL, NULL) = 4372 > [pid 10534] <... recvfrom resumed> > "\24\21\0\0\237\376\352\177\33t\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., > > 8192, 0, NULL, NULL) = 4372 > > I'm also test GELF UDP input, messages comes from logstash (to logstash > messages comes from UDP input with 32k buffer size), there is no problem - > message looks good, Graylog shows full message. > > Where is the problem? > > Some additional information: > OS: CentOS release 6.5 (Final) > Kernel: 2.6.32-431.29.2.el6.centos.plus.x86_64 > Graylog: 1.0.2 > > 2 graylog-server nodes behind load balancer (LVS) + 2 nodes elasticsearch > cluster. > > Thank you! > -- You received this message because you are subscribed to the Google Groups "graylog2" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
