Jochen, thanks for you reply! As I mentioned in my first post, I can see with tcpdump that packet lenght, that comes to Graylog node is more than 8192 bytes: $ sudo tcpdump -n -i tunl0 port 12500 and udp tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on tunl0, link-type RAW (Raw IP), capture size 65535 bytes 18:22:19.062304 IP 10.218.52.42.56745 > 10.218.50.20.12500: UDP, length 54 18:22:19.079891 IP 10.218.52.42.56745 > 10.218.50.20.12500: UDP, length 81 18:22:19.113119 IP 10.218.52.42.56745 > 10.218.50.20.12500: UDP, length 92 18:22:19.117398 IP 10.218.52.42.56745 > 10.218.50.20.12500: UDP, length 93 18:22:19.121636 IP 10.218.52.42.56745 > 10.218.50.20.12500: UDP, length 81 18:22:19.123707 IP 10.218.52.42.56745 > 10.218.50.20.12500: UDP, length 94 18:22:22.092734 IP 10.218.49.5.61843 > 10.218.50.20.12500: UDP, length 108 18:22:22.093300 IP 10.218.49.5.61843 > 10.218.50.20.12500: UDP, length 70 18:22:22.238882 IP 10.218.49.5.61843 > 10.218.50.20.12500: UDP, length 71 18:22:24.067068 IP 10.218.49.4.59298 > 10.218.50.20.12500: UDP, length 87 18:22:26.148394 IP 10.218.52.42.56745 > 10.218.50.20.12500: UDP, length 85 18:22:27.477703 IP 10.218.49.4.59298 > 10.218.50.20.12500: UDP, length 13642 18:22:31.158020 IP 10.218.52.42.56745 > 10.218.50.20.12500: UDP, length 91 18:22:35.945376 IP 10.218.49.6.63104 > 10.218.50.20.12500: UDP, length 69 18:22:35.945489 IP 10.218.49.6.63104 > 10.218.50.20.12500: UDP, length 91 18:22:37.279499 IP 10.218.49.5.61843 > 10.218.50.20.12500: UDP, length 108
Also, I have another node with Graylog2 0.20, messages from same sources comes to logstash UDP input without problems. So, I don't thinks that it is network related problems. On Thursday, May 7, 2015 at 4:04:26 PM UTC+3, Jochen Schalanda wrote: > > Hi Arkadiy, > > not all network devices support UDP packets bigger than 8KiB (8192 bytes) > and this seems to be the case on your network somewhere. To circumvent this > restriction, Graylog or more specifically the GELF format supports chunking > which means splitting a large message into multiple UDP packets of a > certain size (see https://www.graylog.org/resources/gelf for details on > the GELF format). So in order to solve your problem you could collect the > logs on the same machine which produces those raw log lines with a log > shipper like nxlog or logstash and send them to Graylog via GELF. > Alternatively you'll have to switch from UDP to TCP which doesn't suffer > this kind of restriction. > > Cheers, > Jochen > > On Thursday, 7 May 2015 13:56:25 UTC+2, Arkadiy Shinkarev wrote: >> >> Hi! >> >> I'm trying to send messages to raw udp input in Graylog 1.0.2. >> The message size is 1k-20k, but Graylog only shown first 8k of message. >> >> I have configured "recv_buffer_size: 10485760" for input, also >> set net.core.rmem_max = 26214400 in sysctl.conf. >> >> When I run tcpdump, I see that message len is ok (>8k). >> When I run strace -e trace=network i see the following: >> [pid 10539] recvfrom(365, >> "\24\21\0\0\370\370\352\1778q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., >> >> 8192, 0, NULL, NULL) = 4372 >> [pid 10539] recvfrom(365, >> "\24\21\0\0\371\370\352\1779q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., >> >> 8192, 0, NULL, NULL) = 4372 >> [pid 10537] <... recvfrom resumed> >> "\24\21\0\0O\372\352\177\235q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., >> >> 8192, 0, NULL, NULL) = 4372 >> [pid 10537] recvfrom(365, >> "\24\21\0\0X\372\352\177\236q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., >> >> 8192, 0, NULL, NULL) = 4372 >> [pid 10541] recvfrom(365, >> "\24\21\0\0\200\372\352\177\240q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., >> >> 8192, 0, NULL, NULL) = 4372 >> [pid 10541] recvfrom(365, >> "\24\21\0\0\201\372\352\177\241q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., >> >> 8192, 0, NULL, NULL) = 4372 >> [pid 10534] recvfrom(365, >> "\24\21\0\0\302\372\352\177\330q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., >> >> 8192, 0, NULL, NULL) = 4372 >> [pid 10534] recvfrom(365, >> "\24\21\0\0\303\372\352\177\331q\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., >> >> 8192, 0, NULL, NULL) = 4372 >> [pid 10540] <... recvfrom resumed> >> "\24\21\0\0$\373\352\177-r\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., >> 8192, 0, NULL, NULL) = 4372 >> [pid 10540] <... recvfrom resumed> >> "\24\21\0\0&\373\352\177/r\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., >> 8192, 0, NULL, NULL) = 4372 >> [pid 10535] recvfrom(362, >> "\24\21\0\0b\373\352\177jr\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., >> 8192, 0, NULL, NULL) = 4372 >> [pid 10535] recvfrom(362, >> "\24\21\0\0c\373\352\177kr\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., >> 8192, 0, NULL, NULL) = 4372 >> [pid 10537] recvfrom(362, >> "\24\21\0\0\315\375\352\177_s\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., >> >> 8192, 0, NULL, NULL) = 4372 >> [pid 10537] recvfrom(362, >> "\24\21\0\0\316\375\352\177`s\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., >> >> 8192, 0, NULL, NULL) = 4372 >> [pid 10538] recvfrom(362, >> "\24\21\0\0\v\376\352\177\224s\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., >> >> 8192, 0, NULL, NULL) = 4372 >> [pid 10538] recvfrom(362, >> "\24\21\0\0\f\376\352\177\225s\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., >> >> 8192, 0, NULL, NULL) = 4372 >> [pid 10534] <... recvfrom resumed> >> "\24\21\0\0\234\376\352\177\30t\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., >> >> 8192, 0, NULL, NULL) = 4372 >> [pid 10534] <... recvfrom resumed> >> "\24\21\0\0\237\376\352\177\33t\1\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., >> >> 8192, 0, NULL, NULL) = 4372 >> >> I'm also test GELF UDP input, messages comes from logstash (to logstash >> messages comes from UDP input with 32k buffer size), there is no problem - >> message looks good, Graylog shows full message. >> >> Where is the problem? >> >> Some additional information: >> OS: CentOS release 6.5 (Final) >> Kernel: 2.6.32-431.29.2.el6.centos.plus.x86_64 >> Graylog: 1.0.2 >> >> 2 graylog-server nodes behind load balancer (LVS) + 2 nodes elasticsearch >> cluster. >> >> Thank you! >> > -- You received this message because you are subscribed to the Google Groups "graylog2" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
