I've had to answer this sort of question for our devs in the past, here's what I looked at:
I'd do a packet dump (tcpdump, etc) on the interface and see if the machine is even seeing the syslog packets. If it is, you know it's something on the graylog server/host causing the packet loss. I'd look at any firewall rules that might be interfering, IE an ACL list that is missing entries for some of your expected clients, a forgotten blacklist, etc. If the packets aren't there in the raw packet dump, look at the network. A switch or router may be jettisoning the UDP syslog packets because they're considered lower priority than TCP traffic, or there may be a routing issue. A simpler test is to verify each client can ping the graylog server interfaces, sometimes clients have routes missing. For the client logs that are showing up in graylog, are they consistently present from a subset of your clients? My server has multiple interfaces, use 'sudo netstat -nlp' or equivalent to verify the graylog server is actually listening on all of the expected interfaces. Are all your machines using IPv4 only, or do you have IPv6 in the mix? I've seen Java apps misbehave when the JVM decided it would grok only IPv6 and ignore IPv4. There's a JVM command-line option to address this, " -Djava.net.preferIPv4Stack=true" but this probably isn't universal. On Tue, Dec 1, 2015 at 11:18 AM, HockeyFan0000 <[email protected]> wrote: > One correction. In my last post, I stated syslog was collecting 55k > messages per minute. That is incorrect. I had collected 55k in the > previous 8hrs. Now, the problem seems even worse. I've only collected > 3300 messages in the past 8hrs with no changes being made and still no sign > of any problems that I can find. Any help is greatly appreciated. > > -- > You received this message because you are subscribed to the Google Groups > "Graylog Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/graylog2/16d868b4-1087-485b-a8ee-c76994a1cc29%40googlegroups.com > <https://groups.google.com/d/msgid/graylog2/16d868b4-1087-485b-a8ee-c76994a1cc29%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- No matter what we think of Linux versus FreeBSD, etc., the one thing I really like about Linux is that it has Microsoft worried. Anything that kicks a monopoly in the pants has got to be good for something. - Chris Johnson -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/CAL5rfGUzY4%2B0aJXdJGtKtT_FjdbuT_xEmXZnzH8Y3JGG%3DDbWXQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
