Our graylog server receives logging from a great many different devices. One manager, who handles one particularly chatty device, wanted a stream created so he could setup alerts and such. The other day I noticed that Graylog had stopped that stream with a message saying it stopped the stream because it was taking too long to handle.
I find this odd, because the only rule on the stream is that the source field match a particular string. No regexp, no parsing the message field, nothing... So, I unpause the stream, and add another rule to exclude 97% of the messages sent (because this manager insists on running his gear with logging set to level=DEBUG...)... In the 'Streams' display, the stream is showing as active and receiving 20-30 messages/second. However, when I actually click on the name of the stream to see the messages, I get nothing. Nada. Zilch. I waited a while and tried again, still nothing. I checked the System/Nodes page, and see that there is only a 50-100 message backlog in the processing queue, so messages on this stream should have been processed well before the time I checked. I've spent over two hours now trying to figure out why this stream no longer works. I deleted the rule I'd added, and even simplified the source test, to no avail. The GUI continues to say the stream is receiving messages, yet doing a search on that stream (by clicking its name in the Streams list) displays nothing, and no errors appear in the logs. (Other active streams display their expected contents!) I've used the manual load a test message tab in the add-rules page to specify a message I want selected, and it's green as I expected. I selected a message I wanted excluded, and it was excluded, as expected. So I'm certain the problem isn't in the rules. I've browsed through server.log and application.log and can't see any errors there related to the definition and use of this stream. Does anyone have any suggestions how to debug this? I'm at wit's end by now. Bueller? Bueller? -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/770f86b9-0f20-47d2-bad3-8368374d5038%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
