Here are some examples (I copy/pasted from graylog2 and sanitized the IP address):
2014-01-03 19:30:31.430 65.xx.xx.xx <38> Jan 3 14:30:30 managedapi-qa-node-2 sshd 10593 - - Failed password for invalid user magic from 196.200.176.66 port 57342 ssh2 2014-01-03 19:30:31.428 65.xx.xx.xx <38> Jan 3 14:30:30 managedapi-qa-node-2 sshd 10593 - - Failed password for invalid user magic from 196.200.176.66 port 57342 ssh2 2014-01-03 19:30:29.820 65.xx.xx.xx <84> Jan 3 14:30:28 node1-qa-api sshd 22245 - - pam_unix(sshd:auth): check pass; user unknown 2014-01-03 19:30:29.815 65.xx.xx.xx <84> Jan 3 14:30:28 node1-qa-api sshd 22245 - - pam_unix(sshd:auth): check pass; user unknown 2014-01-03 19:30:29.778 65.xx.xx.xx <84> Jan 3 14:30:28 managedapi-qa-node-2 sshd 10593 - - pam_unix(sshd:auth): check pass; user unknown 2014-01-03 19:30:29.775 65.xx.xx.xx <84> Jan 3 14:30:28 managedapi-qa-node-2 sshd 10593 - - pam_unix(sshd:auth): check pass; user unknown 2014-01-03 19:30:29.631 65.xx.xx.xx <38> Jan 3 14:30:28 node1-qa-api sshd 22245 - - Failed password for invalid user magic from 196.200.176.66 port 42391 ssh2 2014-01-03 19:30:29.627 65.xx.xx.xx <38> Jan 3 14:30:28 node1-qa-api sshd 22245 - - Failed password for invalid user magic from 196.200.176.66 port 42391 ssh2 2014-01-03 19:30:29.613 65.xx.xx.xx <38> Jan 3 14:30:28 managedapi-qa-node-2 sshd 10593 - - Failed password for invalid user magic from 196.200.176.66 port 57342 ssh2 2014-01-03 19:30:29.609 65.xx.xx.xx <38> Jan 3 14:30:28 managedapi-qa-node-2 sshd 10593 - - Failed password for invalid user magic from 196.200.176.66 port 57342 ssh2 -- Jeff Schoolcraft On Friday, January 3, 2014 at 12:12 PM, Lennart Koopmann wrote: > Hmmm, I need to see the actual message that is being sent in. Can you > please start your server in normal mode again (not debug)? Then > terminate the syslog input and start a raw/plaintext input on the > exact same port. That should show you the messages in the original > form how they are coming in. > > On Fri, Jan 3, 2014 at 5:29 PM, Jeff Schoolcraft > <[email protected] (mailto:[email protected])> wrote: > > No worries, I appreciate the help. > > > > Here's what I get from running with --debug: > > > > 2014-01-03 16:15:02,755 DEBUG: org.graylog2.inputs.syslog.SyslogProcessor - > > Skipping incomplete message. > > 2014-01-03 16:15:02,756 DEBUG: org.graylog2.inputs.syslog.SyslogProcessor - > > Skipping incomplete message. > > 2014-01-03 16:15:02,756 DEBUG: org.graylog2.inputs.syslog.SyslogProcessor - > > Skipping incomplete message. > > 2014-01-03 16:15:02,757 DEBUG: org.graylog2.inputs.syslog.SyslogProcessor - > > Skipping incomplete message. > > 2014-01-03 16:15:02,757 DEBUG: org.graylog2.inputs.syslog.SyslogProcessor - > > Skipping incomplete message. > > 2014-01-03 16:15:02,758 DEBUG: org.graylog2.inputs.syslog.SyslogProcessor - > > Skipping incomplete message. > > 2014-01-03 16:15:02,759 DEBUG: org.graylog2.inputs.syslog.SyslogProcessor - > > Skipping incomplete message. > > 2014-01-03 16:15:02,759 DEBUG: org.graylog2.inputs.syslog.SyslogProcessor - > > Skipping incomplete message. > > 2014-01-03 16:15:02,760 DEBUG: org.graylog2.inputs.syslog.SyslogProcessor - > > Skipping incomplete message. > > > > > > -- > > Jeff Schoolcraft > > > > On Friday, January 3, 2014 at 10:41 AM, Lennart Koopmann wrote: > > > > Argh, sorry: That log level changing is not included in preview.8 yet > > AFAIR. It will be included in preview.9. The logs would have appeared > > in your local logfile on disk and not in Graylog2. Graylog2 is not > > logging into itself for several reasons. ;) > > > > Please start graylog2-server like this: java -jar graylog2-server.jar > > --debug > > > > That should print debug messages to STDOUT. > > > > On Fri, Jan 3, 2014 at 4:37 PM, Jeff Schoolcraft > > <[email protected] (mailto:[email protected])> > > wrote: > > > > There isn't a script in between… > > > > I don't see any log currently from graylog2-server. > > > > In both chrome and safari I can't switch the log level of any of the > > subsystems to anything. I try to switch to Debug and there's no update. > > > > -- > > Jeff Schoolcraft > > > > On Friday, January 3, 2014 at 10:27 AM, Lennart Koopmann wrote: > > > > That should usually work. What is the script in between doing? > > > > I suggest you lower the log level for a moment and see why messages > > are rejected. Go to "System" -> "Logging" and set the log level of the > > Graylog2 subsystem to DEBUG. You should now see a way more verbose > > output in your graylog2-server log file. > > > > On Fri, Jan 3, 2014 at 4:09 PM, Jeff Schoolcraft > > <[email protected] (mailto:[email protected])> > > wrote: > > > > So, I turned off the RAW input and turned on the syslog input, adding this > > to the end of my /etc/rsyslog.conf file: > > > > *.* @graylog_server_ip:514 > > > > I'm on ubuntu (Linux node1-qa-api 3.5.0-44-generic #67-Ubuntu SMP Tue Nov 12 > > 19:36:14 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux) and just forwarding > > messages along without some intermediate script... > > > > And I'm still not getting messages. Any ideas? > > > > > > org.graylog2.inputs.syslog.udp.SyslogUDPInput.250e3b5e-34e1-4c3a-b992-fea3132ac74b.incomingMessages > > > > Meter > > > > > > Total:108 eventsMean:0.17 events/sec1 minute avg:0.22 events/sec5 minute > > avg:0.16 events/sec15 minute avg:0.16 events/sec > > > > org.graylog2.inputs.syslog.udp.SyslogUDPInput.250e3b5e-34e1-4c3a-b992-fea3132ac74b.incompleteMessages > > > > Meter > > > > > > Total:108 eventsMean:0.17 events/sec1 minute avg:0.22 events/sec5 minute > > avg:0.16 events/sec15 minute avg:0.16 events/sec > > > > org.graylog2.inputs.syslog.udp.SyslogUDPInput.250e3b5e-34e1-4c3a-b992-fea3132ac74b.parsingFailures > > > > Meter > > > > > > Total:0 eventsMean:0 events/sec1 minute avg:0 events/sec5 minute avg:0 > > events/sec15 minute avg:0 events/sec > > > > org.graylog2.inputs.syslog.udp.SyslogUDPInput.250e3b5e-34e1-4c3a-b992-fea3132ac74b.processedMessages > > > > Meter > > > > > > Total:0 eventsMean:0 events/sec1 minute avg:0 events/sec5 minute avg:0 > > events/sec > > 15 minute avg:0 events/sec > > > > > > -- > > Jeff Schoolcraft > > > > On Thursday, January 2, 2014 at 7:44 PM, Lennart Koopmann wrote: > > > > Great to hear it worked! :) Just ping the mailing list if any more > > questions should arise. > > > > On Fri, Jan 3, 2014 at 1:33 AM, Jeff Schoolcraft > > <[email protected] (mailto:[email protected])> > > wrote: > > > > Hi Lennart, > > > > Thanks, "Raw/Plaintext UDP" listening on 514 did the trick, I see messages! > > > > Now to go from syslog to GELF without throwing away 1/2 GB RAM using > > logstash :) > > > > -- > > Jeff Schoolcraft > > > > On Thursday, January 2, 2014 at 6:09 PM, Lennart Koopmann wrote: > > > > Hey Jeff, > > > > from what I can see you spawned UDP syslog inputs. Those inputs expect > > syslog RFC compliant messages. Do you send such messages from your > > script? All 530 messages you sent were rejected as incomplete > > (non-compliant). > > > > I suggest you try this: Start a "Raw/Plaintext UDP" input and point > > your script to it. Those inputs do not expect any specific format and > > just store any text they get until the first newline delimiter (\n). I > > am pretty sure this will show the messages. They won't have any > > information extracted to fields though. There are several ways to > > address that and the easiest might be: > > > > * Send GELF from your script. It is easy to construct GELF messages > > in your favorite language: http://graylog2.org/gelf#libraries > > * Keep sending raw/plaintext messages and use the Graylog2 > > extractors to extract data to fields. > > > > Thanks for attaching the metrics. This made debugging this really > > easy. Awesome! :) > > > > Cheers, > > Lennart > > > > On Thu, Jan 2, 2014 at 11:11 PM, Jeff Schoolcraft > > <[email protected] (mailto:[email protected])> > > wrote: > > > > I have a newly installed graylog2 server (The stats from the rsyslog > > listener: graylog2-web-interface v0.20.0-preview.8) and I've set up 2 > > listeners udp gelf and udp rsyslog. > > > > I used a small script to pipe output from an nginx access log to the graylog > > server over udp and it's getting messages but not storing them. > > > > I also switched a running app currently logging messages to a v0.11.0 server > > to the new graylog server, it too shows up in stats but isn't storing any > > events. > > > > How can I debug this? > > > > Here are the metrics from the syslog parser. > > > > > > > > org.graylog2.inputs.syslog.udp.SyslogUDPInput.3bd9b418-3441-4bdd-b1aa-125a2ffa04c8.incomingMessages > > > > Meter > > > > > > Total:530 eventsMean:0.09 events/sec1 minute avg:0.1 events/sec5 minute > > avg:0.13 events/sec15 minute avg:0.13 events/sec > > > > org.graylog2.inputs.syslog.udp.SyslogUDPInput.3bd9b418-3441-4bdd-b1aa-125a2ffa04c8.incompleteMessages > > > > Meter > > > > > > Total:530 eventsMean:0.09 events/sec1 minute avg:0.1 events/sec5 minute > > avg:0.13 events/sec15 minute avg:0.13 events/sec > > > > org.graylog2.inputs.syslog.udp.SyslogUDPInput.3bd9b418-3441-4bdd-b1aa-125a2ffa04c8.parsingFailures > > > > Meter > > > > > > Total:0 eventsMean:0 events/sec1 minute avg:0 events/sec5 minute avg:0 > > events/sec15 minute avg:0 events/sec > > > > org.graylog2.inputs.syslog.udp.SyslogUDPInput.3bd9b418-3441-4bdd-b1aa-125a2ffa04c8.processedMessages > > > > Meter > > > > > > Total:0 eventsMean:0 events/sec1 minute avg:0 events/sec5 minute avg:0 > > events/sec15 minute avg:0 events/sec > > > > -- > > Jeff Schoolcraft > > > > -- > > You received this message because you are subscribed to the Google Groups > > "graylog2" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [email protected] > > (mailto:[email protected]). > > For more options, visit https://groups.google.com/groups/opt_out. > > > > > > -- > > You received this message because you are subscribed to the Google Groups > > "graylog2" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [email protected] > > (mailto:[email protected]). > > For more options, visit https://groups.google.com/groups/opt_out. > > > > > > -- > > You received this message because you are subscribed to the Google Groups > > "graylog2" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [email protected] > > (mailto:[email protected]). > > For more options, visit https://groups.google.com/groups/opt_out. > > > > > > -- > > You received this message because you are subscribed to the Google Groups > > "graylog2" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [email protected] > > (mailto:[email protected]). > > For more options, visit https://groups.google.com/groups/opt_out. > > > > > > -- > > You received this message because you are subscribed to the Google Groups > > "graylog2" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [email protected] > > (mailto:[email protected]). > > For more options, visit https://groups.google.com/groups/opt_out. > > > > > > -- > > You received this message because you are subscribed to the Google Groups > > "graylog2" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [email protected] > > (mailto:[email protected]). > > For more options, visit https://groups.google.com/groups/opt_out. > > > > > > -- > > You received this message because you are subscribed to the Google Groups > > "graylog2" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [email protected] > > (mailto:[email protected]). > > For more options, visit https://groups.google.com/groups/opt_out. > > > > > > -- > > You received this message because you are subscribed to the Google Groups > > "graylog2" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [email protected] > > (mailto:[email protected]). > > For more options, visit https://groups.google.com/groups/opt_out. > > > > > > -- > > You received this message because you are subscribed to the Google Groups > > "graylog2" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [email protected] > > (mailto:[email protected]). > > For more options, visit https://groups.google.com/groups/opt_out. > > > > > -- > You received this message because you are subscribed to the Google Groups > "graylog2" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > (mailto:[email protected]). > For more options, visit https://groups.google.com/groups/opt_out. > > -- You received this message because you are subscribed to the Google Groups "graylog2" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
