Looks like nxlog will suit my purpose just fine. Thanks for the help!
On Friday, June 6, 2014 6:06:02 AM UTC-4, Arie wrote: > > Hi Stephen, > > I do hop I'll get you right. > Take a look at nxlog, you can send with it in a CSV format, and give names > to your fields to. > > See: http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html#xm_xml >> > > *Example 6.12. Parsing a CSV file and sending it to Graylog2 in GELF* > On Thursday, June 5, 2014 9:48:21 PM UTC+2, Stephen Hosom wrote: >> >> I'm looking at using Graylog2 for a pretty specific project. It's only >> going to about 10 different types of log files, but I need to get extremely >> specific within those log files. I also do not have the ability to change >> the format of the logs. The logs come as tab delimited and contain a header >> section. >> >> I'm trying to use the CSV to fields converter, but I keep getting the >> feeling that it doesn't do what I think it does. Is there a way to pass >> Graylog2 a bunch of headers and to tell it what fields are what, and then >> have it parse them? >> >> For example, I'd really like to parse the following: >> >> #fields ts uid id.orig_h id.orig_p id.resp_h >> id.resp_p proto service duration orig_bytes resp_bytes >> conn_state local_orig missed_bytes history orig_pkts >> orig_ip_bytes resp_pkts resp_ip_bytes tunnel_parents orig_cc >> resp_cc sensorname >> 1401997562.606254 C17WYF4RAfPEzCDg23 10.246.50.7 137 >> 10.246.50.255 137 udp dns 4.180676 300 0 S0 >> T 0 D 6 468 00 (empty) - - >> so-test-eth0 >> 1401997536.549391 CLaDQnmYfW34xG7Bd 10.246.50.11 123 >> 204.235.61.9 123 udp - 0.046794 0 48 SHR >> T 0 Cd 0 0 176 (empty) - US >> so-test-eth0 >> 1401997550.087390 CavY0m1XCa42ydnBO1 10.246.50.32 68 >> 255.255.255.255 67 udp dhcp - - - S0 T >> 0 D 1 328 0 0(empty) - - >> so-test-eth0 >> 1401997479.316667 C5oU7l4fRLIZXNlaJf 10.246.50.32 57059 >> 239.255.255.250 1900 udp - 74.496845 1596 0 S0 >> T 0 D 12 193200 (empty) - - >> so-test-eth0 >> >> I know that I can manually define out these fields and then craft regular >> expressions for each of them, however, I'm trying to avoid that. >> > -- You received this message because you are subscribed to the Google Groups "graylog2" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
