Has anyone been able to successfully use the Graylog Extended Log Format input to accept logs? The documentation on RTD states:
> Parses a payload containing JSON in the Graylog2 Extended Format > specficiation. http://graylog2.org/resources/gelf/specification"; The link in the docs take you to the Graylog2 website, where they provide a simple console command to test log ingest: Start an UDP GELF input on port 12200 and send the message over: > echo '{"version": "1.1","host":"example.org","short_message":"A short > message that helps you identify what is going on","full_message":"Backtrace > here\n\nmore > stuff","level":1,"_user_id":9001,"_some_info":"foo","_some_env_var":"bar"}' > | nc -w 1 -u localhost 12200 > Search for “stuff” in the Graylog Web Interface and you should find the > message. This is an uncompressed and non-chuked GELF message. Note that no > timestamp field is set so you will find the message by searching in the > last 5 minutes. My thinking was that by setting up a simple Heka test case I should be able to get this same console command to echo to stdout via Heka. The configuration I used was as follows: [gelf_decoder] > type = "SandboxDecoder" > script_type = "lua" > filename = "lua_decoders/graylog_extended.lua" > [gelf_decoder.config] > type = "gelf" > payload_keep = true > # Udp socket to receive application logs on, as GELF records > [udp:4890] > type = "UdpInput" > address = ":4890" > decoder = "gelf_decoder" > [ESLogstashV0Encoder] [LogOutput] > message_matcher = "TRUE" #Type == 'gelf'" > encoder = "ESLogstashV0Encoder" > payload_only = false Basically accept on UDP:4890, decode as a GELF object, and echo back to stdout in the ES JSON format. However this simple example just throws an error related to the decoder: > 2015/07/22 11:16:50 Decoder 'udp:4890-gelf_decoder' error: Failed > parsing: payload: {"version": "1.1","host":"example.org","short_message":"A > short message that helps you identify what is going > on","full_message":"Backtrace here\n\nmore > stuff","level":1,"_user_id":9001,"_some_info":"foo","_some_env_var":"bar"} > So my question is am I doing something wrong? To the best of my current understanding this should work, but it apparently doesn't. As an additional point, for those who read the GELF specification page, it states that GELF records can be sent uncompressed, or GZIP'd or ZLIB'd, and that GZIP is the protocol default, but neither GZIP or ZLIB appear to be supported as part of the Graylog Extended Log Format lua script rendering most GELF output libraries (such as https://github.com/severb/graypy) initially unusable without later customization. Thanks for your time and consideration. - Alexander
_______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
