Alright I got it… my “logtime” table below was causing formatting issues. Once I removed that from the capture then it started working.
Thanks for looking into this! A great learning experience for me. - Justin On 10/22/15, 3:11 PM, "Heka on behalf of Vogt, Justin" <[email protected] on behalf of [email protected]> wrote: >Now I’m not quite sure what I’m missing… I keep getting this error: > >inject_message() could not encode protobuf - unsupported type: nil > >I’m trying the bare minimum of msg here: > >local l = require "lpeg" >local dt = require "date_time" >local sp = l.space > >l.locale(l) > >local pri = l.P"<" * l.Cg(l.R"09"^0, "pri") * l.P">" >local logtime = l.Cg(dt.build_strftime_grammar("%b %d %X"), "logtime") * sp >local hostname = l.Cg((1 - sp)^1, "hostname") * sp >local logname = l.Cg((1 - sp)^1, "logname") * sp >local message = l.Cg(l.P(1)^0, "message") >msg = pri * logtime * hostname * logname * message > >stack = l.Ct(msg) > >local msg_type = read_config("type") > >local msg = { >Type = msg_type, >Payload = nil, >Fields = nil >} > >function process_message () > local log = read_message("Payload") > local fields = stack:match(log) > > if not fields then return -1 end > > msg.Payload = log > msg.Fields = fields > > inject_message(msg) > > return 0 >end > > >- Justin > >From: Michael Trinkala >Date: Thursday, October 22, 2015 at 2:47 PM >To: Justin Vogt >Cc: "[email protected]<mailto:[email protected]>", heka >Subject: Re: [heka] Lua Decoder Parse Error > >The code is failing on the inject_message line. The message Fields table must >follow this: >https://hekad.readthedocs.org/en/latest/sandbox/index.html#lua-message-hash-based-field-structure > schema. Also, you could trap the error from the pcall and return it with the >status code to avoid some of the confusion. > >Trink > >On Thu, Oct 22, 2015 at 11:59 AM, Vogt, Justin ><[email protected]<mailto:[email protected]>> wrote: >Here’s the whole lua file: > >local l = require "lpeg" >local dt = require "date_time" >local sp = l.space > >l.locale(l) > >local pri = l.P"<" * l.Cg(l.R"09"^0, "pri") * l.P">" >local logtime = l.Cg(dt.build_strftime_grammar("%b %d %X"), "logtime") * sp >local hostname = l.Cg((1 - sp)^1, "hostname") * sp >local logname = l.Cg((1 - sp)^1, "logname") * sp >local message = l.Cg(l.P(1)^0, "message") >msg = pri * logtime * hostname * logname * message > >stack = l.Ct(msg) > >local msg_type = read_config("type") > >local msg = { > Timestamp = nil, > Type = msg_type, > Hostname = nil, > Payload = nil, > Fields = nil >} > >function process_message () > local log = read_message("Payload") > local flds = stack:match(log) > > if not flds then return -1 end > > if flds.hostname then > msg.Hostname = flds.hostname > flds.hostname = nil > end > > msg.Payload = log > msg.Fields = flds > > if not pcall(inject_message, msg) then return -1 end > return 0 >end > >I tried a few more things, but still seems to be failing. I’ll take a look at >the rsyslog decoder, but I’m trying to learn how to do, rather than borrow :-) > >- Justin > >From: Heka on behalf of Michael Trinkala >Date: Thursday, October 22, 2015 at 11:37 AM >To: >"[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>" >Cc: heka >Subject: Re: [heka] Lua Decoder Parse Error > >It is probably not the grammar failing, I am betting you return -1 somewhere >else in the decoder. Can you share the code? > >Trink > >On Wed, Oct 21, 2015 at 11:23 AM, Justin Vogt ><[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>> > wrote: > >Hello Heka Community, > >I’m getting a weird error… I’ve written a custom decoder in Lua, and it works >find when I test it on the LPEG grammar tester, but when I try to run it in >Heka, I continually get a “Decoder error, failed parsing” message. I’ve tried >just about everything I can think of and have been pulling my hair out to >figure out the issue. Any help would be greatly appreciated!! > > >Here's the grammar > >local l = require "lpeg" > >local dt = require "date_time" > >local sp = l.space > >l.locale(l) > >local pri = l.P"<" * lpeg.Cg(lpeg.R"09"^0, "pri") * lpeg.P">" > >local logtime = l.Cg(dt.build_strftime_grammar("%b %d %X"), "logtime") * sp > >local hostname = l.Cg((1 - sp)^1, "hostname") * sp > >local logname = l.Cg((1 - sp)^1, "logname") * sp > >local message = l.Cg(l.P(1)^0, "message") > >msg = pri * logtime * hostname * logname * message > >grammar = l.Ct(msg) > > >And a sample log line: > ><14>Oct 16 02:26:17 node-85 keystone-all 192.168.0.2 - - [16/Oct/2015 >02:26:17] "GET /v3/auth/tokens HTTP/1.1" 200 10162 0.125928 > > >I can literally copy and paste the grammar out of the decoder and the log line >from the failed parse message into the LPEG tester and it works. > > >Thanks in advance! > >_______________________________________________ >Heka mailing list >[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>> >https://mail.mozilla.org/listinfo/heka > > >_______________________________________________ >Heka mailing list >[email protected]<mailto:[email protected]> >https://mail.mozilla.org/listinfo/heka > >_______________________________________________ >Heka mailing list >[email protected] >https://mail.mozilla.org/listinfo/heka _______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
