Hello Timur,
I'm not sure why you're calling read_message("raw") in your decoder.
Usually a Lua decoder calls read_message("Payload") which has been filled
by the input plugin, processes what's in there to set fields in the Heka
message's table and finally calls the inject_message() function.
IIRC read_message("raw") returns the Protobuf encoding of the Heka message
but the encoding is done after the decoder has done its job. In your case,
read_message() probably returns nil which is why the decode_message()
function complains.
HTH
SimonOn Mon, Oct 12, 2015 at 4:17 PM, Timur Batyrshin <[email protected]> wrote: > Hi, > > I’ve been writing a decoder for myself and have hit the following issue > which I can’t understand. > > When I start Heka it produces the following error message in logs: > > 2015/10/12 13:54:47 SubDecoder > ‘zerogw-zerogw_decoder-stdout-zerogw_rotate_fields’ error: FATAL: > process_message() /usr/share/heka/lua_decoders/rotate_fields.lua:30: bad > argument #0 to ‘decode_message’ (must have one string argument) > > At the same time the code for decoder is the following: > > -- the only lines above are comments which are skipped > metric_field = read_config("metric_field") or "metric" > value_field = read_config("value_field") or "value" > > function process_message() > local fields = {} > > raw = read_message("raw”) # line 29 > msg = decode_message(raw) # line 30 > > -- other part of code is probably irrelevant as crash is seen in the above > line > > (I’ve tried writing that as decode_message(read_message("raw”)) with the > same effect) > > What’s really weird is the exactly the same decoder works fine on other > hosts. > > I’m using the following Heka config: > > [zerogw] > type = "ProcessInput" > ticker_interval = 0 > splitter = "on_newline" > decoder = "zerogw_decoder" > stdout = true > stderr = false > > [zerogw.command.0] > bin = "/usr/local/bin/zerogw_collector.py" > args = ["-s", "tcp://127.0.0.1:5111"] > > [on_newline] > type = "TokenSplitter" > delimiter = "\n" > > [estp_decoder] > type = "PayloadRegexDecoder" > match_regex = '^(?P<Name>[^\s]+) (?P<Timestamp>\d+) (?P<Value>\d+)' > timestamp_layout = "Epoch" > > [estp_decoder.message_fields] > Service = "Zerogw" > Metric = "%Name%" > Value = "%Value%" > > [zerogw_decoder] > type = "MultiDecoder" > subs = ["estp_decoder", "zerogw_rotate_fields"] > cascade_strategy = "all" > > [zerogw_rotate_fields] > type = "SandboxDecoder" > filename = "lua_decoders/rotate_fields.lua" > > [zerogw_rotate_fields.config] > metric_field = "Metric" > value_field = "Value" > > zerogw_collector.py produces about a dozen of lines to stdout every 5 > seconds in the format as seen in message payload (see below). > > As MultiDecoder has cascade_strategy = "all" Heka dumps messages > processed by the first decoder in the chain to stdout which are the > following: > > 2015/10/12 14:04:22 > :Timestamp: 2015-10-12 14:04:22 +0000 UTC > :Type: ProcessInput > :Hostname: t-eu-zgw > :Pid: 5212 > :Uuid: 2c7deb23-7961-49dc-8f57-da716d851439 > :Logger: zerogw > :Payload: zerogw.connections.total 1444658662 4 > > :EnvVersion: > :Severity: 7 > :Fields: > | name:"ProcessInputName" type:string value:"zerogw.stdout" > | name:"ExitStatus" type:integer value:0 > | name:"Value" type:string value:"4" > | name:"Service" type:string value:"Zerogw" > | name:"Metric" type:string value:"zerogw.connections.total" > > In plain Lua I’d dump the result of read_message("raw") to stdout, add > some prints everywhere and see what happens inside but don’t know how to do. > > Any clues on how I should debug such cases? > > Thanks, > Timur > > > _______________________________________________ > Heka mailing list > [email protected] > https://mail.mozilla.org/listinfo/heka > >
_______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
