On Thu, Jul 9, 2015 at 1:35 PM Ali <[email protected]> wrote: > Hi, all. > > So, I finally got a custom Lua decoder to parse events coming in via > TcpInput. On the other end, it's just nxlog watching a bunch of log files > and then sending the new events to Heka via TCP. The log files are all > named SystemOut.log and are located throughout a directory tree, one file > per server. > > Once the data gets to Heka, Heka sends it to Elasticsearch and I am > viewing the results in Kibana. > > I started examining the data and found that I had no way of determining > which SystemOut.log file a given event came from. :-) I have been playing > with nxlog, rewriting the message on that end and then letting Heka parse > out the SystemOut.log path that I'm tacking onto the end of the message. > That seemed really wrong, though. I thought, if I'm doing this -- > modifying the data at all -- I should really be doing it in a structured > way, like using nxlog's JSON output and then adding an attribute for > "filepath" or something. > > Which begs the question, what is the right way to handle this on the Heka > side?? As I said, I already wrote the Lua decoder that parses out the > payload and the funny timestamp so I'm thinking I'll still have to use it. > But should I use that to parse out the JSON? It doesn't look like the Lua > standard library has JSON parsing. Should I use one decoder to parse the > JSON and the custom Lua decoder to parse out the timestamp? Seems like > this would be simple to do in a Go plug-in. > > Not sure if this addresses everything, but cjson is provided by the default sandbox build. Just require it: https://gist.github.com/davidbirdsong/593fbccb1bd1f1ab8716
> TIA, > Ali > _______________________________________________ > Heka mailing list > [email protected] > https://mail.mozilla.org/listinfo/heka >
_______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
