I managed to get a working config but I want to understand what's going on:
[syslog-decoder] type = "MultiDecoder" subs = ['nginx-access-decoder', 'rsyslog-decoder'] cascade_strategy = "first-wins" log_sub_errors = false In the nginx-access I'm deconding the corresponding access.log entries from my rsyslog and in the rsyslog decoder I'm capturing any other rsyslog entries and discarding them. That works well but in my first attempt I tried with the config extracted from the documentation: [shipped-nginx-decoder] type = "MultiDecoder" subs = ['ProtobufDecoder', 'nginx-access-decoder'] cascade_strategy = "all" log_sub_errors = true [ProtobufDecoder] I would rather this config than the previous one, because it can log me the errors of my nginx decoding. The problem is that when using the ProtobufDecoder the speed of decoding is really slow, and my nginx logs doesn't keep up with the current events in time, and it's always behind the current time. This doesn't happen with the rsyslog-decoder config, it parses the logs really fast. I thought it will be much faster using the internal ProtobufDecoder than a lua one but it's not the case. What's the reason for this? On Fri, Jan 30, 2015 at 11:31 AM, Victor Castell <[email protected]> wrote: > Didn't know of that! Life saver > > Thanks! > El 30/1/2015 11:17, "Krzysztof Krzyżaniak" <[email protected]> > escribió: > > W dni pią 30 sty, 2015 o 10∶34 użytkownik Victor Castell < >> [email protected]> napisał: >> >> Hi! >> >> I have a centralized rsyslog formatted logfile and I'm extracting nginx >> logs from there using heka and the nginx access log decoder. >> >> The problem is that the parser also logs every other log message out to >> heka.log. >> >> The volume of non nginx logs mixed in my rsyslog log is really huge so >> heka.log file is growing like crazy (I have logrotating before you ask) >> >> Is there a way to conditionally/intentionally suppress the parsing errors >> of a given decoder? >> >> >> You probably want to use MultiDecoder which split nginx logs from the >> rest and use log_sub_errors = false in MultiDecoder section. >> >> eloy >> > -- V
_______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
