Peter, it's a bit difficult to tell what's going on without seeing the rest of the syslog-ng configuration and your script's source code.
However, a couple possibilities come to mind: - Your script is only reading one line at a time. syslog-ng starts a program() output persistently and expects that it can send multiple messages into its pipe to your script's stdin. - Messages are being buffered inside of syslog-ng. Check out the flush_lines() and flush_timeout() flags to syslog-ng's program() output. Find the right page for your version, but here's v3.3.: http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-3.3-guides/en/syslog-ng-ose-v3.3-guide-admin-en/html/reference_destination_program.html - Messages are being buffered in your shell or script. Maybe try some non-blocking IO with a smallish buffer to see data as it comes in before a whole line or block fills and flushes in. To Anurag's question about open source log management with a WebUI, I agree with Blake: logstash ingesting syslog and inputting it into elasticsearch makes for a great backend for Kibana. The logstash grok filter is great for pulling apart and indexing weird vendor-specific logging formats: http://logstash.net/docs/1.4.1/filters/grok Cheers, jof On Sat, May 10, 2014 at 2:24 AM, Peter Persson <web...@webbax.se> wrote: > Hey, > > I got a weird problem with my syslog-ng setup, im logging from alot of > cisco machines and that works great. > The problem is that when i "pass" this further to a shell program, some > lines disapere. > > My destination looks like this > destination hosts { > file("/var/log/ciscorouters/$HOST.log" > owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes)); > program("/scripts/irc/syslog_wrapper_new.sh" template(t_irctempl)); > }; > The "/var/log/ciscorouters/$HOST.log" writes correct, but the data thats > putted trough to "/scripts/irc/syslog_wrapper_new.sh" only get the first > line, if it gets flooded (like 5 rows per second). > > Do anyone of you have any idea of what might be the problem? > > Regards, > Peter