On Wed, 8 Feb 2012, Vlad Grigorescu wrote:
- - Finally, I'm a bit confused on how rsyslog receives commit errors with the new transactional plugin system. If there's a batch of 5 messages, and only message 4 is successfully committed during endTransaction, how would one convey that information back to rsyslog? I know Radu mentioned calling a program with omprog, and sending messages to ES from there, but in my setup, data integrity is paramount, and I don't want to re-implement rsyslog's reliable method delivery and failover systems.
the approach that rsyslog takes is that it tries to push the entire batch, if that fails it cuts the batch size in half and tries to send that, and if it fails with a batch size of 1 it decides that message is bad.
I don't remember what happens with a bad message (it's either that a rsyslog message is logged about the problem and the message is skipped, or the output plugin will stall forever). I do remember that there is a way to differentiate between a temporary failure (the server is down) and a permanent failure (the message is malformed in such a way that it will never succeed)
this is based on the assumption that a batch is handled as a single transaction, so if any of the messages fail, none of them are inserted.
David Lang _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/
