2016-03-20 17:39 GMT+01:00 David Lang <[email protected]>: > On Sun, 20 Mar 2016, Eugene Prokopiev wrote: > >> 2016-03-20 2:24 GMT+03:00 David Lang : >> >>> with current versions of rsyslog, there is a batch mode that was created >>> specifically for this, however, rsyslog works to minimize the latency in >>> getting the logs into the database, so if the log rate is low enough to >>> be >>> inserted one at a time, that's what will happen. As the log volume >>> increases, things will not keep up (either on the database side or the >>> rsyslog side) and messages will be processed in batches. >> >> >> Thank you! So, switching to batch mode must be automatic? On which >> criteria (messages per seconds or queue size) it will happen? > > > the worker thread that is processing messages grabs up to dequeuebatchsize > messages from the queue (main Q or action Q), processes them, and repeats. > > So, if there are more messages arriving than can be processed one at a time, > the worker queue will find multiple messages in the queue and process them > in a batch. > > As the rate of new messages goes up, and as the time taken to process a > message goes up, the probability that multiple messages will be sitting in > the queue also goes up. >
This is all correct, but it means messages are processed in batches. It does not mean that the output module actually processes a batch as a single transaction. I have just checked, and ompgsql does not do so. I have added a enhancement request to our bug tracker: https://github.com/rsyslog/rsyslog/issues/895 Note that I will not have time nor a use case to work on that issue. So in order to make this happen any time soon, a contribution is required. Rainer _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
