On Mon, 31 May 2010, [email protected] wrote: > On Mon, 31 May 2010, Rainer Gerhards wrote: > >>> I agree that doing it in the output would be far better in many ways, >>> but >>> since there isn't a way to do a plugin there (at least not as far as I >>> know, it would be good to get confirmation or a better idea) >> >> David, can you tell me what you have on your mind for this functionality? I >> have thought a bit about it, and I probably have one approach myself. But I >> would prefer to hear your idea before I push you into a direction. > > > two options > > 1. something that would work similar to the existing format > string, but would call a C subroutine that could read the existing > properties and would create the output string in a buffer > > 2. something that could also modify the exisitng properties (more > powerful, but also more dangerous and could involve locking to prevent > other things from trying to read properties at the same time) > > we haven't gone too far down the road of researching the output > performance (since the input and queue locking has dominated so far), but > it is clear that the output currently takes significantly more CPU time > than input, it may be that being able to use C to define the output format > instead of interpreting the format string may be a noticable improvement. > Is there a relativly easy way to test this? (say, hard-code a format or > two and test writes to file and network with the hard-coded format vs a > format string that produces the same output?)
for the traditional output formats the difference may not be that much, but if there is extensive parsing involved (as the initial poster is doing, or what I would expect is common for specific log types into a database) the difference can be much more significant since it can replace multiple regex statements with a much faster single pass that looks for word breaks and inserts standard filler in those spots. With the new syslog format where the data is 'supposed to be' in a series of name=value tuples, something like this would be a pretty efficiant way of extracting particular portions of the data to be output (although the properties could be extended to do this sort of thing by providing something similar to a perl hash) David Lang _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com
