> It may be worthwhile doing some testing on how much time is spent in > the > output portion vs time spent on the selection/string manipulation > (especially with the new high-speed fixed templates). I am speculating > that there is a significant amount of time spent if the > selection/template > stage crafting the strings that are then written out (possibly by being > sent through zip, accessing a dynafile, etc). If so there may be a > significant win in just moving the lock aquisition from the beginning > of > the action to just before writing (by allowing one thread to be > crafting > the string while another is writing it out)
I agree, and that is my plan. Hopefully for next week. As you have probably seen, I made some changes with good results, but there are correctness issues (e.g. I see some duplicated strings). I will now focus on getting the code correct again (which probably means some loss of performance) and then I'll dig further into that direction. I have lots of ideas, and I think there is lots of potential for speedup. It requires partial redesign, but that doesn't matter. > I would expect the win to be less with more complex output > (dynafiles/compression) but I also suspect that high speed sites will > tend > to not use these features as much (especially if they make a noticable > difference in the speed ;-) My overall route will be to make rsyslog look at the parameters actually being used for a given problem (output, queue, whatever) and select the best algorithm for these parameters. So if someone needs complex features, they will cost performance, but those who only use simpler features will gain speed benefits. That, however, doesn't mean I don't care about complex features. I still will try to make them as fast as possible (and we seem not to be too bad at this if I read third-party comparisons). But the "one algorithm fits all needs" approach so far used doesn't give us excellent performance in all cases. If it is possible to avoid the more costly features, that's a good choice for a user. If not, one needs to pay a price. My goal, however, is that rsyslog offers complex features far less expensive than other syslogds. Using multiple algorithms is obviously more development effort, and this is the premier reason I avoided it so far. But looking at what we currently have reached, I think it pays of and is the right thing to do (it would not have been three years ago). Rainer > > something to try a quick test on sometime. > > David Lang > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com
