On Sun, 2009-01-18 at 04:33 -0800, [email protected] wrote: > On Sun, 18 Jan 2009, Rainer Gerhards wrote: > > > On Sat, 2009-01-17 at 16:55 -0800, [email protected] wrote: > > > >> > >> what I think that you are seeing is that the default rsyslog memory queue > >> only uses as much ram as it needs to hold the data (even though it's > >> described as a array it seems to grow dynamicly, I'm not sure about it > >> shrinking) > > > > If you use "fixedarray" mode, the pointer array is allocated statically, > > no matter how many messages are in the queue. HOWEVER, this is only the > > pointers, so quite few memory. Actual messages are dynamically allocated > > and freed when processed - in any mode. > > that makes sense. It would be interesting to see what would happen to the > enqueue/dequeue timings if the message memory was staticly allocated > > from what I remember seeing of the memory footprint it does appear as if > you allocate the max size for the message each time, not the minimum sized > needed to hold the message > yes, that's right. This is done to prevent an additional copy to clean things up (realloc might work, too) and memory fragmentation. The later is really nasty, I've seen that some memory areas remain allocated for quite some while due to fragmentation.
> if that shows a noticable difference it may be worth allocating the memory > in chunks substantially larger than a single message That's a good suggestion. The basic classes are able to trim strings. It may be worth putting a config option into it. The current approach works well for small queues, but obviously does provide sub-optimal performance as soon as the queues grow considerably. So it may even make sense to start trimming messages only after a certain amount of messages are in-queue. Rainer _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com
