On Sun, 18 Jan 2009, Rainer Gerhards wrote: > On Sun, 2009-01-18 at 17:30 -0800, [email protected] wrote: >> >> what I was thinking was that instead of allocating memory for one message >> at a time, initially allocate memory for 100 messages, then if this needs >> to be extended increase the allocation by 50-100%. this minimizes the >> number of allocations needed and the fragmentation of system memory. >> >> just like the fixed-array queue option is significantly faster than the >> linked list queue option (I assume from a combination of having to chase >> pointers and allocate/deallocate memory), there may be similar benifits >> from doing the same thing for the message content itself. > > I have to admit I am skeptic about this. The reason is that there are > many non-fixed fields within the message and they are allocated as > needed (with some initial size that fits most messages, but it may > extend on an as-needed basis). So there is no real fixed size for any > message. It also depends on template formatting and other factors. > > I think if I'd try to prealloc at least the initial chunks, I'd probably > do pretty much the same that the malloc()/free() runtime does. That, > however, will probably be less performant than the runtime is (at least > I hope so, these parts of the code should be heavily tweaked). This is > also an error-prone task. > > There may be a compromise in between (e.g. allocating a fixed chunk of > message text together with the message blobs), but I still think the > necessary complexity is not outweight by similar benefits. > > All in all, I think, we have seen that the in-user-space computing needs > (and malloc counts as such) are not really the bottlenecks. Implementing > e.g. a "bunch writer" (which enables submission of multiple messages at > once to an action) seems to be (just) equally complex but promises far > better results.
always possible. > In any case, I'd finally like to track down that dangling race before I > do any further optimization. It looks like Lorenzo seems to have a > relatively stable environment for reproduction and I'd like to take > advantage of that. agreed, tracking down a reproducable problem takes precidence over new improvements/tweaks any day. David Lang _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com
