On Sun, 2009-01-18 at 17:30 -0800, [email protected] wrote: > I'm not sure that we're saying the same thing. let me try again.
You are right, we weren't... > > what I was thinking was that instead of allocating memory for one message > at a time, initially allocate memory for 100 messages, then if this needs > to be extended increase the allocation by 50-100%. this minimizes the > number of allocations needed and the fragmentation of system memory. > > just like the fixed-array queue option is significantly faster than the > linked list queue option (I assume from a combination of having to chase > pointers and allocate/deallocate memory), there may be similar benifits > from doing the same thing for the message content itself. I have to admit I am skeptic about this. The reason is that there are many non-fixed fields within the message and they are allocated as needed (with some initial size that fits most messages, but it may extend on an as-needed basis). So there is no real fixed size for any message. It also depends on template formatting and other factors. I think if I'd try to prealloc at least the initial chunks, I'd probably do pretty much the same that the malloc()/free() runtime does. That, however, will probably be less performant than the runtime is (at least I hope so, these parts of the code should be heavily tweaked). This is also an error-prone task. There may be a compromise in between (e.g. allocating a fixed chunk of message text together with the message blobs), but I still think the necessary complexity is not outweight by similar benefits. All in all, I think, we have seen that the in-user-space computing needs (and malloc counts as such) are not really the bottlenecks. Implementing e.g. a "bunch writer" (which enables submission of multiple messages at once to an action) seems to be (just) equally complex but promises far better results. In any case, I'd finally like to track down that dangling race before I do any further optimization. It looks like Lorenzo seems to have a relatively stable environment for reproduction and I'd like to take advantage of that. Rainer _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com
