On Mon, 7 Jun 2010, Rainer Gerhards wrote: > I have now shifted my focus to enhancing the multi-core utilization with > imtcp. So far, we have a single epoll-loop (or select, if epoll is not > supported), which obviously limits concurrency in some environments. I intend > to remove that limit, or at least move the actual value much further. > > There are a couple of things to think about. There is one relatively simple > approach, but if it works is pretty much depending on how things are deployed > in practice. > > So I need your help. I would appreciate if you could read > > http://blog.gerhards.net/2010/06/further-improving-tcp-input-performance.html > > and share your comments. I have created a blogpost because that makes it > easier for me to keep the text as reference. While the post is not exactly > short, it also is not exhaustive, so please don't feel discouraged from > reading it just because my thoughts are on a web site rather than included > inline.
for some reason I can't comment on the blogpost, so I'll do it here. I'm surprised to see this as a problem (especially as my experiance has been that the bottlenecks are on the output side, not the input side) the data is serialized as it arrives over the wire (at least if you have a single ethernet port in use), and with epoll I would expect a single thread to have no problem pulling the data from the network stack and putting it somewhere. I think that more research needs to be done on what is eating up the time in your test cases. If it's DNS lookups, they can be disabled (and/or a name cache can be created as we have discussed before) It may be that the parsing that's being done is what's taking the time here, so I would consider soemthing like the following one thread to pull the data from the wire and dispatch it to N worker threads that would parse the message and put the result into the main queue. even late last year with UDP messages I was able to saturate a Gig-E network with packets and receive them with <25% of a single cpu. I would not expect that TCP would have noticably more overhead. David Lang _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com
