On Sat, 24 Jan 2015, David Lang wrote:
- RELP or not
* My benchmarks on a 1G 20-core box achieved maximum of 27 MB/s
(uncompressed) with RELP with multiple producers. Compare it with ptcp: 98
MB/s uncompressed, 220 MB/s with stream-compression. So efficiency is one
of the points of contention.
* Per message ack does not bring a whole lot of value over seq_no range
based acks. Entire range will have to be re-transmitted in case of error,
but if errors are infrequent, that'll not be a problem.
so let's look at improving RELP, something that will help many people who
aren't going to do the millions of logs/sec with the dozens of nodes per
layer than you are describing.
Where are the bottlenecks in RELP? it does allow a window of X messages to be
waiting for acks. Does that window need to be larger? Do we need to allow for
multiple acks to be sent as one message?
Another datapoint that may be related, there was a post here where someone has
the batch size set to 1000 and RELP didn't work for them until that setting was
removed.
There's no fundamental reason why RELP, with a large enough window, should be
significantly slower than plain TCP, as long as the network and CPUs are not
being saturated. There is a little overhead when the message is sent that eats
some bandwideth, but the acks back should be effectively free (they will
piggyback on the TCP level ack packets that are sent, eating a little bandwidth
in a direction that is otherwise pretty much idle). If the window is large
enough, the acks should be arriving fast enough so that transmission of new
messages never needs to pause.
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
