On Thu, 2007-04-19 at 09:00 +0200, Albert Strasheim wrote: > Hello all, > > On Wed, 18 Apr 2007, davidmc wrote: > > <excellent summary snipped> > > > My case was exactly as yours: > > -the behaviour is different between a local broker and a remote broker, > > because: > > *Windows sockets that connect to localhost don't seem to be really sockets, > > or either, the TCP behaves differently (no Nagle's algorithm). > > *Linux localhost sockets also behave a bit differently from Linux remote > > sockets when it comes to Nagle / Delayed ACK issues. > > -the behaviour is different in Linux and in Windows, because > > *in windows the Delayed ACK timeout is 200ms. So if your broker is in > > Windows, you will notice messages with a roundtrip (latency) time of 200ms. > > *in Linux this timeout is 40ms "only". > > -message size matters, because big messages fill the buffer and get sent > > immediately regardless of ACKs received or not. > > Also I have noticed some strange behaviour with very little packets (less > > than 512 or so bytes), with those packets, disabling Nagle might not work. > > The size below which problems happen is different in a Linux local scenario > > than in a Linux remote scenario. > > This describes almost exactly what I'm seeing! The speed depends > greatly on whether I'm using a broker on Linux or Windows and Stomp or > Openwire. In order of most to least messages/sec (without TCP_NODELAY > being set, as far as I can tell) I get: > > 1. AMQCPP Linux Stomp -> Local Broker Linux > 2. AMQCPP Linux Stomp -> Remote Broker Windows > 3. AMQCPP Linux Openwire -> Remote Broker Windows > 4. AMQCPP Linux Openwire -> Local Broker Linux > > Also, 4 is more than an order of magnitude slower than 1. with 1, 2 and > 3 all in the same ballpark. This is with a reasonably large message, > that might end up compressing quite well with Openwire. > > From what I can tell, AMQCPP supports a broker URL property for setting > TCP_NODELAY, but actually setting it doesn't change anything at socket > creation time (although I could be mistaken, I just did a quick browse > through the code). Maybe TCP_NODELAY should even be the default? >
As of last night I have submitted all the changes to implement allowing TCP_NODELAY to be set, along with some unit tests. If you haven't grabbed trunk in a day or so, you won't have all those changes. We could default it to on if we find its at the root of some of the performance issues. > > How to check if this is really the problem: > > -Go to AMQCPP source code and add a line to your socket creation code, where > > you explicitely set TCP_NODELAY to True always (to disable Nagle's > > algorithm). Rebuild and compare. > > I'll give this a go today. > > > -Capture traffic with Wireshark (former Ethereal), or tcpdump, and see if > > the broker takes a fixed time (40ms in Linux or 200ms in Windows) to send > > ACKs back. > > > > Hope this helps :) > > I think it might. Thanks very much. > > Cheers. > > Albert
