Thanks Emmanuel, more comments below. On 20 Oct 2011, at 16:10, Emmanuel Lecharny wrote:
> On 10/20/11 3:37 PM, Zigor Salvador wrote: >> Hello everyone, >> >> I'm benchmarking a distributed application developed with MINA 2.0.4. >> >> I use a simple text protocol (TextLineCodecFactory over TCP) to transmit >> JSON formatted messages (Using Jackson 1.9.0) with a payload size of 472 >> bytes (that is the length of the String I write to the MINA session). >> >> Two applications running on different machines that communicate with said >> protocol over Gigabit Ethernet achieve a maximum stable throughput of 36.000 >> events/second (measured by my benchmarking component). >> >> Assuming 20 bytes for IP and another 20 bytes for TCP headers, that is 512 >> bytes per message so roughly 17.57 MBytes/second. Is the header size assumption correct? (20 bytes for TCP and 20 for IP) >> >> I was expecting *more* of our Gigabit switch (and this is without any other >> traffic in the network) so I'm looking for ways to improve performance. The >> machines have plenty of power (8 core Xeons) and RAM (24 Gb each). > > Is it a Linux OS, Windows or MacOS, or anything more exotic ? > Which Linux flavor ? Ubuntu 11.10 (64 bit). >> >> I have used VisualVM to profile the applications and I see that most of the >> time is spent in NioProcessor.select() followed (quite far) by >> NioProcessor.write(). >> >> Should I understand that there's nothing left for me to do as MINA is taking >> most of the processing time? >> >> I haven't customized any aspect of MINA (buffer size, thread pool, things >> like that...). > Should not be necessary at this step. > > What are the CPU metrics on your server ? >> >> Any hints would be GREATLY appreciated. >> >> I guess I'm using very small packages that have an impact in the maximum >> achieved bandwidth. Maybe using a message size closer to the MTU of 1500 >> bytes would bring total bandwidth up at the cost of a lowered throughput? > > Each message sent and received on ethernet will use 1500 bytes, even if you > transmit only one byte of data. That means for a message being send and a > response being provided, you will eat 2 x 1500 bytes. With 36 000 messages > sent per second, that is 36 000 x 2 x 1500 = 108 000 000 bytes per second. > This will pretty much saturate your Gb ethernet network... I thought MTU was actually the *maximum* size of the unit... Anyhow, your calculations have made it clear that I was only taking into account one way throughput/bandwidth (silly me). The good news is that If ethernet frames have a 1500 byte payload regardless of smaller TCP packets, throughput has already been maxed for this setup. Great! :-D > > Just try to send a 1500 bytes long messages (actually, less than 1500, I > don't remember what is the size of the max payload you can store in a PDU), > to see if you can still process 36 000 messages per second. The response must > have the same size. If so, you should totally saturate your Gb ethernet, and > have the same throughput. I'll try this next... Thank you so much! Zigor.
