On Mon, Mar 8, 2010 at 12:01 PM, Terry Smith <Terry.Smith at sun.com> wrote:
> Hi Mike
>
> I also have done some timing tests and come to the same conclusions broadly
> as Martin.
>
>
> On 08/03/2010 17:55, Mike Gerdts wrote:
>> 2) Why is it only getting 540 Mbits/sec (1 / 14.8 * 8 * 1000) of
>> network throughput?
>>
>> During the memory copy is a CPU pegged ("mpstat 1", possibly "prstat
>> -mLc -n 5 1")? ?If so a fix probably requires code changes to offload
>> crypto. If not, there is is a pretty good chance that a bit of network
>> tuning will get your throughput much closer to wire speed.
>>
>> http://unix.derkeiler.com/Newsgroups/comp.unix.solaris/2007-04/msg00439.html
>> suggests several parameters that are pretty common to set - go look at
>> disclosures for pretty much any benchmark published by Sun. ?Note that
>> I haven't read that post closely, but the parameters set at the top
>> are consistent with what my experience suggests is needed to get wire
>> speed on gigabit NICs.
>>
>>>
> The system is not network limited - but limited by the amount of work to be
> done by the primary domain in doing memory compression and moving bits
> around.Interesting... If it is not able to push enough (compressed) bits across the wire to do more than 55% of wire speed, how much benefit is there in compression? That is, if the data were not compressed, it would be pushing 540 Mbit/sec. With compression it is probably pushing much less than 300 Mbit/sec. I posit that with network tuning, removal of compression, and the use of hardware encryption[1], the memory transfer could be nearly twice as fast. 1. According to http://staff.science.uva.nl/~jesshope/Downloads/NiagaraII_vs_IntelI7.pdf, Niagara II can do 36 Gbit/sec AES128. Even if that is 36 Gbit per chip / 8 cores per chip, that is still well above 1 Gbit/sec. -- Mike Gerdts http://mgerdts.blogspot.com/
