Henrik Johansen wrote: > Dai Ngo wrote: >> Henrik Johansen wrote: >>> [Dai Ngo] wrote: >>>> Henrik Johansen wrote: >>>>> Henrik Johansen wrote: >>>>>> Piyush Shivam wrote: >>>>>>> On 08/05/09 15:53, Henrik Johansen wrote: >>>>>>>> Hi list, >>>>>>>> >>>>>>>> I have 2 servers which are directly connected via ixgbe based >>>>>>>> nics, both >>>>>>>> running OpenSolaris 2009.06. >>>>>>>> >>>>>>>> The actual network connection seems fine, iperf reports ~6.3 >>>>>>>> Gbits/sec >>>>>>>> in terms of throughput and nicstat seems to agree that the nics >>>>>>>> are ~63% >>>>>>>> utilized. >>>>>>>> Iperf : henrik at opensolaris:~# ./iperf-2.0.4/src/iperf -c >>>>>>>> 10.10.10.2 -N -t 40 >>>>>>>> ------------------------------------------------------------ >>>>>>>> Client connecting to 10.10.10.2, TCP port 5001 >>>>>>>> TCP window size: 391 KByte (default) >>>>>>>> ------------------------------------------------------------ >>>> Can you verify the TCP window size on both client and server system >>>> with this command: >>>> >>>> # ndd -get /dev/tcp tcp_xmit_hiwat >>>> >>>> # ndd -get /dev/tcp tcp_recv_hiwat >>> >>> Both client and server have both set to 400000. >>> >> Can you try 1024000, this is the default in snv_119. >> Did you verify the new setting take effect on NFS connections? > > I have just updated them to 1024000 (on both client and server). > henrik at opensolaris:/# ptime cp /tmpfs/testfile_5g /nfs > > real 1:17.571581221 > user 0.016147465 > sys 13.271892825 > > That gives ~66 mb/s. If I read your previous mail correctly, then you got 170 MB/s with the previous TCP window setting (400000) and with 1024000 you got 66 MB/s? > - the most I have been able to write via NFS is 170 mb/s. Something is wrong with this picture. I'd ask again, how did you make sure the new setting takes effect on both client & server system?
-Dai
