Thanks - already done, as you say
> On May 15, 2017, at 7:32 AM, Håkon Bugge <hakon.bu...@gmail.com> wrote:
>
> Dear Open MPIers,
>
>
> Automatic tuning of socket buffers has been in the linux kernel since
> 2.4.17/2.6.7. That is some time ago. I remember, at the time, that we removed
> the default setsockopt() for SO_SNDBUF and SO_RCVBUF in Scali MPI.
>
> Today, running Open MPI 1.10.2 using the TCP BTL, on a 10Gbit/ ethernet, I
> get:
>
> # OSU MPI Bi-Directional Bandwidth Test
> # Size Bi-Bandwidth (MB/s)
> 1 1.72
> 2 4.58
> 4 9.06
> 8 18.43
> 16 35.68
> 32 68.47
> 64 135.20
> 128 259.30
> 256 450.59
> 512 703.55
> 1024 935.58
> 2048 1020.04
> 4096 1191.23
> 8192 1192.13
> 16384 1155.97
> 32768 1181.74
>
> and by strace I see that Open MPI sets said buffer sizes:
>
> setsockopt(12, SOL_SOCKET, SO_SNDBUF, [131072], 4) = 0
> setsockopt(12, SOL_SOCKET, SO_RCVBUF, [131072], 4) = 0
>
> Now, by adding “—mca btl_tcp_rcvbuf 0 —mca btl_tcp_sndbuf 0” to the above
> command line, I get:
>
> # OSU MPI Bi-Directional Bandwidth Test
> # Size Bi-Bandwidth (MB/s)
> 1 1.60
> 2 4.56
> 4 9.03
> 8 11.66
> 16 35.54
> 32 68.36
> 64 133.70
> 128 247.69
> 256 466.75
> 512 885.40
> 1024 1557.51
> 2048 2115.40
> 4096 2226.65
> 8192 2288.82
> 16384 2318.11
> 32768 2334.19
>
> (and strace shows no setsockopt for SO_{RCV,SND}BUF)
>
> Roughly, the performance doubles.
>
> Just a humble suggestion to remove the setting of the socket buffer sizes if
> not already done in newer versions.
>
>
> Thxs, Håkon
>
> _______________________________________________
> devel mailing list
> devel@lists.open-mpi.org
> https://rfd.newmexicoconsortium.org/mailman/listinfo/devel
_______________________________________________
devel mailing list
devel@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/devel
