-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 10/26/2017 09:58 AM, Sebastian Eastham wrote:
> As it happens, you are exactly right, and it turns out this was a
> case of "error exists between keyboard and chair". In particular I
> wanted to thank John Hearns for pointing out the difference
> between pure Infiniband and IPOIB. Once I saw that, I realized I
> should be looking at total > transfer rates in iftop rather than
> at the transfer broken down by device. I could then see that there
> was a big spike in traffic on ib0 exactly when communication
> started occurring, which I take to mean that communication is
> happening over IB, but that iftop is unable to disaggregate the
> traffic.
I have made the exact same mistake in the past -- in my case, it was the
MVAPICH2 mailing list that set me straight. While I'm not sure of a
way to check RDMA traffic counters/rates like iftop does, if you turn
on additional debugging in MVAPICH2, you can see which interface is
selected or at least that RDMA is -- it's been awhile.
I suspect for most MPI, the default configuration is to use the best
suited interface, at least for Infiniband/RDMA.
- --
____
|| \\UTGERS, |----------------------*O*------------------------
||_// the State | Ryan Novosielski - novos...@rutgers.edu
|| \\ University | Sr. Technologist - 973/972.0922 ~*~ RBHS Campus
|| \\ of NJ | Office of Advanced Res. Comp. - MSB C630, Newark
`'
-----BEGIN PGP SIGNATURE-----
iFwEARECABwFAlnyBRwVHG5vdm9zaXJqQHJ1dGdlcnMuZWR1AAoJEJm/oGnRHLG+
8t4AnAmuukJlH7g7RkTQOY9083Qpf84lAJwKtpgElgiEHmao/xIJ6jIFZQnpkw==
=TSSh
-----END PGP SIGNATURE-----
