Infiniband has much lower latencies when performing RDMA and native IB traffic. Doing IPoIB adds all the Ethernet stuff that has to be done in software. Still it is comparable to Ethernet even with this disadvantage. Once Ceph has the ability to do native RDMA, Infiniband should have an edge.
Robert LeBlanc Sent from a mobile device please excuse any typos. On Nov 7, 2014 4:25 AM, "Stefan Priebe - Profihost AG" < [email protected]> wrote: > Hi, > > this is with intel 10GBE bondet (2x10Gbit/s) network. > rtt min/avg/max/mdev = 0.053/0.107/0.184/0.034 ms > > I thought that the mellanox stuff had lower latencies. > > Stefan > > Am 06.11.2014 um 18:09 schrieb Robert LeBlanc: > > rtt min/avg/max/mdev = 0.130/0.157/0.190/0.016 ms > > > > IPoIB Mellanox ConnectX-3 MT27500 FDR adapter and Mellanox IS5022 QDR > > switch MTU set to 65520. CentOS 7.0.1406 > > running 3.17.2-1.el7.elrepo.x86_64 on Intel(R) Atom(TM) CPU C2750 with > > 32 GB of RAM. > > > > On Thu, Nov 6, 2014 at 9:46 AM, Udo Lembke <[email protected] > > <mailto:[email protected]>> wrote: > > > > Hi, > > no special optimizations on the host. > > In this case the pings are from an proxmox-ve host to ceph-osds > > (ubuntu + debian). > > > > The pings from one osd to the others are comparable. > > > > Udo > > > > On 06.11.2014 15:00, Irek Fasikhov wrote: > >> Hi,Udo. > >> Good value :) > >> > >> Whether an additional optimization on the host? > >> Thanks. > >> > >> Thu Nov 06 2014 at 16:57:36, Udo Lembke <[email protected] > >> <mailto:[email protected]>>: > >> > >> Hi, > >> from one host to five OSD-hosts. > >> > >> NIC Intel 82599EB; jumbo-frames; single Switch IBM G8124 > >> (blade network). > >> > >> rtt min/avg/max/mdev = 0.075/0.114/0.231/0.037 ms > >> rtt min/avg/max/mdev = 0.088/0.164/0.739/0.072 ms > >> rtt min/avg/max/mdev = 0.081/0.141/0.229/0.030 ms > >> rtt min/avg/max/mdev = 0.083/0.115/0.183/0.030 ms > >> rtt min/avg/max/mdev = 0.087/0.144/0.190/0.028 ms > >> > >> > >> Udo > >> > > > > > > _______________________________________________ > > ceph-users mailing list > > [email protected] <mailto:[email protected]> > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > > > > > > > _______________________________________________ > > ceph-users mailing list > > [email protected] > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > >
_______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
