Re: [ceph-users] Ceph on RDMA

2017-08-30 Thread Haomai Wang
On Wed, Aug 30, 2017 at 7:53 AM, Jeroen Oldenhof wrote: > Hi, > > I used 'https://community.mellanox.com/docs/DOC-2721', and was under the > impression I followed all steps.. but I somehow skipped over the > /usr/lib/systemd/system/ (/lib/systemd/system/ in case of Ubuntu) > ceph-xxx@.service fil

Re: [ceph-users] Ceph on RDMA

2017-08-30 Thread Jeroen Oldenhof
Hi, I used  'https://community.mellanox.com/docs/DOC-2721', and was under the impression I followed all steps.. but I somehow skipped over the /usr/lib/systemd/system/ (/lib/systemd/system/ in case of Ubuntu) ceph-xxx@.service files. I did update the /etc/security/limits.conf file for unlimite

Re: [ceph-users] Ceph on RDMA

2017-08-28 Thread Haomai Wang
do you follow this instruction(https://community.mellanox.com/docs/DOC-2693)? On Mon, Aug 28, 2017 at 6:40 AM, Jeroen Oldenhof wrote: > Hi All! > > I'm trying to run CEPH over RDMA, using a batch of Infiniband Mellanox > MT25408 20GBit (4x DDR) cards. > > RDMA is running, rping works between all

[ceph-users] Ceph on RDMA

2017-08-28 Thread Jeroen Oldenhof
Hi All! I'm trying to run CEPH over RDMA, using a batch of Infiniband Mellanox MT25408 20GBit (4x DDR) cards. RDMA is running, rping works between all hosts, and I've configured 10.0.0.x addressing on the ib0 interfaces. But when enabling RMDA in ceph.conf:   ms_type = async+rdma   ms_asyn