Re: [ceph-users] Ceph InfiniBand Cluster - Jewel - Performance

2016-04-07 Thread Mark Nelson
On 04/07/2016 03:09 PM, German Anders wrote: also jewel does not supposed to get more 'performance', since it used bluestore in order to store metadata. Or do I need to specify during install to use bluestore? Bluestore is not used by default in jewel. It's just tech preview for now. It also

Re: [ceph-users] Ceph InfiniBand Cluster - Jewel - Performance

2016-04-07 Thread Florian Haas
On Thu, Apr 7, 2016 at 10:09 PM, German Anders wrote: > also jewel does not supposed to get more 'performance', since it used > bluestore in order to store metadata. Or do I need to specify during install > to use bluestore? Do the words "enable experimental unrecoverable data corrupting features

Re: [ceph-users] Ceph InfiniBand Cluster - Jewel - Performance

2016-04-07 Thread Mark Nelson
On 04/07/2016 02:43 PM, German Anders wrote: Hi Cephers, I've setup a production environment Ceph cluster with the Jewel release (10.1.0 (96ae8bd25f31862dbd5302f304ebf8bf1166aba6)) consisting of 3 MON Servers and 6 OSD Servers: 3x MON Servers: 2x Intel Xeon E5-2630v3@2.40Ghz 384GB RAM 2x 200G I

Re: [ceph-users] Ceph InfiniBand Cluster - Jewel - Performance

2016-04-07 Thread German Anders
also jewel does not supposed to get more 'performance', since it used bluestore in order to store metadata. Or do I need to specify during install to use bluestore? Thanks, *German* 2016-04-07 16:55 GMT-03:00 Robert LeBlanc : > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Ceph is not

Re: [ceph-users] Ceph InfiniBand Cluster - Jewel - Performance

2016-04-07 Thread Robert LeBlanc
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Ceph is not able to use native Infiniband protocols yet and so it is only leveraging IPoIB at the moment. The most likely reason you are only getting ~10 Gb performance is that IPoIB heavily leverages multicast in Infiniband (if you do so research in

[ceph-users] Ceph InfiniBand Cluster - Jewel - Performance

2016-04-07 Thread German Anders
Hi Cephers, I've setup a production environment Ceph cluster with the Jewel release (10.1.0 (96ae8bd25f31862dbd5302f304ebf8bf1166aba6)) consisting of 3 MON Servers and 6 OSD Servers: 3x MON Servers: 2x Intel Xeon E5-2630v3@2.40Ghz 384GB RAM 2x 200G Intel DC3700 in RAID-1 for OS 1x InfiniBand Conn