Hi Michal, You can have a look at a thread I started a few days ago : http://lists.ceph.com/pipermail/ceph-users-ceph.com/2015-December/006494.html
I had some interrogations about performances as well and I think the explanations apply to your case. Also, your SSD does not seem to be DC grade, which is recommended for Ceph for both performance and durability. Adrien On Mon, Dec 14, 2015 at 1:15 PM, Michał Chybowski < [email protected]> wrote: > Hi > > I've set up a small (5-node) cluster of Ceph. I'm trying to benchmark more > real-life performance of ceph's block storage, but I'm seeing very weird > (low) values of my benchmark setup. > > My cluster consists of 5 nodes, every node has: > 2 x 3TB HGST SATA drive > 1x Samsung SM 841 120GB SSD for journal and system (3 partitions set on > this drive, 1st is OS, 2nd and third are journals) > 2 networks (public and cluster), both on 1 interface (IPoverInfiniBand on > Mellanox InfiniHost III cards, up to 12Gbit/s) > ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299) (latest > available as debian-infernalis) > > I'm using command shown below to benchmark storage (both my existing > solution and new ones, which could possibly be adopted to production): > fio --name=testrw --ioengine=libaio --iodepth=16 --rw=randwrite --bs=4k > --direct=1 --size=1024M --numjobs=5 --runtime=240 --group_reporting > > Unfortunately, in ceph's RBD I'm getting only ~400 - 450 IOPS in this test. > > My ceph.conf: > [global] > fsid = fcea4035-8a96-4e99-ae79-72e316197604 > mon_initial_members = testing-1 > mon_host = 10.31.7.21 > auth_cluster_required = cephx > auth_service_required = cephx > auth_client_required = cephx > filestore_xattr_use_omap = true > public network = 10.31.7.21/24 > cluster network = 10.32.7.21/24 > osd pool default size = 2 > osd pool default min size = 1 > > Is there anything I could do to at least get 10*1HDD performance on single > RBD mapping? > > -- > Pozdrawiam > Michał Chybowski > Tiktalik.com > > _______________________________________________ > ceph-users mailing list > [email protected] > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- ----------------------------------------------------------------------------------------- Adrien GILLARD +33 (0)6 29 06 16 31 [email protected]
_______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
