Disk models, other hardware information including CPU, network config? You say you're using Luminous, but then say journal on same device. I'm assuming you mean that you just have the bluestore OSD configured without a separate WAL or DB partition? Any more specifics you can give will be helpful.
On Mon, Jan 22, 2018 at 11:20 AM Steven Vacaroaia <ste...@gmail.com> wrote: > Hi, > > I'll appreciate if you can provide some guidance / suggestions regarding > perfomance issues on a test cluster ( 3 x DELL R620, 1 Entreprise SSD, 3 x > 600 GB ,Entreprise HDD, 8 cores, 64 GB RAM) > > I created 2 pools ( replication factor 2) one with only SSD and the other > with only HDD > ( journal on same disk for both) > > The perfomance is quite similar although I was expecting to be at least 5 > times better > No issues noticed using atop > > What should I check / tune ? > > Many thanks > Steven > > > > HDD based pool ( journal on the same disk) > > ceph osd pool get scbench256 all > > size: 2 > min_size: 1 > crash_replay_interval: 0 > pg_num: 256 > pgp_num: 256 > crush_rule: replicated_rule > hashpspool: true > nodelete: false > nopgchange: false > nosizechange: false > write_fadvise_dontneed: false > noscrub: false > nodeep-scrub: false > use_gmt_hitset: 1 > auid: 0 > fast_read: 0 > > > rbd bench --io-type write image1 --pool=scbench256 > bench type write io_size 4096 io_threads 16 bytes 1073741824 pattern > sequential > SEC OPS OPS/SEC BYTES/SEC > 1 46816 46836.46 191842139.78 > 2 90658 45339.11 185709011.80 > 3 133671 44540.80 182439126.08 > 4 177341 44340.36 181618100.14 > 5 217300 43464.04 178028704.54 > 6 259595 42555.85 174308767.05 > elapsed: 6 ops: 262144 ops/sec: 42694.50 bytes/sec: 174876688.23 > > fio /home/cephuser/write_256.fio > write-4M: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=rbd, > iodepth=32 > fio-2.2.8 > Starting 1 process > rbd engine: RBD version: 1.12.0 > Jobs: 1 (f=1): [r(1)] [100.0% done] [66284KB/0KB/0KB /s] [16.6K/0/0 iops] > [eta 00m:00s] > > > fio /home/cephuser/write_256.fio > write-4M: (g=0): rw=write, bs=4K-4K/4K-4K/4K-4K, ioengine=rbd, iodepth=32 > fio-2.2.8 > Starting 1 process > rbd engine: RBD version: 1.12.0 > Jobs: 1 (f=1): [W(1)] [100.0% done] [0KB/14464KB/0KB /s] [0/3616/0 iops] > [eta 00m:00s] > > > SSD based pool > > > ceph osd pool get ssdpool all > > size: 2 > min_size: 1 > crash_replay_interval: 0 > pg_num: 128 > pgp_num: 128 > crush_rule: ssdpool > hashpspool: true > nodelete: false > nopgchange: false > nosizechange: false > write_fadvise_dontneed: false > noscrub: false > nodeep-scrub: false > use_gmt_hitset: 1 > auid: 0 > fast_read: 0 > > rbd -p ssdpool create --size 52100 image2 > > rbd bench --io-type write image2 --pool=ssdpool > bench type write io_size 4096 io_threads 16 bytes 1073741824 pattern > sequential > SEC OPS OPS/SEC BYTES/SEC > 1 42412 41867.57 171489557.93 > 2 78343 39180.86 160484805.88 > 3 118082 39076.48 160057256.16 > 4 155164 38683.98 158449572.38 > 5 192825 38307.59 156907885.84 > 6 230701 37716.95 154488608.16 > elapsed: 7 ops: 262144 ops/sec: 36862.89 bytes/sec: 150990387.29 > > > [root@osd01 ~]# fio /home/cephuser/write_256.fio > write-4M: (g=0): rw=write, bs=4K-4K/4K-4K/4K-4K, ioengine=rbd, iodepth=32 > fio-2.2.8 > Starting 1 process > rbd engine: RBD version: 1.12.0 > Jobs: 1 (f=1): [W(1)] [100.0% done] [0KB/20224KB/0KB /s] [0/5056/0 iops] > [eta 00m:00s] > > > fio /home/cephuser/write_256.fio > write-4M: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=rbd, > iodepth=32 > fio-2.2.8 > Starting 1 process > rbd engine: RBD version: 1.12.0 > Jobs: 1 (f=1): [r(1)] [100.0% done] [76096KB/0KB/0KB /s] [19.3K/0/0 iops] > [eta 00m:00s] > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
