I have no performance data before we migrated to bluestore. You should start a
separate topic regarding your question.
Could anyone with an more or less equally sized cluster post the output of a
sysbench with the following parameters (either from inside an openstack vm or a
mounted rbd)?
sysbench --file-fsync-freq=1 --threads=16 fileio --file-total-size=1G
--file-test-mode=rndrw --file-rw-ratio=2 prepare
sysbench --file-fsync-freq=1 --threads=16 fileio --file-total-size=1G
--file-test-mode=rndrw --file-rw-ratio=2 run
Thanks in advance.
Regards
Felix
-------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt
-------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------
Am 06.06.19, 15:09 schrieb "Marc Roos" <[email protected]>:
I am also thinking of moving the wal/db to ssd of the sata hdd's. Did
you do tests before and after this change, and know what the difference
is iops? And is the advantage more or less when your sata hdd's are
slower?
-----Original Message-----
From: Stolte, Felix [mailto:[email protected]]
Sent: donderdag 6 juni 2019 10:47
To: ceph-users
Subject: [ceph-users] Expected IO in luminous Ceph Cluster
Hello folks,
we are running a ceph cluster on Luminous consisting of 21 OSD Nodes
with 9 8TB SATA drives and 3 Intel 3700 SSDs for Bluestore WAL and DB
(1:3 Ratio). OSDs have 10Gb for Public and Cluster Network. The cluster
is running stable for over a year. We didn’t had a closer look on IO
until one of our customers started to complain about a VM we migrated
from VMware with Netapp Storage to our Openstack Cloud with ceph
storage. He sent us a sysbench report from the machine, which I could
reproduce on other VMs as well as on a mounted RBD on physical hardware:
sysbench --file-fsync-freq=1 --threads=16 fileio --file-total-size=1G
--file-test-mode=rndrw --file-rw-ratio=2 run sysbench 1.0.11 (using
system LuaJIT 2.1.0-beta3)
Running the test with following options:
Number of threads: 16
Initializing random number generator from current time
Extra file open flags: 0
128 files, 8MiB each
1GiB total file size
Block size 16KiB
Number of IO requests: 0
Read/Write ratio for combined random IO test: 2.00 Periodic FSYNC
enabled, calling fsync() each 1 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
File operations:
reads/s: 36.36
writes/s: 18.18
fsyncs/s: 2318.59
Throughput:
read, MiB/s: 0.57
written, MiB/s: 0.28
General statistics:
total time: 10.0071s
total number of events: 23755
Latency (ms):
min: 0.01
avg: 6.74
max: 1112.58
95th percentile: 26.68
sum: 160022.67
Threads fairness:
events (avg/stddev): 1484.6875/52.59
execution time (avg/stddev): 10.0014/0.00
Are these numbers reasonable for a cluster of our size?
Best regards
Felix
IT-Services
Telefon 02461 61-9243
E-Mail: [email protected]
------------------------------------------------------------------------
-------------
------------------------------------------------------------------------
-------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof.
Dr. Sebastian M. Schmidt
------------------------------------------------------------------------
-------------
------------------------------------------------------------------------
-------------
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
