On 07/02/2019 17:07, jes...@krogh.cc wrote:
Hi Maged

Thanks for your reply.

6k is low as a max write iops value..even for single client. for cluster
of 3 nodes, we see from 10k to 60k write iops depending on hardware.

can you increase your threads to 64 or 128 via -t parameter
I can absolutely get it higher by increasing the parallism. But I
may have missed to explain my purpuse - I'm intested in how close to
putting local SSD/NVMe in servers I can get with RDB. Thus putting
parallel scenarios that I would never see in production in the
tests does not really help my understanding. I think a concurrency level
of 16 is in the top of what I would expect our PostgreSQL databases to do
in real life.

can you run fio with sync=1 on your disks.

can you try with noop scheduler

what is the %utilization on the disks and cpu ?

can you have more than 1 disk per node
I'll have a look at that. Thanks for the suggestion.

Jesper


Thanks for your explanation. In your case, you have low concurrency requirements, so focusing on latency rather than total iops is your goal. Your current setup gives 1.9 ms latency for writes and 0.6 ms for read. These are considered good, it is difficult to go below 1 ms for writes. As Wido pointed, to get latency down you need to insure you have C States in your cpu settings ( or just C1 state ), you have no low frequencies in your P States and get cpu with high GHz frequency rather than more cores (Nick Fisk has a good presentation on this), also avoid dual socket and NUMA. Also if money is no issue, you will get a bit better latency with 40G or 100G network.

/Maged


_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Reply via email to