Is there any way to confirm (beforehand) that using SSDs for journals will
help?
We're seeing very disappointing Ceph performance. We have 10GigE
interconnect (as a shared public/internal network).
We're wondering whether it makes sense to buy SSDs and put journals on
them. But we're looking for
Thanks for your answer, Nick.
Typically it's a single rsync session at a time (sometimes two, but rarely
more concurrently). So it's a single ~5GB typical linux filesystem from one
random VM to another random VM.
Apart from using RBD Cache, is there any other way to improve the overall
On 01-05-15 11:42, Nick Fisk wrote:
Yeah, that’s your problem, doing a single thread rsync when you have
quite poor write latency will not be quick. SSD journals should give you
a fair performance boost, otherwise you need to coalesce the writes at
the client so that Ceph is given bigger IOs
How many Rsync's are doing at a time? If it is only a couple, you will not
be able to take advantage of the full number of OSD's, as each block of data
is only located on 1 OSD (not including replicas). When you look at disk
statistics you are seeing an average over time, so it will look like the
Also remember to drive your Ceph cluster as hard as you got means to, eg.
tuning the VM OSes/IO sub systems like using multiple RBD devices per VM (to
issue more out standing IOPs from VM IO subsystem), best IO scheduler, CPU
power + memory per VM, also ensure low network latency + bandwidth
Yeah, that's your problem, doing a single thread rsync when you have quite
poor write latency will not be quick. SSD journals should give you a fair
performance boost, otherwise you need to coalesce the writes at the client
so that Ceph is given bigger IOs at higher queue depths.
RBD Cache can
Piotr,
You may also investigate if the cache tier made of a couple of ssds could help
you. Not sure how the data is used in your company, but if you have a bunch of
hot data that moves around from one vm to another it might greatly speed up the
rsync. On the other hand, if a lot of rsync data
Hi,
On 01.05.2015 10:30, Piotr Wachowicz wrote:
Is there any way to confirm (beforehand) that using SSDs for journals
will help?
yes SSD-Journal helps a lot (if you use the right SSDs) for write speed,
and I made the experiences that this also helped (but not too much) for
read-performance.
yes SSD-Journal helps a lot (if you use the right SSDs)
What SSDs to avoid for journaling from your experience? Why?
We're seeing very disappointing Ceph performance. We have 10GigE
interconnect (as a shared public/internal network).
Which kind of CPU do you use for the OSD-hosts?
