On Thu, 2 Oct 2014 21:54:54 +0100 (BST) Andrei Mikhailovsky wrote: > Hello Cephers, > > I am a bit lost on the best ways of using ssd and hdds for ceph cluster > which uses rbd + kvm for guest vms. > > At the moment I've got 2 osd servers which currently have 8 hdd osds > (max 16 bays) each and 4 ssd disks. Currently, I am using 2 ssds for osd > journals and I've got 2x512GB ssd spare, which are waiting to be > utilised. I am running Ubuntu 12.04 with 3.13 kernel from Ubuntu 14.04 > and the latest firefly release. > In case you're planning to add more HDDs to those nodes, the obvious use case for those SSDs would be additional journals.
Also depending on your use case, a kernel newer than 3.13(which also is not getting any upstream updates/support) might be a good idea. > I've tried to use ceph cache pool tier and the results were not good. My > small cluster slowed down by quite a bit and i've disabled the cache > tier altogether. > Yeah, this feature is clearly a case of "wait for the next major release or the one after that and try again". > My question is how would one utilise the ssds in the best manner to > achieve a good performance boost compared to a pure hdd setup? Should I > enable block level caching (likes of bcache or similar) using all my > ssds and do not bother using ssd journals? Should I keep the journals on > two ssds and utilse the remaining two ssds for bcache? Or is there a > better alternative? > This has all been discussed very recently here and the results where inconclusive at best. In some cases reads were improved, but for writes it was potentially worse than normal Ceph journals. Have you monitored your storage nodes (I keep recommending atop for this) during a high load time? If your SSDs are becoming the bottleneck and not the actual disks (doubtful, but verify), more journals. Other than that, maybe create a 1TB (usable space) SSD pool for guests with special speed requirements... Christian -- Christian Balzer Network/Systems Engineer [email protected] Global OnLine Japan/Fusion Communications http://www.gol.com/ _______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
