On Mon, Aug 21, 2017 at 3:58 PM, Ronny Aasen <[email protected]> wrote:
> On 21. aug. 2017 07:40, Nick Tan wrote: > >> Hi all, >> >> I'm in the process of building a ceph cluster, primarily to use cephFS. >> At this stage I'm in the planning phase and doing a lot of reading on best >> practices for building the cluster, however there's one question that I >> haven't been able to find an answer to. >> >> Is it better to use many hosts with single OSD's, or fewer hosts with >> multiple OSD's? I'm looking at using 8 or 10TB HDD's as OSD's and hosts >> with up to 12 HDD's. If a host dies, that means up to 120TB of data will >> need to be recovered if the host has 12 x 10TB HDD's. But if smaller hosts >> with single HDD's are used then a single host failure will result in only a >> maximum of 10TB to be recovered, so in this case it looks better to use >> smaller hosts with single OSD's if the failure domain is the host. >> >> Are there other benefits or drawbacks of using many small servers with >> single OSD's vs fewer large servers with lots of OSD's? >> > > > one thing i did not see mentioned in previous emails was that 10TB disks > are often SMR disks. those are not suited for CEPH unless your data is of > the write once, archive forever type. This is not a ceph problem exactly > more of how the SMR disks deal with lots of random writes. > > http://ceph.com/planet/do-not-use-smr-disks-with-ceph/ > > > generally more nodes are better. (but more expencive) due to how ceph > spreads the load over all the nodes. depending on your needs you should not > have too much of your data in a single node (eggs vs baskets). > Large nodes are not "wrong" if your needs are tons of archival data, but > most people have more varied needs. Try to avoid having more then 10% of > your data in a single node and allways have enough freespace to deal with > the loss of a whole node. if you can have a cold node standby you could > just as well plug it into the cluster. It would improve performance since > you'd have more nodes to spread load on. > > kind regards > Ronny Aasen > Thanks Ronny for your advice. I've used SMR disks before and will definitely be avoiding them for this project. Thanks, Nick
_______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
