It would work fine either way. I was just curious how people are setting up Ceph in their environments.
Usually when people say they have one OSD per server, then they are using RAID for one reason or another. It not really relevant to the question at hand, but thank you for satisfying my curiosity On Mon, Apr 2, 2018 at 7:13 PM, ST Wong (ITSC) <s...@itsc.cuhk.edu.hk> wrote: > There are multiple disks per server, and will have one OSD for each disk. > Is that okay? > > > > Thanks again. > > > > *From:* Donny Davis [mailto:do...@fortnebula.com] > *Sent:* Tuesday, April 03, 2018 10:12 AM > *To:* ST Wong (ITSC) > *Cc:* Ronny Aasen; ceph-users@lists.ceph.com > > *Subject:* Re: [ceph-users] split brain case > > > > Do you only have one OSD per server? Not that is really matters... because > all of the above stated is true in any case. > > Just curious > > > > On Mon, Apr 2, 2018 at 6:40 PM, ST Wong (ITSC) <s...@itsc.cuhk.edu.hk> > wrote: > > Hi, > > > > >how many servers are your osd's split over ? keep in mind that ceph's > default picks one osd from each host. so you would need minimum 4 osd hosts > in total to be able to use 4+2 pools and with only 4 hosts you have no > failuredomain. but 4 hosts in the minimum sane starting point for a > regular small cluster with 3+2 pools (you can loose a node and ceph > selfheals as long as there are enough freespace. > > > > We’ll have 8 servers to split over (4 in each room). Thanks. > > > > Best Rgds, > > /st wong > > > > *From:* Ronny Aasen [mailto:ronny+ceph-us...@aasen.cx] > *Sent:* Friday, March 30, 2018 3:18 AM > *To:* ST Wong (ITSC); ceph-users@lists.ceph.com > *Subject:* Re: [ceph-users] split brain case > > > > On 29.03.2018 11:13, ST Wong (ITSC) wrote: > > Hi, > > > > Thanks. > > > > > ofcourse the 4 osd's left working now want to selfheal by recreating > all objects stored on the 4 split off osd's and have a huge recovery job. > and you may risk that the osd's goes into too_full error, unless you have > free space in your osd's to recreate all the data in the defective part of > the cluster. or they will be stuck in recovery mode until you get the > second room running, this depends on your crush map. > > > > Means we’ve to made 4 OSD machines sufficient space to hold all data and > thus the usable space will be halved? > > > yes if you want to be able to be able to operatate one room as if it was > the whole cluster (HA) then you need this. > also if you want to have 4+2 instead of 3+2 pool size to avoid the > blocking during recovery, that would take a whole lot of ekstra space. > you can optionally let the cluster run degraded with 4+2 while one room is > down. or temporary set pools to 2+2 while the other room is down, to reduce > the space requirements. > > > > > > point in that slitting the cluster hurts. and if HA is the most > important then you may want to check out rbd mirror. > > > > Will consider when there is budget to setup another ceph cluster for rdb > mirror. > > > i do not know your needs or applications, but while you only have 2 rooms > you may just think of it as a single cluster that just happen to occupy 2 > rooms. but with that few osd's you should perhaps just put the cluster in > a single room > the pain of splitting a cluster down the middle is quite significant. and > i would perhaps use resources to improve the redundancy of the networks > between the buildings instead. have multiple paths between the buildings to > prevent service disruption in the building that does not house the cluster. > > having 5 mons is quite a lot. i think most clusters have 3 mons up into > several hundred osd hosts > > how many servers are your osd's split over ? keep in mind that ceph's > default picks one osd from each host. so you would need minimum 4 osd hosts > in total to be able to use 4+2 pools and with only 4 hosts you have no > failuredomain. but 4 hosts in the minimum sane starting point for a > regular small cluster with 3+2 pools (you can loose a node and ceph > selfheals as long as there are enough freespace. > > kind regards > Ronny Aasen > > > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
