On 21/03/17 17:48, Wes Dillingham wrote: > a min_size of 1 is dangerous though because it means you are 1 hard disk > failure away from losing the objects within that placement group entirely. a > min_size of 2 is generally considered the minimum you want but many people > ignore that advice, some wish they hadn't.
I admit I am having difficulty following why this is the case. From searching about I understand that the min_size parameter prevents I/O to a PG which does not have the required number of replicas, but the justification confuses me - if your min_size is one, and you have a PG which now only exists on one OSD, surely you are one OSD failure away from losing that PG entirely regardless of whether or not you are doing any I/O to it, as that's the last copy of your data? And the OSD itself likely serves many other placement groups which are above the min_size, so it is not as if freezing I/O on that PG prevents the actual disk from doing any activity which could possibly exacerbate a failure. Is the assumption that the other lost OSDs could be coming back with their old copy of the PG so any newer writes to the PG would be lost if you're unlucky enough that the last remaining OSD went down before the others came back? Which is not the same thing as losing the objects in that PG entirely, though obviously it's not at all ideal, and is also completely irrelevant if you know the other OSDs will not be coming back. I am sure I remember having to reduce min_size to 1 temporarily in the past to allow recovery from having two drives irrecoverably die at the same time in one of my clusters. Rich
signature.asc
Description: OpenPGP digital signature
_______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
