If you look at the PGs that are primary on an OSD that has primary affinity 0, you'll find that they are only on OSDs with primary affinity of 0, so 1 of them has to take the reins or nobody would be responsible for the PG. To prevent this from happening, you would need to configure your crush map in a way where all PGs are guaranteed to land on at least 1 OSD that doesn't have a primary affinity of 0.
On Mon, Feb 12, 2018 at 2:45 PM Teun Docter <teun.doc...@brightcomputing.com> wrote: > Hi, > > I'm looking into storing the primary copy on SSDs, and replicas on > spinners. > One way to achieve this should be the primary affinity setting, as > outlined in this post: > > > https://www.sebastien-han.fr/blog/2015/08/06/ceph-get-the-best-of-your-ssd-with-primary-affinity > > So I've deployed a small test cluster and set the affinity to 0 for half > the OSDs and to 1 for the rest: > > # ceph osd tree > ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF > -1 0.07751 root default > -3 0.01938 host osd001 > 1 hdd 0.00969 osd.1 up 1.00000 1.00000 > 4 hdd 0.00969 osd.4 up 1.00000 0 > -7 0.01938 host osd002 > 2 hdd 0.00969 osd.2 up 1.00000 1.00000 > 6 hdd 0.00969 osd.6 up 1.00000 0 > -9 0.01938 host osd003 > 3 hdd 0.00969 osd.3 up 1.00000 1.00000 > 7 hdd 0.00969 osd.7 up 1.00000 0 > -5 0.01938 host osd004 > 0 hdd 0.00969 osd.0 up 1.00000 1.00000 > 5 hdd 0.00969 osd.5 up 1.00000 0 > > Then I've created a pool. The summary at the end of "ceph pg dump" looks > like this: > > sum 0 0 0 0 0 0 0 0 > OSD_STAT USED AVAIL TOTAL HB_PEERS PG_SUM PRIMARY_PG_SUM > 7 1071M 9067M 10138M [0,1,2,3,4,5,6] 192 26 > 6 1072M 9066M 10138M [0,1,2,3,4,5,7] 198 18 > 5 1071M 9067M 10138M [0,1,2,3,4,6,7] 192 21 > 4 1076M 9062M 10138M [0,1,2,3,5,6,7] 202 15 > 3 1072M 9066M 10138M [0,1,2,4,5,6,7] 202 121 > 2 1072M 9066M 10138M [0,1,3,4,5,6,7] 195 114 > 1 1076M 9062M 10138M [0,2,3,4,5,6,7] 161 95 > 0 1071M 9067M 10138M [1,2,3,4,5,6,7] 194 102 > sum 8587M 72524M 81111M > > Now, the OSDs for which the primary affinity is set to zero are acting as > primary a lot less than the others. > > But what I'm wondering about is this: > > For those OSDs that have primary affinity set to zero, why is the > PRIMARY_PG_SUM column not zero? > > # ceph -v > ceph version 12.2.2 (cf0baeeeeba3b47f9427c6c97e2144b094b7e5ba) luminous > (stable) > > Note that I've created the pool after setting the primary affinity, and no > data is stored yet. > > Thanks, > Teun > > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
