Root affected got more than 70TB free. The only solution is manual reweight
the OSD. But in this situacion balancer in unmap mode should move data to
get all HEALTHY
Hope some fix come in the next 14.2.X to fix that issue.
Ceph 14.2.2 Centos 7.6
cluster:
id: e1ee8086-7cce-43fd-a252-3d677af22428
health: HEALTH_ERR
2 nearfull osd(s)
2 pool(s) nearfull
Degraded data redundancy (low space): 9 pgs backfill_toofull
services:
mon: 3 daemons, quorum CEPH-MON01,CEPH002,CEPH003 (age 19m)
mgr: CEPH001(active, since 12h), standbys: CEPH-MON01
osd: 90 osds: 90 up (since 4h), 90 in (since 34h); 9 remapped pgs
rgw: 1 daemon active (ceph-rgw03)
data:
pools: 18 pools, 8252 pgs
objects: 105.59M objects, 294 TiB
usage: 340 TiB used, 84 TiB / 424 TiB avail
pgs: 197930/116420259 objects misplaced (0.170%)
8243 active+clean
9 active+remapped+backfill_toofull
ID CLASS WEIGHT REWEIGHT SIZE RAW USE DATA OMAP META AVAIL
%USE VAR PGS STATUS TYPE NAME
-41 392.89362 - 393 TiB 317 TiB 317 TiB 19 MiB 561 GiB 75
TiB 0 0 - root archive
-44 130.95793 - 131 TiB 106 TiB 106 TiB 608 KiB 188 GiB 25
TiB 81.29 1.01 - host CEPH005
83 archive 10.91309 1.00000 11 TiB 9.2 TiB 9.1 TiB 56 KiB 16 GiB 1.7
TiB 83.97 1.05 131 up osd.83
84 archive 10.91309 1.00000 11 TiB 8.5 TiB 8.5 TiB 56 KiB 15 GiB 2.4
TiB 78.33 0.98 121 up osd.84
85 archive 10.91309 1.00000 11 TiB 9.0 TiB 9.0 TiB 92 KiB 16 GiB 1.9
TiB 82.24 1.03 129 up osd.85
86 archive 10.91309 1.00000 11 TiB 10 TiB 10 TiB 48 KiB 18 GiB 597
GiB 94.66 1.18 148 up osd.86
87 archive 10.91399 1.00000 11 TiB 10 TiB 10 TiB 88 KiB 18 GiB 596
GiB 94.66 1.18 149 up osd.87
88 archive 10.91309 1.00000 11 TiB 8.3 TiB 8.3 TiB 56 KiB 14 GiB 2.6
TiB 76.02 0.95 119 up osd.88
97 archive 10.91309 1.00000 11 TiB 7.7 TiB 7.7 TiB 44 KiB 14 GiB 3.2
TiB 70.44 0.88 109 up osd.97
98 archive 10.91309 1.00000 11 TiB 8.4 TiB 8.4 TiB 72 KiB 15 GiB 2.5
TiB 77.18 0.96 126 up osd.98
99 archive 10.91309 1.00000 11 TiB 8.0 TiB 8.0 TiB 24 KiB 15 GiB 2.9
TiB 73.29 0.91 116 up osd.99
100 archive 10.91309 1.00000 11 TiB 9.5 TiB 9.4 TiB 32 KiB 17 GiB 1.4
TiB 86.72 1.08 132 up osd.100
101 archive 10.91309 1.00000 11 TiB 8.4 TiB 8.4 TiB 12 KiB 15 GiB 2.5
TiB 76.71 0.96 125 up osd.101
102 archive 10.91309 1.00000 11 TiB 8.9 TiB 8.8 TiB 28 KiB 15 GiB 2.1
TiB 81.20 1.01 125 up osd.102
-17 0 - 0 B 0 B 0 B 0 B 0 B 0
B 0 0 - host CEPH006
-26 130.96783 - 131 TiB 104 TiB 104 TiB 6.7 MiB 184 GiB 27
TiB 79.21 0.99 - host CEPH007
14 archive 10.91399 1.00000 11 TiB 8.3 TiB 8.3 TiB 28 KiB 15 GiB 2.6
TiB 76.06 0.95 126 up osd.14
15 archive 10.91399 1.00000 11 TiB 8.9 TiB 8.9 TiB 84 KiB 16 GiB 2.0
TiB 81.72 1.02 130 up osd.15
16 archive 10.91399 1.00000 11 TiB 8.7 TiB 8.7 TiB 80 KiB 15 GiB 2.2
TiB 79.98 1.00 127 up osd.16
39 archive 10.91399 1.00000 11 TiB 8.1 TiB 8.1 TiB 3.4 MiB 14 GiB 2.8
TiB 74.26 0.93 118 up osd.39
40 archive 10.91399 1.00000 11 TiB 9.2 TiB 9.2 TiB 53 KiB 16 GiB 1.7
TiB 84.53 1.05 132 up osd.40
44 archive 10.91399 1.00000 11 TiB 8.1 TiB 8.1 TiB 2.6 MiB 15 GiB 2.8
TiB 74.40 0.93 117 up osd.44
48 archive 10.91399 1.00000 11 TiB 9.7 TiB 9.7 TiB 44 KiB 17 GiB 1.2
TiB 89.02 1.11 135 up osd.48
49 archive 10.91399 1.00000 11 TiB 8.6 TiB 8.6 TiB 132 KiB 15 GiB 2.3
TiB 78.90 0.98 126 up osd.49
52 archive 10.91399 1.00000 11 TiB 9.3 TiB 9.3 TiB 28 KiB 17 GiB 1.6
TiB 85.28 1.06 134 up osd.52
77 archive 10.91399 1.00000 11 TiB 8.1 TiB 8.1 TiB 73 KiB 15 GiB 2.8
TiB 74.44 0.93 118 up osd.77
89 archive 10.91399 1.00000 11 TiB 7.2 TiB 7.2 TiB 60 KiB 13 GiB 3.7
TiB 66.22 0.83 106 up osd.89
90 archive 10.91399 1.00000 11 TiB 9.4 TiB 9.3 TiB 48 KiB 16 GiB 1.6
TiB 85.68 1.07 137 up osd.90
-31 130.96783 - 131 TiB 107 TiB 107 TiB 12 MiB 189 GiB 24
TiB 81.86 1.02 - host CEPH008
5 archive 10.91399 1.00000 11 TiB 9.6 TiB 9.6 TiB 2.7 MiB 17 GiB 1.3
TiB 87.81 1.10 135 up osd.5
6 archive 10.91399 1.00000 11 TiB 8.4 TiB 8.4 TiB 3.9 MiB 16 GiB 2.5
TiB 77.19 0.96 124 up osd.6
11 archive 10.91399 1.00000 11 TiB 8.9 TiB 8.8 TiB 48 KiB 16 GiB 2.1
TiB 81.11 1.01 128 up osd.11
45 archive 10.91399 1.00000 11 TiB 9.5 TiB 9.4 TiB 48 KiB 17 GiB 1.5
TiB 86.66 1.08 138 up osd.45
46 archive 10.91399 1.00000 11 TiB 8.9 TiB 8.8 TiB 4.8 MiB 16 GiB 2.1
TiB 81.21 1.01 126 up osd.46
47 archive 10.91399 1.00000 11 TiB 8.4 TiB 8.4 TiB 32 KiB 15 GiB 2.5
TiB 77.22 0.96 118 up osd.47
55 archive 10.91399 1.00000 11 TiB 9.0 TiB 9.0 TiB 28 KiB 16 GiB 1.9
TiB 82.33 1.03 125 up osd.55
70 archive 10.91399 1.00000 11 TiB 9.0 TiB 9.0 TiB 80 KiB 16 GiB 1.9
TiB 82.20 1.03 132 up osd.70
71 archive 10.91399 1.00000 11 TiB 8.2 TiB 8.2 TiB 32 KiB 15 GiB 2.7
TiB 75.47 0.94 118 up osd.71
78 archive 10.91399 1.00000 11 TiB 8.8 TiB 8.8 TiB 61 KiB 15 GiB 2.1
TiB 80.40 1.00 119 up osd.78
79 archive 10.91399 1.00000 11 TiB 9.7 TiB 9.7 TiB 49 KiB 17 GiB 1.2
TiB 89.03 1.11 134 up osd.79
91 archive 10.91399 1.00000 11 TiB 8.9 TiB 8.9 TiB 24 KiB 16 GiB 2.0
TiB 81.69 1.02 130 up osd.91
Regards,
Manuel
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
