Hi,
Ceph version 12.2.13.
Ceph status show ‘Health_Warn’ nearfull osd(s) and pool(s). No data in this
nvme class pools and nvme osd’s, completely free. But meta values is so big in
nvme osds. Why does meta values on OSDs shows huge disk usage?
$ ceph health detail
HEALTH_WARN 1 nearfull osd(s); 6 pool(s) nearfull
OSD_NEARFULL 1 nearfull osd(s)
osd.49 is near full
POOL_NEARFULL 6 pool(s) nearfull
pool 'VmImages_NVMe_FdChassis_Rp3' is nearfull
pool '.rgw.root' is nearfull
pool 'default.rgw.control' is nearfull
pool 'default.rgw.meta' is nearfull
pool 'default.rgw.log' is nearfull
pool 'default.rgw.buckets.non-ec' is nearfull
--
$ ceph df --cluster ceph
GLOBAL:
SIZE AVAIL RAW USED %RAW USED
501TiB 354TiB 147TiB 29.42
POOLS:
NAME ID USED
%USED MAX AVAIL OBJECTS
VmImages_SSD_FdHost_Rp3 6 25.2TiB 33.26 50.6TiB
6990169
VmImages_NVMe_FdChassis_Rp3 8 19B 0 242GiB
2
VmImages_HDD_FdHost_Rp3 15 21.3TiB 35.33 39.0TiB
5755928
.rgw.root 16 1.77KiB
0 16.3TiB 11
default.rgw.control 17 0B
0 16.3TiB 8
default.rgw.meta 18 17.6KiB
0 16.3TiB 80
default.rgw.log 19 162B
0 16.3TiB 210
default.rgw.buckets.index 20 0B 0
50.6TiB 252
default.rgw.buckets.data 21 95.8GiB 0.24
39.0TiB 1561373
default.rgw.buckets.non-ec 22 0B 0
16.3TiB 4
default.rgw.buckets.data.ssd 25 0B 0
50.6TiB 0
$ ceph osd df tree | grep nvme
ID CLASS WEIGHT REWEIGHT SIZE USE DATA OMAP META AVAIL
%USE VAR PGS TYPE NAME
0 nvme 0.36378 1.00000 373GiB 313GiB 57.3MiB 25.8MiB 313GiB 59.0GiB
84.15 2.86 79 osd.0
7 nvme 0.36378 1.00000 373GiB 307GiB 57.4MiB 22.9MiB 307GiB 65.8GiB
82.32 2.80 70 osd.7
14 nvme 0.36378 1.00000 373GiB 313GiB 57.3MiB 25.9MiB 313GiB 59.3GiB
84.08 2.86 79 osd.14
21 nvme 0.36378 1.00000 373GiB 312GiB 57.4MiB 24.2MiB 312GiB 60.8GiB
83.68 2.85 78 osd.21
28 nvme 0.36378 1.00000 373GiB 316GiB 57.3MiB 29.8MiB 316GiB 56.4GiB
84.87 2.89 85 osd.28
35 nvme 0.36378 1.00000 373GiB 312GiB 57.4MiB 26.5MiB 312GiB 60.5GiB
83.75 2.85 80 osd.35
42 nvme 0.36378 1.00000 373GiB 311GiB 57.5MiB 25.5MiB 311GiB 61.9GiB
83.39 2.84 79 osd.42
49 nvme 0.36378 1.00000 373GiB 318GiB 57.3MiB 27.7MiB 318GiB 54.8GiB
85.28 2.90 90 osd.49
56 nvme 0.36378 1.00000 373GiB 303GiB 57.3MiB 20.2MiB 303GiB 69.5GiB
81.33 2.77 71 osd.56
63 nvme 0.36378 1.00000 373GiB 309GiB 57.4MiB 28.5MiB 309GiB 63.2GiB
83.02 2.82 82 osd.63
70 nvme 0.36378 1.00000 373GiB 312GiB 57.3MiB 26.2MiB 312GiB 60.5GiB
83.76 2.85 88 osd.70
77 nvme 0.36378 1.00000 373GiB 299GiB 57.4MiB 22.7MiB 299GiB 73.6GiB
80.24 2.73 69 osd.77
84 nvme 0.36378 1.00000 373GiB 312GiB 57.4MiB 31.7MiB 312GiB 60.2GiB
83.84 2.85 76 osd.84
91 nvme 0.36378 1.00000 373GiB 301GiB 57.3MiB 21.1MiB 301GiB 71.5GiB
80.80 2.75 76 osd.91
98 nvme 0.36378 1.00000 373GiB 285GiB 57.3MiB 15.7MiB 285GiB 87.6GiB
76.47 2.60 54 osd.98
105 nvme 0.36378 1.00000 373GiB 308GiB 57.3MiB 30.8MiB 308GiB 64.6GiB
82.67 2.81 86 osd.105
112 nvme 0.36378 1.00000 373GiB 264GiB 57.3MiB 26.0MiB 264GiB 108GiB
70.99 2.42 84 osd.112
119 nvme 0.36378 1.00000 373GiB 297GiB 57.3MiB 23.6MiB 297GiB 75.1GiB
79.85 2.72 74 osd.119
126 nvme 0.36378 1.00000 373GiB 292GiB 57.3MiB 22.8MiB 292GiB 80.2GiB
78.48 2.67 68 osd.126
133 nvme 0.36378 1.00000 373GiB 294GiB 57.3MiB 24.1MiB 294GiB 78.7GiB
78.87 2.68 70 osd.133
_______________________________________________
ceph-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
