Hi,
I have a question about lost objects. I have a placement group in the
active+recovering state with a missing object. Should I wait until the
placement group is active+clean before doing anything about the missing
objects?
# ceph -v
ceph version 0.94.5 (9764da52395923e0b32908d83a9f7304401fee43)
# ceph health
HEALTH_ERR 67 pgs inconsistent; 48 pgs recovering; 48 pgs stuck unclean; 5
requests are blocked > 32 sec; recovery 24677/534737733 objects degraded
(0.005%); recovery 1011/90652026 unfound (0.001%); 5537 scrub errors; noout
flag(s) set
Whe I query a pg with a missing object, I see:
{
"offset": {
"oid": "",
"key": "",
"snapid": 0,
"hash": 0,
"max": 0,
"pool": -1,
"namespace": ""
},
"num_missing": 1,
"num_unfound": 1,
"objects": [
{
"oid": {
"oid":
"default.325674.107__shadow_PSI_COMMON\/DATA\/REFLECTANCE_DAILY\/DOTMATs\/data_MOD09GA_h10v05_500m_daily_2003.mat.2~skoVlnbFGILyQRSgfrvjDHIvE8Mm1gT.10_111",
"key": "",
"snapid": -2,
"hash": 4261306676,
"max": 0,
"pool": 70,
"namespace": ""
},
"need": "73544'201734",
"have": "0'0",
"locations": [
"322(2)"
]
}
],
"more": 0
}
Is there a way to tell if the object can be reverted or if it is a new
object that is now lost?
The recovery state of the placement group is maybe not encouraging (because
it shows the OSD are already probed--if I interpret this correctly):
"recovery_state": [
{
"name": "Started\/Primary\/Active",
"enter_time": "2016-02-05 08:25:16.558771",
"might_have_unfound": [
{
"osd": "7(3)",
"status": "already probed"
},
{
"osd": "17(5)",
"status": "already probed"
},
{
"osd": "43(1)",
"status": "already probed"
},
{
"osd": "172(4)",
"status": "already probed"
},
{
"osd": "322(2)",
"status": "already probed"
}
],
"recovery_progress": {
"backfill_targets": [],
"waiting_on_backfill": [],
"last_backfill_started": "0\/\/0\/\/-1",
"backfill_info": {
"begin": "0\/\/0\/\/-1",
"end": "0\/\/0\/\/-1",
"objects": []
},
"peer_backfill_info": [],
"backfills_in_flight": [],
"recovering": [],
"pg_backend": {
"recovery_ops": [],
"read_ops": []
}
},
"scrub": {
"scrubber.epoch_start": "143439",
"scrubber.active": 0,
"scrubber.waiting_on": 0,
"scrubber.waiting_on_whom": []
}
},
{
"name": "Started",
"enter_time": "2016-02-05 08:22:37.288281"
}
Thanks,
Jeff
--
Jeffrey McDonald, PhD
Assistant Director for HPC Operations
Minnesota Supercomputing Institute
University of Minnesota Twin Cities
599 Walter Library email: [email protected]
117 Pleasant St SE phone: +1 612 625-6905
Minneapolis, MN 55455 fax: +1 612 624-8861
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
