Hi,
I have a question about lost objects. I have a placement group in the active+recovering state with a missing object. Should I wait until the placement group is active+clean before doing anything about the missing objects? # ceph -v ceph version 0.94.5 (9764da52395923e0b32908d83a9f7304401fee43) # ceph health HEALTH_ERR 67 pgs inconsistent; 48 pgs recovering; 48 pgs stuck unclean; 5 requests are blocked > 32 sec; recovery 24677/534737733 objects degraded (0.005%); recovery 1011/90652026 unfound (0.001%); 5537 scrub errors; noout flag(s) set Whe I query a pg with a missing object, I see: { "offset": { "oid": "", "key": "", "snapid": 0, "hash": 0, "max": 0, "pool": -1, "namespace": "" }, "num_missing": 1, "num_unfound": 1, "objects": [ { "oid": { "oid": "default.325674.107__shadow_PSI_COMMON\/DATA\/REFLECTANCE_DAILY\/DOTMATs\/data_MOD09GA_h10v05_500m_daily_2003.mat.2~skoVlnbFGILyQRSgfrvjDHIvE8Mm1gT.10_111", "key": "", "snapid": -2, "hash": 4261306676, "max": 0, "pool": 70, "namespace": "" }, "need": "73544'201734", "have": "0'0", "locations": [ "322(2)" ] } ], "more": 0 } Is there a way to tell if the object can be reverted or if it is a new object that is now lost? The recovery state of the placement group is maybe not encouraging (because it shows the OSD are already probed--if I interpret this correctly): "recovery_state": [ { "name": "Started\/Primary\/Active", "enter_time": "2016-02-05 08:25:16.558771", "might_have_unfound": [ { "osd": "7(3)", "status": "already probed" }, { "osd": "17(5)", "status": "already probed" }, { "osd": "43(1)", "status": "already probed" }, { "osd": "172(4)", "status": "already probed" }, { "osd": "322(2)", "status": "already probed" } ], "recovery_progress": { "backfill_targets": [], "waiting_on_backfill": [], "last_backfill_started": "0\/\/0\/\/-1", "backfill_info": { "begin": "0\/\/0\/\/-1", "end": "0\/\/0\/\/-1", "objects": [] }, "peer_backfill_info": [], "backfills_in_flight": [], "recovering": [], "pg_backend": { "recovery_ops": [], "read_ops": [] } }, "scrub": { "scrubber.epoch_start": "143439", "scrubber.active": 0, "scrubber.waiting_on": 0, "scrubber.waiting_on_whom": [] } }, { "name": "Started", "enter_time": "2016-02-05 08:22:37.288281" } Thanks, Jeff -- Jeffrey McDonald, PhD Assistant Director for HPC Operations Minnesota Supercomputing Institute University of Minnesota Twin Cities 599 Walter Library email: jeffrey.mcdon...@msi.umn.edu 117 Pleasant St SE phone: +1 612 625-6905 Minneapolis, MN 55455 fax: +1 612 624-8861
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com