I tried to remove the whole image as i didn't need it.
But it seems it doesn't get cleared: 106dd406b8b4567 was the id of the
old deleted rbd image.
ceph-57]# find . -name "*106dd406b8b4567*" -exec ls -la "{}" \;
-rw-r--r-- 1 ceph ceph 4194304 Aug 5 09:40
./current/3.61a_head/DIR_A/DIR_1/DIR_6/DIR_8/rbd\udata.106dd406b8b4567.0000000000002315__9d5e4_9E65861A__3
-rw-r--r-- 1 ceph ceph 4194304 Aug 5 09:40
./current/3.61a_head/DIR_A/DIR_1/DIR_6/DIR_8/rbd\udata.106dd406b8b4567.0000000000002315__9d84a_9E65861A__3
-rw-r--r-- 1 ceph ceph 0 Aug 5 11:47
./current/3.61a_head/DIR_A/DIR_1/DIR_6/DIR_8/rbd\udata.106dd406b8b4567.0000000000002315__snapdir_9E65861A__3
-rw-r--r-- 1 ceph ceph 4194304 Aug 5 11:50
./current/3.61a_head/DIR_A/DIR_1/DIR_6/DIR_A/rbd\udata.106dd406b8b4567.000000000000018c__9d455_BCB2A61A__3
-rw-r--r-- 1 ceph ceph 1400832 Aug 5 09:40
./current/3.61a_head/DIR_A/DIR_1/DIR_6/DIR_A/rbd\udata.106dd406b8b4567.000000000000018c__9d5e4_BCB2A61A__3
-rw-r--r-- 1 ceph ceph 1400832 Aug 5 09:40
./current/3.61a_head/DIR_A/DIR_1/DIR_6/DIR_A/rbd\udata.106dd406b8b4567.000000000000018c__9d84a_BCB2A61A__3
-rw-r--r-- 1 ceph ceph 0 Aug 5 11:47
./current/3.61a_head/DIR_A/DIR_1/DIR_6/DIR_A/rbd\udata.106dd406b8b4567.000000000000018c__snapdir_BCB2A61A__3
Stefan
Am 05.08.2017 um 11:02 schrieb Stefan Priebe - Profihost AG:
> Is there a way to remove that object from all osds? As it is unexpected
> it should not harm.
>
> Greets,
> Stefan
>
> Am 05.08.2017 um 09:03 schrieb Stefan Priebe - Profihost AG:
>> Hello,
>>
>> i'm trying to fix a cluster where one pg is in
>> active+clean+inconsistent+snaptrim
>>
>> state.
>>
>> The log says:
>> 2017-08-05 08:57:43.240030 osd.20 [ERR] 3.61a repair 0 missing, 1
>> inconsistent objects
>> 2017-08-05 08:57:43.240044 osd.20 [ERR] 3.61a repair 4 errors, 2 fixed
>> 2017-08-05 08:57:43.242828 osd.20 [ERR] trim_object Snap 9d455 not in clones
>>
>> find says:
>>
>> osd.20]# find . -name "*9d455*" -exec ls -la "{}" \;
>> -rw-r--r-- 1 ceph ceph 0 Aug 5 08:57
>> ./current/3.61a_head/DIR_A/DIR_1/DIR_6/DIR_A/rbd\udata.106dd406b8b4567.000000000000018c__9d455_BCB2A61A__3
>>
>> find . -name "*9d455*" -exec ls -la "{}" \;
>> osd.70]# -rw-r--r-- 1 ceph ceph 4194304 Aug 5 08:57
>> ./current/3.61a_head/DIR_A/DIR_1/DIR_6/DIR_A/rbd\udata.106dd406b8b4567.000000000000018c__9d455_BCB2A61A__3
>>
>> osd.57]# find . -name "*9d455*" -exec ls -la "{}" \;
>> # no file found
>>
>> I tried the following stuff to solve this issue.
>>
>> stop osd.y
>> copy file from osd.x to osd.y
>> start osd.y
>>
>> But all that does not help. It stays in this state.
>>
>> The output of a ceph pg repair is:
>> 2017-08-05 08:57:25.715812 osd.20 [ERR] 3.61a shard 20: soid
>> 3:58654d3d:::rbd_data.106dd406b8b4567.000000000000018c:9d455 data_digest
>> 0xffffffff != data_digest 0xbd9585ca from auth oi
>> 3:58654d3d:::rbd_data.106dd406b8b4567.000000000000018c:9d455(818765'26612094
>> osd.20.0:562819 [9d455] dirty|data_digest|omap_digest s 4194304 uv
>> 26585239 dd bd9585ca od ffffffff alloc_hint [0 0]), size 0 != size
>> 4194304 from auth oi
>> 3:58654d3d:::rbd_data.106dd406b8b4567.000000000000018c:9d455(818765'26612094
>> osd.20.0:562819 [9d455] dirty|data_digest|omap_digest s 4194304 uv
>> 26585239 dd bd9585ca od ffffffff alloc_hint [0 0])
>> 2017-08-05 08:57:25.715817 osd.20 [ERR] 3.61a shard 70: soid
>> 3:58654d3d:::rbd_data.106dd406b8b4567.000000000000018c:9d455 data_digest
>> 0x43d61c5d != data_digest 0xffffffff from shard 20, data_digest
>> 0x43d61c5d != data_digest 0xbd9585ca from auth oi
>> 3:58654d3d:::rbd_data.106dd406b8b4567.000000000000018c:9d455(818765'26612094
>> osd.20.0:562819 [9d455] dirty|data_digest|omap_digest s 4194304 uv
>> 26585239 dd bd9585ca od ffffffff alloc_hint [0 0]), size 4194304 != size
>> 0 from shard 20
>> 2017-08-05 08:57:25.715903 osd.20 [ERR] repair 3.61a
>> 3:58654d3d:::rbd_data.106dd406b8b4567.000000000000018c:9d455 is an
>> unexpected clone
>>
>> How to get this fixed?
>>
>> Greets,
>> Stefan
>> _______________________________________________
>> ceph-users mailing list
>> ceph-users@lists.ceph.com
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>
> _______________________________________________
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
