Re: [ceph-users] corrupt OSD: BlueFS.cc: 828: FAILED assert

2018-07-05 Thread Jake Grimmett
Hi Igor, Many thanks for the quick reply. Your advice concurs with my own thoughts, given the damage, probably safest to wipe the OSD's and start over. thanks again, Jake On 05/07/18 14:28, Igor Fedotov wrote: > Hi Jake, > > IMO it doesn't make sense to recover from this drive/data as the da

Re: [ceph-users] corrupt OSD: BlueFS.cc: 828: FAILED assert

2018-07-05 Thread Igor Fedotov
Hi Jake, IMO it doesn't make sense to recover from this drive/data as the damage coverage looks pretty wide. By modifying BlueFS code you can bypass that specific assertion but most probably BlueFS and  other BlueStore stuff are pretty inconsistent and most probably are unrecoverable at the

[ceph-users] corrupt OSD: BlueFS.cc: 828: FAILED assert

2018-07-05 Thread Jake Grimmett
Dear All, I have a Mimic (13.2.0) cluster, which, due to a bad disk controller, corrupted three Bluestore OSD's on one node. Unfortunately these three OSD's crash when they try to start. systemctl start ceph-osd@193 (snip) /BlueFS.cc: 828: FAILED assert(r != q->second->file_map.end()) Full log