Thanks Sam, I'll take a look. Seems sensible enough and worth a shot.
We'll probably call it a day after this and flatten in, but I'm
wondering if it's possible some rbd devices may miss these pg's and
could be exportable? Will have a tinker!
On Wed, Mar 11, 2015 at 7:06 PM, Samuel Just
For each of those pgs, you'll need to identify the pg copy you want to
be the winner and either
1) Remove all of the other ones using ceph-objectstore-tool and
hopefully the winner you left alone will allow the pg to recover and go
active.
2) Export the winner using ceph-objectstore-tool, use
For clarity too, I've tried to drop the min_size before as suggested,
doesn't make a difference unfortunately
On Wed, Mar 11, 2015 at 9:50 AM, joel.merr...@gmail.com
joel.merr...@gmail.com wrote:
Sure thing, n.b. I increased pg count to see if it would help. Alas not. :)
Thanks again!
Ok, you lost all copies from an interval where the pgs went active. The
recovery from this is going to be complicated and fragile. Are the
pools valuable?
-Sam
On 03/11/2015 03:35 AM, joel.merr...@gmail.com wrote:
For clarity too, I've tried to drop the min_size before as suggested,
doesn't
What do you mean by unblocked but still stuck?
-Sam
On Mon, 2015-03-09 at 22:54 +, joel.merr...@gmail.com wrote:
On Mon, Mar 9, 2015 at 2:28 PM, Samuel Just sj...@redhat.com wrote:
You'll probably have to recreate osds with the same ids (empty ones),
let them boot, stop them, and mark
Stuck unclean and stuck inactive. I can fire up a full query and
health dump somewhere useful if you want (full pg query info on ones
listed in health detail, tree, osd dump etc). There were blocked_by
operations that no longer exist after doing the OSD addition.
Side note, spent some time
Yeah, get a ceph pg query on one of the stuck ones.
-Sam
On Tue, 2015-03-10 at 14:41 +, joel.merr...@gmail.com wrote:
Stuck unclean and stuck inactive. I can fire up a full query and
health dump somewhere useful if you want (full pg query info on ones
listed in health detail, tree, osd
You'll probably have to recreate osds with the same ids (empty ones),
let them boot, stop them, and mark them lost. There is a feature in the
tracker to improve this behavior: http://tracker.ceph.com/issues/10976
-Sam
On Mon, 2015-03-09 at 12:24 +, joel.merr...@gmail.com wrote:
Hi,
I'm
On Mon, Mar 9, 2015 at 2:28 PM, Samuel Just sj...@redhat.com wrote:
You'll probably have to recreate osds with the same ids (empty ones),
let them boot, stop them, and mark them lost. There is a feature in the
tracker to improve this behavior: http://tracker.ceph.com/issues/10976
-Sam
Thanks
