Re: [ceph-users] PGs stuck in active+clean+replay

2015-11-09 Thread Andras Pataki
Hi Greg, I’ve tested the patch below on top of the 0.94.5 hammer sources, and it works beautifully. No more active+clean+replay stuck PGs. Thanks! Andras On 10/27/15, 4:46 PM, "Andras Pataki" wrote: >Yes, this definitely sounds plausible (the

Re: [ceph-users] PGs stuck in active+clean+replay

2015-10-27 Thread Andras Pataki
Hi Greg, No, unfortunately I haven¹t found any resolution to it. We are using cephfs, the whole installation is on 0.94.4. What I did notice is that performance is extremely poor when backfilling is happening. I wonder if timeouts of some kind could cause PG¹s to get stuck in replay. I

Re: [ceph-users] PGs stuck in active+clean+replay

2015-10-27 Thread Gregory Farnum
On Tue, Oct 27, 2015 at 11:03 AM, Gregory Farnum wrote: > On Thu, Oct 22, 2015 at 3:58 PM, Andras Pataki > wrote: >> Hi ceph users, >> >> We’ve upgraded to 0.94.4 (all ceph daemons got restarted) – and are in the >> middle of doing some

Re: [ceph-users] PGs stuck in active+clean+replay

2015-10-27 Thread Gregory Farnum
On Tue, Oct 27, 2015 at 11:22 AM, Andras Pataki wrote: > Hi Greg, > > No, unfortunately I haven¹t found any resolution to it. We are using > cephfs, the whole installation is on 0.94.4. What I did notice is that > performance is extremely poor when backfilling is

Re: [ceph-users] PGs stuck in active+clean+replay

2015-10-27 Thread Gregory Farnum
On Thu, Oct 22, 2015 at 3:58 PM, Andras Pataki wrote: > Hi ceph users, > > We’ve upgraded to 0.94.4 (all ceph daemons got restarted) – and are in the > middle of doing some rebalancing due to crush changes (removing some disks). > During the rebalance, I see that

Re: [ceph-users] PGs stuck in active+clean+replay

2015-10-27 Thread Andras Pataki
Yes, this definitely sounds plausible (the peering/activating process does take a long time). At the moment I’m trying to get our cluster back to a more working state. Once everything works, I could try building a patched set of ceph processes from source (currently I’m using the pre-built

Re: [ceph-users] pgs stuck in active+clean+replay state

2014-09-25 Thread Gregory Farnum
I imagine you aren't actually using the data/metadata pool that these PGs are in, but it's a previously-reported bug we haven't identified: http://tracker.ceph.com/issues/8758 They should go away if you restart the OSDs that host them (or just remove those pools), but it's not going to hurt

Re: [ceph-users] pgs stuck in active+clean+replay state

2014-09-25 Thread Pavel V. Kaygorodov
Hi! I imagine you aren't actually using the data/metadata pool that these PGs are in, but it's a previously-reported bug we haven't identified: http://tracker.ceph.com/issues/8758 They should go away if you restart the OSDs that host them (or just remove those pools), but it's not going to