Re: [ceph-users] mds isn't working anymore after osd's running full

2014-11-12 Thread Jasper Siero
. The problem is solved now :) Thanks! Jasper Van: Gregory Farnum [g...@gregs42.com] Verzonden: dinsdag 11 november 2014 19:19 Aan: Jasper Siero CC: ceph-users Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full On Tue, Nov 11, 2014

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-11-11 Thread Jasper Siero
[g...@gregs42.com] Verzonden: maandag 10 november 2014 18:24 Aan: Jasper Siero CC: ceph-users; John Spray Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full It's supposed to do that; deep scrubbing is an ongoing consistency-check mechanism. If you really want to disable

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-11-11 Thread Gregory Farnum
On Tue, Nov 11, 2014 at 5:06 AM, Jasper Siero jasper.si...@target-holding.nl wrote: No problem thanks for helping. I don't want to disable the deep scrubbing process itself because its very useful but one placement group (3.30) is continuously deep scrubbing and it should finish after some

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-11-10 Thread Jasper Siero
7 november 2014 22:42 Aan: Jasper Siero CC: ceph-users; John Spray Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full On Thu, Nov 6, 2014 at 11:49 AM, John Spray john.sp...@redhat.com wrote: This is still an issue on master, so a fix will be coming soon. Follow

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-11-10 Thread Jasper Siero
Van: Gregory Farnum [g...@gregs42.com] Verzonden: vrijdag 7 november 2014 22:42 Aan: Jasper Siero CC: ceph-users; John Spray Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full On Thu, Nov 6, 2014 at 11:49 AM, John Spray john.sp

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-11-10 Thread Gregory Farnum
isn't working anymore after osd's running full On Wed, Oct 29, 2014 at 7:51 AM, Jasper Siero jasper.si...@target-holding.nl wrote: Hello Greg, I added the debug options which you mentioned and started the process again: [root@th1-mon001 ~]# /usr/bin/ceph-mds -i th1-mon001 --pid-file /var

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-11-07 Thread Gregory Farnum
Van: gregory.far...@inktank.com [gregory.far...@inktank.com] namens Gregory Farnum [gfar...@redhat.com] Verzonden: donderdag 30 oktober 2014 1:03 Aan: Jasper Siero CC: John Spray; ceph-users Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-11-06 Thread John Spray
...@inktank.com [gregory.far...@inktank.com] namens Gregory Farnum [gfar...@redhat.com] Verzonden: donderdag 30 oktober 2014 1:03 Aan: Jasper Siero CC: John Spray; ceph-users Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full On Wed, Oct 29, 2014 at 7:51 AM, Jasper

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-11-06 Thread John Spray
Gregory Farnum [gfar...@redhat.com] Verzonden: donderdag 30 oktober 2014 1:03 Aan: Jasper Siero CC: John Spray; ceph-users Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full On Wed, Oct 29, 2014 at 7:51 AM, Jasper Siero jasper.si...@target-holding.nl wrote: Hello

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-11-03 Thread Jasper Siero
...@inktank.com [gregory.far...@inktank.com] namens Gregory Farnum [gfar...@redhat.com] Verzonden: donderdag 30 oktober 2014 1:03 Aan: Jasper Siero CC: John Spray; ceph-users Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full On Wed, Oct 29, 2014 at 7:51 AM, Jasper Siero

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-10-30 Thread Jasper Siero
...@inktank.com [gregory.far...@inktank.com] namens Gregory Farnum [gfar...@redhat.com] Verzonden: donderdag 30 oktober 2014 1:03 Aan: Jasper Siero CC: John Spray; ceph-users Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full On Wed, Oct 29, 2014 at 7:51 AM, Jasper Siero

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-10-29 Thread Jasper Siero
...@inktank.com [gregory.far...@inktank.com] namens Gregory Farnum [gfar...@redhat.com] Verzonden: dinsdag 28 oktober 2014 19:26 Aan: Jasper Siero CC: John Spray; ceph-users Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full You'll need to gather a log with the offsets visible

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-10-29 Thread Gregory Farnum
On Wed, Oct 29, 2014 at 7:51 AM, Jasper Siero jasper.si...@target-holding.nl wrote: Hello Greg, I added the debug options which you mentioned and started the process again: [root@th1-mon001 ~]# /usr/bin/ceph-mds -i th1-mon001 --pid-file /var/run/ceph/mds.th1-mon001.pid -c

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-10-28 Thread Gregory Farnum
Van: john.sp...@inktank.com [john.sp...@inktank.com] namens John Spray [john.sp...@redhat.com] Verzonden: donderdag 16 oktober 2014 12:23 Aan: Jasper Siero CC: Gregory Farnum; ceph-users Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-10-24 Thread Jasper Siero
this. Aborted Jasper Van: Gregory Farnum [g...@inktank.com] Verzonden: dinsdag 14 oktober 2014 23:40 Aan: Jasper Siero CC: ceph-users Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full ceph-mds --undump-journal rank

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-10-16 Thread Jasper Siero
Farnum; ceph-users Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full Following up: firefly fix for undump is: https://github.com/ceph/ceph/pull/2734 Jasper: if you still need to try undumping on this existing firefly cluster, then you can download ceph-mds packages

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-10-16 Thread John Spray
Van: Gregory Farnum [g...@inktank.com] Verzonden: dinsdag 14 oktober 2014 23:40 Aan: Jasper Siero CC: ceph-users Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full ceph-mds --undump-journal rank journal-file Looks like it accidentally

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-10-15 Thread Jasper Siero
this. Aborted Jasper Van: Gregory Farnum [g...@inktank.com] Verzonden: dinsdag 14 oktober 2014 23:40 Aan: Jasper Siero CC: ceph-users Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full ceph-mds --undump-journal rank journal-file Looks like

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-10-15 Thread John Spray
23:40 Aan: Jasper Siero CC: ceph-users Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full ceph-mds --undump-journal rank journal-file Looks like it accidentally (or on purpose? you can break things with it) got left out of the help text. On Tue, Oct 14, 2014

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-10-14 Thread Jasper Siero
Van: Gregory Farnum [g...@inktank.com] Verzonden: vrijdag 10 oktober 2014 23:45 Aan: Jasper Siero CC: ceph-users Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full Ugh, debug journaler, not debug journaled. That said, the filer output tells me

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-10-14 Thread Gregory Farnum
: Gregory Farnum [g...@inktank.com] Verzonden: vrijdag 10 oktober 2014 23:45 Aan: Jasper Siero CC: ceph-users Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full Ugh, debug journaler, not debug journaled. That said, the filer output tells me that you're missing

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-10-10 Thread Gregory Farnum
oktober 2014 19:45 Aan: Jasper Siero CC: ceph-users Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full Sorry; I guess this fell off my radar. The issue here is not that it's waiting for an osdmap; it got the requested map and went into replay mode almost

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-10-07 Thread Gregory Farnum
16:43 Aan: Gregory Farnum Onderwerp: RE: [ceph-users] mds isn't working anymore after osd's running full I did restart it but you are right about the epoch number which has changed but the situation looks the same. 2014-08-21 16:33:06.032366 7f9b5f3cd700 1 mds.0.27 need osdmap epoch 1994

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-08-20 Thread Jasper Siero
Van: Gregory Farnum [g...@inktank.com] Verzonden: dinsdag 19 augustus 2014 19:49 Aan: Jasper Siero CC: ceph-users@lists.ceph.com Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full On Mon, Aug 18, 2014 at 6:56 AM, Jasper Siero jasper.si...@target-holding.nl

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-08-20 Thread Gregory Farnum
Onderwerp: Re: [ceph-users] mds isn't working anymore after osd's running full On Mon, Aug 18, 2014 at 6:56 AM, Jasper Siero jasper.si...@target-holding.nl wrote: Hi all, We have a small ceph cluster running version 0.80.1 with cephfs on five nodes. Last week some osd's were full and shut itself

Re: [ceph-users] mds isn't working anymore after osd's running full

2014-08-19 Thread Gregory Farnum
On Mon, Aug 18, 2014 at 6:56 AM, Jasper Siero jasper.si...@target-holding.nl wrote: Hi all, We have a small ceph cluster running version 0.80.1 with cephfs on five nodes. Last week some osd's were full and shut itself down. To help de osd's start again I added some extra osd's and moved some

[ceph-users] mds isn't working anymore after osd's running full

2014-08-18 Thread Jasper Siero
Hi all, We have a small ceph cluster running version 0.80.1 with cephfs on five nodes. Last week some osd's were full and shut itself down. To help de osd's start again I added some extra osd's and moved some placement group directories on the full osd's (which has a copy on another osd) to