Re: [ceph-users] All client writes block when 2 of 3 OSDs down

2015-03-26 Thread Gregory Farnum
Has the OSD actually been detected as down yet? You'll also need to set that min size on your existing pools (ceph osd pool pool set min_size 1 or similar) to change their behavior; the config option only takes effect for newly-created pools. (Thus the default.) On Thu, Mar 26, 2015 at 1:29 PM,

[ceph-users] All client writes block when 2 of 3 OSDs down

2015-03-26 Thread Lee Revell
I added the osd pool default min size = 1 to test the behavior when 2 of 3 OSDs are down, but the behavior is exactly the same as without it: when the 2nd OSD is killed, all client writes start to block and these pipe.(stuff).fault messages begin: 2015-03-26 16:08:50.775848 7fce177fe700 0

Re: [ceph-users] All client writes block when 2 of 3 OSDs down

2015-03-26 Thread Lee Revell
​Ah, thanks, got it, I wasn't thinking that mons and osds on the same node isn't a likely real world thing. You have to admit that pipe/fault log message is a bit cryptic. Thanks, Lee ___ ceph-users mailing list ceph-users@lists.ceph.com

Re: [ceph-users] All client writes block when 2 of 3 OSDs down

2015-03-26 Thread Lee Revell
On Thu, Mar 26, 2015 at 4:40 PM, Gregory Farnum g...@gregs42.com wrote: Has the OSD actually been detected as down yet? I believe it has, however I can't directly check because ceph health starts to hang when I down the second node. You'll also need to set that min size on your existing

Re: [ceph-users] All client writes block when 2 of 3 OSDs down

2015-03-26 Thread Gregory Farnum
On Thu, Mar 26, 2015 at 2:30 PM, Lee Revell rlrev...@gmail.com wrote: On Thu, Mar 26, 2015 at 4:40 PM, Gregory Farnum g...@gregs42.com wrote: Has the OSD actually been detected as down yet? I believe it has, however I can't directly check because ceph health starts to hang when I down the

Re: [ceph-users] All client writes block when 2 of 3 OSDs down

2015-03-26 Thread Steffen W Sørensen
To: Somnath Roy Cc: Lee Revell; ceph-users@lists.ceph.com Subject: Re: [ceph-users] All client writes block when 2 of 3 OSDs down On Thu, Mar 26, 2015 at 3:22 PM, Somnath Roy somnath@sandisk.com wrote: Greg, Couple of dumb question may be. 1. If you see , the clients are connecting

Re: [ceph-users] All client writes block when 2 of 3 OSDs down

2015-03-26 Thread Gregory Farnum
Message- From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Gregory Farnum Sent: Thursday, March 26, 2015 2:40 PM To: Lee Revell Cc: ceph-users@lists.ceph.com Subject: Re: [ceph-users] All client writes block when 2 of 3 OSDs down On Thu, Mar 26, 2015 at 2:30 PM, Lee

Re: [ceph-users] All client writes block when 2 of 3 OSDs down

2015-03-26 Thread Gregory Farnum
Revell; ceph-users@lists.ceph.com Subject: Re: [ceph-users] All client writes block when 2 of 3 OSDs down On Thu, Mar 26, 2015 at 3:22 PM, Somnath Roy somnath@sandisk.com wrote: Greg, Couple of dumb question may be. 1. If you see , the clients are connecting fine with two monitors

Re: [ceph-users] All client writes block when 2 of 3 OSDs down

2015-03-26 Thread Somnath Roy
: Gregory Farnum [mailto:g...@gregs42.com] Sent: Thursday, March 26, 2015 3:29 PM To: Somnath Roy Cc: Lee Revell; ceph-users@lists.ceph.com Subject: Re: [ceph-users] All client writes block when 2 of 3 OSDs down On Thu, Mar 26, 2015 at 3:22 PM, Somnath Roy somnath@sandisk.com wrote: Greg, Couple

Re: [ceph-users] All client writes block when 2 of 3 OSDs down

2015-03-26 Thread Somnath Roy
, March 26, 2015 3:29 PM To: Somnath Roy Cc: Lee Revell; ceph-users@lists.ceph.com Subject: Re: [ceph-users] All client writes block when 2 of 3 OSDs down On Thu, Mar 26, 2015 at 3:22 PM, Somnath Roy somnath@sandisk.com wrote: Greg, Couple of dumb question may be. 1. If you see

Re: [ceph-users] All client writes block when 2 of 3 OSDs down

2015-03-26 Thread Gregory Farnum
, March 26, 2015 3:41 PM To: Somnath Roy Cc: Lee Revell; ceph-users@lists.ceph.com Subject: Re: [ceph-users] All client writes block when 2 of 3 OSDs down On Thu, Mar 26, 2015 at 3:36 PM, Somnath Roy somnath@sandisk.com wrote: Got most portion of it, thanks ! But, still not able to get when

Re: [ceph-users] All client writes block when 2 of 3 OSDs down

2015-03-26 Thread Somnath Roy
-Original Message- From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Gregory Farnum Sent: Thursday, March 26, 2015 2:40 PM To: Lee Revell Cc: ceph-users@lists.ceph.com Subject: Re: [ceph-users] All client writes block when 2 of 3 OSDs down On Thu, Mar 26, 2015 at 2:30 PM