Re: [ceph-users] OSDs not coming up on one host

On Wed, Jun 15, 2016 at 10:21 AM, Kostis Fardelas wrote: > Hello Jacob, Gregory, > > did you manage to start up those OSDs at last? I came across a very > much alike incident [1] (no flags preventing the OSDs from getting UP > in the cluster though, no hardware problems

Re: [ceph-users] OSDs not coming up on one host

Hello Jacob, Gregory, did you manage to start up those OSDs at last? I came across a very much alike incident [1] (no flags preventing the OSDs from getting UP in the cluster though, no hardware problems reported) and I wonder if you found out what was the culprit in your case. [1]

Re: [ceph-users] OSDs not coming up on one host

The monitor looks like it's not generating a new OSDMap including the booting OSDs. I could say with more certainty what's going on with the monitor log file, but I'm betting you've got one of the noin or noup family of flags set. I *think* these will be output in ceph -w or in ceph osd dump,

Re: [ceph-users] OSDs not coming up on one host

On Wed, Apr 08, 2015 at 03:42:29PM +, Gregory Farnum wrote: Im on my phone so can't check exactly what those threads are trying to do, but the osd has several threads which are stuck. The FileStore threads are certainly trying to access the disk/local filesystem. You may not have a

Re: [ceph-users] OSDs not coming up on one host

On Thu, Apr 09, 2015 at 08:46:07AM -0700, Gregory Farnum wrote: On Thu, Apr 9, 2015 at 8:14 AM, Jacob Reid lists-c...@jacob-reid.co.uk wrote: On Thu, Apr 09, 2015 at 06:43:45AM -0700, Gregory Farnum wrote: You can turn up debugging (debug osd = 10 and debug filestore = 10 are probably

Re: [ceph-users] OSDs not coming up on one host

On Thu, Apr 9, 2015 at 8:14 AM, Jacob Reid lists-c...@jacob-reid.co.uk wrote: On Thu, Apr 09, 2015 at 06:43:45AM -0700, Gregory Farnum wrote: You can turn up debugging (debug osd = 10 and debug filestore = 10 are probably enough, or maybe 20 each) and see what comes out to get more information

Re: [ceph-users] OSDs not coming up on one host

You can turn up debugging (debug osd = 10 and debug filestore = 10 are probably enough, or maybe 20 each) and see what comes out to get more information about why the threads are stuck. But just from the log my answer is the same as before, and now I don't trust that controller (or maybe its

Re: [ceph-users] OSDs not coming up on one host

Im on my phone so can't check exactly what those threads are trying to do, but the osd has several threads which are stuck. The FileStore threads are certainly trying to access the disk/local filesystem. You may not have a hardware fault, but it looks like something in your stack is not behaving

[ceph-users] OSDs not coming up on one host

I have a cluster of 3 servers (recently updated from 0.80.5 to 0.80.9), each running 4-6 osds as single disks, journaled to a partition each on an SSD, with 3 mons on separate hosts. Recently, I started taking the hosts down to move disks between controllers and add extra disk capacity before