On Tue, Aug 27, 2013 at 5:29 PM, Sage Weil <s...@inktank.com> wrote:

> On Tue, 27 Aug 2013, Francesc Alted wrote:
> > Hi again,
> >
> > I continue to try debugging the problem reported before.  Now, I have
> been
> > trying to use a couple of VM for doing this (one with Ubuntu 12.04
> 64-bit,
> > and the other with Ubuntu 12.10 64-bit, and I use the ceph.com repos for
> > installing the Ceph libraries).  And, unfortunately, I am getting into
> the
> > same problem: the keyring do not appear where they should (i.e.
> > bootstrap-mds and bootstrap-osd in /var/lib/ceph).
> >
> > I have followed the preflight check list
> > (http://ceph.com/docs/next/start/quick-start-preflight/), and the ceph
> user
> > in the admin box can login perfectly well on the server box, so not sure
> > what's going on here.
> >
> > I have even tried to use a single ceph server for installing everything
> > (adding the 'osd crush chooseleaf type = 0' line into the ceph conf file)
> > but then again the keyrings do not appear.
> >
> > Nobody is having the same problems than me (using latest Ceph Dumpling
> > 0.67.2 release here)?
> >
> > Thanks for any insight!
> There are several possible pitfalls here; the missing keys are just the
> most visible symptom of the monitors not forming an initial quorum.
> Can you post the contents of your ceph.conf and output from 'ceph daemon
> mon.`hostnam` mon-status' on each of the mon nodes?

Okay, I tracked down my problem.  It turned out that I was setting
different names for the ceph servers in /etc/hosts than their own
`hostname`.  These log lines when creating the monitor gave me the clue:

[ceph-server2][INFO  ] creating keyring file:
[ceph-server2][INFO  ] create the monitor keyring file
[ceph-server2][INFO  ] Running command: ceph-mon --cluster ceph --mkfs -i
vagrant --keyring /var/lib/ceph/tmp/ceph-vagrant.mon.keyring
[ceph-server2][INFO  ] ceph-mon: mon.noname-a is
local, renaming to mon.vagrant
[ceph-server2][INFO  ] ceph-mon: set fsid to

I was calling this 'vagrant' hostname 'ceph-server2' in my /etc/hosts and I
realized this was fooling cephs.  So I changed all my /etc/hosts to follow
the original hostnames (changed to 'quantal64'), and pum! everything works
as intended:

[quantal64][INFO  ] creating keyring file:
[quantal64][INFO  ] create the monitor keyring file
[quantal64][INFO  ] Running command: ceph-mon --cluster ceph --mkfs -i
quantal64 --keyring /var/lib/ceph/tmp/ceph-quantal64.mon.keyring
[quantal64][INFO  ] ceph-mon: mon.noname-a is local,
renaming to mon.quantal64
[quantal64][INFO  ] ceph-mon: set fsid to

Now I can gather the keys normally:

$ ceph-deploy gatherkeys quantal64
[ceph_deploy.gatherkeys][DEBUG ] Checking quantal64 for
[ceph_deploy.gatherkeys][DEBUG ] Got ceph.client.admin.keyring key from
[ceph_deploy.gatherkeys][DEBUG ] Have ceph.mon.keyring
[ceph_deploy.gatherkeys][DEBUG ] Checking quantal64 for
[ceph_deploy.gatherkeys][DEBUG ] Got ceph.bootstrap-osd.keyring key from
[ceph_deploy.gatherkeys][DEBUG ] Checking quantal64 for
[ceph_deploy.gatherkeys][DEBUG ] Got ceph.bootstrap-mds.keyring key from

Well, thanks anyways.  Now it is time to make some more progress and create
some ODSs :)

Francesc Alted
ceph-users mailing list

Reply via email to