Hi Ron,
Can you try running :
ceph-deploy mon create {ceph-node}
ceph-deploy gatherkeys {ceph-node}
as two different commands, instead of running
ceph-deploy mon create-initial
and let me know if the problem still persists.
Actually it is not able to find "client.admin.keyring" file as it has
not been created yet.
Thanks
Ashish Chandra
On Fri, Feb 14, 2014 at 6:15 PM, Ron Gage <[email protected]> wrote:
> Greetings all!
> I am trying to deploy a proof of concept system for a replicated fault
> tolerant CIFS server cluster using CEPH as the back end. I am having
> significant trouble getting the CEPH part to install correctly.
> Specifically, I can't get the monitor to come up into a useful state due to
> the apparent lack of a keyring or two. Here is the background:
>
> Everything is running on a standalone VMware server with internal
> storage. The idea here is to see it work, not to see how well it performs.
>
> Current configuraton: 4 x OSD, 1 x monitor/manager
> OSD: 2 x CPU, 8 gig memory, internal only networking, 16 gig sda for
> boot/root, 160 gig sdb for CEPH store, Centos 6.5 x64 fully up to date,
> minimal install
> mon/mgr: 2 x CPU, 2 gig memory, internal and external networking, 16 gig
> sda, Centos 6.5 x64 fully up to date, minimal install
>
> Since I have made several attempts already to get this working, I execute
> the following commands from the mgr to reset everything (logged in as user
> ceph)
> ceph-deploy purgedata c1 c2 c3 c4 cm
> ceph-deploy forgetkeys
> ceph-deploy purge c1 c2 c3 c4 cm
> sudo rm -f /etc/ceph/*
> sudo rm -f /var/lib/ceph/mon/*.*
> cd ~/my-cluster
> rm -f *
>
> Now to deploy a straight forward single monitor node:
> ceph-deploy new cm
>
> And to add the public interface to the ceph.conf
> vi ceph.conf
> (add to end of global section) public network = 172.24.0.0/16
>
> Now to install the ceph software in the 5 nodes:
> ceph-deploy install cm c1 c2 c3 c4
> (waiting...waiting...waiting...)
>
> And here is where the problems begin - creating the initial monitor:
> [ceph@cm my-cluster]$ ceph-deploy mon create-initial
> [ceph_deploy.cli][INFO ] Invoked (1.3.5): /usr/bin/ceph-deploy mon
> create-initial
> [ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts cm
> [ceph_deploy.mon][DEBUG ] detecting platform for host cm ...
> [cm][DEBUG ] connected to host: cm
> [cm][DEBUG ] detect platform information from remote host
> [cm][DEBUG ] detect machine type
> [ceph_deploy.mon][INFO ] distro info: CentOS 6.5 Final
> [cm][DEBUG ] determining if provided host has same hostname in remote
> [cm][DEBUG ] get remote short hostname
> [cm][DEBUG ] deploying mon to cm
> [cm][DEBUG ] get remote short hostname
> [cm][DEBUG ] remote hostname: cm
> [cm][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
> [cm][DEBUG ] create the mon path if it does not exist
> [cm][DEBUG ] checking for done path: /var/lib/ceph/mon/ceph-cm/done
> [cm][DEBUG ] create a done file to avoid re-doing the mon deployment
> [cm][DEBUG ] create the init path if it does not exist
> [cm][DEBUG ] locating the `service` executable...
> [cm][INFO ] Running command: sudo /sbin/service ceph -c
> /etc/ceph/ceph.conf start mon.cm
> [cm][DEBUG ] === mon.cm ===
> [cm][DEBUG ] Starting Ceph mon.cm on cm...
> [cm][DEBUG ] Starting ceph-create-keys on cm...
> [cm][WARNIN] No data was received after 7 seconds, disconnecting...
> [cm][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon
> /var/run/ceph/ceph-mon.cm.asok mon_status
> [cm][DEBUG ]
> ********************************************************************************
>
> [cm][DEBUG ] status for monitor: mon.cm
> [cm][DEBUG ] {
> [cm][DEBUG ] "election_epoch": 1,
> [cm][DEBUG ] "extra_probe_peers": [],
> [cm][DEBUG ] "monmap": {
> [cm][DEBUG ] "created": "0.000000",
> [cm][DEBUG ] "epoch": 1,
> [cm][DEBUG ] "fsid": "b11875f0-e611-40b9-ae81-e7cba7586ed5",
> [cm][DEBUG ] "modified": "0.000000",
> [cm][DEBUG ] "mons": [
> [cm][DEBUG ] {
> [cm][DEBUG ] "addr": "172.24.12.91:6789/0",
> [cm][DEBUG ] "name": "cm",
> [cm][DEBUG ] "rank": 0
> [cm][DEBUG ] }
> [cm][DEBUG ] ]
> [cm][DEBUG ] },
> [cm][DEBUG ] "name": "cm",
> [cm][DEBUG ] "outside_quorum": [],
> [cm][DEBUG ] "quorum": [
> [cm][DEBUG ] 0
> [cm][DEBUG ] ],
> [cm][DEBUG ] "rank": 0,
> [cm][DEBUG ] "state": "leader",
> [cm][DEBUG ] "sync_provider": []
> [cm][DEBUG ] }
> [cm][DEBUG ]
> ********************************************************************************
>
> [cm][INFO ] monitor: mon.cm is running
> [cm][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon
> /var/run/ceph/ceph-mon.cm.asok mon_status
> [ceph_deploy.mon][INFO ] processing monitor mon.cm
> [cm][DEBUG ] connected to host: cm
> [cm][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon
> /var/run/ceph/ceph-mon.cm.asok mon_status
> [ceph_deploy.mon][INFO ] mon.cm monitor has reached quorum!
> [ceph_deploy.mon][INFO ] all initial monitors are running and have formed
> quorum
> [ceph_deploy.mon][INFO ] Running gatherkeys...
> [ceph_deploy.gatherkeys][DEBUG ] Checking cm for
> /etc/ceph/ceph.client.admin.keyring
> [cm][DEBUG ] connected to host: cm
> [cm][DEBUG ] detect platform information from remote host
> [cm][DEBUG ] detect machine type
> [cm][DEBUG ] fetch remote file
> [ceph_deploy.gatherkeys][WARNIN] Unable to find
> /etc/ceph/ceph.client.admin.keyring on ['cm']
> [ceph_deploy.gatherkeys][DEBUG ] Have ceph.mon.keyring
> [ceph_deploy.gatherkeys][DEBUG ] Checking cm for
> /var/lib/ceph/bootstrap-osd/ceph.keyring
> [cm][DEBUG ] connected to host: cm
> [cm][DEBUG ] detect platform information from remote host
> [cm][DEBUG ] detect machine type
> [cm][DEBUG ] fetch remote file
> [ceph_deploy.gatherkeys][WARNIN] Unable to find
> /var/lib/ceph/bootstrap-osd/ceph.keyring on ['cm']
> [ceph_deploy.gatherkeys][DEBUG ] Checking cm for
> /var/lib/ceph/bootstrap-mds/ceph.keyring
> [cm][DEBUG ] connected to host: cm
> [cm][DEBUG ] detect platform information from remote host
> [cm][DEBUG ] detect machine type
> [cm][DEBUG ] fetch remote file
> [ceph_deploy.gatherkeys][WARNIN] Unable to find
> /var/lib/ceph/bootstrap-mds/ceph.keyring on ['cm']
> [ceph@cm my-cluster]$
>
>
> From my untrained eye, it looks like several key files aren't either
> being created or copied to the expected place.
>
> Contents of ~/my-cluster:
> [ceph@cm my-cluster]$ ls
> ceph.conf ceph.log ceph.mon.keyring epel-release-6-8.noarch.rpm
>
> Contents of /var/lib/ceph/mon/ceph-cm:
>
> [ceph@cm ceph-cm]$ ls -R .: done keyring store.db sysvinit
>
> ./store.db: 000016.sst 000021.sst CURRENT LOG 000018.sst 000022.log
> LOCK MANIFEST-000020
>
> Contents of /etc/ceph:
>
> [ceph@cm ceph]$ ls
> ceph.client.admin.keyring.5704.tmp ceph.conf rbdmap
>
> As you can plainly see, I have followed the directions in the "quick
> start" deployment guide on ceph.com (
> http://ceph.com/docs/master/start/quick-ceph-deploy/#create-a-cluster).
>
> What am I missing? Are the directions incomplete (or wrong)?
>
>
>
> HELP!
>
>
> Ron Gage
> Westland, Michigan
>
> Please be sure to check out My Photography Website<http://www.prgstudios.com>
>
>
> _______________________________________________
> ceph-users mailing list
> [email protected]
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
>
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
