I am trying to deploy a cluster with ceph-deploy. I installed ceph
0.72.2 from the rpm repositories. Running "ceph-deploy mon
create-initial" creates /var/lib/ceph etc. on all the nodes, but on
all nodes I get a warning:
[hvrrzceph2][DEBUG ] Starting Ceph mon.hvrrzceph2 on hvrrzceph2...
[hvrrzceph2][DEBUG ] Starting ceph-create-keys on hvrrzceph2...
[hvrrzceph2][WARNING] No data was received after 7 seconds,
disconnecting...
and afterwards, ceph -s cannot connect to the cluster. No
client.admin keyring is created in ceph.conf.
Then I attempted to create one monitor only on one node. Again, no
keys for client.admin, bootstrap-mds and bootstrap-osd were created.
Here is the complete log of this attempt:
hvrrzceph1:~/my-cluster # ceph-deploy new hvrrzceph1
[ceph_deploy.cli][INFO ] Invoked (1.4.0): /usr/bin/ceph-deploy new
hvrrzceph1
[ceph_deploy.new][DEBUG ] Creating new cluster named ceph
[ceph_deploy.new][DEBUG ] Resolving host hvrrzceph1
[ceph_deploy.new][DEBUG ] Monitor hvrrzceph1 at 10.1.1.239
[ceph_deploy.new][INFO ] making sure passwordless SSH succeeds
[ceph_deploy.new][DEBUG ] Monitor initial members are ['hvrrzceph1']
[ceph_deploy.new][DEBUG ] Monitor addrs are ['10.1.1.239']
[ceph_deploy.new][DEBUG ] Creating a random mon key...
[ceph_deploy.new][DEBUG ] Writing initial config to ceph.conf...
[ceph_deploy.new][DEBUG ] Writing monitor keyring to
ceph.mon.keyring...
hvrrzceph1:~/my-cluster # vi ceph.conf
[added "cluster network" + "public network"; leaving them away doesnt
change anything]
hvrrzceph1:~/my-cluster # cat ceph.conf
[global]
auth_service_required = cephx
filestore_xattr_use_omap = true
auth_client_required = cephx
auth_cluster_required = cephx
mon_host = 10.1.1.239
mon_initial_members = hvrrzceph1
fsid = ec73a230-f645-456f-9523-9a03621d18dd
cluster_network = 10.112.0.0/16
public_network = 10.111.0.0/16
hvrrzceph1:~/my-cluster # ceph-deploy mon create-initial
[ceph_deploy.cli][INFO ] Invoked (1.4.0): /usr/bin/ceph-deploy mon
create-initial
[ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts
hvrrzceph1
[ceph_deploy.mon][DEBUG ] detecting platform for host hvrrzceph1 ...
[hvrrzceph1][DEBUG ] connected to host: hvrrzceph1
[hvrrzceph1][DEBUG ] detect platform information from remote host
[hvrrzceph1][DEBUG ] detect machine type
[ceph_deploy.mon][INFO ] distro info: SUSE Linux Enterprise Server
11 x86_64
[hvrrzceph1][DEBUG ] determining if provided host has same hostname
in remote
[hvrrzceph1][DEBUG ] get remote short hostname
[hvrrzceph1][DEBUG ] deploying mon to hvrrzceph1
[hvrrzceph1][DEBUG ] get remote short hostname
[hvrrzceph1][DEBUG ] remote hostname: hvrrzceph1
[hvrrzceph1][DEBUG ] write cluster configuration to
/etc/ceph/{cluster}.conf
[hvrrzceph1][DEBUG ] create the mon path if it does not exist
[hvrrzceph1][DEBUG ] checking for done path:
/var/lib/ceph/mon/ceph-hvrrzceph1/done
[hvrrzceph1][DEBUG ] done path does not exist:
/var/lib/ceph/mon/ceph-hvrrzceph1/done
[hvrrzceph1][INFO ] creating keyring file:
/var/lib/ceph/tmp/ceph-hvrrzceph1.mon.keyring
[hvrrzceph1][DEBUG ] create the monitor keyring file
[hvrrzceph1][INFO ] Running command: ceph-mon --cluster ceph --mkfs
-i hvrrzceph1 --keyring /var/lib/ceph/tmp/ceph-hvrrzceph1.mon.keyring
[hvrrzceph1][DEBUG ] ceph-mon: set fsid to
ec73a230-f645-456f-9523-9a03621d18dd
[hvrrzceph1][DEBUG ] ceph-mon: created monfs at
/var/lib/ceph/mon/ceph-hvrrzceph1 for mon.hvrrzceph1
[hvrrzceph1][INFO ] unlinking keyring file
/var/lib/ceph/tmp/ceph-hvrrzceph1.mon.keyring
[hvrrzceph1][DEBUG ] create a done file to avoid re-doing the mon
deployment
[hvrrzceph1][DEBUG ] create the init path if it does not exist
[hvrrzceph1][INFO ] Running command: rcceph -c /etc/ceph/ceph.conf
start mon.hvrrzceph1
[hvrrzceph1][DEBUG ] === mon.hvrrzceph1 ===
[hvrrzceph1][DEBUG ] Starting Ceph mon.hvrrzceph1 on hvrrzceph1...
[hvrrzceph1][DEBUG ] Starting ceph-create-keys on hvrrzceph1...
[hvrrzceph1][WARNIN] No data was received after 7 seconds,
disconnecting...
[hvrrzceph1][INFO ] Running command: ceph --cluster=ceph
--admin-daemon /var/run/ceph/ceph-mon.hvrrzceph1.asok mon_status
[hvrrzceph1][DEBUG ]
**********************************************************************
**********
[hvrrzceph1][DEBUG ] status for monitor: mon.hvrrzceph1
[hvrrzceph1][DEBUG ] {
[hvrrzceph1][DEBUG ] "election_epoch": 2,
[hvrrzceph1][DEBUG ] "extra_probe_peers": [
[hvrrzceph1][DEBUG ] "10.1.1.239:6789/0"
[hvrrzceph1][DEBUG ] ],
[hvrrzceph1][DEBUG ] "monmap": {
[hvrrzceph1][DEBUG ] "created": "0.000000",
[hvrrzceph1][DEBUG ] "epoch": 1,
[hvrrzceph1][DEBUG ] "fsid":
"ec73a230-f645-456f-9523-9a03621d18dd",
[hvrrzceph1][DEBUG ] "modified": "0.000000",
[hvrrzceph1][DEBUG ] "mons": [
[hvrrzceph1][DEBUG ] {
[hvrrzceph1][DEBUG ] "addr": "10.111.3.1:6789/0",
[hvrrzceph1][DEBUG ] "name": "hvrrzceph1",
[hvrrzceph1][DEBUG ] "rank": 0
[hvrrzceph1][DEBUG ] }
[hvrrzceph1][DEBUG ] ]
[hvrrzceph1][DEBUG ] },
[hvrrzceph1][DEBUG ] "name": "hvrrzceph1",
[hvrrzceph1][DEBUG ] "outside_quorum": [],
[hvrrzceph1][DEBUG ] "quorum": [
[hvrrzceph1][DEBUG ] 0
[hvrrzceph1][DEBUG ] ],
[hvrrzceph1][DEBUG ] "rank": 0,
[hvrrzceph1][DEBUG ] "state": "leader",
[hvrrzceph1][DEBUG ] "sync_provider": []
[hvrrzceph1][DEBUG ] }
[hvrrzceph1][DEBUG ]
**********************************************************************
**********
[hvrrzceph1][INFO ] monitor: mon.hvrrzceph1 is running
[hvrrzceph1][INFO ] Running command: ceph --cluster=ceph
--admin-daemon /var/run/ceph/ceph-mon.hvrrzceph1.asok mon_status
[ceph_deploy.mon][INFO ] processing monitor mon.hvrrzceph1
[hvrrzceph1][DEBUG ] connected to host: hvrrzceph1
[hvrrzceph1][INFO ] Running command: ceph --cluster=ceph
--admin-daemon /var/run/ceph/ceph-mon.hvrrzceph1.asok mon_status
[ceph_deploy.mon][INFO ] mon.hvrrzceph1 monitor has reached quorum!
[ceph_deploy.mon][INFO ] all initial monitors are running and have
formed quorum
[ceph_deploy.mon][INFO ] Running gatherkeys...
[ceph_deploy.gatherkeys][DEBUG ] Checking hvrrzceph1 for
/etc/ceph/ceph.client.admin.keyring
[hvrrzceph1][DEBUG ] connected to host: hvrrzceph1
[hvrrzceph1][DEBUG ] detect platform information from remote host
[hvrrzceph1][DEBUG ] detect machine type
[hvrrzceph1][DEBUG ] fetch remote file
[ceph_deploy.gatherkeys][WARNIN] Unable to find
/etc/ceph/ceph.client.admin.keyring on ['hvrrzceph1']
[ceph_deploy.gatherkeys][DEBUG ] Have ceph.mon.keyring
[ceph_deploy.gatherkeys][DEBUG ] Checking hvrrzceph1 for
/var/lib/ceph/bootstrap-osd/ceph.keyring
[hvrrzceph1][DEBUG ] connected to host: hvrrzceph1
[hvrrzceph1][DEBUG ] detect platform information from remote host
[hvrrzceph1][DEBUG ] detect machine type
[hvrrzceph1][DEBUG ] fetch remote file
[ceph_deploy.gatherkeys][WARNIN] Unable to find
/var/lib/ceph/bootstrap-osd/ceph.keyring on ['hvrrzceph1']
[ceph_deploy.gatherkeys][DEBUG ] Checking hvrrzceph1 for
/var/lib/ceph/bootstrap-mds/ceph.keyring
[hvrrzceph1][DEBUG ] connected to host: hvrrzceph1
[hvrrzceph1][DEBUG ] detect platform information from remote host
[hvrrzceph1][DEBUG ] detect machine type
[hvrrzceph1][DEBUG ] fetch remote file
[ceph_deploy.gatherkeys][WARNIN] Unable to find
/var/lib/ceph/bootstrap-mds/ceph.keyring on ['hvrrzceph1']
hvrrzceph1:~/my-cluster #
I had already a running cluster, and ceph-deploy had worked ... :-(
Any hints where I can find what is missing?
TIA.
--
Diedrich Ehlerding, Fujitsu Technology Solutions GmbH,
FTS CE SC PS&IS W, Hildesheimer Str 25, D-30880 Laatzen
Fon +49 511 8489-1806, Fax -251806, Mobil +49 173 2464758
Firmenangaben: http://de.ts.fujitsu.com/imprint.html
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
