I fixed it by starting again deleting everything and using the "--skip-mon-network” option to cephadm bootstrap, I think the config was not finished before.
The logging is very verbose by default. I have reduced most of but can’t reduce cluster 2020-12-23T12:26:48.142993+0000 mgr.host1.xsqlhs (mgr.34116) 238 : cluster [DBG] pgmap v234: 1 pgs: 1 This logs every 2 seconds I tried setting "mon_cluster_log_file_level = info” which I found on the web, but this has no effect. Maybe because its logging to stderr/dout because its running in a container? > On 23 Dec 2020, at 09:16, Duncan Bellamy <[email protected]> wrote: > > Hi, > I am trying to set up a new cluster with cephadm using a docker backend. > > The initial boot strap did not finish cleanly and it errored out waiting for > the mon-ip, I used the command: > cephadm bootstrap --mon-ip 192.168.0.1 > > With 192.168.0.1 being the ip address of this first host. > > I tried the command again but it failed as the new ceph node was actually > running so it could not bind to the ports. > > After a bit of searching I was able to use "sudo cephadm shell —“ commands to > change the username and password for the dashboard and login to it. > > I then used cephadm to add a new host with "sudo cephadm shell — ceph orch > host add host2” > > Now in the inventory of the dashboard, and "ceph orch device ls” only devices > on host2 are listed not host1. > > In the Cluster/Hosts section of the dashboard host1 has its root volume drive > listed in devices, and host2 has the root volume drive and drive for the OSD > listed. > > I successfully added an OSD with a drive on host2, trying the same command > adjusted for host1 I get the following in the log: > > Dec 23 08:55:47 localhost systemd[1]: > var-lib-docker-overlay2-91e9dffa86c333353dd6b445021c852d7ce8da6237d0d4d95909d68ef3d4fe23\x2dinit-merged.mount: > Succeeded. > Dec 23 08:55:47 localhost systemd[24638]: > var-lib-docker-overlay2-91e9dffa86c333353dd6b445021c852d7ce8da6237d0d4d95909d68ef3d4fe23\x2dinit-merged.mount: > Succeeded. > Dec 23 08:55:47 localhost containerd[1470]: > time="2020-12-23T08:55:47.369773808Z" level=info msg="shim containerd-shim > started" > address=/containerd-shim/80f876072532ebebdfef341a5c793654e27766f2d1708991a6f25599b24b6557.sock > debug=false pid=28597 > Dec 23 08:55:47 localhost bash[8745]: debug 2020-12-23T08:55:47.517+0000 > ffff73d7a200 1 mon.host1@0(leader).osd e12 _set_new_cache_sizes > cache_size:1020054731 inc_alloc: 71303168 full_alloc: 71303168 kv_alloc: > 876609536 > Dec 23 08:55:47 localhost containerd[1470]: > time="2020-12-23T08:55:47.621748606Z" level=info msg="shim reaped" > id=69a786e4a61605c1e6eca5a6e0e5ed0900635a214b0f1c96a4f26ea7911a12ff > Dec 23 08:55:47 localhost dockerd[2930]: > time="2020-12-23T08:55:47.631479207Z" level=info msg="ignoring event" > module=libcontainerd namespace=moby topic=/tasks/delete > type="*events.TaskDelete" > Dec 23 08:55:47 localhost systemd[24638]: > var-lib-docker-overlay2-91e9dffa86c333353dd6b445021c852d7ce8da6237d0d4d95909d68ef3d4fe23-merged.mount: > Succeeded. > Dec 23 08:55:47 localhost systemd[1]: > var-lib-docker-overlay2-91e9dffa86c333353dd6b445021c852d7ce8da6237d0d4d95909d68ef3d4fe23-merged.mount: > Succeeded. > Dec 23 08:55:47 localhost systemd[24638]: > var-lib-docker-overlay2-64bb135bc0cdab187566992dc9870068dee1430062e1a2b484381c19e03da895\x2dinit-merged.mount: > Succeeded. > Dec 23 08:55:47 localhost systemd[1]: > var-lib-docker-overlay2-64bb135bc0cdab187566992dc9870068dee1430062e1a2b484381c19e03da895\x2dinit-merged.mount: > Succeeded. > Dec 23 08:55:47 localhost containerd[1470]: > time="2020-12-23T08:55:47.972437378Z" level=info msg="shim containerd-shim > started" > address=/containerd-shim/4a61d63e1f46722ffa7a950c31145d167c5c69087d003e5928a6aa3a4831f031.sock > debug=false pid=28659 > Dec 23 08:55:48 localhost bash[8745]: cluster 2020-12-23T08:55:46.892633+0000 > mgr.host1.kkssvi (mgr.24098) 24278 : cluster [DBG] pgmap v24212: 1 pgs: 1 > undersized+peered; 0 B data, 112 KiB used, 931 GiB / 932 GiB avail > Dec 23 08:55:48 localhost bash[8756]: debug 2020-12-23T08:55:48.889+0000 > ffff93573700 0 log_channel(cluster) log [DBG] : pgmap v24213: 1 pgs: 1 > undersized+peered; 0 B data, 112 KiB used, 931 GiB / 932 GiB avail > Dec 23 08:55:49 localhost bash[8756]: debug 2020-12-23T08:55:49.085+0000 > ffff9056f700 0 log_channel(audit) log [DBG] : from='client.24206 -' > entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": > "host1:/dev/nvme0n1", "target": ["mon-mgr", ""]}]: dispatch > Dec 23 08:55:49 localhost bash[8745]: debug 2020-12-23T08:55:49.085+0000 > ffff71575200 0 mon.host1@0(leader) e2 handle_command mon_command({"prefix": > "osd tree", "states": ["destroyed"], "format": "json"} v 0) v1 > Dec 23 08:55:49 localhost bash[8745]: debug 2020-12-23T08:55:49.085+0000 > ffff71575200 0 log_channel(audit) log [DBG] : from='mgr.24098 > 192.168.0.1:0/2486989775' entity='mgr.host1.kkssvi' cmd=[{"prefix": "osd > tree", "states": ["destroyed"], "format": "json"}]: dispatch > Dec 23 08:55:49 localhost bash[8756]: debug 2020-12-23T08:55:49.089+0000 > ffff8ed6d700 0 log_channel(cephadm) log [INF] : Found osd claims -> {} > Dec 23 08:55:49 localhost bash[8756]: debug 2020-12-23T08:55:49.089+0000 > ffff8ed6d700 0 log_channel(cephadm) log [INF] : Found osd claims for > drivegroup None -> {} > Dec 23 08:55:49 localhost containerd[1470]: > time="2020-12-23T08:55:49.331868093Z" level=info msg="shim reaped" > id=780a38dd49fce4a823c4c3d834abdd1cc17bbe0c0aa4f2dd7caeddf8dce1708e > Dec 23 08:55:49 localhost dockerd[2930]: > time="2020-12-23T08:55:49.341765820Z" level=info msg="ignoring event" > module=libcontainerd namespace=moby topic=/tasks/delete > type="*events.TaskDelete" > Dec 23 08:55:49 localhost systemd[24638]: > var-lib-docker-overlay2-64bb135bc0cdab187566992dc9870068dee1430062e1a2b484381c19e03da895-merged.mount: > Succeeded. > Dec 23 08:55:49 localhost systemd[1]: > var-lib-docker-overlay2-64bb135bc0cdab187566992dc9870068dee1430062e1a2b484381c19e03da895-merged.mount: > Succeeded. > Dec 23 08:55:49 localhost bash[8745]: audit 2020-12-23T08:55:49.091014+0000 > mon.host1 (mon.0) 1093 : audit [DBG] from='mgr.24098 > 192.168.0.1:0/2486989775' entity='mgr.host1.kkssvi' cmd=[{"prefix": "osd > tree", "states": ["destroyed"], "format": "json"}]: dispatch > Dec 23 08:55:50 localhost bash[8745]: cluster 2020-12-23T08:55:48.893433+0000 > mgr.host1.kkssvi (mgr.24098) 24279 : cluster [DBG] pgmap v24213: 1 pgs: 1 > undersized+peered; 0 B data, 112 KiB used, 931 GiB / 932 GiB avail > Dec 23 08:55:50 localhost bash[8745]: audit 2020-12-23T08:55:49.087597+0000 > mgr.host1.kkssvi (mgr.24098) 24280 : audit [DBG] from='client.24206 -' > entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": > "host1:/dev/nvme0n1", "target": ["mon-mgr", ""]}]: dispatch > Dec 23 08:55:50 localhost bash[8745]: cephadm 2020-12-23T08:55:49.093552+0000 > mgr.host1.kkssvi (mgr.24098) 24281 : cephadm [INF] Found osd claims -> {} > Dec 23 08:55:50 localhost bash[8745]: cephadm 2020-12-23T08:55:49.093933+0000 > mgr.host1.kkssvi (mgr.24098) 24282 : cephadm [INF] Found osd claims for > drivegroup None -> {} > > The other problem is logging is set to debug for both hosts, I tried "sudo > cephadm shell -- ceph daemon mon.host1 config set mon_cluster_log_file_level > info” which reports success but logging remains at debug level. > > If I try the same command with mon.host2 I get > INFO:cephadm:Inferring fsid ae111111-1111-1111-1111-f1111a11111a > INFO:cephadm:Inferring config > /var/lib/ceph/ae147088-4486-11eb-9044-f1337a55707a/mon.host1/config > INFO:cephadm:Using recent ceph image ceph/ceph:v15 > admin_socket: exception getting command descriptions: [Errno 2] No such file > or directory > > Which looks like it is trying to use the config for host1 on host2? > > Thanks, > > Duncan > _______________________________________________ > ceph-users mailing list -- [email protected] > To unsubscribe send an email to [email protected] _______________________________________________ ceph-users mailing list -- [email protected] To unsubscribe send an email to [email protected]
