Re: [ceph-users] {Disarmed} Re: Adding Monitor

2015-03-14 Thread Georgios Dimitrakakis
My problem was that commands like ceph -s fail to connect and therefore I couldn't extract monmap. I could get it from the running pid though and I 've used it along with the documentation and the example of how a monmap looks like in order to create a new and inject it into the second monitor.

Re: [ceph-users] {Disarmed} Re: Adding Monitor

2015-03-13 Thread Georgios Dimitrakakis
This is the message that is flooding the ceph-mon.log now: 2015-03-14 08:16:39.286823 7f9f6920b700 1 mon.fu@0(electing).elector(1) init, last seen epoch 1 2015-03-14 08:16:42.736674 7f9f6880a700 1 mon.fu@0(electing) e2 adding peer 15.12.6.21:6789/0 to list of hints 2015-03-14

Re: [ceph-users] {Disarmed} Re: Adding Monitor

2015-03-13 Thread Sage Weil
On Sat, 14 Mar 2015, Georgios Dimitrakakis wrote: This is the message that is flooding the ceph-mon.log now: 2015-03-14 08:16:39.286823 7f9f6920b700 1 mon.fu@0(electing).elector(1) init, last seen epoch 1 2015-03-14 08:16:42.736674 7f9f6880a700 1 mon.fu@0(electing) e2 adding peer

Re: [ceph-users] {Disarmed} Re: Adding Monitor

2015-03-13 Thread Sage Weil
On Sat, 14 Mar 2015, Georgios Dimitrakakis wrote: Sage, correct me if I am wrong but this is when you have a surviving monitor ! right? Yes. By surviving I mean that the mon data directory has not been deleted. My problem is that I cannot extract the monmap from any! Do you mean that

Re: [ceph-users] {Disarmed} Re: Adding Monitor

2015-03-13 Thread Georgios Dimitrakakis
This is the output from CEPH HEALTH # ceph health 2015-03-14 09:16:54.435458 7f507843b700 0 -- :/1048223 15.12.6.21:6789/0 pipe(0x7f5074022250 sd=3 :0 s=1 pgs=0 cs=0 l=1 c=0x7f50740224e0).fault 2015-03-14 09:16:57.433435 7f507833a700 0 -- :/1048223 192.168.1.100:6789/0

Re: [ceph-users] {Disarmed} Re: Adding Monitor

2015-03-13 Thread Georgios Dimitrakakis
I can no longer start my OSDs :-@ failed: 'timeout 30 /usr/bin/ceph -c /etc/ceph/ceph.conf --name=osd.6 --keyring=/var/lib/ceph/osd/ceph-6/keyring osd crush create-or-move -- 6 3.63 host=fu root=default' Please help!!! George ceph mon add stops at this: [jin][INFO ] Running command:

Re: [ceph-users] {Disarmed} Re: Adding Monitor

2015-03-13 Thread Georgios Dimitrakakis
This is the log for monitor (ceph-mon.log) when I try to restart the monitor: 2015-03-14 07:47:26.384561 7f1f1dc0f700 -1 mon.fu@0(probing) e2 *** Got Signal Terminated *** 2015-03-14 07:47:26.384593 7f1f1dc0f700 1 mon.fu@0(probing) e2 shutdown 2015-03-14 07:47:26.384654 7f1f1dc0f700 0

Re: [ceph-users] {Disarmed} Re: Adding Monitor

2015-03-13 Thread Georgios Dimitrakakis
Not a healthy monitor means that I can not get a monmap from none of them! and none of the commands ceph health etc. are working. Best, George Yes Sage! Priority is to fix things! Right now I don't have a healthy monitor! Can I remove all of them and add the first one from scratch? What

Re: [ceph-users] {Disarmed} Re: Adding Monitor

2015-03-13 Thread Georgios Dimitrakakis
Guyn any help much appreciated because my cluster is down :-( After trying ceph mon add which didn't complete since it was stuck for ever here: [jin][WARNIN] 2015-03-14 07:07:14.964265 7fb4be6f5700 0 monclient: hunting for new mon ^CKilled by signal 2. [ceph_deploy][ERROR ]

Re: [ceph-users] {Disarmed} Re: Adding Monitor

2015-03-13 Thread Georgios Dimitrakakis
ceph mon add stops at this: [jin][INFO ] Running command: sudo ceph mon getmap -o /var/lib/ceph/tmp/ceph.raijin.monmap and never gets over it! Any help?? Thanks, George Guyn any help much appreciated because my cluster is down :-( After trying ceph mon add which didn't complete

Re: [ceph-users] {Disarmed} Re: Adding Monitor

2015-03-13 Thread Sage Weil
On Sat, 14 Mar 2015, Georgios Dimitrakakis wrote: Guyn any help much appreciated because my cluster is down :-( After trying ceph mon add which didn't complete since it was stuck for ever here: [jin][WARNIN] 2015-03-14 07:07:14.964265 7fb4be6f5700 0 monclient: hunting for new mon

Re: [ceph-users] {Disarmed} Re: Adding Monitor

2015-03-13 Thread Georgios Dimitrakakis
Yes Sage! Priority is to fix things! Right now I don't have a healthy monitor! Can I remove all of them and add the first one from scratch? What would that mean about the data?? Best, George On Sat, 14 Mar 2015, Georgios Dimitrakakis wrote: This is the message that is flooding the

Re: [ceph-users] {Disarmed} Re: Adding Monitor

2015-03-13 Thread Sage Weil
On Sat, 14 Mar 2015, Georgios Dimitrakakis wrote: Not a healthy monitor means that I can not get a monmap from none of them! If you look at the procedure at http://docs.ceph.com/docs/master/rados/operations/add-or-rm-mons/#removing-monitors-from-an-unhealthy-cluster you'll notice that you do