On Mon, 24 Jun 2013, Mandell Degerness wrote: > I'm testing the change (actually re-starting the monitors after the > monitor removal), but this brings up the issue with why we didn't want > to do this in the first place: When reducing the number of monitors > from 5 to 3, we are guaranteed to have a service outage for the time > it takes to restart at least one of the monitors (and, possibly, for > two of the restarts, now that I think on it). In theory, the > stop/start cycle is very short and should complete in a reasonable > time. What I'm concerned about, however, is the case that something > is wrong with our re-written config file. In that case, the outage is > immediate and will last until the problem is corrected on the first > server to have the monitor restarted.
I'm jumping into this thread late, but: why would you follow the second removal procedure for broken clusters? To go from 5->3 mons, you should just stop 2 of the mons and do 'ceph mon rm <addr1>' 'ceph mon rm <addr2>'. sage > > On Mon, Jun 24, 2013 at 10:07 AM, John Nielsen <[email protected]> wrote: > > On Jun 21, 2013, at 5:00 PM, Mandell Degerness <[email protected]> > > wrote: > > > >> There is a scenario where we would want to remove a monitor and, at a > >> later date, re-add the monitor (using the same IP address). Is there > >> a supported way to do this? I tried deleting the monitor directory > >> and rebuilding from scratch following the add monitor procedures from > >> the web, but the monitor still suicide's when started. > > > > > > I assume you're already referencing this: > > http://ceph.com/docs/master/rados/operations/add-or-rm-mons/ > > > > I have done what you describe before. There were a couple hiccups, let's > > see if I remember the specifics: > > > > Remove: > > Follow the first two steps under "removing a monitor (manual) at the link > > above: > > service ceph stop mon.N > > ceph mon remove N > > Comment out the monitor entry in ceph.conf on ALL mon, osd and client hosts. > > Restart services as required to make everyone happy with the smaller set of > > monitors > > > > Re-add: > > Wipe the old monitor's directory and re-create it > > Follow the steps for "adding a monitor (manual) at the link above. Instead > > of adding a new entry you can just un-comment your old ones in ceph.conf. > > You can also start the monitor with "service ceph start mon N" on the > > appropriate host instead of running yourself (step 8). Note that you DO > > need to run ceph-mon as specified in step 5. I was initially confused about > > the '--mkfs' flag there--it doesn't refer to the OS's filesystem, you > > should use a directory or mountpoint that is already prepared/mounted. > > > > HTH. If you run into trouble post exactly the steps you followed and > > additional details about your setup. > > > > JN > > > _______________________________________________ > ceph-users mailing list > [email protected] > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > _______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
