Re: [ceph-users] Monitor as local VM on top of the server pool cluster?
For large cluster , there will be a lot of change at any time, this means the pressure of mon will be big at some time, because all change will go through leader , so for this , the local storage for mon should be good enough, I think this maybe a conderation . On Tue, Jul 11, 2017 at 11:29 AM, Brad Hubbardwrote: > On Tue, Jul 11, 2017 at 3:44 AM, David Turner wrote: >> Mons are a paxos quorum and as such want to be in odd numbers. 5 is >> generally what people go with. I think I've heard of a few people use 7 >> mons, but you do not want to have an even number of mons or an ever growing > > Unless your cluster is very large three should be sufficient. > >> number of mons. The reason you do not want mons running on the same >> hardware as osds is resource contention during recovery. As long as the Xen >> servers you are putting the mons on are not going to cause any source of >> resource limitation/contention, then virtualizing them should be fine for >> you. Make sure that you aren't configuring the mon to run using an RBD for >> its storage, that would be very bad. >> >> The mon Quorum elects a leader and that leader will be in charge of the >> quorum. Having local mons doesn't do anything as the clients will still be >> talking to the mons as a quorum and won't necessarily talk to the mon >> running on them. The vast majority of communication to the cluster that >> your Xen servers will be doing is to the OSDs anyway, very little >> communication to the mons. >> >> On Mon, Jul 10, 2017 at 1:21 PM Massimiliano Cuttini >> wrote: >>> >>> Hi everybody, >>> >>> i would like to separate MON from OSD as reccomended. >>> In order to do so without new hardware I'm planning to create all the >>> monitor as a Virtual Machine on top of my hypervisors (Xen). >>> I'm testing a pool of 8 nodes of Xen. >>> >>> I'm thinking about create 8 monitor and pin one monitor for one Xen node. >>> So, i'm guessing, every Ceph monitor'll be local for each node client. >>> This should speed up the system by local connecting monitors with a >>> little overflown for the monitors sync between nodes. >>> >>> Is it a good idea have a local monitor virtualized on top of each >>> hypervisor node? >>> Did you see any understimation or wrong design in this? >>> >>> Thanks for every helpfull info. >>> >>> >>> Regards, >>> Max >>> >>> ___ >>> ceph-users mailing list >>> ceph-users@lists.ceph.com >>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> >> >> ___ >> ceph-users mailing list >> ceph-users@lists.ceph.com >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> > > > > -- > Cheers, > Brad > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Monitor as local VM on top of the server pool cluster?
On Tue, Jul 11, 2017 at 3:44 AM, David Turnerwrote: > Mons are a paxos quorum and as such want to be in odd numbers. 5 is > generally what people go with. I think I've heard of a few people use 7 > mons, but you do not want to have an even number of mons or an ever growing Unless your cluster is very large three should be sufficient. > number of mons. The reason you do not want mons running on the same > hardware as osds is resource contention during recovery. As long as the Xen > servers you are putting the mons on are not going to cause any source of > resource limitation/contention, then virtualizing them should be fine for > you. Make sure that you aren't configuring the mon to run using an RBD for > its storage, that would be very bad. > > The mon Quorum elects a leader and that leader will be in charge of the > quorum. Having local mons doesn't do anything as the clients will still be > talking to the mons as a quorum and won't necessarily talk to the mon > running on them. The vast majority of communication to the cluster that > your Xen servers will be doing is to the OSDs anyway, very little > communication to the mons. > > On Mon, Jul 10, 2017 at 1:21 PM Massimiliano Cuttini > wrote: >> >> Hi everybody, >> >> i would like to separate MON from OSD as reccomended. >> In order to do so without new hardware I'm planning to create all the >> monitor as a Virtual Machine on top of my hypervisors (Xen). >> I'm testing a pool of 8 nodes of Xen. >> >> I'm thinking about create 8 monitor and pin one monitor for one Xen node. >> So, i'm guessing, every Ceph monitor'll be local for each node client. >> This should speed up the system by local connecting monitors with a >> little overflown for the monitors sync between nodes. >> >> Is it a good idea have a local monitor virtualized on top of each >> hypervisor node? >> Did you see any understimation or wrong design in this? >> >> Thanks for every helpfull info. >> >> >> Regards, >> Max >> >> ___ >> ceph-users mailing list >> ceph-users@lists.ceph.com >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- Cheers, Brad ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Monitor as local VM on top of the server pool cluster?
Mons are a paxos quorum and as such want to be in odd numbers. 5 is generally what people go with. I think I've heard of a few people use 7 mons, but you do not want to have an even number of mons or an ever growing number of mons. The reason you do not want mons running on the same hardware as osds is resource contention during recovery. As long as the Xen servers you are putting the mons on are not going to cause any source of resource limitation/contention, then virtualizing them should be fine for you. Make sure that you aren't configuring the mon to run using an RBD for its storage, that would be very bad. The mon Quorum elects a leader and that leader will be in charge of the quorum. Having local mons doesn't do anything as the clients will still be talking to the mons as a quorum and won't necessarily talk to the mon running on them. The vast majority of communication to the cluster that your Xen servers will be doing is to the OSDs anyway, very little communication to the mons. On Mon, Jul 10, 2017 at 1:21 PM Massimiliano Cuttiniwrote: > Hi everybody, > > i would like to separate MON from OSD as reccomended. > In order to do so without new hardware I'm planning to create all the > monitor as a Virtual Machine on top of my hypervisors (Xen). > I'm testing a pool of 8 nodes of Xen. > > I'm thinking about create 8 monitor and pin one monitor for one Xen node. > So, i'm guessing, every Ceph monitor'll be local for each node client. > This should speed up the system by local connecting monitors with a > little overflown for the monitors sync between nodes. > > Is it a good idea have a local monitor virtualized on top of each > hypervisor node? > Did you see any understimation or wrong design in this? > > Thanks for every helpfull info. > > > Regards, > Max > > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] Monitor as local VM on top of the server pool cluster?
Hi everybody, i would like to separate MON from OSD as reccomended. In order to do so without new hardware I'm planning to create all the monitor as a Virtual Machine on top of my hypervisors (Xen). I'm testing a pool of 8 nodes of Xen. I'm thinking about create 8 monitor and pin one monitor for one Xen node. So, i'm guessing, every Ceph monitor'll be local for each node client. This should speed up the system by local connecting monitors with a little overflown for the monitors sync between nodes. Is it a good idea have a local monitor virtualized on top of each hypervisor node? Did you see any understimation or wrong design in this? Thanks for every helpfull info. Regards, Max ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com