Re: [pve-devel] adding a vm workload scheduler feature

[Alexandre DERUMIER]

>>Unless all cluster nodes have identical hardware, how do you determine if a 
>>given node is a suitable target for a vm?

I think we could add a manual "host cpu weight" option, because it's difficult 
to compare cpus performance. (frequencies/nb cores/intel|amd).


>>Also, should there be a flag where you can set the vm to 'allow 
>>auto-migration' vs. 'stick to current node'? 

yes !


>>Last but not least, how do you keep the load that migration generates from 
>>impacting auto-migration decisions? 

Good question . I think we should use rrds to do average stats of cpu usage.



- Mail original -
De: "Martin Waschbüsch" <serv...@waschbuesch.it>
À: "pve-devel" <pve-devel@pve.proxmox.com>
Envoyé: Mardi 17 Novembre 2015 13:04:16
Objet: Re: [pve-devel] adding a vm workload scheduler feature

> Am 17.11.2015 um 08:23 schrieb Alexandre DERUMIER <aderum...@odiso.com>: 
> 
> Hi, 
> 
> For next year, 
> I would like to implement a new feature : workload scheduler 
> 
> This could allow auto migration of vms, to try to balance the vms across the 
> cluster, 
> from defined rules (for example, try to balance cpu usage) 

Unless all cluster nodes have identical hardware, how do you determine if a 
given node is a suitable target for a vm? 

Also, should there be a flag where you can set the vm to 'allow auto-migration' 
vs. 'stick to current node'? 

Last but not least, how do you keep the load that migration generates from 
impacting auto-migration decisions? 

Martin 
___ 
pve-devel mailing list 
pve-devel@pve.proxmox.com 
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] adding a vm workload scheduler feature

[Martin Waschbüsch]

> Am 17.11.2015 um 08:23 schrieb Alexandre DERUMIER :
> 
> Hi,
> 
> For next year,
> I would like to implement a new feature : workload scheduler
> 
> This could allow auto migration of vms, to try to balance the vms across the 
> cluster,
> from defined rules (for example, try to balance cpu usage)

Unless all cluster nodes have identical hardware, how do you determine if a 
given node is a suitable target for a vm?

Also, should there be a flag where you can set the vm to 'allow auto-migration' 
vs. 'stick to current node'?

Last but not least, how do you keep the load that migration generates from 
impacting auto-migration decisions?

Martin
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] adding a vm workload scheduler feature

[Dietmar Maurer]

> On November 17, 2015 at 4:37 PM Dietmar Maurer  wrote:
> 
> 
> > >>Last but not least, how do you keep the load that migration generates from
> > >>impacting auto-migration decisions? 
> > 
> > Good question . I think we should use rrds to do average stats of cpu usage.
> 
> I think that any load/cpu metric will be difficult(unstable). 
> I would simply use static values like cpu count or max. memory.

or static weight.

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] adding a vm workload scheduler feature

[Dietmar Maurer]

> On November 17, 2015 at 4:43 PM Michael Rasmussen  wrote:
> 
> 
> On Tue, 17 Nov 2015 11:01:58 +0100 (CET)
> Alexandre DERUMIER  wrote:
> 
> > 
> > I'm thinked to use rrd files to have stats on a long time. (maybe do average
> > on last x minutes)
> > 
> > (for example we don't want to migrate a vm if the host is overload for 1 or
> > 2 minutes,
> > because of a spiky cpu vm)
> > 
> > 
> Is it not the point with this to take care of short time load? Long
> time load should be handled by human intervention.

I thought exactly the opposite. You will get a 'unstable' system
if you react to short time load changes.

See the theory of control systems:

https://en.wikipedia.org/wiki/Control_system

Any electronic engineer can teach you how simple it is to build an oscillator
;-)

Consider that a VM has a highly unpredictable behavior.

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] adding a vm workload scheduler feature

[Michael Rasmussen]

On Tue, 17 Nov 2015 11:01:58 +0100 (CET)
Alexandre DERUMIER  wrote:

> 
> I'm thinked to use rrd files to have stats on a long time. (maybe do average 
> on last x minutes)
> 
> (for example we don't want to migrate a vm if the host is overload for 1 or 2 
> minutes,
> because of a spiky cpu vm)
> 
> 
Is it not the point with this to take care of short time load? Long
time load should be handled by human intervention.

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael  rasmussen  cc
http://pgp.mit.edu:11371/pks/lookup?op=get=0xD3C9A00E
mir  datanom  net
http://pgp.mit.edu:11371/pks/lookup?op=get=0xE501F51C
mir  miras  org
http://pgp.mit.edu:11371/pks/lookup?op=get=0xE3E80917
--
/usr/games/fortune -es says:
May you do Good Magic with Perl.
-- Larry Wall's blessing


pgp5fdXVGwUk7.pgp
Description: OpenPGP digital signature
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] adding a vm workload scheduler feature

[Michael Rasmussen]

On Tue, 17 Nov 2015 11:01:58 +0100 (CET)
Alexandre DERUMIER  wrote:

> yes, sure. I just want to start with cpu, but memory could be add too.
> 
>  I'm not sure for io-wait, as migrating the vm don't change the storage ?
> 
Is that also the case for Gluster?

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael  rasmussen  cc
http://pgp.mit.edu:11371/pks/lookup?op=get=0xD3C9A00E
mir  datanom  net
http://pgp.mit.edu:11371/pks/lookup?op=get=0xE501F51C
mir  miras  org
http://pgp.mit.edu:11371/pks/lookup?op=get=0xE3E80917
--
/usr/games/fortune -es says:
May you do Good Magic with Perl.
-- Larry Wall's blessing


pgpeLj3xizrEP.pgp
Description: OpenPGP digital signature
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] adding a vm workload scheduler feature

[Dietmar Maurer]

> >>Last but not least, how do you keep the load that migration generates from
> >>impacting auto-migration decisions? 
> 
> Good question . I think we should use rrds to do average stats of cpu usage.

I think that any load/cpu metric will be difficult(unstable). 
I would simply use static values like cpu count or max. memory.

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] adding a vm workload scheduler feature

[Thomas Lamprecht]

Am 17.11.2015 um 14:56 schrieb Martin Waschbüsch:

Am 17.11.2015 um 14:20 schrieb Alexandre DERUMIER :


Unless all cluster nodes have identical hardware, how do you determine if a 
given node is a suitable target for a vm?

I think we could add a manual "host cpu weight" option, because it's difficult 
to compare cpus performance. (frequencies/nb cores/intel|amd).
Manual weighting VMs and (maybe also) Nodes would general be an good 
option (which Dietmar proposed once to me as an idea) shift the control 
more to the admin, which is normally better able to determine how the 
load should be spread, as there are also different use cases.



Good point. Though, I was more thinking about situations where the cpu-type is 
not set to default (kvm64, I think?) but to something like 'IvyBridge' or 
Opteron_G5. (The primary use I had for using non-default cpu types was to 
expose features such as AES-NI to a vm.)

Another thing that just occurred to me: Backup schedules and settings are not 
necessarily the same for each node, which means auto-migration could lead to 
double, or (worse!) no backup for a vm that was moved around?

Dunno if these are just corner cases?


Configuration like done in the HA-Manager could be used, so that certain 
nodes are in a group and the VM is bound to this group, we could in fact 
use the group API from the HA-Manager.
Also migrations could be blocked when a service is doing stuff like a 
online backup.




___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] adding a vm workload scheduler feature

[Alexandre DERUMIER]

>>Also migrations could be blocked when a service is doing stuff like a 
online backup.

block job too (like mirroring). Maybe just look if the vm have a lock should be 
enough.

- Mail original -
De: "Thomas Lamprecht" <t.lampre...@proxmox.com>
À: "pve-devel" <pve-devel@pve.proxmox.com>
Envoyé: Mardi 17 Novembre 2015 15:26:15
Objet: Re: [pve-devel] adding a vm workload scheduler feature

Am 17.11.2015 um 14:56 schrieb Martin Waschbüsch: 
>> Am 17.11.2015 um 14:20 schrieb Alexandre DERUMIER <aderum...@odiso.com>: 
>> 
>>>> Unless all cluster nodes have identical hardware, how do you determine if 
>>>> a given node is a suitable target for a vm? 
>> I think we could add a manual "host cpu weight" option, because it's 
>> difficult to compare cpus performance. (frequencies/nb cores/intel|amd). 
Manual weighting VMs and (maybe also) Nodes would general be an good 
option (which Dietmar proposed once to me as an idea) shift the control 
more to the admin, which is normally better able to determine how the 
load should be spread, as there are also different use cases. 

> Good point. Though, I was more thinking about situations where the cpu-type 
> is not set to default (kvm64, I think?) but to something like 'IvyBridge' or 
> Opteron_G5. (The primary use I had for using non-default cpu types was to 
> expose features such as AES-NI to a vm.) 
> 
> Another thing that just occurred to me: Backup schedules and settings are not 
> necessarily the same for each node, which means auto-migration could lead to 
> double, or (worse!) no backup for a vm that was moved around? 
> 
> Dunno if these are just corner cases? 

Configuration like done in the HA-Manager could be used, so that certain 
nodes are in a group and the VM is bound to this group, we could in fact 
use the group API from the HA-Manager. 
Also migrations could be blocked when a service is doing stuff like a 
online backup. 



___ 
pve-devel mailing list 
pve-devel@pve.proxmox.com 
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] adding a vm workload scheduler feature

[Alexandre DERUMIER]

>>Good point. Though, I was more thinking about situations where the cpu-type 
>>is not set to default (kvm64, I think?) but to something like 'IvyBridge' or 
>>Opteron_G5. (The primary use I had for using non-default cpu types was >>to 
>>expose features such as AES-NI to a vm.) 

I think we could find a way to find compatible host.
Another way could be to start the migration, and with the new cpu enforce 
feature in proxmox 4, the vm migration will stop directly if target host is not 
compatible and we can try another host or another vm.
But this is solvable. 

>>Another thing that just occurred to me: Backup schedules and settings are not 
>>necessarily the same for each node, which means auto-migration could lead to 
>>double, or (worse!) no backup for a vm that was moved around? 

This occur only if you restrict a backup job for specific node + specific vms.
(By default, you can backup vms, and it's work if they are on any nodes)

For this case, I think we can exclude this vms from auto-migration.


- Mail original -
De: "Martin Waschbüsch" <serv...@waschbuesch.it>
À: "pve-devel" <pve-devel@pve.proxmox.com>
Envoyé: Mardi 17 Novembre 2015 14:56:39
Objet: Re: [pve-devel] adding a vm workload scheduler feature

> Am 17.11.2015 um 14:20 schrieb Alexandre DERUMIER <aderum...@odiso.com>: 
> 
>>> Unless all cluster nodes have identical hardware, how do you determine if a 
>>> given node is a suitable target for a vm? 
> 
> I think we could add a manual "host cpu weight" option, because it's 
> difficult to compare cpus performance. (frequencies/nb cores/intel|amd). 

Good point. Though, I was more thinking about situations where the cpu-type is 
not set to default (kvm64, I think?) but to something like 'IvyBridge' or 
Opteron_G5. (The primary use I had for using non-default cpu types was to 
expose features such as AES-NI to a vm.) 

Another thing that just occurred to me: Backup schedules and settings are not 
necessarily the same for each node, which means auto-migration could lead to 
double, or (worse!) no backup for a vm that was moved around? 

Dunno if these are just corner cases? 

Martin 
___ 
pve-devel mailing list 
pve-devel@pve.proxmox.com 
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] adding a vm workload scheduler feature

[Alexandre DERUMIER]

>> I think that any load/cpu metric will be difficult(unstable). 
>> I would simply use static values like cpu count or max. memory. 
>
>or static weight. 

For my personnal usage, I don't think it'll work, because I have a lot of 
differents workload at different time,
and some vms can use sometime 0% cpu and 80% at spike hours.

But indeed, dynamic stats is not easy to resolve.


Ovirt has a planned feature which use optaplanner

http://www.ovirt.org/Features/Optaplanner
This seem to do some kind of heuristics and maths (out of my competence ;), to 
known which vm migrate.





Another interesting whitepaper with algorithms
"Dynamic Load Balancing of Virtual Machines using QEMU-KVM"
http://research.ijcaonline.org/volume46/number6/pxc3879263.pdf


- Mail original -
De: "dietmar" <diet...@proxmox.com>
À: "aderumier" <aderum...@odiso.com>, "pve-devel" <pve-devel@pve.proxmox.com>
Envoyé: Mardi 17 Novembre 2015 16:51:23
Objet: Re: [pve-devel] adding a vm workload scheduler feature

> On November 17, 2015 at 4:37 PM Dietmar Maurer <diet...@proxmox.com> wrote: 
> 
> 
> > >>Last but not least, how do you keep the load that migration generates 
> > >>from 
> > >>impacting auto-migration decisions? 
> > 
> > Good question . I think we should use rrds to do average stats of cpu 
> > usage. 
> 
> I think that any load/cpu metric will be difficult(unstable). 
> I would simply use static values like cpu count or max. memory. 

or static weight. 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] adding a vm workload scheduler feature

[Dietmar Maurer]

> But indeed, dynamic stats is not easy to resolve.
> 
> 
> Ovirt has a planned feature which use optaplanner
> 
> http://www.ovirt.org/Features/Optaplanner
> This seem to do some kind of heuristics and maths (out of my competence ;), to
> known which vm migrate.

IMHO you can always generate a load which leads to endless migrations...

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] adding a vm workload scheduler feature

[Alexandre DERUMIER]

>>I assume it is hard to get this stable (just a feeling). 
yes, same for me.

>>On the other side, this would be simple 
>>to implement. Each node is responsible to move its own VMs, so you do not 
>>even 
>>need a lock. 
I was more thinking about a lock, to avoid node2 migrate vm to node1, when 
node1 try to migrate a vm to node3 for example.


>>My plan was to integrate this into the HA manager, but then you only have the 
>>feature for HA enabled VMs. 

Could be great to have it without HA too.


>>But the CRM code shows how to use a cluster wide lock to implement a 'master' 
>>role. 
Ok,thanks, I'll try to have a look at it.


Don't have time for now, but I'll begin to do test in January.



- Mail original -
De: "dietmar" <diet...@proxmox.com>
À: "aderumier" <aderum...@odiso.com>, "pve-devel" <pve-devel@pve.proxmox.com>
Envoyé: Mardi 17 Novembre 2015 08:40:19
Objet: Re: [pve-devel] adding a vm workload scheduler feature

> What do you think about it ? 

interesting 

> 
> As we don't have master node, I don't known how to implement this: 
> 
> 1) each node try to migrate his own vms to another node with less cpu usage. 
> maybe with a global cluster lock to not have 2 nodes migrating in both way 
> at the same time ? 

I assume it is hard to get this stable (just a feeling). On the other side, 
this 
would be simple 
to implement. Each node is responsible to move its own VMs, so you do not even 
need a lock. 

> 2) have some kind of master service in the cluster (maybe with corosync 
> service ?), 
> which read global stats of all nodes, and through an algorithm, do the 
> migrations. 
> 
> Don't known which way is better ? 

My plan was to integrate this into the HA manager, but then you only have the 
feature 
for HA enabled VMs. 

But the CRM code shows how to use a cluster wide lock to implement a 'master' 
role. 

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] adding a vm workload scheduler feature

[Thomas Lamprecht]

Il 17 novembre 2015 08:40:19 CET, Dietmar Maurer  ha 
scritto:
>
>> What do you think about it ?
>
>interesting
> 
>> 
>> As we don't have master node, I don't known how to implement this:
>> 
>> 1) each node try to migrate his own vms to another node with less cpu
>usage.
>>maybe with a global cluster lock to not have 2 nodes migrating in
>both way
>> at the same time ?
>
>I assume it is hard to get this stable (just a feeling). On the other
>side, this
>would be simple
>to implement. Each node is responsible to move its own VMs, so you do
>not even
>need a lock.

A lock maybe should be there to only let one rebalancing action happen at any 
time to avoid out of control feedback loops.

>> 2) have some kind of master service in the cluster (maybe with
>corosync
>> service ?),
>>which read global stats of all nodes, and through an algorithm, do
>the
>> migrations.
>> 
>> Don't known which way is better ?
>
>My plan was to integrate this into the HA manager, but then you only
>have the
>feature
>for HA enabled VMs.

That could still be done, adding a new service which uses the HA Environment 
class. With a lock and a status/command file a master wouldn't be necessarly 
needed.

>
>But the CRM code shows how to use a cluster wide lock to implement a
>'master'
>role.
>
>___
>pve-devel mailing list
>pve-devel@pve.proxmox.com
>http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] adding a vm workload scheduler feature

[datanom.net]

Hi all,

On 2015-11-17 08:23, Alexandre DERUMIER wrote:



What do you think about it ?

Sounds great, but I think memory and io-wait should be part of the list 
as well.




As we don't have master node, I don't known how to implement this:

1) each node try to migrate his own vms to another node with less cpu 
usage.

   maybe with a global cluster lock to not have 2 nodes migrating in
both way at the same time ?

How would you distinguish between operator initiated migration and 
automatic migration?
I should think that operator initiated migration should always overrule 
automatic migration.




2) have some kind of master service in the cluster (maybe with
corosync service ?),
   which read global stats of all nodes, and through an algorithm, do
the migrations.

Why not keep it simple? You could extend pvestatd to save the 
performance numbers in a file in a specific folder in /etc/pve since 
pvestatd already has these numbers. Each node names this file by node 
name and if writing the numbers through Data::Dumper then this could be 
a persisted hash like

{
'cpu' => {
   'cur' => 12,
   'max' => 80
 },
'mem' => {
   'cur' => 28,
   'max' => 96
 },
'wait' => {
'cur' => 2,
'max' => 10
  }
}
cur is the reading from pvestatd and max is the configured threshold on 
each node.


Another daemon on each node on a regular interval assembles a hash 
reading every file in the /etc/pve folder to:

{
   'node1' => {
'cpu' => {
   'cur' => 12,
   'max' => 80
 },
'mem' => {
   'cur' => 28,
   'max' => 96
 },
'wait' => {
'cur' => 2,
'max' => 10
  }
  },
..
}
and performs decisions according to a well defined algorithm which 
should also take into account that some VM's by configuration can be 
configured to be locked to a specific node.


As for locking I agree that there should be some kind of global locking 
scheme.


Just some quick thoughts.

--
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael  rasmussen  cc
http://pgp.mit.edu:11371/pks/lookup?op=get=0xD3C9A00E
mir  datanom  net
http://pgp.mit.edu:11371/pks/lookup?op=get=0xE501F51C
mir  miras  org
http://pgp.mit.edu:11371/pks/lookup?op=get=0xE3E80917
--



This mail was virus scanned and spam checked before delivery.
This mail is also DKIM signed. See header dkim-signature.

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] adding a vm workload scheduler feature

[Alexandre DERUMIER]

> What do you think about it ?
> 
>>Sounds great, but I think memory and io-wait should be part of the list 
>>as well.
yes, sure. I just want to start with cpu, but memory could be add too.

 I'm not sure for io-wait, as migrating the vm don't change the storage ?


>>Why not keep it simple? You could extend pvestatd to save the 
>>performance numbers in a file in a specific folder in /etc/pve since 
>>pvestatd already has these numbers. Each node names this file by node 
>>name and if writing the numbers through Data::Dumper then this could be 
>>a persisted hash like 

I'm thinked to use rrd files to have stats on a long time. (maybe do average on 
last x minutes)

(for example we don't want to migrate a vm if the host is overload for 1 or 2 
minutes,
because of a spiky cpu vm)




- Mail original -
De: "datanom.net" <m...@datanom.net>
À: "pve-devel" <pve-devel@pve.proxmox.com>
Envoyé: Mardi 17 Novembre 2015 10:11:40
Objet: Re: [pve-devel] adding a vm workload scheduler feature

Hi all, 

On 2015-11-17 08:23, Alexandre DERUMIER wrote: 
> 
> 
> What do you think about it ? 
> 
Sounds great, but I think memory and io-wait should be part of the list 
as well. 

> 
> As we don't have master node, I don't known how to implement this: 
> 
> 1) each node try to migrate his own vms to another node with less cpu 
> usage. 
> maybe with a global cluster lock to not have 2 nodes migrating in 
> both way at the same time ? 
> 
How would you distinguish between operator initiated migration and 
automatic migration? 
I should think that operator initiated migration should always overrule 
automatic migration. 

> 
> 2) have some kind of master service in the cluster (maybe with 
> corosync service ?), 
> which read global stats of all nodes, and through an algorithm, do 
> the migrations. 
> 
Why not keep it simple? You could extend pvestatd to save the 
performance numbers in a file in a specific folder in /etc/pve since 
pvestatd already has these numbers. Each node names this file by node 
name and if writing the numbers through Data::Dumper then this could be 
a persisted hash like 
{ 
'cpu' => { 
'cur' => 12, 
'max' => 80 
}, 
'mem' => { 
'cur' => 28, 
'max' => 96 
}, 
'wait' => { 
'cur' => 2, 
'max' => 10 
} 
} 
cur is the reading from pvestatd and max is the configured threshold on 
each node. 

Another daemon on each node on a regular interval assembles a hash 
reading every file in the /etc/pve folder to: 
{ 
'node1' => { 
'cpu' => { 
'cur' => 12, 
'max' => 80 
}, 
'mem' => { 
'cur' => 28, 
'max' => 96 
}, 
'wait' => { 
'cur' => 2, 
'max' => 10 
} 
}, 
.. 
} 
and performs decisions according to a well defined algorithm which 
should also take into account that some VM's by configuration can be 
configured to be locked to a specific node. 

As for locking I agree that there should be some kind of global locking 
scheme. 

Just some quick thoughts. 

-- 
Hilsen/Regards 
Michael Rasmussen 

Get my public GnuPG keys: 
michael  rasmussen  cc 
http://pgp.mit.edu:11371/pks/lookup?op=get=0xD3C9A00E 
mir  datanom  net 
http://pgp.mit.edu:11371/pks/lookup?op=get=0xE501F51C 
mir  miras  org 
http://pgp.mit.edu:11371/pks/lookup?op=get=0xE3E80917 
-- 

 

This mail was virus scanned and spam checked before delivery. 
This mail is also DKIM signed. See header dkim-signature. 

___ 
pve-devel mailing list 
pve-devel@pve.proxmox.com 
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] adding a vm workload scheduler feature

[Alexandre DERUMIER]

Hi,

For next year,
I would like to implement a new feature : workload scheduler

This could allow auto migration of vms, to try to balance the vms across the 
cluster,
from defined rules (for example, try to balance cpu usage)

ovirt and vmware have this kind of feature :

http://www.ovirt.org/Features/oVirtScheduler
https://www.vmware.com/fr/products/vsphere/features/drs-dpm

I would like to start with balancing cpu usage.

for example:

node1 : 90% cpu usage
node2 : 30% cpu usage
node3 : 60% cpu usage.

and we define a target under 70% cpu usage for each node.

so node1 will try to migrate 1 or more vms (need to algorithm to find which vm 
migrate), until it each max 70% cpu usage.


What do you think about it ?


As we don't have master node, I don't known how to implement this:

1) each node try to migrate his own vms to another node with less cpu usage.
   maybe with a global cluster lock to not have 2 nodes migrating in both way 
at the same time ?


2) have some kind of master service in the cluster (maybe with corosync service 
?),
   which read global stats of all nodes, and through an algorithm, do the 
migrations.

Don't known which way is better ?


Regards,

alexandre

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] adding a vm workload scheduler feature

[Dietmar Maurer]

> What do you think about it ?

interesting
 
> 
> As we don't have master node, I don't known how to implement this:
> 
> 1) each node try to migrate his own vms to another node with less cpu usage.
>maybe with a global cluster lock to not have 2 nodes migrating in both way
> at the same time ?

I assume it is hard to get this stable (just a feeling). On the other side, this
would be simple
to implement. Each node is responsible to move its own VMs, so you do not even
need a lock.

> 2) have some kind of master service in the cluster (maybe with corosync
> service ?),
>which read global stats of all nodes, and through an algorithm, do the
> migrations.
> 
> Don't known which way is better ?

My plan was to integrate this into the HA manager, but then you only have the
feature
for HA enabled VMs.

But the CRM code shows how to use a cluster wide lock to implement a 'master'
role.

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel