Re: [Engine-devel] Network related hooks in vdsm

2012-10-10 Thread Andrew Cathrow


- Original Message -
> From: "Yaniv Kaul" 
> To: "Igor Lvovsky" 
> Cc: "Dan Yasny" , "engine-devel" , 
> "vdsm-devel"
> 
> Sent: Wednesday, October 10, 2012 10:53:19 AM
> Subject: Re: [Engine-devel] Network related hooks in vdsm
> 
> On 10/10/2012 04:47 PM, Igor Lvovsky wrote:
> >Hi everyone,
> > As you know vdsm has hooks mechanism and we already support dozen
> > of hooks for different needs.
> > Now it's a network's time.
> > We would like to get your comments regarding our proposition for
> > network related hooks.
> >
> > In general we are planning to prepare framework for future support
> > of bunch network related hooks.
> > Some of them already proposed by Itzik Brown [1] and Dan Yasny [2].
> >
> > Below you can find the additional hooks list that we propose:
> >
> > Note: In the first stage we can implement these hooks without any
> > parameters, just to provide an entry point
> >   for simple hooks.
> >
> > Networks manipulation:
> > - before_add_network(conf={}, customProperty={})
> > - after_add_network(conf={}, customProperty={})
> > - before_del_network(conf={}, customProperty={})
> > - after_del_network(conf={}, customProperty={})
> > - before_edit_network(conf={}, customProperty={})
> > - after_edit_network(conf={}, customProperty={})
> > - TBD
> >
> > Bondings manipulations:
> 
> Bond might be interesting because it may require switch configuration
> -
> but so will VLAN changes, so perhaps triggers in VLAN changes are
> worthwhile as well.
> Y.
> 
> > - before_add_bond(conf={}, customProperty={})
> > - after_add_bond(conf={}, customProperty={})
> > - before_del_bond(conf={}, customProperty={})
> > - after_del_bond(conf={}, customProperty={})
> > - before_edit_bond(conf={}, customProperty={})
> > - after_edit_bond(conf={}, customProperty={})
> > - TBD
> >
> > General purpose:
> > - before_persist_network
> > - after_persist_network

What about some way of doing a push that's not tied to an event - if we want to 
"push" something



> >
> >
> > Now we just need to figure out the use cases.
> >
> > Your input more than welcome...
> >
> > [1] http://gerrit.ovirt.org/#/c/7224/   - Adding hooks support for
> > NIC hotplug
> > [2] http://gerrit.ovirt.org/#/c/7547/   - Hook: Cisco VM-FEX
> > support
> >
> >
> > Regards,
> >  Igor Lvovsky
> > ___
> > Engine-devel mailing list
> > Engine-devel@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> ___
> Engine-devel mailing list
> Engine-devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Network related hooks in vdsm

2012-10-10 Thread Simon Grinberg


- Original Message -
> From: "Igor Lvovsky" 
> To: "Simon Grinberg" 
> Cc: "Dan Yasny" , "engine-devel" , 
> "vdsm-devel"
> 
> Sent: Wednesday, October 10, 2012 5:27:13 PM
> Subject: Re: [Engine-devel] Network related hooks in vdsm
> 
> 
> 
> - Original Message -
> > From: "Simon Grinberg" 
> > To: "Igor Lvovsky" 
> > Cc: "Dan Yasny" , "vdsm-devel"
> > , "engine-devel"
> > 
> > Sent: Wednesday, October 10, 2012 5:03:52 PM
> > Subject: Re: [Engine-devel] Network related hooks in vdsm
> > 
> > 
> > 
> > - Original Message -
> > > From: "Igor Lvovsky" 
> > > To: "vdsm-devel" ,
> > > "engine-devel" 
> > > Cc: "Dan Yasny" 
> > > Sent: Wednesday, October 10, 2012 4:47:28 PM
> > > Subject: [Engine-devel] Network related hooks in vdsm
> > > 
> > > Hi everyone,
> > > As you know vdsm has hooks mechanism and we already support dozen
> > > of
> > > hooks for different needs.
> > > Now it's a network's time.
> > > We would like to get your comments regarding our proposition for
> > > network related hooks.
> > > 
> > > In general we are planning to prepare framework for future
> > > support
> > > of
> > > bunch network related hooks.
> > > Some of them already proposed by Itzik Brown [1] and Dan Yasny
> > > [2].
> > > 
> > > Below you can find the additional hooks list that we propose:
> > > 
> > > Note: In the first stage we can implement these hooks without any
> > > parameters, just to provide an entry point
> > >  for simple hooks.
> > > 
> > > Networks manipulation:
> > > - before_add_network(conf={}, customProperty={})
> > > - after_add_network(conf={}, customProperty={})
> > > - before_del_network(conf={}, customProperty={})
> > > - after_del_network(conf={}, customProperty={})
> > > - before_edit_network(conf={}, customProperty={})
> > > - after_edit_network(conf={}, customProperty={})
> > > - TBD
> > 
> > + VM networking related points in addition to before/after vm
> > start/stop
> > before_hotplug_nic(conf={}, customProperty={})
> > *after_hotplug_nic(conf={}, customProperty={})
> > before_hotunplug_nic(conf={}, customProperty={})
> > *after_hotunplug_nic(conf={}, customProperty={})
> > 
> > * Not sure about use case for those two
> > 
> 
> Yep, part of them already proposed by Itzik (look at [1])

This is what happen when you miss out the end of the email :)

But if I'm reading the comment correctly it indeed doesn't implement two of the 
4 above. My '*' was in the wrong location, indeed the two suggested in the 
patch are the two I can find a use case for, while the other two I'm not sure 
of but I think should be implemented for completeness.  


> 
> > The above will require VM IDs and networks? Sorry that I did not
> > look
> > into the actual implementation of the above naming is more of a
> > guess work, but I think the meaning is clear. There may be other VM
> > networking related entry points I've missed.
> > 
> > > 
> > > Bondings manipulations:
> > > - before_add_bond(conf={}, customProperty={})
> > > - after_add_bond(conf={}, customProperty={})
> > > - before_del_bond(conf={}, customProperty={})
> > > - after_del_bond(conf={}, customProperty={})
> > > - before_edit_bond(conf={}, customProperty={})
> > > - after_edit_bond(conf={}, customProperty={})
> > > - TBD
> > > 
> > > General purpose:
> > > - before_persist_network
> > > - after_persist_network
> > > 
> > > 
> > > Now we just need to figure out the use cases.
> > > 
> > > Your input more than welcome...
> > > 
> > > [1] http://gerrit.ovirt.org/#/c/7224/   - Adding hooks support
> > > for
> > > NIC hotplug
> > > [2] http://gerrit.ovirt.org/#/c/7547/   - Hook: Cisco VM-FEX
> > > support
> > > 
> > > 
> > > Regards,
> > > Igor Lvovsky
> > > ___
> > > Engine-devel mailing list
> > > Engine-devel@ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > > 
> > 
> ___
> Engine-devel mailing list
> Engine-devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Network related hooks in vdsm

2012-10-10 Thread Igor Lvovsky


- Original Message -
> From: "Yaniv Kaul" 
> To: "Igor Lvovsky" 
> Cc: "vdsm-devel" , "engine-devel" 
> , "Dan Yasny"
> 
> Sent: Wednesday, October 10, 2012 4:53:19 PM
> Subject: Re: [Engine-devel] Network related hooks in vdsm
> 
> On 10/10/2012 04:47 PM, Igor Lvovsky wrote:
> >Hi everyone,
> > As you know vdsm has hooks mechanism and we already support dozen
> > of hooks for different needs.
> > Now it's a network's time.
> > We would like to get your comments regarding our proposition for
> > network related hooks.
> >
> > In general we are planning to prepare framework for future support
> > of bunch network related hooks.
> > Some of them already proposed by Itzik Brown [1] and Dan Yasny [2].
> >
> > Below you can find the additional hooks list that we propose:
> >
> > Note: In the first stage we can implement these hooks without any
> > parameters, just to provide an entry point
> >   for simple hooks.
> >
> > Networks manipulation:
> > - before_add_network(conf={}, customProperty={})
> > - after_add_network(conf={}, customProperty={})
> > - before_del_network(conf={}, customProperty={})
> > - after_del_network(conf={}, customProperty={})
> > - before_edit_network(conf={}, customProperty={})
> > - after_edit_network(conf={}, customProperty={})
> > - TBD
> >
> > Bondings manipulations:
> 
> Bond might be interesting because it may require switch configuration
> -
> but so will VLAN changes, so perhaps triggers in VLAN changes are
> worthwhile as well.

You are right, but in our implementation VLAN changes are part of network 
manipulation.
So, it's already covered by list above

> Y.
> 
> > - before_add_bond(conf={}, customProperty={})
> > - after_add_bond(conf={}, customProperty={})
> > - before_del_bond(conf={}, customProperty={})
> > - after_del_bond(conf={}, customProperty={})
> > - before_edit_bond(conf={}, customProperty={})
> > - after_edit_bond(conf={}, customProperty={})
> > - TBD
> >
> > General purpose:
> > - before_persist_network
> > - after_persist_network
> >
> >
> > Now we just need to figure out the use cases.
> >
> > Your input more than welcome...
> >
> > [1] http://gerrit.ovirt.org/#/c/7224/   - Adding hooks support for
> > NIC hotplug
> > [2] http://gerrit.ovirt.org/#/c/7547/   - Hook: Cisco VM-FEX
> > support
> >
> >
> > Regards,
> >  Igor Lvovsky
> > ___
> > Engine-devel mailing list
> > Engine-devel@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> 
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Network related hooks in vdsm

2012-10-10 Thread Igor Lvovsky


- Original Message -
> From: "Simon Grinberg" 
> To: "Igor Lvovsky" 
> Cc: "Dan Yasny" , "vdsm-devel" 
> , "engine-devel"
> 
> Sent: Wednesday, October 10, 2012 5:03:52 PM
> Subject: Re: [Engine-devel] Network related hooks in vdsm
> 
> 
> 
> - Original Message -
> > From: "Igor Lvovsky" 
> > To: "vdsm-devel" ,
> > "engine-devel" 
> > Cc: "Dan Yasny" 
> > Sent: Wednesday, October 10, 2012 4:47:28 PM
> > Subject: [Engine-devel] Network related hooks in vdsm
> > 
> > Hi everyone,
> > As you know vdsm has hooks mechanism and we already support dozen
> > of
> > hooks for different needs.
> > Now it's a network's time.
> > We would like to get your comments regarding our proposition for
> > network related hooks.
> > 
> > In general we are planning to prepare framework for future support
> > of
> > bunch network related hooks.
> > Some of them already proposed by Itzik Brown [1] and Dan Yasny [2].
> > 
> > Below you can find the additional hooks list that we propose:
> > 
> > Note: In the first stage we can implement these hooks without any
> > parameters, just to provide an entry point
> >  for simple hooks.
> > 
> > Networks manipulation:
> > - before_add_network(conf={}, customProperty={})
> > - after_add_network(conf={}, customProperty={})
> > - before_del_network(conf={}, customProperty={})
> > - after_del_network(conf={}, customProperty={})
> > - before_edit_network(conf={}, customProperty={})
> > - after_edit_network(conf={}, customProperty={})
> > - TBD
> 
> + VM networking related points in addition to before/after vm
> start/stop
> before_hotplug_nic(conf={}, customProperty={})
> *after_hotplug_nic(conf={}, customProperty={})
> before_hotunplug_nic(conf={}, customProperty={})
> *after_hotunplug_nic(conf={}, customProperty={})
> 
> * Not sure about use case for those two
> 

Yep, part of them already proposed by Itzik (look at [1])

> The above will require VM IDs and networks? Sorry that I did not look
> into the actual implementation of the above naming is more of a
> guess work, but I think the meaning is clear. There may be other VM
> networking related entry points I've missed.
> 
> > 
> > Bondings manipulations:
> > - before_add_bond(conf={}, customProperty={})
> > - after_add_bond(conf={}, customProperty={})
> > - before_del_bond(conf={}, customProperty={})
> > - after_del_bond(conf={}, customProperty={})
> > - before_edit_bond(conf={}, customProperty={})
> > - after_edit_bond(conf={}, customProperty={})
> > - TBD
> > 
> > General purpose:
> > - before_persist_network
> > - after_persist_network
> > 
> > 
> > Now we just need to figure out the use cases.
> > 
> > Your input more than welcome...
> > 
> > [1] http://gerrit.ovirt.org/#/c/7224/   - Adding hooks support for
> > NIC hotplug
> > [2] http://gerrit.ovirt.org/#/c/7547/   - Hook: Cisco VM-FEX
> > support
> > 
> > 
> > Regards,
> > Igor Lvovsky
> > ___
> > Engine-devel mailing list
> > Engine-devel@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > 
> 
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] allowing user taking vm from vmpool in user-api

2012-10-10 Thread Michael Pasternak
On 10/03/2012 07:16 PM, Simon Grinberg wrote:
> 
> 
> - Original Message -
>> From: "Michael Pasternak" 
>> To: "engine-devel" , "Miki Kenneth" 
>> 
>> Cc: "Michal Skrivanek" 
>> Sent: Wednesday, October 3, 2012 6:13:39 PM
>> Subject: [Engine-devel] allowing user taking vm from vmpool in user-api
>>
>>
>> Apparently we have gap in vmpool for user-api, we need make users
>> being
>> able allocating vm/s to themselves,
>>
>> it can be done via new action on vmpool,
>>
>> URI: /api/vmpools/xxx/allocate|rel=allocate
>>
>> thoughts?
> 
> What is the response a VM ref?  

yes.

> 
>>
>>
>> --
>>
>> Michael Pasternak
>> RedHat, ENG-Virtualization R&D
>> ___
>> Engine-devel mailing list
>> Engine-devel@ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>


-- 

Michael Pasternak
RedHat, ENG-Virtualization R&D
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Network related hooks in vdsm

2012-10-10 Thread Simon Grinberg


- Original Message -
> From: "Igor Lvovsky" 
> To: "vdsm-devel" , "engine-devel" 
> 
> Cc: "Dan Yasny" 
> Sent: Wednesday, October 10, 2012 4:47:28 PM
> Subject: [Engine-devel] Network related hooks in vdsm
> 
> Hi everyone,
> As you know vdsm has hooks mechanism and we already support dozen of
> hooks for different needs.
> Now it's a network's time.
> We would like to get your comments regarding our proposition for
> network related hooks.
> 
> In general we are planning to prepare framework for future support of
> bunch network related hooks.
> Some of them already proposed by Itzik Brown [1] and Dan Yasny [2].
> 
> Below you can find the additional hooks list that we propose:
> 
> Note: In the first stage we can implement these hooks without any
> parameters, just to provide an entry point
>  for simple hooks.
> 
> Networks manipulation:
> - before_add_network(conf={}, customProperty={})
> - after_add_network(conf={}, customProperty={})
> - before_del_network(conf={}, customProperty={})
> - after_del_network(conf={}, customProperty={})
> - before_edit_network(conf={}, customProperty={})
> - after_edit_network(conf={}, customProperty={})
> - TBD

+ VM networking related points in addition to before/after vm start/stop
before_hotplug_nic(conf={}, customProperty={})
*after_hotplug_nic(conf={}, customProperty={})
before_hotunplug_nic(conf={}, customProperty={})
*after_hotunplug_nic(conf={}, customProperty={})

* Not sure about use case for those two 

The above will require VM IDs and networks? Sorry that I did not look into the 
actual implementation of the above naming is more of a guess work, but I think 
the meaning is clear. There may be other VM networking related entry points 
I've missed. 

> 
> Bondings manipulations:
> - before_add_bond(conf={}, customProperty={})
> - after_add_bond(conf={}, customProperty={})
> - before_del_bond(conf={}, customProperty={})
> - after_del_bond(conf={}, customProperty={})
> - before_edit_bond(conf={}, customProperty={})
> - after_edit_bond(conf={}, customProperty={})
> - TBD
> 
> General purpose:
> - before_persist_network
> - after_persist_network
> 
> 
> Now we just need to figure out the use cases.
> 
> Your input more than welcome...
> 
> [1] http://gerrit.ovirt.org/#/c/7224/   - Adding hooks support for
> NIC hotplug
> [2] http://gerrit.ovirt.org/#/c/7547/   - Hook: Cisco VM-FEX support
> 
> 
> Regards,
> Igor Lvovsky
> ___
> Engine-devel mailing list
> Engine-devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Network related hooks in vdsm

2012-10-10 Thread Yaniv Kaul

On 10/10/2012 04:47 PM, Igor Lvovsky wrote:

   Hi everyone,
As you know vdsm has hooks mechanism and we already support dozen of hooks for 
different needs.
Now it's a network's time.
We would like to get your comments regarding our proposition for network 
related hooks.

In general we are planning to prepare framework for future support of bunch 
network related hooks.
Some of them already proposed by Itzik Brown [1] and Dan Yasny [2].

Below you can find the additional hooks list that we propose:

Note: In the first stage we can implement these hooks without any parameters, 
just to provide an entry point
  for simple hooks.

Networks manipulation:
- before_add_network(conf={}, customProperty={})
- after_add_network(conf={}, customProperty={})
- before_del_network(conf={}, customProperty={})
- after_del_network(conf={}, customProperty={})
- before_edit_network(conf={}, customProperty={})
- after_edit_network(conf={}, customProperty={})
- TBD

Bondings manipulations:


Bond might be interesting because it may require switch configuration - 
but so will VLAN changes, so perhaps triggers in VLAN changes are 
worthwhile as well.

Y.


- before_add_bond(conf={}, customProperty={})
- after_add_bond(conf={}, customProperty={})
- before_del_bond(conf={}, customProperty={})
- after_del_bond(conf={}, customProperty={})
- before_edit_bond(conf={}, customProperty={})
- after_edit_bond(conf={}, customProperty={})
- TBD

General purpose:
- before_persist_network
- after_persist_network


Now we just need to figure out the use cases.

Your input more than welcome...

[1] http://gerrit.ovirt.org/#/c/7224/   - Adding hooks support for NIC hotplug
[2] http://gerrit.ovirt.org/#/c/7547/   - Hook: Cisco VM-FEX support


Regards,
 Igor Lvovsky
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


[Engine-devel] Network related hooks in vdsm

2012-10-10 Thread Igor Lvovsky
  Hi everyone,
As you know vdsm has hooks mechanism and we already support dozen of hooks for 
different needs.
Now it's a network's time.
We would like to get your comments regarding our proposition for network 
related hooks.

In general we are planning to prepare framework for future support of bunch 
network related hooks.
Some of them already proposed by Itzik Brown [1] and Dan Yasny [2].

Below you can find the additional hooks list that we propose:

Note: In the first stage we can implement these hooks without any parameters, 
just to provide an entry point
 for simple hooks.

Networks manipulation:
- before_add_network(conf={}, customProperty={})
- after_add_network(conf={}, customProperty={})
- before_del_network(conf={}, customProperty={})
- after_del_network(conf={}, customProperty={})
- before_edit_network(conf={}, customProperty={})
- after_edit_network(conf={}, customProperty={})
- TBD

Bondings manipulations:
- before_add_bond(conf={}, customProperty={})
- after_add_bond(conf={}, customProperty={})
- before_del_bond(conf={}, customProperty={})
- after_del_bond(conf={}, customProperty={})
- before_edit_bond(conf={}, customProperty={})
- after_edit_bond(conf={}, customProperty={})
- TBD

General purpose:
- before_persist_network
- after_persist_network


Now we just need to figure out the use cases.

Your input more than welcome...

[1] http://gerrit.ovirt.org/#/c/7224/   - Adding hooks support for NIC hotplug
[2] http://gerrit.ovirt.org/#/c/7547/   - Hook: Cisco VM-FEX support


Regards,
Igor Lvovsky
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] [vdsm] Mom Balloon policy issue

2012-10-10 Thread Dor Laor

On 10/10/2012 11:23 AM, Noam Slomianko wrote:

Regarding qemu-kvm memory allocation behaviour, this is the clearest 
explanation I found:

"Unless using explicit hugepages for the guest, KVM will
*not* initially allocate all RAM from the host OS. Each
page of RAM is only allocated when the guest OS touches
that page. So if you set memory=8G and currentMemory=4G,
although you might expect resident usage of KVM on the
host to be 4 G, it can in fact be pretty much any value
between 0 and 4 GB depending on whether the guest OS
has touched the memory pages, or as much as 8 GB if the
guest OS has no balloon driver and touches all memory
immediately."[1]


A simpler explanation is that we use demand paging with allocating the 
guest RAM and thus we don't instantiate any page prior to its usage by 
the guest. It's the same with huge pages as long as prealloc isn't called.




And as far as I've seen it behave the same after the guest is ballooned and 
then deflated.

Conclusion, the time in which the guest is allocated the resident memory is not 
deterministic and dependent on use and OS
For example Windows7 uses all available memory as cache while fedora does not.


(not cache but zero the pages).


So an inactive guest running windows7 will consume all possible resident memory 
right away while an inactive guest running fedora might never grow in memory 
size.


It takes time for win7 to touch all the pages (during its boot) and to 
KVM to allocate it so it's not instantaneous.


One can measure exactly the current size of qemu's rss so MOM can be 
aware of it.




[1] http://www.redhat.com/archives/virt-tools-list/2011-January/msg1.html

--
Noam Slomianko
Red Hat Enterprise Virtualization, SLA team

- Original Message -
From: "Adam Litke" 
To: "Noam Slomianko" 
Cc: "Doron Fediuck" , vdsm-de...@lists.fedorahosted.org, 
engine-devel@ovirt.org
Sent: Tuesday, October 9, 2012 8:06:02 PM
Subject: Re: Mom Balloon policy issue

Thanks for writing this.  Some thoughts inline, below.  Also, cc'ing some lists
in case other folks want to participate in the discussion.

On Tue, Oct 09, 2012 at 01:12:30PM -0400, Noam Slomianko wrote:

Greetings,

I've fiddled around with ballooning and wanted to raise a question for debate.

Currently as long as the host is under memory pressure, MOM will try and 
reclaim back memory from all guests with more free memory then a given 
threshold.

Main issue: Guest allocated memory is not the same as the resident (physical) 
memory used by qemu.
This means that when memory is reclaimed back (the balloon is inflated) we 
might not get as much memory as planed back (or non at all).

  *Example1 no memory is reclaimed back:
 name | allocated memory | used by the vm | resident memory used in the 
host by qemu
 Vm1  |   4G |   4G,  |4G
 Vm2  |   4G |   1G   |1G
  - MOM will inflate the balloon in vm2 (as vm has no free memory) and will 
gain no memory


One thing to keep in mind is that VMs having less host RSS than their memory
allocation is a temporary condition.  All VMs will eventually consume their full
allocation if allowed to run.  I'd be curious to know how long this process
takes in general.

We might be able to handle this case by refusing to inflate the balloon if:
 (VM free memory - planned balloon inflation) > host RSS



  *Example1 memory is reclaimed partially:
 name | allocated memory | used by the vm | resident memory used in the 
host by qemu
 Vm1  |   4G |   4G,  |4G
 Vm2  |   4G |   1G   |1G
 Vm3  |   4G |   1G   |4G
  - MOM will inflate the balloon in vm2 and vm3 slowly gaining only from vm3


The above rule extension may help here too.


this behaviour might in the cause us to:
  * spend time reclaiming memory from many guests when we can reclaim only from 
a subgroup
  * be under the impression that we have more potential memory to reclaim when 
we do
  * bring inactive VMs dangerously low as they are constantly reclaimed (I've 
had guests crashing from kernel out of memory)


To address this I suggest that we collect guest memory stats from libvirt as 
well, so we have the option to use them in our calculations.
This can be achieved with the command "virsh dommemstat " which returns
 actual 3915372 (allocated)
 rss 2141580 (resident memory used by qemu)


I would suggest adding these two fields to the VmStats that are collected by
vdsm.  Then, to try it out, add the fields to the GuestMemory Collector.  (Note:
MOM does have a collector that gathers RSS for VMs.  It's called GuestQemuProc).
You can then extend the Balloon policy to add a snippet to check if the proposed
balloon adjustment should be carried out.  You could add the logic to the
change_big_enough function.


additional topic:
  * should we include per guest config (

Re: [Engine-devel] RFE: Netwrok Main Tab

2012-10-10 Thread Simon Grinberg


- Original Message -
> From: "Alona Kaplan" 
> To: "Simon Grinberg" 
> Cc: engine-devel@ovirt.org, "Muli Salem" , "Moti Asayag" 
> , "Avi Tal"
> , "Yaniv Kaul" 
> Sent: Wednesday, October 10, 2012 2:12:10 PM
> Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
> 
> So the conclusion is to leave the General tab but make the vm tab the
> default one?

Depends on other inputs as well, currently you have one that thinks is should 
be removed and one the thinks it should remain for future values :)

> 
> There is no indication for the management networks except of the
> name. The name can be bold or we can add extra column to indicate
> it.

+1 for another column, remember that in the future we should probably allow 
different names for this network.
In addition this tab is common to all the DC, meaning more then one of these 
networks. Hopefully when you'll get sort by column this should be one of the 
attributes to sort by. 


> 
> - Original Message -
> > From: "Simon Grinberg" 
> > To: "Alona Kaplan" 
> > Cc: engine-devel@ovirt.org, "Muli Salem" , "Moti
> > Asayag" , "Avi Tal"
> > , "Yaniv Kaul" 
> > Sent: Wednesday, October 10, 2012 1:40:18 PM
> > Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
> > 
> > 
> > 
> > - Original Message -
> > > From: "Alona Kaplan" 
> > > To: "Simon Grinberg" 
> > > Cc: engine-devel@ovirt.org, "Muli Salem" ,
> > > "Moti
> > > Asayag" , "Avi Tal"
> > > , "Yaniv Kaul" 
> > > Sent: Wednesday, October 10, 2012 10:39:50 AM
> > > Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
> > > 
> > > I agree.
> > > 
> > > Simon, what do you think?
> > > 
> > > - Original Message -
> > > > From: "Yaniv Kaul" 
> > > > To: "Alona Kaplan" 
> > > > Cc: engine-devel@ovirt.org, "Muli Salem" ,
> > > > "Moti
> > > > Asayag" , "Avi Tal"
> > > > , "Simon Grinberg" 
> > > > Sent: Tuesday, October 9, 2012 5:01:39 PM
> > > > Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
> > > > 
> > > > On 10/09/2012 11:34 AM, Alona Kaplan wrote:
> > > > > The wiki was updated with the final design.
> > > > > http://wiki.ovirt.org/wiki/Feature/NetworkMainTab
> > > > >
> > > > > Thanks for the comments,
> > > > > Alona.
> > > > 
> > > > Is the general sub-tab providing any information not available
> > > > in
> > > > the
> > > > grid already?
> > > > If not, lets drop it.
> > 
> > How complicated it will be to re-add it later?
> > 
> > There is no added information in this tab since right now there is
> > not much to report, everything can easily fit into the columns. But
> > thinking forward you may have other values and properties.
> > 
> > Just throwing some here not necessary for the near future so don't
> > be
> > alarmed, some may not/probably won't land in the general tab or the
> > network main tab:
> > - VM network / Storage / Migration / Management (*PS what is the
> > current indication in this tab for management networks except for
> > their names?)
> > - Provider type - like external manager (Plug-in)/internal/etc (not
> > sure it's a main tab item, but ... )
> > - Mutually exclusive? Redundancy? other properties.
> > - Last modified by ___ (It's a property that I would have loved to
> > see on all objects instead of searching the event log)
> > - Modification date
> > - Global SLA Policy (for hierarchical SLA)
> > 
> > I think you got the drift
> > And let's not forget 'Description' may not fully fit in it's column
> > 
> > I think it's worth to have it now then to add it later when the
> > need
> > compels it. But not feeling too strongly one way or the other.
> > 
> > 
> > > > And while it's also consistent to show the clusters as the
> > > > second
> > > > (or
> > > > now, after the above suggestion first) sub-tab, it's not an
> > > > interesting
> > > > tab. I suggest the default when clicking on a network is to
> > > > show
> > > > the
> > > > VMs
> > > > tab (so while the order of the tabs is not changed, the default
> > > > shown
> > > > is).
> > 
> > +1
> > 
> > > > Y.
> > > > 
> > > > 
> > > > >
> > > > > - Original Message -
> > > > >> From: "Simon Grinberg" 
> > > > >> To: "Alona Kaplan" 
> > > > >> Cc: engine-devel@ovirt.org
> > > > >> Sent: Tuesday, October 2, 2012 7:36:44 PM
> > > > >> Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
> > > > >>
> > > > >> Trying to sum up some of the suggestions ('coincidently'
> > > > >> dropping
> > > > >> those which I think a bit too much for first implementation
> > > > >> :))
> > > > >> and
> > > > >> add some suggestions of my own
> > > > >>
> > > > >> 1. On the hosts subtab:
> > > > >> 1.1 Have a radio button that will show either
> > > > >> 1.1.1 All the hosts that this network is attached to
> > > > >> 1.1.2 All the hosts where this network attached to
> > > > >> the
> > > > >> cluster
> > > > >> but not to the host (Very important for non-required
> > > > >> where
> > > > >> the host status does not indicate something is
> > > > >> missin

Re: [Engine-devel] RFE: Netwrok Main Tab

2012-10-10 Thread Alona Kaplan
So the conclusion is to leave the General tab but make the vm tab the default 
one?

There is no indication for the management networks except of the name. The name 
can be bold or we can add extra column to indicate it.

- Original Message -
> From: "Simon Grinberg" 
> To: "Alona Kaplan" 
> Cc: engine-devel@ovirt.org, "Muli Salem" , "Moti Asayag" 
> , "Avi Tal"
> , "Yaniv Kaul" 
> Sent: Wednesday, October 10, 2012 1:40:18 PM
> Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
> 
> 
> 
> - Original Message -
> > From: "Alona Kaplan" 
> > To: "Simon Grinberg" 
> > Cc: engine-devel@ovirt.org, "Muli Salem" , "Moti
> > Asayag" , "Avi Tal"
> > , "Yaniv Kaul" 
> > Sent: Wednesday, October 10, 2012 10:39:50 AM
> > Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
> > 
> > I agree.
> > 
> > Simon, what do you think?
> > 
> > - Original Message -
> > > From: "Yaniv Kaul" 
> > > To: "Alona Kaplan" 
> > > Cc: engine-devel@ovirt.org, "Muli Salem" ,
> > > "Moti
> > > Asayag" , "Avi Tal"
> > > , "Simon Grinberg" 
> > > Sent: Tuesday, October 9, 2012 5:01:39 PM
> > > Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
> > > 
> > > On 10/09/2012 11:34 AM, Alona Kaplan wrote:
> > > > The wiki was updated with the final design.
> > > > http://wiki.ovirt.org/wiki/Feature/NetworkMainTab
> > > >
> > > > Thanks for the comments,
> > > > Alona.
> > > 
> > > Is the general sub-tab providing any information not available in
> > > the
> > > grid already?
> > > If not, lets drop it.
> 
> How complicated it will be to re-add it later?
> 
> There is no added information in this tab since right now there is
> not much to report, everything can easily fit into the columns. But
> thinking forward you may have other values and properties.
> 
> Just throwing some here not necessary for the near future so don't be
> alarmed, some may not/probably won't land in the general tab or the
> network main tab:
> - VM network / Storage / Migration / Management (*PS what is the
> current indication in this tab for management networks except for
> their names?)
> - Provider type - like external manager (Plug-in)/internal/etc (not
> sure it's a main tab item, but ... )
> - Mutually exclusive? Redundancy? other properties.
> - Last modified by ___ (It's a property that I would have loved to
> see on all objects instead of searching the event log)
> - Modification date
> - Global SLA Policy (for hierarchical SLA)
> 
> I think you got the drift
> And let's not forget 'Description' may not fully fit in it's column
> 
> I think it's worth to have it now then to add it later when the need
> compels it. But not feeling too strongly one way or the other.
> 
> 
> > > And while it's also consistent to show the clusters as the second
> > > (or
> > > now, after the above suggestion first) sub-tab, it's not an
> > > interesting
> > > tab. I suggest the default when clicking on a network is to show
> > > the
> > > VMs
> > > tab (so while the order of the tabs is not changed, the default
> > > shown
> > > is).
> 
> +1
> 
> > > Y.
> > > 
> > > 
> > > >
> > > > - Original Message -
> > > >> From: "Simon Grinberg" 
> > > >> To: "Alona Kaplan" 
> > > >> Cc: engine-devel@ovirt.org
> > > >> Sent: Tuesday, October 2, 2012 7:36:44 PM
> > > >> Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
> > > >>
> > > >> Trying to sum up some of the suggestions ('coincidently'
> > > >> dropping
> > > >> those which I think a bit too much for first implementation
> > > >> :))
> > > >> and
> > > >> add some suggestions of my own
> > > >>
> > > >> 1. On the hosts subtab:
> > > >> 1.1 Have a radio button that will show either
> > > >> 1.1.1 All the hosts that this network is attached to
> > > >> 1.1.2 All the hosts where this network attached to the
> > > >> cluster
> > > >> but not to the host (Very important for non-required
> > > >> where
> > > >> the host status does not indicate something is
> > > >> missing)
> > > >> 1.2 Have a remove button for 1.1.1, ManageNetworks button
> > > >> for
> > > >> 1.1.2. Simple add will not do since you don't know where
> > > >> to
> > > >> add.
> > > >>
> > > >> 2. On the vms subtab
> > > >> 2.1 Have a radio button that will show either
> > > >> 2.1.1 All the vms that this network is attached to
> > > >> 2.1.2 All the vms where this network is not attached
> > > >> to
> > > >> 2.2 Have a 'remove' button for 2.1.1, Have 'add' button
> > > >> for
> > > >> 2.1.2
> > > >> 2.3 Allow multiple selection for both actions of 2.2
> > > >> 2.4 Add remove from all button.
> > > >> 2.5 I would not bother to show all the nics attached to
> > > >> the
> > > >> VM
> > > >> from the same network, it's too rare. Just make sure there
> > > >> is
> > > >> no
> > > >> exception if this does exist. So the columns should have
> > > >> 'nic'
> > > >> as
> > > >> the first and there

Re: [Engine-devel] RFE: Netwrok Main Tab

2012-10-10 Thread Simon Grinberg


- Original Message -
> From: "Alona Kaplan" 
> To: "Simon Grinberg" 
> Cc: engine-devel@ovirt.org, "Muli Salem" , "Moti Asayag" 
> , "Avi Tal"
> , "Yaniv Kaul" 
> Sent: Wednesday, October 10, 2012 10:39:50 AM
> Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
> 
> I agree.
> 
> Simon, what do you think?
> 
> - Original Message -
> > From: "Yaniv Kaul" 
> > To: "Alona Kaplan" 
> > Cc: engine-devel@ovirt.org, "Muli Salem" , "Moti
> > Asayag" , "Avi Tal"
> > , "Simon Grinberg" 
> > Sent: Tuesday, October 9, 2012 5:01:39 PM
> > Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
> > 
> > On 10/09/2012 11:34 AM, Alona Kaplan wrote:
> > > The wiki was updated with the final design.
> > > http://wiki.ovirt.org/wiki/Feature/NetworkMainTab
> > >
> > > Thanks for the comments,
> > > Alona.
> > 
> > Is the general sub-tab providing any information not available in
> > the
> > grid already?
> > If not, lets drop it.

How complicated it will be to re-add it later?

There is no added information in this tab since right now there is not much to 
report, everything can easily fit into the columns. But thinking forward you 
may have other values and properties.

Just throwing some here not necessary for the near future so don't be alarmed, 
some may not/probably won't land in the general tab or the network main tab:
- VM network / Storage / Migration / Management (*PS what is the current 
indication in this tab for management networks except for their names?)
- Provider type - like external manager (Plug-in)/internal/etc (not sure it's a 
main tab item, but ... )
- Mutually exclusive? Redundancy? other properties.
- Last modified by ___ (It's a property that I would have loved to see on all 
objects instead of searching the event log)
- Modification date 
- Global SLA Policy (for hierarchical SLA)  

I think you got the drift
And let's not forget 'Description' may not fully fit in it's column  

I think it's worth to have it now then to add it later when the need compels 
it. But not feeling too strongly one way or the other. 


> > And while it's also consistent to show the clusters as the second
> > (or
> > now, after the above suggestion first) sub-tab, it's not an
> > interesting
> > tab. I suggest the default when clicking on a network is to show
> > the
> > VMs
> > tab (so while the order of the tabs is not changed, the default
> > shown
> > is).

+1 

> > Y.
> > 
> > 
> > >
> > > - Original Message -
> > >> From: "Simon Grinberg" 
> > >> To: "Alona Kaplan" 
> > >> Cc: engine-devel@ovirt.org
> > >> Sent: Tuesday, October 2, 2012 7:36:44 PM
> > >> Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
> > >>
> > >> Trying to sum up some of the suggestions ('coincidently'
> > >> dropping
> > >> those which I think a bit too much for first implementation :))
> > >> and
> > >> add some suggestions of my own
> > >>
> > >> 1. On the hosts subtab:
> > >> 1.1 Have a radio button that will show either
> > >> 1.1.1 All the hosts that this network is attached to
> > >> 1.1.2 All the hosts where this network attached to the
> > >> cluster
> > >> but not to the host (Very important for non-required
> > >> where
> > >> the host status does not indicate something is missing)
> > >> 1.2 Have a remove button for 1.1.1, ManageNetworks button
> > >> for
> > >> 1.1.2. Simple add will not do since you don't know where to
> > >> add.
> > >>
> > >> 2. On the vms subtab
> > >> 2.1 Have a radio button that will show either
> > >> 2.1.1 All the vms that this network is attached to
> > >> 2.1.2 All the vms where this network is not attached to
> > >> 2.2 Have a 'remove' button for 2.1.1, Have 'add' button for
> > >> 2.1.2
> > >> 2.3 Allow multiple selection for both actions of 2.2
> > >> 2.4 Add remove from all button.
> > >> 2.5 I would not bother to show all the nics attached to the
> > >> VM
> > >> from the same network, it's too rare. Just make sure there
> > >> is
> > >> no
> > >> exception if this does exist. So the columns should have
> > >> 'nic'
> > >> as
> > >> the first and there should be only one VM per line. If there
> > >> are
> > >> more then one nic per VM then just indicate 'multiple'
> > >>
> > >> 3. Templates subtab - same as VM, drop the expansion of the NICs
> > >> list.
> > >>
> > >> 4. Clusters subtab
> > >> Allow assign to multiple clusters - same as the edit in the
> > >> data
> > >> center tab (Just use the same dialogue)
> > >>
> > >> 5. Main: You have enough space then why not add the MTU column?
> > >>
> > >> 6. Queries for the sub tabs: For each needs the reverse query as
> > >> well
> > >> (Probably more important when adding new network)
> > >>
> > >>
> > >> Oops, I think I mostly added and dropped (almost) nothing :)
> > >>
> > >> Regards,
> > >> Simon
> > >>   
> > >>
> > >> - Original Message -
> > >>> From: "Avi Tal" 
> > 

Re: [Engine-devel] oVirt UI Plugins Presentation: PDF slides

2012-10-10 Thread Itamar Heim

On 10/09/2012 12:39 PM, Vojtech Szocs wrote:

Hi guys,

please find the slides for today's UI Plugins presentation attached to this 
email.

Regards,
Vojtech



___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel



great deck!

few comments/questions:
1. you are using internal entities ("VDS") - the ui API must only use
   entities and actions which match the REST API entities/actions.
   or to begin with, make all these entities share IEntity, and just
   pass the IEntity.EntityId property for them to the plugin).
2. slides 21-25
   iiuc, these are part of the plugin api infrastrcture, not of a
   sample plugin. i think a sample, simple, javascript plugin git repo
   would be great (like we have sample hooks for vdsm).
3. nit(?): the event you fire in the sample is not HostActivated,
   rather HostActivateRequest? i.e., need to distinguish between a
   request for an action, to a real status change of an object (which
   would be a change for a host entity in the grid from one status to a
   another status)?
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Mom Balloon policy issue

2012-10-10 Thread Yaniv Kaul

On 10/10/2012 11:23 AM, Noam Slomianko wrote:

Regarding qemu-kvm memory allocation behaviour, this is the clearest 
explanation I found:

"Unless using explicit hugepages for the guest, KVM will
*not* initially allocate all RAM from the host OS. Each
page of RAM is only allocated when the guest OS touches
that page. So if you set memory=8G and currentMemory=4G,
although you might expect resident usage of KVM on the
host to be 4 G, it can in fact be pretty much any value
between 0 and 4 GB depending on whether the guest OS
has touched the memory pages, or as much as 8 GB if the
guest OS has no balloon driver and touches all memory
immediately."[1]


Windows zeros out all memory on startup, so it essentially touches all 
of its pages right away.

Y.



And as far as I've seen it behave the same after the guest is ballooned and 
then deflated.

Conclusion, the time in which the guest is allocated the resident memory is not 
deterministic and dependent on use and OS
For example Windows7 uses all available memory as cache while fedora does not.
So an inactive guest running windows7 will consume all possible resident memory 
right away while an inactive guest running fedora might never grow in memory 
size.

[1] http://www.redhat.com/archives/virt-tools-list/2011-January/msg1.html

--
Noam Slomianko
Red Hat Enterprise Virtualization, SLA team

- Original Message -
From: "Adam Litke" 
To: "Noam Slomianko" 
Cc: "Doron Fediuck" , vdsm-de...@lists.fedorahosted.org, 
engine-devel@ovirt.org
Sent: Tuesday, October 9, 2012 8:06:02 PM
Subject: Re: Mom Balloon policy issue

Thanks for writing this.  Some thoughts inline, below.  Also, cc'ing some lists
in case other folks want to participate in the discussion.

On Tue, Oct 09, 2012 at 01:12:30PM -0400, Noam Slomianko wrote:

Greetings,

I've fiddled around with ballooning and wanted to raise a question for debate.

Currently as long as the host is under memory pressure, MOM will try and 
reclaim back memory from all guests with more free memory then a given 
threshold.

Main issue: Guest allocated memory is not the same as the resident (physical) 
memory used by qemu.
This means that when memory is reclaimed back (the balloon is inflated) we 
might not get as much memory as planed back (or non at all).

  *Example1 no memory is reclaimed back:
 name | allocated memory | used by the vm | resident memory used in the 
host by qemu
 Vm1  |   4G |   4G,  |4G
 Vm2  |   4G |   1G   |1G
  - MOM will inflate the balloon in vm2 (as vm has no free memory) and will 
gain no memory

One thing to keep in mind is that VMs having less host RSS than their memory
allocation is a temporary condition.  All VMs will eventually consume their full
allocation if allowed to run.  I'd be curious to know how long this process
takes in general.

We might be able to handle this case by refusing to inflate the balloon if:
 (VM free memory - planned balloon inflation) > host RSS



  *Example1 memory is reclaimed partially:
 name | allocated memory | used by the vm | resident memory used in the 
host by qemu
 Vm1  |   4G |   4G,  |4G
 Vm2  |   4G |   1G   |1G
 Vm3  |   4G |   1G   |4G
  - MOM will inflate the balloon in vm2 and vm3 slowly gaining only from vm3

The above rule extension may help here too.


this behaviour might in the cause us to:
  * spend time reclaiming memory from many guests when we can reclaim only from 
a subgroup
  * be under the impression that we have more potential memory to reclaim when 
we do
  * bring inactive VMs dangerously low as they are constantly reclaimed (I've 
had guests crashing from kernel out of memory)


To address this I suggest that we collect guest memory stats from libvirt as 
well, so we have the option to use them in our calculations.
This can be achieved with the command "virsh dommemstat " which returns
 actual 3915372 (allocated)
 rss 2141580 (resident memory used by qemu)

I would suggest adding these two fields to the VmStats that are collected by
vdsm.  Then, to try it out, add the fields to the GuestMemory Collector.  (Note:
MOM does have a collector that gathers RSS for VMs.  It's called GuestQemuProc).
You can then extend the Balloon policy to add a snippet to check if the proposed
balloon adjustment should be carried out.  You could add the logic to the
change_big_enough function.


additional topic:
  * should we include per guest config (for example a hard minimum memory cap, 
this vm cannot run effectively with less then 1G memory)

Yes.  This is probably something we want to do.  There is a whole topic around
VM tagging that we should consider.  In the future we will want to be able to do
many different things in policy based on a VMs tag.  For example, some VMs may
be completely exempt from bal

Re: [Engine-devel] Mom Balloon policy issue

2012-10-10 Thread Noam Slomianko
Regarding qemu-kvm memory allocation behaviour, this is the clearest 
explanation I found:

"Unless using explicit hugepages for the guest, KVM will
*not* initially allocate all RAM from the host OS. Each
page of RAM is only allocated when the guest OS touches
that page. So if you set memory=8G and currentMemory=4G,
although you might expect resident usage of KVM on the
host to be 4 G, it can in fact be pretty much any value
between 0 and 4 GB depending on whether the guest OS
has touched the memory pages, or as much as 8 GB if the
guest OS has no balloon driver and touches all memory
immediately."[1]

And as far as I've seen it behave the same after the guest is ballooned and 
then deflated.

Conclusion, the time in which the guest is allocated the resident memory is not 
deterministic and dependent on use and OS 
For example Windows7 uses all available memory as cache while fedora does not.
So an inactive guest running windows7 will consume all possible resident memory 
right away while an inactive guest running fedora might never grow in memory 
size.

[1] http://www.redhat.com/archives/virt-tools-list/2011-January/msg1.html

--
Noam Slomianko
Red Hat Enterprise Virtualization, SLA team

- Original Message -
From: "Adam Litke" 
To: "Noam Slomianko" 
Cc: "Doron Fediuck" , vdsm-de...@lists.fedorahosted.org, 
engine-devel@ovirt.org
Sent: Tuesday, October 9, 2012 8:06:02 PM
Subject: Re: Mom Balloon policy issue

Thanks for writing this.  Some thoughts inline, below.  Also, cc'ing some lists
in case other folks want to participate in the discussion.

On Tue, Oct 09, 2012 at 01:12:30PM -0400, Noam Slomianko wrote:
> Greetings,
> 
> I've fiddled around with ballooning and wanted to raise a question for debate.
> 
> Currently as long as the host is under memory pressure, MOM will try and 
> reclaim back memory from all guests with more free memory then a given 
> threshold.
> 
> Main issue: Guest allocated memory is not the same as the resident (physical) 
> memory used by qemu.
> This means that when memory is reclaimed back (the balloon is inflated) we 
> might not get as much memory as planed back (or non at all).
> 
>  *Example1 no memory is reclaimed back:
> name | allocated memory | used by the vm | resident memory used in the 
> host by qemu
> Vm1  |   4G |   4G,  |4G
> Vm2  |   4G |   1G   |1G
>  - MOM will inflate the balloon in vm2 (as vm has no free memory) and will 
> gain no memory

One thing to keep in mind is that VMs having less host RSS than their memory
allocation is a temporary condition.  All VMs will eventually consume their full
allocation if allowed to run.  I'd be curious to know how long this process
takes in general.

We might be able to handle this case by refusing to inflate the balloon if:
(VM free memory - planned balloon inflation) > host RSS


>  *Example1 memory is reclaimed partially:
> name | allocated memory | used by the vm | resident memory used in the 
> host by qemu
> Vm1  |   4G |   4G,  |4G
> Vm2  |   4G |   1G   |1G
> Vm3  |   4G |   1G   |4G
>  - MOM will inflate the balloon in vm2 and vm3 slowly gaining only from vm3

The above rule extension may help here too.

> this behaviour might in the cause us to:
>  * spend time reclaiming memory from many guests when we can reclaim only 
> from a subgroup
>  * be under the impression that we have more potential memory to reclaim when 
> we do
>  * bring inactive VMs dangerously low as they are constantly reclaimed (I've 
> had guests crashing from kernel out of memory)
> 
> 
> To address this I suggest that we collect guest memory stats from libvirt as 
> well, so we have the option to use them in our calculations.
> This can be achieved with the command "virsh dommemstat " which 
> returns
> actual 3915372 (allocated)
> rss 2141580 (resident memory used by qemu)

I would suggest adding these two fields to the VmStats that are collected by
vdsm.  Then, to try it out, add the fields to the GuestMemory Collector.  (Note:
MOM does have a collector that gathers RSS for VMs.  It's called GuestQemuProc).
You can then extend the Balloon policy to add a snippet to check if the proposed
balloon adjustment should be carried out.  You could add the logic to the
change_big_enough function.

> additional topic:
>  * should we include per guest config (for example a hard minimum memory cap, 
> this vm cannot run effectively with less then 1G memory)

Yes.  This is probably something we want to do.  There is a whole topic around
VM tagging that we should consider.  In the future we will want to be able to do
many different things in policy based on a VMs tag.  For example, some VMs may
be completely exempt from ballooning.  Others may have a minimum limit.

I want to avoid passing in the raw gues

Re: [Engine-devel] RFE: Netwrok Main Tab

2012-10-10 Thread Alona Kaplan
I agree.

Simon, what do you think?

- Original Message -
> From: "Yaniv Kaul" 
> To: "Alona Kaplan" 
> Cc: engine-devel@ovirt.org, "Muli Salem" , "Moti Asayag" 
> , "Avi Tal"
> , "Simon Grinberg" 
> Sent: Tuesday, October 9, 2012 5:01:39 PM
> Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
> 
> On 10/09/2012 11:34 AM, Alona Kaplan wrote:
> > The wiki was updated with the final design.
> > http://wiki.ovirt.org/wiki/Feature/NetworkMainTab
> >
> > Thanks for the comments,
> > Alona.
> 
> Is the general sub-tab providing any information not available in the
> grid already?
> If not, lets drop it.
> And while it's also consistent to show the clusters as the second (or
> now, after the above suggestion first) sub-tab, it's not an
> interesting
> tab. I suggest the default when clicking on a network is to show the
> VMs
> tab (so while the order of the tabs is not changed, the default shown
> is).
> Y.
> 
> 
> >
> > - Original Message -
> >> From: "Simon Grinberg" 
> >> To: "Alona Kaplan" 
> >> Cc: engine-devel@ovirt.org
> >> Sent: Tuesday, October 2, 2012 7:36:44 PM
> >> Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
> >>
> >> Trying to sum up some of the suggestions ('coincidently' dropping
> >> those which I think a bit too much for first implementation :))
> >> and
> >> add some suggestions of my own
> >>
> >> 1. On the hosts subtab:
> >> 1.1 Have a radio button that will show either
> >> 1.1.1 All the hosts that this network is attached to
> >> 1.1.2 All the hosts where this network attached to the
> >> cluster
> >> but not to the host (Very important for non-required where
> >> the host status does not indicate something is missing)
> >> 1.2 Have a remove button for 1.1.1, ManageNetworks button for
> >> 1.1.2. Simple add will not do since you don't know where to
> >> add.
> >>
> >> 2. On the vms subtab
> >> 2.1 Have a radio button that will show either
> >> 2.1.1 All the vms that this network is attached to
> >> 2.1.2 All the vms where this network is not attached to
> >> 2.2 Have a 'remove' button for 2.1.1, Have 'add' button for
> >> 2.1.2
> >> 2.3 Allow multiple selection for both actions of 2.2
> >> 2.4 Add remove from all button.
> >> 2.5 I would not bother to show all the nics attached to the VM
> >> from the same network, it's too rare. Just make sure there is
> >> no
> >> exception if this does exist. So the columns should have 'nic'
> >> as
> >> the first and there should be only one VM per line. If there
> >> are
> >> more then one nic per VM then just indicate 'multiple'
> >>
> >> 3. Templates subtab - same as VM, drop the expansion of the NICs
> >> list.
> >>
> >> 4. Clusters subtab
> >> Allow assign to multiple clusters - same as the edit in the
> >> data
> >> center tab (Just use the same dialogue)
> >>
> >> 5. Main: You have enough space then why not add the MTU column?
> >>
> >> 6. Queries for the sub tabs: For each needs the reverse query as
> >> well
> >> (Probably more important when adding new network)
> >>
> >>
> >> Oops, I think I mostly added and dropped (almost) nothing :)
> >>
> >> Regards,
> >> Simon
> >>   
> >>
> >> - Original Message -
> >>> From: "Avi Tal" 
> >>> To: "Yaniv Kaul" 
> >>> Cc: engine-devel@ovirt.org
> >>> Sent: Monday, September 24, 2012 10:13:52 AM
> >>> Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
> >>>
> >>>
> >>>
> >>> - Original Message -
>  From: "Yaniv Kaul" 
>  To: "Moti Asayag" 
>  Cc: engine-devel@ovirt.org
>  Sent: Sunday, September 23, 2012 6:16:47 PM
>  Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
> 
>  On 09/23/2012 06:12 PM, Moti Asayag wrote:
> > On 09/23/2012 05:01 PM, Yaniv Kaul wrote:
> >> On 09/23/2012 04:55 PM, Alona Kaplan wrote:
> >>> - Original Message -
>  From: "Avi Tal" 
>  To: "Alona Kaplan" 
>  Cc: engine-devel@ovirt.org
>  Sent: Sunday, September 23, 2012 4:17:26 PM
>  Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
> 
> 
> 
>  - Original Message -
> > From: "Alona Kaplan" 
> > To: "Avi Tal" 
> > Cc: engine-devel@ovirt.org
> > Sent: Sunday, September 23, 2012 1:31:32 PM
> > Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
> >
> > 1. What actions do you suggest to add?
>  Add, Edit and remove actions of each component.
> 
> >>> Host:
> >>> Add- What will add action on Networks->Hosts sub tab do?
> >>> Choose
> >>> an
> >>> existing host and attach the network to one of its nics?
> >>> How will it work? after choosing the host you will open
> >>> setupNetworks
> >>> window and just enable dragging of the selected network (in
> >>> the
> >>> main
> >>> tab) to a nic? I think it is too much.
> >>>
> >>> Edi