Re: [Engine-devel] NUMA support action items

2014-04-06 Thread Chegu Vinod

On 4/3/2014 7:11 AM, Gilad Chaplik wrote:

- Original Message -

From: Chegu Vinod chegu_vi...@hp.com
To: Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ) xiao-lei@hp.com
Cc: Einav Cohen eco...@redhat.com, Shang-Chun Liang (David Liang, 
HPservers-Core-OE-PSC)
shangchun.li...@hp.com, Chuan Liao (Jason Liao, HPservers-Core-OE-PSC) 
chuan.l...@hp.com, msi...@redhat.com,
Da-huai Tang (Gary, MCXS-CQ) da-huai.t...@hp.com, Malini Rao m...@redhat.com, 
Eldan Hildesheim
ehild...@redhat.com, Doron Fediuck dfedi...@redhat.com, sher...@redhat.com, 
Alexander Wels
aw...@redhat.com, Gilad Chaplik gchap...@redhat.com
Sent: Thursday, April 3, 2014 3:28:03 PM
Subject: RE: NUMA support action items

Hi Bruce,

The virtual NUMA layout in the guest is a very simple one (not multi-level
etc). It is generated by qemu+seabios... and there is no relationship with
the host NUMA node distances etc.  Let us not worry about gathering Virtual
NUMA node distances for now.

Vinod


CC'ing devel list as well.

Having said that, I don't see a reason why not to prepare an infrastructure for 
that (if it's free) for future versions (guest agent will collect vNuma data in 
some point in time).


If you think having this Virtual NUMA topology (along with the virtual 
numa node *distance* info.) really helps some future use cases then pl. 
go ahead...


Vinod





Thanks,
Gilad.


-Original Message-
From: Shi, Xiao-Lei (Bruce, HP Servers-PSC-CQ)
Sent: Thursday, April 03, 2014 12:41 AM
To: Vinod, Chegu
Cc: Einav Cohen; Liang, Shang-Chun (David Liang, HPservers-Core-OE-PSC);
Liao, Chuan (Jason Liao, HPservers-Core-OE-PSC); msi...@redhat.com; Tang,
Da-huai (Gary, MCXS-CQ); Malini Rao; Eldan Hildesheim; Doron Fediuck;
sher...@redhat.com; Alexander Wels; Gilad Chaplik
Subject: RE: NUMA support action items

Hi Vinod,

Is it meaningful for us to collect the distance information of vm numa node
(maybe in future, not now)?
In my understanding, vm numa topology is a simulation of numa topology, since
the vcpus are just threads, I don't know how the vm numa node distances are
calculated in vm. Is there any relationship between the vNode distances and
host node distances?

Thanks  Best Regards
Shi, Xiao-Lei (Bruce)

Hewlett-Packard Co., Ltd.
HP Servers Core Platform Software China Telephone +86 23 65683093 Mobile +86
18696583447 Email xiao-lei@hp.com


-Original Message-
From: Vinod, Chegu
Sent: Thursday, April 03, 2014 7:18 AM
To: Gilad Chaplik
Cc: Einav Cohen; Liang, Shang-Chun (David Liang, HPservers-Core-OE-PSC);
Liao, Chuan (Jason Liao, HPservers-Core-OE-PSC); msi...@redhat.com; Shi,
Xiao-Lei (Bruce, HP Servers-PSC-CQ); Tang, Da-huai (Gary, MCXS-CQ); Malini
Rao; Eldan Hildesheim; Doron Fediuck; sher...@redhat.com; Alexander Wels
Subject: RE: NUMA support action items

Not sure what the correct way to do this isbut here is a suggestion.

Let a given host server diagram shown be very generic...i.e. show the N
sockets/nodes numbered from 0 thru N-1.  Show the amount of memory and the
list of CPUs in each of those sockets/nodes.
Draw a generic Interconnect fabric [box] in between which all the sockets
connect to

Ideally ... Under that host diagram we could show the NUMA node distances in
text format (as you know this is derived from the numactl -H and then
conveyed from VDSM- oVIrt engine etc).
That distance info. will tell the user what the distance between a pair of
sockets/nodes are (and they can then do what they wish after that :)).

Vinod

-Original Message-
From: Gilad Chaplik [mailto:gchap...@redhat.com]
Sent: Wednesday, April 02, 2014 4:09 PM
To: Vinod, Chegu
Cc: Einav Cohen; Liang, Shang-Chun (David Liang, HPservers-Core-OE-PSC);
Liao, Chuan (Jason Liao, HPservers-Core-OE-PSC); msi...@redhat.com; Shi,
Xiao-Lei (Bruce, HP Servers-PSC-CQ); Tang, Da-huai (Gary, MCXS-CQ); Malini
Rao; Eldan Hildesheim; Doron Fediuck; sher...@redhat.com; Alexander Wels
Subject: Re: NUMA support action items

Thank you Vinod for the much elaborate explanation.
GUI-wise, do you want to show those numbers? maybe for first phase, enough to
show them via API?

A thought, According to your example there could be up to 2 distances, so
maybe the 'closer' nodes can be on the same column or sth; I mean to try an
illustrate it graphically rather than with numbers (we have enough of those
:)).

Thanks,
Gilad.

- Original Message -

From: Chegu Vinod chegu_vi...@hp.com
To: Einav Cohen eco...@redhat.com
Cc: Gilad Chaplik gchap...@redhat.com, Shang-Chun Liang (David Liang,
HPservers-Core-OE-PSC)
shangchun.li...@hp.com, Chuan Liao (Jason Liao,
HPservers-Core-OE-PSC) chuan.l...@hp.com, msi...@redhat.com, Xiao-Lei
Shi (Bruce, HP Servers-PSC-CQ) xiao-lei@hp.com, Da-huai Tang
(Gary, MCXS-CQ)
da-huai.t...@hp.com, Malini Rao m...@redhat.com, Eldan Hildesheim
ehild...@redhat.com, Doron Fediuck
dfedi...@redhat.com, sher...@redhat.com, Alexander Wels
aw...@redhat.com
Sent: Saturday, March 29, 2014 8:15:56 AM
Subject: Re: NUMA support 

Re: [Engine-devel] NUMA support action items

2014-04-06 Thread Gilad Chaplik
- Original Message -
 From: Chegu Vinod chegu_vi...@hp.com
 To: Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ) xiao-lei@hp.com
 Cc: Einav Cohen eco...@redhat.com, Shang-Chun Liang (David Liang, 
 HPservers-Core-OE-PSC)
 shangchun.li...@hp.com, Chuan Liao (Jason Liao, HPservers-Core-OE-PSC) 
 chuan.l...@hp.com, msi...@redhat.com,
 Da-huai Tang (Gary, MCXS-CQ) da-huai.t...@hp.com, Malini Rao 
 m...@redhat.com, Eldan Hildesheim
 ehild...@redhat.com, Doron Fediuck dfedi...@redhat.com, 
 sher...@redhat.com, Alexander Wels
 aw...@redhat.com, Gilad Chaplik gchap...@redhat.com
 Sent: Thursday, April 3, 2014 3:28:03 PM
 Subject: RE: NUMA support action items
 
 Hi Bruce,
 
 The virtual NUMA layout in the guest is a very simple one (not multi-level
 etc). It is generated by qemu+seabios... and there is no relationship with
 the host NUMA node distances etc.  Let us not worry about gathering Virtual
 NUMA node distances for now.
 
 Vinod
 

CC'ing devel list as well.

Having said that, I don't see a reason why not to prepare an infrastructure for 
that (if it's free) for future versions (guest agent will collect vNuma data in 
some point in time).

Thanks, 
Gilad.

 
 -Original Message-
 From: Shi, Xiao-Lei (Bruce, HP Servers-PSC-CQ)
 Sent: Thursday, April 03, 2014 12:41 AM
 To: Vinod, Chegu
 Cc: Einav Cohen; Liang, Shang-Chun (David Liang, HPservers-Core-OE-PSC);
 Liao, Chuan (Jason Liao, HPservers-Core-OE-PSC); msi...@redhat.com; Tang,
 Da-huai (Gary, MCXS-CQ); Malini Rao; Eldan Hildesheim; Doron Fediuck;
 sher...@redhat.com; Alexander Wels; Gilad Chaplik
 Subject: RE: NUMA support action items
 
 Hi Vinod,
 
 Is it meaningful for us to collect the distance information of vm numa node
 (maybe in future, not now)?
 In my understanding, vm numa topology is a simulation of numa topology, since
 the vcpus are just threads, I don't know how the vm numa node distances are
 calculated in vm. Is there any relationship between the vNode distances and
 host node distances?
 
 Thanks  Best Regards
 Shi, Xiao-Lei (Bruce)
 
 Hewlett-Packard Co., Ltd.
 HP Servers Core Platform Software China Telephone +86 23 65683093 Mobile +86
 18696583447 Email xiao-lei@hp.com
 
 
 -Original Message-
 From: Vinod, Chegu
 Sent: Thursday, April 03, 2014 7:18 AM
 To: Gilad Chaplik
 Cc: Einav Cohen; Liang, Shang-Chun (David Liang, HPservers-Core-OE-PSC);
 Liao, Chuan (Jason Liao, HPservers-Core-OE-PSC); msi...@redhat.com; Shi,
 Xiao-Lei (Bruce, HP Servers-PSC-CQ); Tang, Da-huai (Gary, MCXS-CQ); Malini
 Rao; Eldan Hildesheim; Doron Fediuck; sher...@redhat.com; Alexander Wels
 Subject: RE: NUMA support action items
 
 Not sure what the correct way to do this isbut here is a suggestion.
 
 Let a given host server diagram shown be very generic...i.e. show the N
 sockets/nodes numbered from 0 thru N-1.  Show the amount of memory and the
 list of CPUs in each of those sockets/nodes.
 Draw a generic Interconnect fabric [box] in between which all the sockets
 connect to
 
 Ideally ... Under that host diagram we could show the NUMA node distances in
 text format (as you know this is derived from the numactl -H and then
 conveyed from VDSM- oVIrt engine etc).
 That distance info. will tell the user what the distance between a pair of
 sockets/nodes are (and they can then do what they wish after that :)).
 
 Vinod
 
 -Original Message-
 From: Gilad Chaplik [mailto:gchap...@redhat.com]
 Sent: Wednesday, April 02, 2014 4:09 PM
 To: Vinod, Chegu
 Cc: Einav Cohen; Liang, Shang-Chun (David Liang, HPservers-Core-OE-PSC);
 Liao, Chuan (Jason Liao, HPservers-Core-OE-PSC); msi...@redhat.com; Shi,
 Xiao-Lei (Bruce, HP Servers-PSC-CQ); Tang, Da-huai (Gary, MCXS-CQ); Malini
 Rao; Eldan Hildesheim; Doron Fediuck; sher...@redhat.com; Alexander Wels
 Subject: Re: NUMA support action items
 
 Thank you Vinod for the much elaborate explanation.
 GUI-wise, do you want to show those numbers? maybe for first phase, enough to
 show them via API?
 
 A thought, According to your example there could be up to 2 distances, so
 maybe the 'closer' nodes can be on the same column or sth; I mean to try an
 illustrate it graphically rather than with numbers (we have enough of those
 :)).
 
 Thanks,
 Gilad.
 
 - Original Message -
  From: Chegu Vinod chegu_vi...@hp.com
  To: Einav Cohen eco...@redhat.com
  Cc: Gilad Chaplik gchap...@redhat.com, Shang-Chun Liang (David Liang,
  HPservers-Core-OE-PSC)
  shangchun.li...@hp.com, Chuan Liao (Jason Liao,
  HPservers-Core-OE-PSC) chuan.l...@hp.com, msi...@redhat.com, Xiao-Lei
  Shi (Bruce, HP Servers-PSC-CQ) xiao-lei@hp.com, Da-huai Tang
  (Gary, MCXS-CQ)
  da-huai.t...@hp.com, Malini Rao m...@redhat.com, Eldan Hildesheim
  ehild...@redhat.com, Doron Fediuck
  dfedi...@redhat.com, sher...@redhat.com, Alexander Wels
  aw...@redhat.com
  Sent: Saturday, March 29, 2014 8:15:56 AM
  Subject: Re: NUMA support action items
  
  On 3/27/2014 10:42 AM, Einav Cohen wrote:
   Hi Vinod, thank you very 

Re: [Engine-devel] Please help us to review our database schema design with NUMA feature on ovirt

2014-04-06 Thread Gilad Chaplik
- Original Message -
 From: Eli Mesika emes...@redhat.com
 To: Omer Frenkel ofren...@redhat.com
 Cc: Gilad Chaplik gchap...@redhat.com, Xiao-Lei Shi (Bruce, HP 
 Servers-PSC-CQ) xiao-lei@hp.com, Roy
 Golan rgo...@redhat.com, Chuan Liao (Jason Liao, HPservers-Core-OE-PSC) 
 chuan.l...@hp.com, Doron Fediuck
 dfedi...@redhat.com, Chegu Vinod chegu_vi...@hp.com, Shang-Chun Liang 
 (David Liang, HPservers-Core-OE-PSC)
 shangchun.li...@hp.com, Yaniv Dary yd...@redhat.com, 
 engine-devel@ovirt.org
 Sent: Thursday, April 3, 2014 4:51:38 PM
 Subject: Re: Please help us to review our database schema design with NUMA 
 feature on ovirt
 
 
 
 - Original Message -
  From: Omer Frenkel ofren...@redhat.com
  To: Gilad Chaplik gchap...@redhat.com
  Cc: Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ) xiao-lei@hp.com, Roy
  Golan rgo...@redhat.com, Eli Mesika
  emes...@redhat.com, Chuan Liao (Jason Liao, HPservers-Core-OE-PSC)
  chuan.l...@hp.com, Doron Fediuck
  dfedi...@redhat.com, Chegu Vinod chegu_vi...@hp.com, Shang-Chun
  Liang (David Liang, HPservers-Core-OE-PSC)
  shangchun.li...@hp.com, Yaniv Dary yd...@redhat.com,
  engine-devel@ovirt.org
  Sent: Thursday, April 3, 2014 4:45:49 PM
  Subject: Re: Please help us to review our database schema design with NUMA
  feature on ovirt
  
  
  
  - Original Message -
   From: Gilad Chaplik gchap...@redhat.com
   To: Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ) xiao-lei@hp.com, Roy
   Golan rgo...@redhat.com, Omer Frenkel
   ofren...@redhat.com
   Cc: Eli Mesika emes...@redhat.com, Roy Golan rgo...@redhat.com,
   Chuan Liao (Jason Liao,
   HPservers-Core-OE-PSC) chuan.l...@hp.com, Doron Fediuck
   dfedi...@redhat.com, Chegu Vinod
   chegu_vi...@hp.com, Shang-Chun Liang (David Liang,
   HPservers-Core-OE-PSC) shangchun.li...@hp.com, Yaniv Dary
   yd...@redhat.com, engine-devel@ovirt.org
   Sent: Monday, March 31, 2014 4:31:17 PM
   Subject: Re: Please help us to review our database schema design with
   NUMA
   feature on ovirt
   
   adding Roy  Omer.
   
   why CPU topology is in dynamic?
   
  
  the guideline we (try to) follow today is (i know you can find exceptions
  to
  this, but this is how it suppose to be):
  vm/vds static - user configurable information (whatever you have in
  add/edit)
  vm dynamic - information related to a running instance, the is changed in a
  somewhat small frequency
  vds dynamic - host capabilities, information from the host that doesn't
  change while the host is up.
  statistics - all the rest, mostly statistics, and other fast changing
  information.
 
 IMO it is a little more complicated and the terms static/dynamic does not
 reflect today the meaning of those terms rather, if engine knows the values
 by itself , this is considered static and if it has to ask the VDSM for
 the value then this is considered dynamic.
 For example cpu_cores is a static information but it is stored in
 vds_dynamic since we get the value from VDSM.
 I found those terms very confusing and a good candidate for a serious
 refactoring in future
 

+1, I've opened another thread about it in devel list, and cc'ed Omer.

 
  
   Thanks,
   Gilad.
   
   - Original Message -
From: Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ) xiao-lei@hp.com
To: Eli Mesika emes...@redhat.com
Cc: Gilad Chaplik gchap...@redhat.com, Roy Golan
rgo...@redhat.com,
Chuan Liao (Jason Liao,
HPservers-Core-OE-PSC) chuan.l...@hp.com, Doron Fediuck
dfedi...@redhat.com, Chegu Vinod
chegu_vi...@hp.com, Shang-Chun Liang (David Liang,
HPservers-Core-OE-PSC) shangchun.li...@hp.com, Yaniv Dary
yd...@redhat.com, engine-devel@ovirt.org
Sent: Monday, March 31, 2014 3:20:33 PM
Subject: RE: Please help us to review our database schema design with
NUMA
feature on ovirt

Thanks Eli.
I will move the vm level NUMA fields to vm_dynamic, and the related
database
schema will be updated accordingly.

Thanks  Best Regards
Shi, Xiao-Lei (Bruce)

Hewlett-Packard Co., Ltd.
HP Servers Core Platform Software China
Telephone +86 23 65683093
Mobile +86 18696583447
Email xiao-lei@hp.com

-Original Message-
From: Eli Mesika [mailto:emes...@redhat.com]
Sent: Monday, March 31, 2014 5:49 PM
To: Shi, Xiao-Lei (Bruce, HP Servers-PSC-CQ)
Cc: Gilad Chaplik; Roy Golan; Liao, Chuan (Jason Liao,
HPservers-Core-OE-PSC); Doron Fediuck; Vinod, Chegu; Liang, Shang-Chun
(David Liang, HPservers-Core-OE-PSC); Yaniv Dary;
engine-devel@ovirt.org
Subject: Re: Please help us to review our database schema design with
NUMA
feature on ovirt



- Original Message -
 From: Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ) xiao-lei@hp.com
 To: Gilad Chaplik gchap...@redhat.com, Eli Mesika
 emes...@redhat.com, Roy Golan rgo...@redhat.com
 Cc: Chuan Liao (Jason Liao, HPservers-Core-OE-PSC)
 chuan.l...@hp.com, 

Re: [Engine-devel] Please help us to review our database schema design with NUMA feature on ovirt

2014-04-06 Thread Eli Mesika


- Original Message -
 From: Omer Frenkel ofren...@redhat.com
 To: Gilad Chaplik gchap...@redhat.com
 Cc: Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ) xiao-lei@hp.com, Roy 
 Golan rgo...@redhat.com, Eli Mesika
 emes...@redhat.com, Chuan Liao (Jason Liao, HPservers-Core-OE-PSC) 
 chuan.l...@hp.com, Doron Fediuck
 dfedi...@redhat.com, Chegu Vinod chegu_vi...@hp.com, Shang-Chun Liang 
 (David Liang, HPservers-Core-OE-PSC)
 shangchun.li...@hp.com, Yaniv Dary yd...@redhat.com, 
 engine-devel@ovirt.org
 Sent: Thursday, April 3, 2014 4:45:49 PM
 Subject: Re: Please help us to review our database schema design with NUMA 
 feature on ovirt
 
 
 
 - Original Message -
  From: Gilad Chaplik gchap...@redhat.com
  To: Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ) xiao-lei@hp.com, Roy
  Golan rgo...@redhat.com, Omer Frenkel
  ofren...@redhat.com
  Cc: Eli Mesika emes...@redhat.com, Roy Golan rgo...@redhat.com,
  Chuan Liao (Jason Liao,
  HPservers-Core-OE-PSC) chuan.l...@hp.com, Doron Fediuck
  dfedi...@redhat.com, Chegu Vinod
  chegu_vi...@hp.com, Shang-Chun Liang (David Liang,
  HPservers-Core-OE-PSC) shangchun.li...@hp.com, Yaniv Dary
  yd...@redhat.com, engine-devel@ovirt.org
  Sent: Monday, March 31, 2014 4:31:17 PM
  Subject: Re: Please help us to review our database schema design with NUMA
  feature on ovirt
  
  adding Roy  Omer.
  
  why CPU topology is in dynamic?
  
 
 the guideline we (try to) follow today is (i know you can find exceptions to
 this, but this is how it suppose to be):
 vm/vds static - user configurable information (whatever you have in add/edit)
 vm dynamic - information related to a running instance, the is changed in a
 somewhat small frequency
 vds dynamic - host capabilities, information from the host that doesn't
 change while the host is up.
 statistics - all the rest, mostly statistics, and other fast changing
 information.

IMO it is a little more complicated and the terms static/dynamic does not 
reflect today the meaning of those terms rather, if engine knows the values by 
itself , this is considered static and if it has to ask the VDSM for the 
value then this is considered dynamic.
For example cpu_cores is a static information but it is stored in vds_dynamic 
since we get the value from VDSM.
I found those terms very confusing and a good candidate for a serious 
refactoring in future 


 
  Thanks,
  Gilad.
  
  - Original Message -
   From: Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ) xiao-lei@hp.com
   To: Eli Mesika emes...@redhat.com
   Cc: Gilad Chaplik gchap...@redhat.com, Roy Golan
   rgo...@redhat.com,
   Chuan Liao (Jason Liao,
   HPservers-Core-OE-PSC) chuan.l...@hp.com, Doron Fediuck
   dfedi...@redhat.com, Chegu Vinod
   chegu_vi...@hp.com, Shang-Chun Liang (David Liang,
   HPservers-Core-OE-PSC) shangchun.li...@hp.com, Yaniv Dary
   yd...@redhat.com, engine-devel@ovirt.org
   Sent: Monday, March 31, 2014 3:20:33 PM
   Subject: RE: Please help us to review our database schema design with
   NUMA
   feature on ovirt
   
   Thanks Eli.
   I will move the vm level NUMA fields to vm_dynamic, and the related
   database
   schema will be updated accordingly.
   
   Thanks  Best Regards
   Shi, Xiao-Lei (Bruce)
   
   Hewlett-Packard Co., Ltd.
   HP Servers Core Platform Software China
   Telephone +86 23 65683093
   Mobile +86 18696583447
   Email xiao-lei@hp.com
   
   -Original Message-
   From: Eli Mesika [mailto:emes...@redhat.com]
   Sent: Monday, March 31, 2014 5:49 PM
   To: Shi, Xiao-Lei (Bruce, HP Servers-PSC-CQ)
   Cc: Gilad Chaplik; Roy Golan; Liao, Chuan (Jason Liao,
   HPservers-Core-OE-PSC); Doron Fediuck; Vinod, Chegu; Liang, Shang-Chun
   (David Liang, HPservers-Core-OE-PSC); Yaniv Dary; engine-devel@ovirt.org
   Subject: Re: Please help us to review our database schema design with
   NUMA
   feature on ovirt
   
   
   
   - Original Message -
From: Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ) xiao-lei@hp.com
To: Gilad Chaplik gchap...@redhat.com, Eli Mesika
emes...@redhat.com, Roy Golan rgo...@redhat.com
Cc: Chuan Liao (Jason Liao, HPservers-Core-OE-PSC)
chuan.l...@hp.com, Doron Fediuck dfedi...@redhat.com, Chegu
Vinod
chegu_vi...@hp.com, Shang-Chun Liang (David Liang,
HPservers-Core-OE-PSC)
shangchun.li...@hp.com, Yaniv Dary yd...@redhat.com,
engine-devel@ovirt.org
Sent: Monday, March 31, 2014 12:38:04 PM
Subject: RE: Please help us to review our database schema design with
NUMA feature on ovirt

We put host level NUMA fields in vds_dynamic because these information
are from host itself, and NUMA topology may be changed if the host's
hardware make a change. NUMA information are similar to the host's cpu
topology information like cpu_cores and cpu_sockets which are in
vds_dynamic, we refer to this.
VM level NUMA fields are configured by user, and actually we
originally think they should be in vm_dynamic. But we found that the

Re: [Engine-devel] vds_dynamic refactor

2014-04-06 Thread Liran Zelkha
Don't go down that road.  Status shouldn't be saved in the db.
But anyway statistics is rapidly changing. So it fits...
On Apr 3, 2014 5:14 PM, Gilad Chaplik gchap...@redhat.com wrote:

 - Original Message -
  From: Liran Zelkha liran.zel...@gmail.com
  To: Eli Mesika emes...@redhat.com
  Cc: Gilad Chaplik gchap...@redhat.com, engine-devel 
 engine-devel@ovirt.org
  Sent: Thursday, April 3, 2014 4:36:07 PM
  Subject: Re: [Engine-devel] vds_dynamic refactor
 
  I would be happy if we can lose vds_dynamic all together, and just keep
  vds_static and vds_statistics. Our performance will be happy too ;-)
 

 @Liran, status and pending fields are very fragile ones, IMO need separate
 table.
 @Eli, iirc, you don't have a problem with adding more tables :-)

 
  On Thu, Apr 3, 2014 at 4:33 PM, Eli Mesika emes...@redhat.com wrote:
 
  
  
   - Original Message -
From: Gilad Chaplik gchap...@redhat.com
To: Yair Zaslavsky yzasl...@redhat.com
Cc: engine-devel engine-devel@ovirt.org
Sent: Thursday, April 3, 2014 4:00:25 PM
Subject: Re: [Engine-devel] vds_dynamic refactor
   
- Original Message -
 From: Yair Zaslavsky yzasl...@redhat.com
 To: Liran Zelkha liran.zel...@gmail.com
 Cc: Gilad Chaplik gchap...@redhat.com, engine-devel
 engine-devel@ovirt.org
 Sent: Thursday, April 3, 2014 2:12:59 PM
 Subject: Re: [Engine-devel] vds_dynamic refactor



 - Original Message -
  From: Liran Zelkha liran.zel...@gmail.com
  To: Gilad Chaplik gchap...@redhat.com
  Cc: engine-devel engine-devel@ovirt.org
  Sent: Thursday, April 3, 2014 2:04:29 PM
  Subject: Re: [Engine-devel] vds_dynamic refactor
 
  Why not move it to vds_static?

 +1 on Liran's comment.
  
   +1 as well , vds_static is the place for that
  
 I would prefer not to add more complexity to the  vds tables
 family.
 Such complexity may effect performs of queries/views.
 If you wish, you can create a view on top of vds_static named
   vds_on_boot
 for
 querying of vds on boot info.

 Yair
   
That means moving almost all of vds_dynamic into vds_static except of
   memory,
pending resources and status (maybe more but not much);
and there will not be any db separation between user input and
 on_boot
   data.
  
   Why we should have such separation ?
  
   
Thanks,
Gilad.
   

 
 
  On Thu, Apr 3, 2014 at 2:00 PM, Gilad Chaplik 
 gchap...@redhat.com
  wrote:
 
   Hi list,
  
   I propose to split vds_dynamic table into 2 tables:
   - vds_dynamic
   - vds_on_boot
   We need a place to put all non-dynamic data that gets updated
 once
   the
   host is booted, and I think dynamic isn't the place for it.
   In first phase we will put there NUMA host topoplogy, but
 later on
   migrate
   all non-dynamic host data (cpu, os, etc.).
  
   thoughts?
  
   Thanks,
   Gilad.
   ___
   Engine-devel mailing list
   Engine-devel@ovirt.org
   http://lists.ovirt.org/mailman/listinfo/engine-devel
  
 
  ___
  Engine-devel mailing list
  Engine-devel@ovirt.org
  http://lists.ovirt.org/mailman/listinfo/engine-devel
 

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel
   
   ___
   Engine-devel mailing list
   Engine-devel@ovirt.org
   http://lists.ovirt.org/mailman/listinfo/engine-devel
  
 

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] vds_dynamic refactor

2014-04-06 Thread Gilad Chaplik
- Original Message -
 From: Liran Zelkha liran.zel...@gmail.com
 To: Eli Mesika emes...@redhat.com
 Cc: Gilad Chaplik gchap...@redhat.com, engine-devel 
 engine-devel@ovirt.org
 Sent: Thursday, April 3, 2014 4:36:07 PM
 Subject: Re: [Engine-devel] vds_dynamic refactor
 
 I would be happy if we can lose vds_dynamic all together, and just keep
 vds_static and vds_statistics. Our performance will be happy too ;-)
 

@Liran, status and pending fields are very fragile ones, IMO need separate 
table.
@Eli, iirc, you don't have a problem with adding more tables :-)

 
 On Thu, Apr 3, 2014 at 4:33 PM, Eli Mesika emes...@redhat.com wrote:
 
 
 
  - Original Message -
   From: Gilad Chaplik gchap...@redhat.com
   To: Yair Zaslavsky yzasl...@redhat.com
   Cc: engine-devel engine-devel@ovirt.org
   Sent: Thursday, April 3, 2014 4:00:25 PM
   Subject: Re: [Engine-devel] vds_dynamic refactor
  
   - Original Message -
From: Yair Zaslavsky yzasl...@redhat.com
To: Liran Zelkha liran.zel...@gmail.com
Cc: Gilad Chaplik gchap...@redhat.com, engine-devel
engine-devel@ovirt.org
Sent: Thursday, April 3, 2014 2:12:59 PM
Subject: Re: [Engine-devel] vds_dynamic refactor
   
   
   
- Original Message -
 From: Liran Zelkha liran.zel...@gmail.com
 To: Gilad Chaplik gchap...@redhat.com
 Cc: engine-devel engine-devel@ovirt.org
 Sent: Thursday, April 3, 2014 2:04:29 PM
 Subject: Re: [Engine-devel] vds_dynamic refactor

 Why not move it to vds_static?
   
+1 on Liran's comment.
 
  +1 as well , vds_static is the place for that
 
I would prefer not to add more complexity to the  vds tables family.
Such complexity may effect performs of queries/views.
If you wish, you can create a view on top of vds_static named
  vds_on_boot
for
querying of vds on boot info.
   
Yair
  
   That means moving almost all of vds_dynamic into vds_static except of
  memory,
   pending resources and status (maybe more but not much);
   and there will not be any db separation between user input and on_boot
  data.
 
  Why we should have such separation ?
 
  
   Thanks,
   Gilad.
  
   


 On Thu, Apr 3, 2014 at 2:00 PM, Gilad Chaplik gchap...@redhat.com
 wrote:

  Hi list,
 
  I propose to split vds_dynamic table into 2 tables:
  - vds_dynamic
  - vds_on_boot
  We need a place to put all non-dynamic data that gets updated once
  the
  host is booted, and I think dynamic isn't the place for it.
  In first phase we will put there NUMA host topoplogy, but later on
  migrate
  all non-dynamic host data (cpu, os, etc.).
 
  thoughts?
 
  Thanks,
  Gilad.
  ___
  Engine-devel mailing list
  Engine-devel@ovirt.org
  http://lists.ovirt.org/mailman/listinfo/engine-devel
 

 ___
 Engine-devel mailing list
 Engine-devel@ovirt.org
 http://lists.ovirt.org/mailman/listinfo/engine-devel

   
   ___
   Engine-devel mailing list
   Engine-devel@ovirt.org
   http://lists.ovirt.org/mailman/listinfo/engine-devel
  
  ___
  Engine-devel mailing list
  Engine-devel@ovirt.org
  http://lists.ovirt.org/mailman/listinfo/engine-devel
 
 
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Please help us to review our database schema design with NUMA feature on ovirt

2014-04-06 Thread Omer Frenkel


- Original Message -
 From: Gilad Chaplik gchap...@redhat.com
 To: Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ) xiao-lei@hp.com, Roy 
 Golan rgo...@redhat.com, Omer Frenkel
 ofren...@redhat.com
 Cc: Eli Mesika emes...@redhat.com, Roy Golan rgo...@redhat.com, 
 Chuan Liao (Jason Liao,
 HPservers-Core-OE-PSC) chuan.l...@hp.com, Doron Fediuck 
 dfedi...@redhat.com, Chegu Vinod
 chegu_vi...@hp.com, Shang-Chun Liang (David Liang, HPservers-Core-OE-PSC) 
 shangchun.li...@hp.com, Yaniv Dary
 yd...@redhat.com, engine-devel@ovirt.org
 Sent: Monday, March 31, 2014 4:31:17 PM
 Subject: Re: Please help us to review our database schema design with NUMA 
 feature on ovirt
 
 adding Roy  Omer.
 
 why CPU topology is in dynamic?
 

the guideline we (try to) follow today is (i know you can find exceptions to 
this, but this is how it suppose to be):
vm/vds static - user configurable information (whatever you have in add/edit)
vm dynamic - information related to a running instance, the is changed in a 
somewhat small frequency
vds dynamic - host capabilities, information from the host that doesn't change 
while the host is up.
statistics - all the rest, mostly statistics, and other fast changing 
information.

 Thanks,
 Gilad.
 
 - Original Message -
  From: Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ) xiao-lei@hp.com
  To: Eli Mesika emes...@redhat.com
  Cc: Gilad Chaplik gchap...@redhat.com, Roy Golan rgo...@redhat.com,
  Chuan Liao (Jason Liao,
  HPservers-Core-OE-PSC) chuan.l...@hp.com, Doron Fediuck
  dfedi...@redhat.com, Chegu Vinod
  chegu_vi...@hp.com, Shang-Chun Liang (David Liang,
  HPservers-Core-OE-PSC) shangchun.li...@hp.com, Yaniv Dary
  yd...@redhat.com, engine-devel@ovirt.org
  Sent: Monday, March 31, 2014 3:20:33 PM
  Subject: RE: Please help us to review our database schema design with NUMA
  feature on ovirt
  
  Thanks Eli.
  I will move the vm level NUMA fields to vm_dynamic, and the related
  database
  schema will be updated accordingly.
  
  Thanks  Best Regards
  Shi, Xiao-Lei (Bruce)
  
  Hewlett-Packard Co., Ltd.
  HP Servers Core Platform Software China
  Telephone +86 23 65683093
  Mobile +86 18696583447
  Email xiao-lei@hp.com
  
  -Original Message-
  From: Eli Mesika [mailto:emes...@redhat.com]
  Sent: Monday, March 31, 2014 5:49 PM
  To: Shi, Xiao-Lei (Bruce, HP Servers-PSC-CQ)
  Cc: Gilad Chaplik; Roy Golan; Liao, Chuan (Jason Liao,
  HPservers-Core-OE-PSC); Doron Fediuck; Vinod, Chegu; Liang, Shang-Chun
  (David Liang, HPservers-Core-OE-PSC); Yaniv Dary; engine-devel@ovirt.org
  Subject: Re: Please help us to review our database schema design with NUMA
  feature on ovirt
  
  
  
  - Original Message -
   From: Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ) xiao-lei@hp.com
   To: Gilad Chaplik gchap...@redhat.com, Eli Mesika
   emes...@redhat.com, Roy Golan rgo...@redhat.com
   Cc: Chuan Liao (Jason Liao, HPservers-Core-OE-PSC)
   chuan.l...@hp.com, Doron Fediuck dfedi...@redhat.com, Chegu Vinod
   chegu_vi...@hp.com, Shang-Chun Liang (David Liang,
   HPservers-Core-OE-PSC)
   shangchun.li...@hp.com, Yaniv Dary yd...@redhat.com,
   engine-devel@ovirt.org
   Sent: Monday, March 31, 2014 12:38:04 PM
   Subject: RE: Please help us to review our database schema design with
   NUMA feature on ovirt
   
   We put host level NUMA fields in vds_dynamic because these information
   are from host itself, and NUMA topology may be changed if the host's
   hardware make a change. NUMA information are similar to the host's cpu
   topology information like cpu_cores and cpu_sockets which are in
   vds_dynamic, we refer to this.
   VM level NUMA fields are configured by user, and actually we
   originally think they should be in vm_dynamic. But we found that the
   field of another feature cpuPin which is similar as NUMA feature is in
   vm_static, so we put vm NUMA fields in vm_static.
   Do you think we need to put VM level NUMA fields in vm_dynamic?
  
  I think that in this case we should fix cpuPin to be in vm_dynamic and put
  after that the other NUMA fields in vm_dynamic as well
  
   
   Thanks  Best Regards
   Shi, Xiao-Lei (Bruce)
   
   Hewlett-Packard Co., Ltd.
   HP Servers Core Platform Software China Telephone +86 23 65683093
   Mobile +86 18696583447 Email xiao-lei@hp.com
   
   
   -Original Message-
   From: Gilad Chaplik [mailto:gchap...@redhat.com]
   Sent: Monday, March 31, 2014 5:22 PM
   To: Eli Mesika; Roy Golan
   Cc: Shi, Xiao-Lei (Bruce, HP Servers-PSC-CQ); Liao, Chuan (Jason Liao,
   HPservers-Core-OE-PSC); Doron Fediuck; Vinod, Chegu; Liang, Shang-Chun
   (David Liang, HPservers-Core-OE-PSC); Yaniv Dary; engine-devel@ovirt.org
   Subject: Re: Please help us to review our database schema design with
   NUMA
   feature on ovirt
   
   +1
   
   IMO: vds data should reside in static
   VM need to think about it.
   
   Roy?
   
   Thanks,
   Gilad.
   
   
   - Original Message -
From: Eli Mesika emes...@redhat.com

Re: [Engine-devel] vds_dynamic refactor

2014-04-06 Thread Gilad Chaplik
- Original Message -
 From: Liran Zelkha liran.zel...@gmail.com
 To: Gilad Chaplik gchap...@redhat.com
 Cc: Eli Mesika emes...@redhat.com, engine-devel engine-devel@ovirt.org
 Sent: Thursday, April 3, 2014 5:16:51 PM
 Subject: Re: [Engine-devel] vds_dynamic refactor
 
 Don't go down that road.  Status shouldn't be saved in the db.
 But anyway statistics is rapidly changing. So it fits...

First let's have a notion of caching, then notion of shared caching, then I can 
start thinking of not going down that road...

 On Apr 3, 2014 5:14 PM, Gilad Chaplik gchap...@redhat.com wrote:
 
  - Original Message -
   From: Liran Zelkha liran.zel...@gmail.com
   To: Eli Mesika emes...@redhat.com
   Cc: Gilad Chaplik gchap...@redhat.com, engine-devel 
  engine-devel@ovirt.org
   Sent: Thursday, April 3, 2014 4:36:07 PM
   Subject: Re: [Engine-devel] vds_dynamic refactor
  
   I would be happy if we can lose vds_dynamic all together, and just keep
   vds_static and vds_statistics. Our performance will be happy too ;-)
  
 
  @Liran, status and pending fields are very fragile ones, IMO need separate
  table.
  @Eli, iirc, you don't have a problem with adding more tables :-)
 
  
   On Thu, Apr 3, 2014 at 4:33 PM, Eli Mesika emes...@redhat.com wrote:
  
   
   
- Original Message -
 From: Gilad Chaplik gchap...@redhat.com
 To: Yair Zaslavsky yzasl...@redhat.com
 Cc: engine-devel engine-devel@ovirt.org
 Sent: Thursday, April 3, 2014 4:00:25 PM
 Subject: Re: [Engine-devel] vds_dynamic refactor

 - Original Message -
  From: Yair Zaslavsky yzasl...@redhat.com
  To: Liran Zelkha liran.zel...@gmail.com
  Cc: Gilad Chaplik gchap...@redhat.com, engine-devel
  engine-devel@ovirt.org
  Sent: Thursday, April 3, 2014 2:12:59 PM
  Subject: Re: [Engine-devel] vds_dynamic refactor
 
 
 
  - Original Message -
   From: Liran Zelkha liran.zel...@gmail.com
   To: Gilad Chaplik gchap...@redhat.com
   Cc: engine-devel engine-devel@ovirt.org
   Sent: Thursday, April 3, 2014 2:04:29 PM
   Subject: Re: [Engine-devel] vds_dynamic refactor
  
   Why not move it to vds_static?
 
  +1 on Liran's comment.
   
+1 as well , vds_static is the place for that
   
  I would prefer not to add more complexity to the  vds tables
  family.
  Such complexity may effect performs of queries/views.
  If you wish, you can create a view on top of vds_static named
vds_on_boot
  for
  querying of vds on boot info.
 
  Yair

 That means moving almost all of vds_dynamic into vds_static except of
memory,
 pending resources and status (maybe more but not much);
 and there will not be any db separation between user input and
  on_boot
data.
   
Why we should have such separation ?
   

 Thanks,
 Gilad.

 
  
  
   On Thu, Apr 3, 2014 at 2:00 PM, Gilad Chaplik 
  gchap...@redhat.com
   wrote:
  
Hi list,
   
I propose to split vds_dynamic table into 2 tables:
- vds_dynamic
- vds_on_boot
We need a place to put all non-dynamic data that gets updated
  once
the
host is booted, and I think dynamic isn't the place for it.
In first phase we will put there NUMA host topoplogy, but
  later on
migrate
all non-dynamic host data (cpu, os, etc.).
   
thoughts?
   
Thanks,
Gilad.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel
   
  
   ___
   Engine-devel mailing list
   Engine-devel@ovirt.org
   http://lists.ovirt.org/mailman/listinfo/engine-devel
  
 
 ___
 Engine-devel mailing list
 Engine-devel@ovirt.org
 http://lists.ovirt.org/mailman/listinfo/engine-devel

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel
   
  
 
 
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] ovirtengine connect node

2014-04-06 Thread Itamar Heim

On 03/28/2014 09:36 AM, marco lin wrote:

st node1 is compatible with versions (3.0,3.1,3.2,3.3) and cannot join
Cluster Default which is set to version 3.4. How to solve?



was this resolveD? which OS are you using for node1?

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Please help us to review our database schema design with NUMA feature on ovirt

2014-04-06 Thread Chegu Vinod

On 4/3/2014 3:46 AM, Gilad Chaplik wrote:

- Original Message -

From: Eli Mesika emes...@redhat.com
To: Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ) xiao-lei@hp.com
Cc: Gilad Chaplik gchap...@redhat.com, Roy Golan rgo...@redhat.com, Omer 
Frenkel ofren...@redhat.com,
Chegu Vinod chegu_vi...@hp.com, Chuan Liao (Jason Liao, HPservers-Core-OE-PSC) 
chuan.l...@hp.com, Doron
Fediuck dfedi...@redhat.com, Shang-Chun Liang (David Liang, 
HPservers-Core-OE-PSC) shangchun.li...@hp.com,
Yaniv Dary yd...@redhat.com, engine-devel@ovirt.org
Sent: Thursday, April 3, 2014 10:54:54 AM
Subject: Re: Please help us to review our database schema design with NUMA 
feature on ovirt



- Original Message -

From: Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ) xiao-lei@hp.com
To: Gilad Chaplik gchap...@redhat.com, Eli Mesika
emes...@redhat.com
Cc: Roy Golan rgo...@redhat.com, Omer Frenkel ofren...@redhat.com,
Chegu Vinod chegu_vi...@hp.com, Chuan
Liao (Jason Liao, HPservers-Core-OE-PSC) chuan.l...@hp.com, Doron
Fediuck dfedi...@redhat.com, Shang-Chun
Liang (David Liang, HPservers-Core-OE-PSC) shangchun.li...@hp.com,
Yaniv Dary yd...@redhat.com,
engine-devel@ovirt.org
Sent: Thursday, April 3, 2014 7:25:11 AM
Subject: RE: Please help us to review our database schema design with NUMA
feature on ovirt

Hi all,
Please see my comments in line.

Thanks  Best Regards
Shi, Xiao-Lei (Bruce)

Hewlett-Packard Co., Ltd.
HP Servers Core Platform Software China
Telephone +86 23 65683093
Mobile +86 18696583447
Email xiao-lei@hp.com

-Original Message-
From: Gilad Chaplik [mailto:gchap...@redhat.com]
Sent: Thursday, April 03, 2014 9:05 AM
To: Eli Mesika
Cc: Shi, Xiao-Lei (Bruce, HP Servers-PSC-CQ); Roy Golan; Omer Frenkel;
Vinod,
Chegu; Liao, Chuan (Jason Liao, HPservers-Core-OE-PSC); Doron Fediuck;
Liang, Shang-Chun (David Liang, HPservers-Core-OE-PSC); Yaniv Dary;
engine-devel@ovirt.org
Subject: Re: Please help us to review our database schema design with NUMA
feature on ovirt

Hi all,
Sorry for joining-in late.

My comments (according to the db diagram section in
https://docs.google.com/document/d/1-wdDkm6EDbwyoCIRPPcmbGWAcyQo_ISTY8ykDr0I6VY):
1) Join vm_numa_node and vds_numa_node to a single table (almost
identical),
one of the FKs can be null.
[Bruce] I prefer two tables. Actually host level NUMA node and vm level
NUMA
node are different objects. In my understanding, vm level NUMA node is just
a simulation of host level NUMA node, and host level NUMA node has more
features that not in vm NUMA (like several levels of host NUMA topology
mentioned by Vinod). We need to consider the extensions of host NUMA in the
future.

What future extension are you referring to ?


Not sure how relevant this is to the discussion but a little bit of 
background info. here :


A VM's Virtual NUMA node topology is generated by qemu+seabios and is 
based on options specified at the qemu command line (libvirt translates 
the information in the VM's xml file and invokes the qemu command line 
with the correct options)..


Today there is no support in qemu+seabios for generating multiple levels 
of Virtual NUMA. A vast majority of the hosts out there (i.e. 2 socket 
and 4 socket hosts) have only single level of NUMA topology...so this is 
fine for now. (Multi-level NUMA support in the qemu+seabios is a 
slightly different topic...and may (or may not) be pursued as a 
potential future enhancement for qemu so for now let us not worry 
about such things  over-engineer in oVirt infrastructure etc. for 
multi-level virtual NUMA nodes etc.)


The values for the node distances in the virtual NUMA topology are 
auto-generated defaults (by qemu+seabios) and has no relation with the 
node distances in the host NUMA topology (which is extracted from the 
ACPI SLIT tables and are supposed to be representative of the underlying 
system fabric's inter node latency capabilities etc).


All the guest OS needs to know is that there are multiple [virtual] NUMA 
nodes and these virtual nodes are a single hop away This helps the 
guest to do the right thing with per node data structure 
allocations/locking etc and helps it scale/perform better.




As I mentioned in another email thread : If it makes sense for some 
[current/future] use cases to store this virtual NUMA topology info. 
(along with the node distances) somewhere in the oVirt 
infrastructure...then please feel free to do so.






Let's open the discussion and consider them right now. vNode and Node are the 
same.



Not really sure what I can say here...

A VM's virtual NUMA node should be sized (i.e. cpu count in the node) no 
larger than the host NUMA node. (Ideally they should be of the same size).


Vinod


Vinod?


I agree with Bruce, we have no problem with more tables and constrains should
work as expected and remove entries when a Host or VM is removed.
I do not like tables that have 2 UUIDs when one of them is null , this is
against simple DB normalization


We are going 

Re: [Engine-devel] NUMA Support - GUI Technical Session

2014-04-06 Thread Vojtech Szocs
Hi Gilad,

to my understanding, we already use _HTML5_ Drag'n'Drop support
(exposed by GWT API since 2.4) in Setup Host Networks dialog.

To utilize HTML5 Drag'n'Drop support in GWT widget, just extend
FocusPanel and mark your widget's DOM element as draggable.

For example, in UnassignedNetworksPanel constructor:

  getElement().setDraggable( ... );
  addBitlessDomHandler(new DragEnterHandler() { ... });
  addBitlessDomHandler(new DragOverHandler() { ... });
  addBitlessDomHandler(new DragLeaveHandler() { ... });
  addBitlessDomHandler(new DropHandler() { ... });

In other words, GWT already exposes API for working with HTML5
Drag'n'Drop spec, so you don't need any 3rd party libraries.
The downside is that HTML5 Drag'n'Drop spec is supported only
in recent browsers (but this isn't an issue for us, AFAIK):

  http://caniuse.com/#feat=dragndrop

So in general we have two alternatives:

  1, use standard HTML5 Drag'n'Drop spec

 pros:
 + no need for 3rd party library
 + compliant with existing code, i.e. Setup Host Networks

 cons (not too relevant IMHO):
 - requires browser support of HTML5 Drag'n'Drop spec
 - HTML5 Drag'n'Drop spec deals with dragging data, not widgets
   themselves (no HTML DOM re-parenting after drag finish)

  2, use 3rd party gwt-dnd library

 pros (which I don't think we really need):
 + emulate Drag'n'Drop support in older browsers
 + more advanced functionality, i.e. allows dragging widgets
   so that HTML DOM is dynamically updated

 cons:
 - dependency on 3rd party library
 - this would mean we need to revisit existing code
   (we should do Drag'n'Drop in one consistent way)

It's possible that I might be missing something, but I'd
suggest to try using HTML5 Drag'n'Drop via GWT API as the
first approach.

If we find out that HTML5 Drag'n'Drop doesn't work for us
in given browser(s) or if we need extra functionality, we
can always add gwt-dnd dependency.

Few more comments inline, let me know what you think.

Regards,
Vojtech


- Original Message -
 From: Alexander Wels aw...@redhat.com
 To: Gilad Chaplik gchap...@redhat.com
 Cc: eco...@redhat.com, Vojtech Szocs vsz...@redhat.com, 
 dfedi...@redhat.com, engine-devel@ovirt.org, chegu
 vinod chegu_vi...@hp.com, lver...@redhat.com
 Sent: Tuesday, April 1, 2014 7:24:12 PM
 Subject: Re: NUMA Support - GUI Technical Session
 
 On Tuesday, April 01, 2014 11:34:43 AM Gilad Chaplik wrote:
  Hi all,
  
  Here are the resolutions from the meeting:
  
  * option 1
  1) Use gwt-dnd lib for oVirt's dnd (drag and drop) infrastructure.
  2) Come up with a very simple POC that covers all of NUMA-support UX
  requirements. 3) Either do a POC, or get UX maintainers approval, that
  moving already existing dnd features to new infrastructure (setup-networks
  and scheduling policy dialogs) is feasible and possible.
  
 
 +1 for option 1. None of the drag and drop in the application now looks
 terribly hard. The gwt-dnd library simply makes things easier to control and
 maintain.

Yeah, but the downside is adding 3rd party dependency which predates GWT's
support for (API exposure of) HTML5 Drag'n'Drop spec.

 
  * option 2
  Extract setup network dnd capabilities to a common general infrastructure
  and use that as an infrastructure. NOTE that setup-networks will not use it
  in oVirt-3.5.

GWT's HTML5 Drag'n'Drop API works directly on DOM element level, it's just
a thin API overlay on top of HTML5Drag'n'Drop spec.

Do we really need custom DnD infra on top of that? Can't we just use GWT
APIs like Element.setDraggable, Drag*Handler, Drop*Handler?

  
  I will start with option 1, just need UX team approval that [1] will be
  added to ovirt-3.5, and be used for dnd for now on. In case I fail to
  deliver option 1 (with the help and guidance of the UX team) in a quick
  cycle (a week or so), I will peruse option 2, which is trivial.
  
  Moving forward, all new dnd enabled features will use the new
  infrastructure, and the motivation is to migrate existing ones as well.

I agree, there should be one consistent way to do DnD in oVirt UI.

I'm just saying we should consider existing GWT HTML5 Drag'n'Drop API
before jumping into gwt-dnd 3rd party dependency. If it turns out that
GWT DnD API is too basic (lacks functionality) or has issues with some
browser(s) - we can add gwt-dnd dependency anytime.

  
  Thanks,
  Gilad.
  
  [1] http://code.google.com/p/gwt-dnd/
  
  - Original Message -
  
   From: Gilad Chaplik gchap...@redhat.com
   To: Einav Cohen eco...@redhat.com, Alexander Wels
   aw...@redhat.com, Eyal Edri ee...@redhat.com, Steve Gordon
   sgor...@redhat.com, Eli Mesika emes...@redhat.com, Otavio Luiz
   Ferranti otavio.ferra...@eldorado.org.br, Sandro Bonazzola
   sbona...@redhat.com, Greg Sheremeta gsher...@redhat.com, Doron
   Fediuck dfedi...@redhat.com, Lior Vernia lver...@redhat.com,
   engine-devel engine-devel@ovirt.org, Martin Sivak
   msi...@redhat.com, chuan 

Re: [Engine-devel] NUMA support action items

2014-04-06 Thread Gilad Chaplik
- Original Message -
 From: Chegu Vinod chegu_vi...@hp.com
 To: Gilad Chaplik gchap...@redhat.com
 Cc: Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ) xiao-lei@hp.com, Einav 
 Cohen eco...@redhat.com, Shang-Chun
 Liang (David Liang, HPservers-Core-OE-PSC) shangchun.li...@hp.com, Chuan 
 Liao (Jason Liao,
 HPservers-Core-OE-PSC) chuan.l...@hp.com, msi...@redhat.com, Da-huai Tang 
 (Gary, MCXS-CQ)
 da-huai.t...@hp.com, Malini Rao m...@redhat.com, Eldan Hildesheim 
 ehild...@redhat.com, Doron Fediuck
 dfedi...@redhat.com, sher...@redhat.com, Alexander Wels 
 aw...@redhat.com, engine-devel
 engine-devel@ovirt.org
 Sent: Thursday, April 3, 2014 5:21:49 PM
 Subject: Re: NUMA support action items
 
 On 4/3/2014 7:11 AM, Gilad Chaplik wrote:
  - Original Message -
  From: Chegu Vinod chegu_vi...@hp.com
  To: Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ) xiao-lei@hp.com
  Cc: Einav Cohen eco...@redhat.com, Shang-Chun Liang (David Liang,
  HPservers-Core-OE-PSC)
  shangchun.li...@hp.com, Chuan Liao (Jason Liao, HPservers-Core-OE-PSC)
  chuan.l...@hp.com, msi...@redhat.com,
  Da-huai Tang (Gary, MCXS-CQ) da-huai.t...@hp.com, Malini Rao
  m...@redhat.com, Eldan Hildesheim
  ehild...@redhat.com, Doron Fediuck dfedi...@redhat.com,
  sher...@redhat.com, Alexander Wels
  aw...@redhat.com, Gilad Chaplik gchap...@redhat.com
  Sent: Thursday, April 3, 2014 3:28:03 PM
  Subject: RE: NUMA support action items
 
  Hi Bruce,
 
  The virtual NUMA layout in the guest is a very simple one (not multi-level
  etc). It is generated by qemu+seabios... and there is no relationship with
  the host NUMA node distances etc.  Let us not worry about gathering
  Virtual
  NUMA node distances for now.
 
  Vinod
 
  CC'ing devel list as well.
 
  Having said that, I don't see a reason why not to prepare an infrastructure
  for that (if it's free) for future versions (guest agent will collect
  vNuma data in some point in time).
 
 If you think having this Virtual NUMA topology (along with the virtual
 numa node *distance* info.) really helps some future use cases then pl.
 go ahead...
 
 Vinod
 
 

I really don't know. but IMO, me as a user that gets some machine (transparent 
to the fact it's VM or bare metal), it would be very nice to see the NUMA stats 
outside of the machine.

Thanks, 
Gilad.
 
 
  Thanks,
  Gilad.
 
  -Original Message-
  From: Shi, Xiao-Lei (Bruce, HP Servers-PSC-CQ)
  Sent: Thursday, April 03, 2014 12:41 AM
  To: Vinod, Chegu
  Cc: Einav Cohen; Liang, Shang-Chun (David Liang, HPservers-Core-OE-PSC);
  Liao, Chuan (Jason Liao, HPservers-Core-OE-PSC); msi...@redhat.com; Tang,
  Da-huai (Gary, MCXS-CQ); Malini Rao; Eldan Hildesheim; Doron Fediuck;
  sher...@redhat.com; Alexander Wels; Gilad Chaplik
  Subject: RE: NUMA support action items
 
  Hi Vinod,
 
  Is it meaningful for us to collect the distance information of vm numa
  node
  (maybe in future, not now)?
  In my understanding, vm numa topology is a simulation of numa topology,
  since
  the vcpus are just threads, I don't know how the vm numa node distances
  are
  calculated in vm. Is there any relationship between the vNode distances
  and
  host node distances?
 
  Thanks  Best Regards
  Shi, Xiao-Lei (Bruce)
 
  Hewlett-Packard Co., Ltd.
  HP Servers Core Platform Software China Telephone +86 23 65683093 Mobile
  +86
  18696583447 Email xiao-lei@hp.com
 
 
  -Original Message-
  From: Vinod, Chegu
  Sent: Thursday, April 03, 2014 7:18 AM
  To: Gilad Chaplik
  Cc: Einav Cohen; Liang, Shang-Chun (David Liang, HPservers-Core-OE-PSC);
  Liao, Chuan (Jason Liao, HPservers-Core-OE-PSC); msi...@redhat.com; Shi,
  Xiao-Lei (Bruce, HP Servers-PSC-CQ); Tang, Da-huai (Gary, MCXS-CQ); Malini
  Rao; Eldan Hildesheim; Doron Fediuck; sher...@redhat.com; Alexander Wels
  Subject: RE: NUMA support action items
 
  Not sure what the correct way to do this isbut here is a suggestion.
 
  Let a given host server diagram shown be very generic...i.e. show the N
  sockets/nodes numbered from 0 thru N-1.  Show the amount of memory and the
  list of CPUs in each of those sockets/nodes.
  Draw a generic Interconnect fabric [box] in between which all the sockets
  connect to
 
  Ideally ... Under that host diagram we could show the NUMA node distances
  in
  text format (as you know this is derived from the numactl -H and then
  conveyed from VDSM- oVIrt engine etc).
  That distance info. will tell the user what the distance between a pair of
  sockets/nodes are (and they can then do what they wish after that :)).
 
  Vinod
 
  -Original Message-
  From: Gilad Chaplik [mailto:gchap...@redhat.com]
  Sent: Wednesday, April 02, 2014 4:09 PM
  To: Vinod, Chegu
  Cc: Einav Cohen; Liang, Shang-Chun (David Liang, HPservers-Core-OE-PSC);
  Liao, Chuan (Jason Liao, HPservers-Core-OE-PSC); msi...@redhat.com; Shi,
  Xiao-Lei (Bruce, HP Servers-PSC-CQ); Tang, Da-huai (Gary, MCXS-CQ); Malini
  Rao; Eldan Hildesheim; Doron Fediuck; sher...@redhat.com; 

Re: [Engine-devel] vds_dynamic refactor

2014-04-06 Thread Liran Zelkha
True but that's no reason to have a bad schema
On Apr 3, 2014 5:18 PM, Gilad Chaplik gchap...@redhat.com wrote:

 - Original Message -
  From: Liran Zelkha liran.zel...@gmail.com
  To: Gilad Chaplik gchap...@redhat.com
  Cc: Eli Mesika emes...@redhat.com, engine-devel 
 engine-devel@ovirt.org
  Sent: Thursday, April 3, 2014 5:16:51 PM
  Subject: Re: [Engine-devel] vds_dynamic refactor
 
  Don't go down that road.  Status shouldn't be saved in the db.
  But anyway statistics is rapidly changing. So it fits...

 First let's have a notion of caching, then notion of shared caching, then
 I can start thinking of not going down that road...

  On Apr 3, 2014 5:14 PM, Gilad Chaplik gchap...@redhat.com wrote:
 
   - Original Message -
From: Liran Zelkha liran.zel...@gmail.com
To: Eli Mesika emes...@redhat.com
Cc: Gilad Chaplik gchap...@redhat.com, engine-devel 
   engine-devel@ovirt.org
Sent: Thursday, April 3, 2014 4:36:07 PM
Subject: Re: [Engine-devel] vds_dynamic refactor
   
I would be happy if we can lose vds_dynamic all together, and just
 keep
vds_static and vds_statistics. Our performance will be happy too ;-)
   
  
   @Liran, status and pending fields are very fragile ones, IMO need
 separate
   table.
   @Eli, iirc, you don't have a problem with adding more tables :-)
  
   
On Thu, Apr 3, 2014 at 4:33 PM, Eli Mesika emes...@redhat.com
 wrote:
   


 - Original Message -
  From: Gilad Chaplik gchap...@redhat.com
  To: Yair Zaslavsky yzasl...@redhat.com
  Cc: engine-devel engine-devel@ovirt.org
  Sent: Thursday, April 3, 2014 4:00:25 PM
  Subject: Re: [Engine-devel] vds_dynamic refactor
 
  - Original Message -
   From: Yair Zaslavsky yzasl...@redhat.com
   To: Liran Zelkha liran.zel...@gmail.com
   Cc: Gilad Chaplik gchap...@redhat.com, engine-devel
   engine-devel@ovirt.org
   Sent: Thursday, April 3, 2014 2:12:59 PM
   Subject: Re: [Engine-devel] vds_dynamic refactor
  
  
  
   - Original Message -
From: Liran Zelkha liran.zel...@gmail.com
To: Gilad Chaplik gchap...@redhat.com
Cc: engine-devel engine-devel@ovirt.org
Sent: Thursday, April 3, 2014 2:04:29 PM
Subject: Re: [Engine-devel] vds_dynamic refactor
   
Why not move it to vds_static?
  
   +1 on Liran's comment.

 +1 as well , vds_static is the place for that

   I would prefer not to add more complexity to the  vds tables
   family.
   Such complexity may effect performs of queries/views.
   If you wish, you can create a view on top of vds_static named
 vds_on_boot
   for
   querying of vds on boot info.
  
   Yair
 
  That means moving almost all of vds_dynamic into vds_static
 except of
 memory,
  pending resources and status (maybe more but not much);
  and there will not be any db separation between user input and
   on_boot
 data.

 Why we should have such separation ?

 
  Thanks,
  Gilad.
 
  
   
   
On Thu, Apr 3, 2014 at 2:00 PM, Gilad Chaplik 
   gchap...@redhat.com
wrote:
   
 Hi list,

 I propose to split vds_dynamic table into 2 tables:
 - vds_dynamic
 - vds_on_boot
 We need a place to put all non-dynamic data that gets
 updated
   once
 the
 host is booted, and I think dynamic isn't the place for it.
 In first phase we will put there NUMA host topoplogy, but
   later on
 migrate
 all non-dynamic host data (cpu, os, etc.).

 thoughts?

 Thanks,
 Gilad.
 ___
 Engine-devel mailing list
 Engine-devel@ovirt.org
 http://lists.ovirt.org/mailman/listinfo/engine-devel

   
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel
   
  
  ___
  Engine-devel mailing list
  Engine-devel@ovirt.org
  http://lists.ovirt.org/mailman/listinfo/engine-devel
 
 ___
 Engine-devel mailing list
 Engine-devel@ovirt.org
 http://lists.ovirt.org/mailman/listinfo/engine-devel

   
  
 

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] vds_dynamic refactor

2014-04-06 Thread Liran Zelkha
On Thu, Apr 3, 2014 at 5:33 PM, Gilad Chaplik gchap...@redhat.com wrote:

 *From: *Liran Zelkha liran.zel...@gmail.com
 *To: *Gilad Chaplik gchap...@redhat.com
 *Cc: *Omer Frenkel ofren...@redhat.com, Eli Mesika 
 emes...@redhat.com, engine-devel engine-devel@ovirt.org
 *Sent: *Thursday, April 3, 2014 5:27:56 PM
 *Subject: *Re: [Engine-devel] vds_dynamic refactor

 True but that's no reason to have a bad schema

 I'm open to new ideas and truly want to understand what is bad?

The problem is with both updates and selects.
For selects - to get all the information for the VDS we have multiple
joins. Adding another one will hurt performance even more.
For updates - we have vds_static thats hardly changed. vds_statistics that
changes all the time. vds_dynamic is not changed allot - but is updated all
the time because of the status. I think it's best to split it to the two
existing tables (BTW - relevant for VM as well)

 On Apr 3, 2014 5:18 PM, Gilad Chaplik gchap...@redhat.com wrote:

 - Original Message -
  From: Liran Zelkha liran.zel...@gmail.com
  To: Gilad Chaplik gchap...@redhat.com
  Cc: Eli Mesika emes...@redhat.com, engine-devel 
 engine-devel@ovirt.org
  Sent: Thursday, April 3, 2014 5:16:51 PM
  Subject: Re: [Engine-devel] vds_dynamic refactor
 
  Don't go down that road.  Status shouldn't be saved in the db.
  But anyway statistics is rapidly changing. So it fits...

 First let's have a notion of caching, then notion of shared caching, then
 I can start thinking of not going down that road...

  On Apr 3, 2014 5:14 PM, Gilad Chaplik gchap...@redhat.com wrote:
 
   - Original Message -
From: Liran Zelkha liran.zel...@gmail.com
To: Eli Mesika emes...@redhat.com
Cc: Gilad Chaplik gchap...@redhat.com, engine-devel 
   engine-devel@ovirt.org
Sent: Thursday, April 3, 2014 4:36:07 PM
Subject: Re: [Engine-devel] vds_dynamic refactor
   
I would be happy if we can lose vds_dynamic all together, and just
 keep
vds_static and vds_statistics. Our performance will be happy too ;-)
   
  
   @Liran, status and pending fields are very fragile ones, IMO need
 separate
   table.
   @Eli, iirc, you don't have a problem with adding more tables :-)
  
   
On Thu, Apr 3, 2014 at 4:33 PM, Eli Mesika emes...@redhat.com
 wrote:
   


 - Original Message -
  From: Gilad Chaplik gchap...@redhat.com
  To: Yair Zaslavsky yzasl...@redhat.com
  Cc: engine-devel engine-devel@ovirt.org
  Sent: Thursday, April 3, 2014 4:00:25 PM
  Subject: Re: [Engine-devel] vds_dynamic refactor
 
  - Original Message -
   From: Yair Zaslavsky yzasl...@redhat.com
   To: Liran Zelkha liran.zel...@gmail.com
   Cc: Gilad Chaplik gchap...@redhat.com, engine-devel
   engine-devel@ovirt.org
   Sent: Thursday, April 3, 2014 2:12:59 PM
   Subject: Re: [Engine-devel] vds_dynamic refactor
  
  
  
   - Original Message -
From: Liran Zelkha liran.zel...@gmail.com
To: Gilad Chaplik gchap...@redhat.com
Cc: engine-devel engine-devel@ovirt.org
Sent: Thursday, April 3, 2014 2:04:29 PM
Subject: Re: [Engine-devel] vds_dynamic refactor
   
Why not move it to vds_static?
  
   +1 on Liran's comment.

 +1 as well , vds_static is the place for that

   I would prefer not to add more complexity to the  vds tables
   family.
   Such complexity may effect performs of queries/views.
   If you wish, you can create a view on top of vds_static named
 vds_on_boot
   for
   querying of vds on boot info.
  
   Yair
 
  That means moving almost all of vds_dynamic into vds_static
 except of
 memory,
  pending resources and status (maybe more but not much);
  and there will not be any db separation between user input and
   on_boot
 data.

 Why we should have such separation ?

 
  Thanks,
  Gilad.
 
  
   
   
On Thu, Apr 3, 2014 at 2:00 PM, Gilad Chaplik 
   gchap...@redhat.com
wrote:
   
 Hi list,

 I propose to split vds_dynamic table into 2 tables:
 - vds_dynamic
 - vds_on_boot
 We need a place to put all non-dynamic data that gets
 updated
   once
 the
 host is booted, and I think dynamic isn't the place for
 it.
 In first phase we will put there NUMA host topoplogy, but
   later on
 migrate
 all non-dynamic host data (cpu, os, etc.).

 thoughts?

 Thanks,
 Gilad.
 ___
 Engine-devel mailing list
 Engine-devel@ovirt.org
 http://lists.ovirt.org/mailman/listinfo/engine-devel

   
___
Engine-devel mailing list
Engine-devel@ovirt.org

Re: [Engine-devel] vds_dynamic refactor

2014-04-06 Thread Gilad Chaplik
 From: Liran Zelkha liran.zel...@gmail.com
 To: Gilad Chaplik gchap...@redhat.com
 Cc: Omer Frenkel ofren...@redhat.com, Eli Mesika emes...@redhat.com,
 engine-devel engine-devel@ovirt.org
 Sent: Thursday, April 3, 2014 5:27:56 PM
 Subject: Re: [Engine-devel] vds_dynamic refactor

 True but that's no reason to have a bad schema
I'm open to new ideas and truly want to understand what is bad? 

 On Apr 3, 2014 5:18 PM, Gilad Chaplik  gchap...@redhat.com  wrote:

  - Original Message -
 
   From: Liran Zelkha  liran.zel...@gmail.com 
 
   To: Gilad Chaplik  gchap...@redhat.com 
 
   Cc: Eli Mesika  emes...@redhat.com , engine-devel 
   engine-devel@ovirt.org 
 
   Sent: Thursday, April 3, 2014 5:16:51 PM
 
   Subject: Re: [Engine-devel] vds_dynamic refactor
 
  
 
   Don't go down that road. Status shouldn't be saved in the db.
 
   But anyway statistics is rapidly changing. So it fits...
 

  First let's have a notion of caching, then notion of shared caching, then I
  can start thinking of not going down that road...
 

   On Apr 3, 2014 5:14 PM, Gilad Chaplik  gchap...@redhat.com  wrote:
 
  
 
- Original Message -
 
 From: Liran Zelkha  liran.zel...@gmail.com 
 
 To: Eli Mesika  emes...@redhat.com 
 
 Cc: Gilad Chaplik  gchap...@redhat.com , engine-devel 
 
engine-devel@ovirt.org 
 
 Sent: Thursday, April 3, 2014 4:36:07 PM
 
 Subject: Re: [Engine-devel] vds_dynamic refactor
 

 
 I would be happy if we can lose vds_dynamic all together, and just
 keep
 
 vds_static and vds_statistics. Our performance will be happy too ;-)
 

 
   
 
@Liran, status and pending fields are very fragile ones, IMO need
separate
 
table.
 
@Eli, iirc, you don't have a problem with adding more tables :-)
 
   
 

 
 On Thu, Apr 3, 2014 at 4:33 PM, Eli Mesika  emes...@redhat.com 
 wrote:
 

 
 
 
 
 
  - Original Message -
 
   From: Gilad Chaplik  gchap...@redhat.com 
 
   To: Yair Zaslavsky  yzasl...@redhat.com 
 
   Cc: engine-devel  engine-devel@ovirt.org 
 
   Sent: Thursday, April 3, 2014 4:00:25 PM
 
   Subject: Re: [Engine-devel] vds_dynamic refactor
 
  
 
   - Original Message -
 
From: Yair Zaslavsky  yzasl...@redhat.com 
 
To: Liran Zelkha  liran.zel...@gmail.com 
 
Cc: Gilad Chaplik  gchap...@redhat.com , engine-devel
 
 engine-devel@ovirt.org 
 
Sent: Thursday, April 3, 2014 2:12:59 PM
 
Subject: Re: [Engine-devel] vds_dynamic refactor
 
   
 
   
 
   
 
- Original Message -
 
 From: Liran Zelkha  liran.zel...@gmail.com 
 
 To: Gilad Chaplik  gchap...@redhat.com 
 
 Cc: engine-devel  engine-devel@ovirt.org 
 
 Sent: Thursday, April 3, 2014 2:04:29 PM
 
 Subject: Re: [Engine-devel] vds_dynamic refactor
 

 
 Why not move it to vds_static?
 
   
 
+1 on Liran's comment.
 
 
 
  +1 as well , vds_static is the place for that
 
 
 
I would prefer not to add more complexity to the vds tables
 
family.
 
Such complexity may effect performs of queries/views.
 
If you wish, you can create a view on top of vds_static named
 
  vds_on_boot
 
for
 
querying of vds on boot info.
 
   
 
Yair
 
  
 
   That means moving almost all of vds_dynamic into vds_static
   except
   of
 
  memory,
 
   pending resources and status (maybe more but not much);
 
   and there will not be any db separation between user input and
 
on_boot
 
  data.
 
 
 
  Why we should have such separation ?
 
 
 
  
 
   Thanks,
 
   Gilad.
 
  
 
   
 

 

 
 On Thu, Apr 3, 2014 at 2:00 PM, Gilad Chaplik 
 
gchap...@redhat.com 
 
 wrote:
 

 
  Hi list,
 
 
 
  I propose to split vds_dynamic table into 2 tables:
 
  - vds_dynamic
 
  - vds_on_boot
 
  We need a place to put all non-dynamic data that gets
  updated
 
once
 
  the
 
  host is booted, and I think dynamic isn't the place for it.
 
  In first phase we will put there NUMA host topoplogy, but
 
later on
 
  migrate
 
  all non-dynamic host data (cpu, os, etc.).
 
 
 
  thoughts?
 
 
 
  Thanks,
 
  Gilad.
 
  ___
 
  Engine-devel mailing list
 
  Engine-devel@ovirt.org
 
  http://lists.ovirt.org/mailman/listinfo/engine-devel
 
 
 

 
 ___
 
 Engine-devel mailing list
 
 Engine-devel@ovirt.org
 
 http://lists.ovirt.org/mailman/listinfo/engine-devel
 

 
   
 
   

Re: [Engine-devel] [Devel] vds_dynamic refactor

2014-04-06 Thread Liran Zelkha
On Fri, Apr 4, 2014 at 3:15 PM, Itamar Heim ih...@redhat.com wrote:

 On 04/04/2014 04:30 AM, Liran Zelkha wrote:




 On Thu, Apr 3, 2014 at 8:40 PM, Gilad Chaplik gchap...@redhat.com
 mailto:gchap...@redhat.com wrote:

 - Original Message -
   From: Liran Zelkha liran.zel...@gmail.com
 mailto:liran.zel...@gmail.com
   To: Gilad Chaplik gchap...@redhat.com
 mailto:gchap...@redhat.com
   Cc: Omer Frenkel ofren...@redhat.com
 mailto:ofren...@redhat.com, Eli Mesika emes...@redhat.com
 mailto:emes...@redhat.com, engine-devel engine-devel@ovirt.org
 mailto:engine-devel@ovirt.org,
   de...@linode01.ovirt.org mailto:de...@linode01.ovirt.org
   Sent: Thursday, April 3, 2014 7:51:39 PM
   Subject: Re: [Engine-devel] vds_dynamic refactor
  
   On Thu, Apr 3, 2014 at 5:33 PM, Gilad Chaplik
 gchap...@redhat.com mailto:gchap...@redhat.com wrote:
  
*From: *Liran Zelkha liran.zel...@gmail.com
 mailto:liran.zel...@gmail.com

*To: *Gilad Chaplik gchap...@redhat.com
 mailto:gchap...@redhat.com

*Cc: *Omer Frenkel ofren...@redhat.com
 mailto:ofren...@redhat.com, Eli Mesika 
emes...@redhat.com mailto:emes...@redhat.com, engine-devel
 engine-devel@ovirt.org mailto:engine-devel@ovirt.org

*Sent: *Thursday, April 3, 2014 5:27:56 PM
*Subject: *Re: [Engine-devel] vds_dynamic refactor
   
True but that's no reason to have a bad schema
   
I'm open to new ideas and truly want to understand what is bad?
   
   The problem is with both updates and selects.
   For selects - to get all the information for the VDS we have
 multiple
   joins. Adding another one will hurt performance even more.

 What about creating the VDS list in the db. i.e. your patch [1]
 about constructing VDS objects in the engine, should occur in the db
 within a SP, and should be transparent to the server. that will
 solve the multiple table join.


 The joins are happening on the server. We have a VDS view that brings in
 information from many tables and we retrieve rows from it all the time.
 Just run an explain on a select * from vds and see for yourself.


 thought the distinction was vds_static is updated by user (configuration),
 where as vds_dyanmic/statistics is reported from vdsm (slow/fast updates).

 True - vds_static hardly change. But vds_dynamic is split-brain - it has
practically static data, but also has the status - which changes rapidly.
And since we have 3 tables, we need to do 3 joins to get VDS (actually we
do much more).


 please remember joining them to one table, also means DWH will ETA all of
 data each time. today it will only copy statistics if dynamic did not
 change.

 I'm not saying joining all 3 tables to 1 table, just make them 2 tables.
And since the data is hardly changing - same logic of DWH will stay.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Devel] [vdsm] vdsm sync meeting minutes April 1st, 2014

2014-04-06 Thread ybronhei

Hey all,
back from France

On 04/02/2014 11:29 AM, dan...@redhat.com wrote:

Vdsm sync call April 1st 2014
=

- cpopen:
   - Toni: there's a versioning mismatch: the version in pypi is higher
 than the one on https://github.com/ficoos/cpopen
   - Saggi: there shouldn't be any code difference
   - Yaniv should sync the two.

https://github.com/ficoos/cpopen/pull/23 - committed 1.3 tag
https://pypi.python.org/pypi?:action=displayname=cpopenversion=1.3 - 
uploaded latest code


   - We use two options that are missing from Python3's Popen: umask and
 deathSignal.
 - Toni to re-send his Python3 port for cpopen, with tests running on
   Python3, too.
 - Saggi to accept it.
 - Infra team to suggest missing features to Python.




- fromani described his attempts of consolidating the two
   migration-monitoring threads into one. The motivation is a finer way
   of contolling the migration down time based on progress. Reducing
   thread numbers per se is not the main motivation.

- pep8 has changed. Again. Version 1.5.1 has even more draconic
   requirements. We can comply with them, or, include our version of
   pep8/pyflakes/pylint in our git repo. danken shudders at the thought
   of copying the code, but having a git sub-module is a reasonable
   compromise.
   - Infra team to take this task (unless someone else is faster)
   - Until that happens, one need to use pep8-1.4.6 or --ignore offending
 errors.

- We've been asked to kill the separate vdsm-devel mailing list, and
   join it into devel@ovirt.org. There's some resistence in the audience,
   so we'd do it softly.

   Next week, I'll have vdsm-devel members mass-subscribed to
   devel@ovirt. If you do NOT want to be subscribed, please send me a
   private request.

   Then, for several months, we'd see how it goes, and whether we drown
   in unrelated Engine chatter.

- We had a very (too) heated debate about ignoring failures of
   setDomainRegularRole() in http://gerrit.ovirt.org/24495/ and
   http://gerrit.ovirt.org/25424.

   The pain point is that relying on domain role (master/regular) is
   faulty by design. We cannot avoid the cases where a pool has more than
   one domain with a master role written in its metadata.

   One side argued that oVirt should be fixed to handle this unescapable
   truth, or at least enumerate the places where Vdsm and Engine, both
   current and old, depend on master role uniqueness.

   The other side argued that this is not a priority task, and that we
   should try to garbage-collect known-bad master roles as a courtesy
   to people digging into domain metadata, and as a means to lessen the
   problem until we kill the pool concept in the upcoming version.

   I hope that I present the debate fairly enough.

Dan.




--
Yaniv Bronhaim.
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel


Re: [Devel] [Engine-devel] vds_dynamic refactor

2014-04-06 Thread Itamar Heim

On 04/03/2014 07:51 PM, Liran Zelkha wrote:

The problem is with both updates and selects.
For selects - to get all the information for the VDS we have multiple
joins. Adding another one will hurt performance even more.
For updates - we have vds_static thats hardly changed. vds_statistics
that changes all the time. vds_dynamic is not changed allot - but is
updated all the time because of the status. I think it's best to split
it to the two existing tables (BTW - relevant for VM as well)


but we don't update it unless the status has changed, which is a rare 
occurance?

___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel


Re: [Devel] NUMA Support - GUI Technical Session

2014-04-06 Thread Gilad Chaplik
[top posting]

Thanks you Vojtech for the elaborate expatiation, I will follow on that and 
keep you all posted.

Thanks, 
Gilad.

- Original Message -
 From: Vojtech Szocs vsz...@redhat.com
 To: Gilad Chaplik gchap...@redhat.com
 Cc: devel@ovirt.org, chegu vinod chegu_vi...@hp.com, aw...@redhat.com
 Sent: Saturday, April 5, 2014 12:08:09 AM
 Subject: Re: [Devel] NUMA Support - GUI Technical Session
 
 Forgot to send some references on the matter:
 
   https://groups.google.com/forum/#!topic/google-web-toolkit/q0JUtbiUR8g
   
 http://stackoverflow.com/questions/13681541/best-way-to-implement-a-drag-and-drop-list-in-gwt
 
 Regards,
 Vojtech
 
 
 - Original Message -
  From: Vojtech Szocs vsz...@redhat.com
  To: aw...@redhat.com
  Cc: devel@ovirt.org, chegu vinod chegu_vi...@hp.com
  Sent: Friday, April 4, 2014 11:06:16 PM
  Subject: Re: [Devel] NUMA Support - GUI Technical Session
  
  Hi Gilad,
  
  to my understanding, we already use _HTML5_ Drag'n'Drop support
  (exposed by GWT API since 2.4) in Setup Host Networks dialog.
  
  To utilize HTML5 Drag'n'Drop support in GWT widget, just extend
  FocusPanel and mark your widget's DOM element as draggable.
  
  For example, in UnassignedNetworksPanel constructor:
  
getElement().setDraggable( ... );
addBitlessDomHandler(new DragEnterHandler() { ... });
addBitlessDomHandler(new DragOverHandler() { ... });
addBitlessDomHandler(new DragLeaveHandler() { ... });
addBitlessDomHandler(new DropHandler() { ... });
  
  In other words, GWT already exposes API for working with HTML5
  Drag'n'Drop spec, so you don't need any 3rd party libraries.
  The downside is that HTML5 Drag'n'Drop spec is supported only
  in recent browsers (but this isn't an issue for us, AFAIK):
  
http://caniuse.com/#feat=dragndrop
  
  So in general we have two alternatives:
  
1, use standard HTML5 Drag'n'Drop spec
  
   pros:
   + no need for 3rd party library
   + compliant with existing code, i.e. Setup Host Networks
  
   cons (not too relevant IMHO):
   - requires browser support of HTML5 Drag'n'Drop spec
   - HTML5 Drag'n'Drop spec deals with dragging data, not widgets
 themselves (no HTML DOM re-parenting after drag finish)
  
2, use 3rd party gwt-dnd library
  
   pros (which I don't think we really need):
   + emulate Drag'n'Drop support in older browsers
   + more advanced functionality, i.e. allows dragging widgets
 so that HTML DOM is dynamically updated
  
   cons:
   - dependency on 3rd party library
   - this would mean we need to revisit existing code
 (we should do Drag'n'Drop in one consistent way)
  
  It's possible that I might be missing something, but I'd
  suggest to try using HTML5 Drag'n'Drop via GWT API as the
  first approach.
  
  If we find out that HTML5 Drag'n'Drop doesn't work for us
  in given browser(s) or if we need extra functionality, we
  can always add gwt-dnd dependency.
  
  Few more comments inline, let me know what you think.
  
  Regards,
  Vojtech
  
  
  - Original Message -
   From: Alexander Wels aw...@redhat.com
   To: Gilad Chaplik gchap...@redhat.com
   Cc: eco...@redhat.com, Vojtech Szocs vsz...@redhat.com,
   dfedi...@redhat.com, engine-de...@ovirt.org, chegu
   vinod chegu_vi...@hp.com, lver...@redhat.com
   Sent: Tuesday, April 1, 2014 7:24:12 PM
   Subject: Re: NUMA Support - GUI Technical Session
   
   On Tuesday, April 01, 2014 11:34:43 AM Gilad Chaplik wrote:
Hi all,

Here are the resolutions from the meeting:

* option 1
1) Use gwt-dnd lib for oVirt's dnd (drag and drop) infrastructure.
2) Come up with a very simple POC that covers all of NUMA-support UX
requirements. 3) Either do a POC, or get UX maintainers approval, that
moving already existing dnd features to new infrastructure
(setup-networks
and scheduling policy dialogs) is feasible and possible.

   
   +1 for option 1. None of the drag and drop in the application now looks
   terribly hard. The gwt-dnd library simply makes things easier to control
   and
   maintain.
  
  Yeah, but the downside is adding 3rd party dependency which predates GWT's
  support for (API exposure of) HTML5 Drag'n'Drop spec.
  
   
* option 2
Extract setup network dnd capabilities to a common general
infrastructure
and use that as an infrastructure. NOTE that setup-networks will not
use
it
in oVirt-3.5.
  
  GWT's HTML5 Drag'n'Drop API works directly on DOM element level, it's just
  a thin API overlay on top of HTML5Drag'n'Drop spec.
  
  Do we really need custom DnD infra on top of that? Can't we just use GWT
  APIs like Element.setDraggable, Drag*Handler, Drop*Handler?
  

I will start with option 1, just need UX team approval that [1] will be
added to ovirt-3.5, and be used for dnd for now on. In case I fail to
deliver option 1 (with the help and guidance of the UX team) in a 

Re: [Devel] [Engine-devel] vds_dynamic refactor

2014-04-06 Thread Gilad Chaplik
- Original Message -
 From: Itamar Heim ih...@redhat.com
 To: Liran Zelkha liran.zel...@gmail.com
 Cc: Gilad Chaplik gchap...@redhat.com, de...@linode01.ovirt.org, 
 engine-devel engine-de...@ovirt.org
 Sent: Sunday, April 6, 2014 11:33:12 AM
 Subject: Re: [Engine-devel] vds_dynamic refactor
 
 On 04/06/2014 11:32 AM, Liran Zelkha wrote:
 
 
 
  On Sun, Apr 6, 2014 at 11:22 AM, Itamar Heim ih...@redhat.com
  mailto:ih...@redhat.com wrote:
 
  On 04/03/2014 07:51 PM, Liran Zelkha wrote:
 
  The problem is with both updates and selects.
  For selects - to get all the information for the VDS we have
  multiple
  joins. Adding another one will hurt performance even more.
  For updates - we have vds_static thats hardly changed.
  vds_statistics
  that changes all the time. vds_dynamic is not changed allot - but
  is
  updated all the time because of the status. I think it's best to
  split
  it to the two existing tables (BTW - relevant for VM as well)
 
 
  but we don't update it unless the status has changed, which is a
  rare occurance?
 
  Actually - no. We can definitely see times we are updating vds_dynamic
  with no reason at all. I tried to create patches for that - but it
  happens from many different places in the code.
 
 what would be updated vds_dyanmic for status not originating in update
 run time info?

We have separate DB flows for that (updateStatus and 
updatePartialVdsDynamicCalc and more in VdsDynamicDAODbFacadeImpl).
A question: do you know if we update status in updateVdsDynamic? :-) not sure 
but I found a possible race for pending resources (cpu, mem), LOL :-)

Still holds my original thought for having vds_on_boot.

 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel


Re: [Devel] [Engine-devel] vds_dynamic refactor

2014-04-06 Thread Gilad Chaplik
- Original Message -
 From: Liran Zelkha liran.zel...@gmail.com
 To: Gilad Chaplik gchap...@redhat.com
 Cc: Itamar Heim ih...@redhat.com, de...@linode01.ovirt.org, 
 engine-devel engine-de...@ovirt.org
 Sent: Sunday, April 6, 2014 3:26:24 PM
 Subject: Re: [Engine-devel] vds_dynamic refactor
 
 On Sun, Apr 6, 2014 at 3:18 PM, Gilad Chaplik gchap...@redhat.com wrote:
 
  - Original Message -
   From: Itamar Heim ih...@redhat.com
   To: Liran Zelkha liran.zel...@gmail.com
   Cc: Gilad Chaplik gchap...@redhat.com, de...@linode01.ovirt.org,
  engine-devel engine-de...@ovirt.org
   Sent: Sunday, April 6, 2014 11:33:12 AM
   Subject: Re: [Engine-devel] vds_dynamic refactor
  
   On 04/06/2014 11:32 AM, Liran Zelkha wrote:
   
   
   
On Sun, Apr 6, 2014 at 11:22 AM, Itamar Heim ih...@redhat.com
mailto:ih...@redhat.com wrote:
   
On 04/03/2014 07:51 PM, Liran Zelkha wrote:
   
The problem is with both updates and selects.
For selects - to get all the information for the VDS we have
multiple
joins. Adding another one will hurt performance even more.
For updates - we have vds_static thats hardly changed.
vds_statistics
that changes all the time. vds_dynamic is not changed allot -
  but
is
updated all the time because of the status. I think it's best
  to
split
it to the two existing tables (BTW - relevant for VM as well)
   
   
but we don't update it unless the status has changed, which is a
rare occurance?
   
Actually - no. We can definitely see times we are updating vds_dynamic
with no reason at all. I tried to create patches for that - but it
happens from many different places in the code.
  
   what would be updated vds_dyanmic for status not originating in update
   run time info?
 
  We have separate DB flows for that (updateStatus and
  updatePartialVdsDynamicCalc and more in VdsDynamicDAODbFacadeImpl).
  A question: do you know if we update status in updateVdsDynamic? :-) not
  sure but I found a possible race for pending resources (cpu, mem), LOL :-)
 
  I think we do but not sure. Will check.

Of course it is, that was a rhetorical question :-) (a lot of emoticons and 
LOLs ;-))

 
 
  Still holds my original thought for having vds_on_boot.
 
 
 
 Let's talk f2f on Tuesday?

I'd prefer to reach conclusions here, I'd like everyone to be involved in a 
root issue like this one.

 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel


Re: [Devel] [Engine-devel] vds_dynamic refactor

2014-04-06 Thread Kobi Ianko
Joining in...
From my point of view, in real life a user should have that many VDSs on one 
Engine (from a DB point of view).
Modern DB system handles tables with millions of records and many relations, Do 
we really have a performance issue here?
We could prefer a more easy to maintain implantation in this case over DB 
performance


- Original Message -
 From: Gilad Chaplik gchap...@redhat.com
 To: Liran Zelkha liran.zel...@gmail.com
 Cc: de...@linode01.ovirt.org, engine-devel engine-de...@ovirt.org
 Sent: Sunday, April 6, 2014 3:32:26 PM
 Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor
 
 - Original Message -
  From: Liran Zelkha liran.zel...@gmail.com
  To: Gilad Chaplik gchap...@redhat.com
  Cc: Itamar Heim ih...@redhat.com, de...@linode01.ovirt.org,
  engine-devel engine-de...@ovirt.org
  Sent: Sunday, April 6, 2014 3:26:24 PM
  Subject: Re: [Engine-devel] vds_dynamic refactor
  
  On Sun, Apr 6, 2014 at 3:18 PM, Gilad Chaplik gchap...@redhat.com wrote:
  
   - Original Message -
From: Itamar Heim ih...@redhat.com
To: Liran Zelkha liran.zel...@gmail.com
Cc: Gilad Chaplik gchap...@redhat.com, de...@linode01.ovirt.org,
   engine-devel engine-de...@ovirt.org
Sent: Sunday, April 6, 2014 11:33:12 AM
Subject: Re: [Engine-devel] vds_dynamic refactor
   
On 04/06/2014 11:32 AM, Liran Zelkha wrote:



 On Sun, Apr 6, 2014 at 11:22 AM, Itamar Heim ih...@redhat.com
 mailto:ih...@redhat.com wrote:

 On 04/03/2014 07:51 PM, Liran Zelkha wrote:

 The problem is with both updates and selects.
 For selects - to get all the information for the VDS we have
 multiple
 joins. Adding another one will hurt performance even more.
 For updates - we have vds_static thats hardly changed.
 vds_statistics
 that changes all the time. vds_dynamic is not changed allot -
   but
 is
 updated all the time because of the status. I think it's best
   to
 split
 it to the two existing tables (BTW - relevant for VM as well)


 but we don't update it unless the status has changed, which is a
 rare occurance?

 Actually - no. We can definitely see times we are updating
 vds_dynamic
 with no reason at all. I tried to create patches for that - but it
 happens from many different places in the code.
   
what would be updated vds_dyanmic for status not originating in update
run time info?
  
   We have separate DB flows for that (updateStatus and
   updatePartialVdsDynamicCalc and more in VdsDynamicDAODbFacadeImpl).
   A question: do you know if we update status in updateVdsDynamic? :-) not
   sure but I found a possible race for pending resources (cpu, mem), LOL
   :-)
  
   I think we do but not sure. Will check.
 
 Of course it is, that was a rhetorical question :-) (a lot of emoticons and
 LOLs ;-))
 
  
  
   Still holds my original thought for having vds_on_boot.
  
  
  
  Let's talk f2f on Tuesday?
 
 I'd prefer to reach conclusions here, I'd like everyone to be involved in a
 root issue like this one.
 
  
 ___
 Devel mailing list
 Devel@ovirt.org
 http://lists.ovirt.org/mailman/listinfo/devel
 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel


Re: [Devel] [Engine-devel] vds_dynamic refactor

2014-04-06 Thread Gilad Chaplik
- Original Message -
 From: Liran Zelkha liran.zel...@gmail.com
 To: Gilad Chaplik gchap...@redhat.com
 Cc: Kobi Ianko k...@redhat.com, de...@linode01.ovirt.org, engine-devel 
 engine-de...@ovirt.org
 Sent: Sunday, April 6, 2014 8:51:02 PM
 Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor
 
 On Sun, Apr 6, 2014 at 6:32 PM, Gilad Chaplik gchap...@redhat.com wrote:
 
  - Original Message -
   From: Liran Zelkha liran.zel...@gmail.com
   To: Kobi Ianko k...@redhat.com
   Cc: Gilad Chaplik gchap...@redhat.com, de...@linode01.ovirt.org,
  engine-devel engine-de...@ovirt.org
   Sent: Sunday, April 6, 2014 3:40:13 PM
   Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor
  
   On Sun, Apr 6, 2014 at 3:37 PM, Kobi Ianko k...@redhat.com wrote:
  
Joining in...
From my point of view, in real life a user should have that many VDSs
  on
one Engine (from a DB point of view).
Modern DB system handles tables with millions of records and many
relations, Do we really have a performance issue here?
We could prefer a more easy to maintain implantation in this case over
  DB
performance
   
Yes we do. We make many queries on the VDS view, which is a VERY
  complex
   view.
  
 
  Actually I quite agree with Kobi, what is the plan for VMs? why do we
  start with VDS...
  what is the biggest deploy do you know of?
 
 We start with VDS because in an idle system, with 200 hosts and several
 thousands VMs, this is what you get as the top queries against the
 database. Look at how many times getvds is called.
 [image: Inline image 1]
 BTW - the second query is an example of abusing the dynamic query
 mechanism. The 4th query (an update command) is a set of useless
 update_vds_dynamic commands.
 
 For reference, the explain plan of get VDS is something like this:
 
 QUERY PLAN
 
 ---
  Nested Loop  (cost=9.30..46.75 rows=6 width=9060) (actual
 time=0.063..0.068 rows=1 loops=1)
Join Filter: (vds_static.vds_id = vds_statistics.vds_id)
-  Seq Scan on vds_statistics  (cost=0.00..1.01 rows=1 width=109)
 (actual time=0.008..0.008 rows=1 loops=1)
-  Nested Loop  (cost=9.30..45.64 rows=6 width=8983) (actual
 time=0.048..0.052 rows=1 loops=1)
  Join Filter: (vds_groups.vds_group_id = vds_static.vds_group_id)
  -  Nested Loop Left Join  (cost=0.00..9.29 rows=1 width=1389)
 (actual time=0.013..0.013 rows=1 loops=1)
-  Seq Scan on vds_groups  (cost=0.00..1.01 rows=1
 width=1271) (actual time=0.003..0.003 rows=1 loops=1)
-  Index Scan using pk_storage_pool on storage_pool
  (cost=0.00..8.27 rows=1 width=134) (actual time=0.008..0.008 rows=1
 loops=1)
  Index Cond: (vds_groups.storage_pool_id = id)
  -  Hash Right Join  (cost=9.30..36.28 rows=6 width=7610) (actual
 time=0.033..0.037 rows=1 loops=1)
Hash Cond: (vds_spm_id_map.vds_id = vds_static.vds_id)
-  Seq Scan on vds_spm_id_map  (cost=0.00..22.30 rows=1230
 width=20) (actual time=0.003..0.003 rows=1 loops=1)
-  Hash  (cost=9.29..9.29 rows=1 width=7606) (actual
 time=0.019..0.019 rows=1 loops=1)
  Buckets: 1024  Batches: 1  Memory Usage: 2kB
  -  Nested Loop  (cost=0.00..9.29 rows=1 width=7606)
 (actual time=0.012..0.013 rows=1 loops=1)
-  Seq Scan on vds_dynamic  (cost=0.00..1.01
 rows=1 width=1895) (actual time=0.006..0.006 rows=1 loops=1)
-  Index Scan using pk_vds_static on vds_static
  (cost=0.00..8.27 rows=1 width=5711) (actual time=0.005..0.006 rows=1
 loops=1)
  Index Cond: (vds_id = vds_dynamic.vds_id)
  Total runtime: 0.299 ms
 (19 rows)
 
 It's terrible. Adding any additional join will make this worse. Please
 don't add any more tables...

Thank you for the detailed explanation, my comments:

* a very long time isn't an argument for not adding another table (should be 
neglectable);
currently we have an unrelated problem, we need to solve it.

*  We start with VDS because in an idle system, with 200 hosts and several
 thousands VMs, this is what you get as the top queries against the
 database.

so, if fetching VMs takes 10 minutes? and its get called a single time?

* you didn't reply on my of my suggestion of constructing the VDS records in 
the DB without using joins.


 
 
   
- Original Message -
 From: Gilad Chaplik gchap...@redhat.com
 To: Liran Zelkha liran.zel...@gmail.com
 Cc: de...@linode01.ovirt.org, engine-devel engine-de...@ovirt.org
  
 Sent: Sunday, April 6, 2014 3:32:26 PM
 Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor

 - Original Message -
  From: Liran Zelkha liran.zel...@gmail.com
  To: Gilad Chaplik gchap...@redhat.com
  

Re: [Devel] Vdsm functional tests

2014-04-06 Thread Dan Kenigsberg
On Sun, Apr 06, 2014 at 11:18:19AM +0300, ybronhei wrote:
 On 04/03/2014 07:31 PM, Douglas Schilling Landgraf wrote:
 On 04/03/2014 11:08 AM, Dan Kenigsberg wrote:
 Functional tests are intended to verify that a running Vdsm instance
 does what it should, when treated as a black box, over its public API.
 
 They should be comprehensive and representative of a typical field usage
 of Vdsm. It is a sin to break such a test - but we must be able to know
 when such a sin is committed.
 
 We currently have the following functional tests modules:
 
 - sosPluginTests.py
supervdsmFuncTests.py
 
 Sure, count with me.
 
 any news about it ? need my help around it?

Douglas still owes me a time estimate on when this be done.

 supervdsmFuncTests.py doesn't really check much. we need to add much
 more logic there if we actually want to test the communication
 between vdsm and supervdsm (not sure if its really required.. its
 like checking calls to libvirt or sanlock or testing api calls)

At the moment supervdsmFuncTests do test that supervdsm is reponsive and
that supervdsm.getProxy() works. It's not like testing libvirt api calls
since supervdsm is inside our tree. So it's like testing libvirt api
calls - within the libvirt project.

I would embrace more smarter logic into the test - but I'm not sure what
you have in mind.
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel


Re: [Devel] Vdsm functional tests

2014-04-06 Thread Douglas Schilling Landgraf

On 04/03/2014 12:24 PM, Dan Kenigsberg wrote:

On Thu, Apr 03, 2014 at 12:31:44PM -0400, Douglas Schilling Landgraf wrote:

On 04/03/2014 11:08 AM, Dan Kenigsberg wrote:

Functional tests are intended to verify that a running Vdsm instance
does what it should, when treated as a black box, over its public API.

They should be comprehensive and representative of a typical field usage
of Vdsm. It is a sin to break such a test - but we must be able to know
when such a sin is committed.

We currently have the following functional tests modules:

- sosPluginTests.py
   supervdsmFuncTests.py


Sure, count with me.


Thanks! When do you think you could write a job similar to
http://jenkins.ovirt.org/view/By%20Project/view/vdsm/job/vdsm_network_functional_tests/configure
running whenever there's a change in the modules relevant to
sosPluginTests and supervdsmFuncTests?



Hi,

Instead of to split by domain like (network, infra, storage), why not 
have a single functional test job? If something fail, it should trigger 
the volunteers.


For while, I started a creation of infra based on the above one.
http://jenkins.ovirt.org/job/vdsm_infra_functional_tests

--
Cheers
Douglas
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel


Re: [Devel] [Engine-devel] vds_dynamic refactor

2014-04-06 Thread Liran Zelkha
On Sun, Apr 6, 2014 at 11:03 PM, Gilad Chaplik gchap...@redhat.com wrote:

 - Original Message -
  From: Liran Zelkha liran.zel...@gmail.com
  To: Gilad Chaplik gchap...@redhat.com
  Cc: Kobi Ianko k...@redhat.com, de...@linode01.ovirt.org,
 engine-devel engine-de...@ovirt.org
  Sent: Sunday, April 6, 2014 8:51:02 PM
  Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor
 
  On Sun, Apr 6, 2014 at 6:32 PM, Gilad Chaplik gchap...@redhat.com
 wrote:
 
   - Original Message -
From: Liran Zelkha liran.zel...@gmail.com
To: Kobi Ianko k...@redhat.com
Cc: Gilad Chaplik gchap...@redhat.com, de...@linode01.ovirt.org,
   engine-devel engine-de...@ovirt.org
Sent: Sunday, April 6, 2014 3:40:13 PM
Subject: Re: [Devel] [Engine-devel] vds_dynamic refactor
   
On Sun, Apr 6, 2014 at 3:37 PM, Kobi Ianko k...@redhat.com wrote:
   
 Joining in...
 From my point of view, in real life a user should have that many
 VDSs
   on
 one Engine (from a DB point of view).
 Modern DB system handles tables with millions of records and many
 relations, Do we really have a performance issue here?
 We could prefer a more easy to maintain implantation in this case
 over
   DB
 performance

 Yes we do. We make many queries on the VDS view, which is a VERY
   complex
view.
   
  
   Actually I quite agree with Kobi, what is the plan for VMs? why do we
   start with VDS...
   what is the biggest deploy do you know of?
  
  We start with VDS because in an idle system, with 200 hosts and several
  thousands VMs, this is what you get as the top queries against the
  database. Look at how many times getvds is called.
  [image: Inline image 1]
  BTW - the second query is an example of abusing the dynamic query
  mechanism. The 4th query (an update command) is a set of useless
  update_vds_dynamic commands.
 
  For reference, the explain plan of get VDS is something like this:
 
  QUERY PLAN
 
 
 ---
   Nested Loop  (cost=9.30..46.75 rows=6 width=9060) (actual
  time=0.063..0.068 rows=1 loops=1)
 Join Filter: (vds_static.vds_id = vds_statistics.vds_id)
 -  Seq Scan on vds_statistics  (cost=0.00..1.01 rows=1 width=109)
  (actual time=0.008..0.008 rows=1 loops=1)
 -  Nested Loop  (cost=9.30..45.64 rows=6 width=8983) (actual
  time=0.048..0.052 rows=1 loops=1)
   Join Filter: (vds_groups.vds_group_id = vds_static.vds_group_id)
   -  Nested Loop Left Join  (cost=0.00..9.29 rows=1 width=1389)
  (actual time=0.013..0.013 rows=1 loops=1)
 -  Seq Scan on vds_groups  (cost=0.00..1.01 rows=1
  width=1271) (actual time=0.003..0.003 rows=1 loops=1)
 -  Index Scan using pk_storage_pool on storage_pool
   (cost=0.00..8.27 rows=1 width=134) (actual time=0.008..0.008 rows=1
  loops=1)
   Index Cond: (vds_groups.storage_pool_id = id)
   -  Hash Right Join  (cost=9.30..36.28 rows=6 width=7610)
 (actual
  time=0.033..0.037 rows=1 loops=1)
 Hash Cond: (vds_spm_id_map.vds_id = vds_static.vds_id)
 -  Seq Scan on vds_spm_id_map  (cost=0.00..22.30
 rows=1230
  width=20) (actual time=0.003..0.003 rows=1 loops=1)
 -  Hash  (cost=9.29..9.29 rows=1 width=7606) (actual
  time=0.019..0.019 rows=1 loops=1)
   Buckets: 1024  Batches: 1  Memory Usage: 2kB
   -  Nested Loop  (cost=0.00..9.29 rows=1 width=7606)
  (actual time=0.012..0.013 rows=1 loops=1)
 -  Seq Scan on vds_dynamic  (cost=0.00..1.01
  rows=1 width=1895) (actual time=0.006..0.006 rows=1 loops=1)
 -  Index Scan using pk_vds_static on
 vds_static
   (cost=0.00..8.27 rows=1 width=5711) (actual time=0.005..0.006 rows=1
  loops=1)
   Index Cond: (vds_id =
 vds_dynamic.vds_id)
   Total runtime: 0.299 ms
  (19 rows)
 
  It's terrible. Adding any additional join will make this worse. Please
  don't add any more tables...

 Thank you for the detailed explanation, my comments:

 * a very long time isn't an argument for not adding another table (should
 be neglectable);
 currently we have an unrelated problem, we need to solve it.

Of course it is. A very long time for a query that you execute many times
is THE factor. Who said the join has no performance effect? Have you tested
it? Under load?  Under many writes/updates?


 *  We start with VDS because in an idle system, with 200 hosts and several
  thousands VMs, this is what you get as the top queries against the
  database.

 so, if fetching VMs takes 10 minutes? and its get called a single time?

 Where do you see 10 minutes? If you are looking at the red bar it's the
inherent time - total query time * number of queries.


 * you didn't reply on my of my suggestion of constructing