Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo

2013-06-12 Thread Barak Azulay


- Original Message -
 From: Itamar Heim ih...@redhat.com
 To: Liran Zelkha lzel...@redhat.com
 Cc: engine-devel engine-devel@ovirt.org
 Sent: Tuesday, June 11, 2013 4:00:21 PM
 Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo
 
 On 06/11/2013 03:26 PM, Liran Zelkha wrote:
  Hi all,
 
  I'm checking performance for VdsUpdateRunTimeInfo.
  Naturally, much of the performance surrounds database activity
  (getVmsRunningOnVds queries, updateDeviceRuntimeInfo, updateVmDynamic)
 
  Few questions:
  1. I have implemented batch updates for procedure UpdateVmDeviceRuntimeInfo
  for improved performance.
  2. Seems like the only parameters UpdateVmDeviceRuntimeInfo is getting are
  vm_id,vm_device_id,address and alias. Are those rapidly changing, or will
  it be beneficial to implement caching on those updates (to ensure
  not-required updates do not travel to the database).
 
 slowly changing, but how will you cover all flows changing these devices
 to invalidate the cache (iiuc, this table is modified by engine when
 adding devices to a VM as well?)


I don't think that in the device run time info we need to invalidate once we 
add a device.
This is a specific case where we actually get the information from the VDSM 
(addresses are received from libvirt)
The commands IIRC are first send to VDSM and than update the runtime info only 
on changed info (we can also hash it),
It may put the placeholder in the DB first but it still relies on the data 
received from VDSM.




 
  3. Any additional known performance problems you know of in regards to this
  class?
 
  Thanks...
  ___
  Engine-devel mailing list
  Engine-devel@ovirt.org
  http://lists.ovirt.org/mailman/listinfo/engine-devel
 
 
 ___
 Engine-devel mailing list
 Engine-devel@ovirt.org
 http://lists.ovirt.org/mailman/listinfo/engine-devel
 
 
 
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Add traffic shaping parameters for a network interface.

2013-06-12 Thread Doron Fediuck


- Original Message -
 From: Giuseppe Vallarelli gvall...@redhat.com
 To: Livnat Peer lp...@redhat.com
 Cc: Doron Fediuck dfedi...@redhat.com, engine-devel@ovirt.org, Dan 
 Kenigsberg dan...@redhat.com
 Sent: Tuesday, June 11, 2013 5:34:16 PM
 Subject: Re: [Engine-devel] Add traffic shaping parameters for a network 
 interface.
 
 - Original Message -
 | From: Giuseppe Vallarelli gvall...@redhat.com
 | To: Livnat Peer lp...@redhat.com
 | Cc: Doron Fediuck dfedi...@redhat.com, engine-devel@ovirt.org, Dan
 | Kenigsberg dan...@redhat.com
 | Sent: Tuesday, June 11, 2013 3:07:32 PM
 | Subject: Re: [Engine-devel] Add traffic shaping parameters for a network
 | interface.
 | 
 | Related to QoS parameters reporting to the engine. Looks like they're
 | already
 | available, I tried to use vdsClient with list verb and I've got the devices
 | list where a 'specParams' is defined (it's empty because I didn't try it
 | with
 | my last patch).
 | 
 | devices = [... {'nicModel': 'pv', 'macAddr': '00:1a:4a:22:3f:04',
 | 'network':
 | 'ovirtmgmt', 'alias': 'net0', 'specParams': {},
 | 'deviceId': '76173ffc-603a-496f-8ffc-31dc4d41cef6', 'address': {'slot':
 | '0x03', 'bus': '0x00', 'domain': '0x',
 | 'type': 'pci', 'function': '0x0'}, 'device': 'bridge', 'type': 'interface'}
 | ...]
 | 
 | Perhaps I'm missing something, any ideas/hints ?
 | Thanks Giuseppe
 
 A few pastes:
 creating the vm: http://paste.fedoraproject.org/17953/37096041/
 dumping vm xml: http://paste.fedoraproject.org/17951/37096009/
 
 I've tried this out thanks to Toni, losing sanity with vdsClient..
 
 Giuseppe
 

Thanks Giuseppe.
Is this also reported by vdsm in getVmStats?
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo

2013-06-12 Thread Itamar Heim

On 06/12/2013 09:41 AM, Barak Azulay wrote:



- Original Message -

From: Itamar Heim ih...@redhat.com
To: Liran Zelkha lzel...@redhat.com
Cc: engine-devel engine-devel@ovirt.org
Sent: Tuesday, June 11, 2013 4:00:21 PM
Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo

On 06/11/2013 03:26 PM, Liran Zelkha wrote:

Hi all,

I'm checking performance for VdsUpdateRunTimeInfo.
Naturally, much of the performance surrounds database activity
(getVmsRunningOnVds queries, updateDeviceRuntimeInfo, updateVmDynamic)

Few questions:
1. I have implemented batch updates for procedure UpdateVmDeviceRuntimeInfo
for improved performance.
2. Seems like the only parameters UpdateVmDeviceRuntimeInfo is getting are
vm_id,vm_device_id,address and alias. Are those rapidly changing, or will
it be beneficial to implement caching on those updates (to ensure
not-required updates do not travel to the database).


slowly changing, but how will you cover all flows changing these devices
to invalidate the cache (iiuc, this table is modified by engine when
adding devices to a VM as well?)



I don't think that in the device run time info we need to invalidate once we 
add a device.
This is a specific case where we actually get the information from the VDSM 
(addresses are received from libvirt)
The commands IIRC are first send to VDSM and than update the runtime info only 
on changed info (we can also hash it),
It may put the placeholder in the DB first but it still relies on the data 
received from VDSM.


if this table is only updated from vdsm to save it, i agree.
but isn't the engine also manipulating it?
wasn't there a request to be able to maybe edit the addresses some day?

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Add traffic shaping parameters for a network interface.

2013-06-12 Thread Dan Kenigsberg
On Tue, Jun 11, 2013 at 10:34:16AM -0400, Giuseppe Vallarelli wrote:
 - Original Message -
 | From: Giuseppe Vallarelli gvall...@redhat.com
 | To: Livnat Peer lp...@redhat.com
 | Cc: Doron Fediuck dfedi...@redhat.com, engine-devel@ovirt.org, Dan 
 Kenigsberg dan...@redhat.com
 | Sent: Tuesday, June 11, 2013 3:07:32 PM
 | Subject: Re: [Engine-devel] Add traffic shaping parameters for a network 
 interface.
 | 
 | Related to QoS parameters reporting to the engine. Looks like they're 
 already
 | available, I tried to use vdsClient with list verb and I've got the devices
 | list where a 'specParams' is defined (it's empty because I didn't try it 
 with
 | my last patch).
 | 
 | devices = [... {'nicModel': 'pv', 'macAddr': '00:1a:4a:22:3f:04', 'network':
 | 'ovirtmgmt', 'alias': 'net0', 'specParams': {},
 | 'deviceId': '76173ffc-603a-496f-8ffc-31dc4d41cef6', 'address': {'slot':
 | '0x03', 'bus': '0x00', 'domain': '0x',
 | 'type': 'pci', 'function': '0x0'}, 'device': 'bridge', 'type': 'interface'}
 | ...]
 | 
 | Perhaps I'm missing something, any ideas/hints ?

I believe this is enough.

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Add traffic shaping parameters for a network interface.

2013-06-12 Thread Giuseppe Vallarelli


- Original Message -
| From: Doron Fediuck dfedi...@redhat.com
| To: Giuseppe Vallarelli gvall...@redhat.com
| Cc: Livnat Peer lp...@redhat.com, engine-devel@ovirt.org, Dan 
Kenigsberg dan...@redhat.com
| Sent: Wednesday, June 12, 2013 9:05:35 AM
| Subject: Re: [Engine-devel] Add traffic shaping parameters for a network 
interface.
| 
| 
| 
| - Original Message -
|  From: Giuseppe Vallarelli gvall...@redhat.com
|  To: Livnat Peer lp...@redhat.com
|  Cc: Doron Fediuck dfedi...@redhat.com, engine-devel@ovirt.org, Dan
|  Kenigsberg dan...@redhat.com
|  Sent: Tuesday, June 11, 2013 5:34:16 PM
|  Subject: Re: [Engine-devel] Add traffic shaping parameters for a network
|  interface.
|  
|  - Original Message -
|  | From: Giuseppe Vallarelli gvall...@redhat.com
|  | To: Livnat Peer lp...@redhat.com
|  | Cc: Doron Fediuck dfedi...@redhat.com, engine-devel@ovirt.org, Dan
|  | Kenigsberg dan...@redhat.com
|  | Sent: Tuesday, June 11, 2013 3:07:32 PM
|  | Subject: Re: [Engine-devel] Add traffic shaping parameters for a network
|  | interface.
|  | 
|  | Related to QoS parameters reporting to the engine. Looks like they're
|  | already
|  | available, I tried to use vdsClient with list verb and I've got the
|  | devices
|  | list where a 'specParams' is defined (it's empty because I didn't try it
|  | with
|  | my last patch).
|  | 
|  | devices = [... {'nicModel': 'pv', 'macAddr': '00:1a:4a:22:3f:04',
|  | 'network':
|  | 'ovirtmgmt', 'alias': 'net0', 'specParams': {},
|  | 'deviceId': '76173ffc-603a-496f-8ffc-31dc4d41cef6', 'address': {'slot':
|  | '0x03', 'bus': '0x00', 'domain': '0x',
|  | 'type': 'pci', 'function': '0x0'}, 'device': 'bridge', 'type':
|  | 'interface'}
|  | ...]
|  | 
|  | Perhaps I'm missing something, any ideas/hints ?
|  | Thanks Giuseppe
|  
|  A few pastes:
|  creating the vm: http://paste.fedoraproject.org/17953/37096041/
|  dumping vm xml: http://paste.fedoraproject.org/17951/37096009/
|  
|  I've tried this out thanks to Toni, losing sanity with vdsClient..
|  
|  Giuseppe
|  
| 
| Thanks Giuseppe.
| Is this also reported by vdsm in getVmStats?
| 

Unfortunately is not reported using getVmStats, I'm looking into it.
Giuseppe
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo

2013-06-12 Thread Barak Azulay


- Original Message -
 From: Itamar Heim ih...@redhat.com
 To: Barak Azulay bazu...@redhat.com
 Cc: engine-devel engine-devel@ovirt.org
 Sent: Wednesday, June 12, 2013 10:09:09 AM
 Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo
 
 On 06/12/2013 09:41 AM, Barak Azulay wrote:
 
 
  - Original Message -
  From: Itamar Heim ih...@redhat.com
  To: Liran Zelkha lzel...@redhat.com
  Cc: engine-devel engine-devel@ovirt.org
  Sent: Tuesday, June 11, 2013 4:00:21 PM
  Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo
 
  On 06/11/2013 03:26 PM, Liran Zelkha wrote:
  Hi all,
 
  I'm checking performance for VdsUpdateRunTimeInfo.
  Naturally, much of the performance surrounds database activity
  (getVmsRunningOnVds queries, updateDeviceRuntimeInfo, updateVmDynamic)
 
  Few questions:
  1. I have implemented batch updates for procedure
  UpdateVmDeviceRuntimeInfo
  for improved performance.
  2. Seems like the only parameters UpdateVmDeviceRuntimeInfo is getting
  are
  vm_id,vm_device_id,address and alias. Are those rapidly changing, or will
  it be beneficial to implement caching on those updates (to ensure
  not-required updates do not travel to the database).
 
  slowly changing, but how will you cover all flows changing these devices
  to invalidate the cache (iiuc, this table is modified by engine when
  adding devices to a VM as well?)
 
 
  I don't think that in the device run time info we need to invalidate once
  we add a device.
  This is a specific case where we actually get the information from the VDSM
  (addresses are received from libvirt)
  The commands IIRC are first send to VDSM and than update the runtime info
  only on changed info (we can also hash it),
  It may put the placeholder in the DB first but it still relies on the data
  received from VDSM.
 
 if this table is only updated from vdsm to save it, i agree.
 but isn't the engine also manipulating it?
 wasn't there a request to be able to maybe edit the addresses some day?

Even if there was such a request, we still update when we receive the info from 
libvirt.

Anyway there are obviously a few improvements to be done before caching.
- update only when info received from VDSM change (need to verify it is the 
practice here)
- do batch update for the new updated data received.
- do the hashing (of vm device runtime info) on VDSM for the data , and update 
only when hash changes
- caching ...

Barak
 


 
 ___
 Engine-devel mailing list
 Engine-devel@ovirt.org
 http://lists.ovirt.org/mailman/listinfo/engine-devel
 
 
 
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Introducing limited branding support.

2013-06-12 Thread Laszlo Hornyak
Hi,

A related question to the designers:
How do you decide if a new configuration goes to the database config, an 
existing or a new config file? Is there a concept for this?

Thx,
Laszlo

- Original Message -
 From: Alexander Wels aw...@redhat.com
 To: engine-devel@ovirt.org
 Sent: Tuesday, June 11, 2013 6:39:20 PM
 Subject: [Engine-devel] Introducing limited branding support.
 
 Hi Guys,
 
 We recently merged at a patch (http://gerrit.ovirt.org/#/c/13181/) that
 allows
 for limited branding support of oVirt user portal and web admin. We also
 moved
 the styles needed to support this branding out of the application and into
 its
 own module. The styles can now be found in ovirt-
 engine/packaging/branding/ovirt.brand.
 
 In this directory you will find the following files:
 - branding.properties. This file controls the branding theme.
 - ovirt_messages.properties. A standard java resource bundle properties file
 containing the messages that can be changed.
 - A bunch of .css files that contain the classes that can be altered.
 
 I have created a wiki page with some information and pictures of what parts
 of
 the interface can be changed at this point in time. It is located here:
 http://www.ovirt.org/Feature/Branding
 
 There is also more information in README.branding that got introduced with
 this patch.
 
 Alexander
 
 ps.
 If your user interface looks messed up (missing borders and things of that
 nature) the engine cannot find the default branding. This means you are not
 using the make commands recently introduced. We highly recommend you use this
 to have a complete environment. If you are unwilling or unable to use that
 you
 can make a symlink in /etc/ovirt-engine/branding/00-ovirt.brand to ovirt-
 engine/packaging/branding/ovirt.brand
 
 ___
 Engine-devel mailing list
 Engine-devel@ovirt.org
 http://lists.ovirt.org/mailman/listinfo/engine-devel
 
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Introducing limited branding support.

2013-06-12 Thread Alon Bar-Lev


- Original Message -
 From: Laszlo Hornyak lhorn...@redhat.com
 To: aw...@redhat.com
 Cc: engine-devel@ovirt.org
 Sent: Wednesday, June 12, 2013 11:46:23 AM
 Subject: Re: [Engine-devel] Introducing limited branding support.
 
 Hi,
 
 A related question to the designers:
 How do you decide if a new configuration goes to the database config, an
 existing or a new config file? Is there a concept for this?

Hello Laszlo,

I do not fully understand the question in this context.

Just to make sure I understand, do you suggest that css, images and messages 
will be stored in database?

Regards,
Alon Bar-Lev.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Introducing limited branding support.

2013-06-12 Thread Laszlo Hornyak


- Original Message -
 From: Alon Bar-Lev alo...@redhat.com
 To: Laszlo Hornyak lhorn...@redhat.com
 Cc: aw...@redhat.com, engine-devel@ovirt.org
 Sent: Wednesday, June 12, 2013 10:54:45 AM
 Subject: Re: [Engine-devel] Introducing limited branding support.
 
 
 
 - Original Message -
  From: Laszlo Hornyak lhorn...@redhat.com
  To: aw...@redhat.com
  Cc: engine-devel@ovirt.org
  Sent: Wednesday, June 12, 2013 11:46:23 AM
  Subject: Re: [Engine-devel] Introducing limited branding support.
  
  Hi,
  
  A related question to the designers:
  How do you decide if a new configuration goes to the database config, an
  existing or a new config file? Is there a concept for this?
 
 Hello Laszlo,
 
 I do not fully understand the question in this context.
 
 Just to make sure I understand, do you suggest that css, images and messages
 will be stored in database?

Oh no, I wasn't thinking of blobs local FS is just OK. I mean some 
configuration parameters are stored in /etc/somewhere, some in 
/etc/somewhereelse and some in the database, and I'd just like to understand 
what is the concept.

 
 Regards,
 Alon Bar-Lev.
 
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


[Engine-devel] Database configuration vs File based configuration (Was: Introducing limited branding support)

2013-06-12 Thread Alon Bar-Lev


- Original Message -
 From: Laszlo Hornyak lhorn...@redhat.com
 To: Alon Bar-Lev alo...@redhat.com
 Cc: aw...@redhat.com, engine-devel@ovirt.org
 Sent: Wednesday, June 12, 2013 12:32:40 PM
 Subject: Re: [Engine-devel] Introducing limited branding support.
 
 
 
 - Original Message -
  From: Alon Bar-Lev alo...@redhat.com
  To: Laszlo Hornyak lhorn...@redhat.com
  Cc: aw...@redhat.com, engine-devel@ovirt.org
  Sent: Wednesday, June 12, 2013 10:54:45 AM
  Subject: Re: [Engine-devel] Introducing limited branding support.
  
  
  
  - Original Message -
   From: Laszlo Hornyak lhorn...@redhat.com
   To: aw...@redhat.com
   Cc: engine-devel@ovirt.org
   Sent: Wednesday, June 12, 2013 11:46:23 AM
   Subject: Re: [Engine-devel] Introducing limited branding support.
   
   Hi,
   
   A related question to the designers:
   How do you decide if a new configuration goes to the database config, an
   existing or a new config file? Is there a concept for this?
  
  Hello Laszlo,
  
  I do not fully understand the question in this context.
  
  Just to make sure I understand, do you suggest that css, images and
  messages
  will be stored in database?
 
 Oh no, I wasn't thinking of blobs local FS is just OK. I mean some
 configuration parameters are stored in /etc/somewhere, some in
 /etc/somewhereelse and some in the database, and I'd just like to understand
 what is the concept.

There is no single concept... however I can present you one...
I hope it will answer your question.

1. Packaging considerations
Why: During package installation and upgrade the database cannot be accessed.
Example: Plugin installation can drop a file but not modify database.

2. Default values should not be stored at database
Why: Product upgrade may update default, if stored in database the upgrade 
process must also access the database to update defaults and add new defaults.
Solution: Application should query database, if missing revert to default 
within file, if missing revert to default within application.
Current state: We do store defaults within database, this should be fixed.

3. Options that are not controlled by application (and owner) but controlled by 
system and sysadmin should not be stored in database:
Example: File locations, no sense to keep these in database if application does 
not control the location, especially if may be modified between installations 
and distributions.
Example: Packaging, product version.
Example: Host name, should actually be acquired from system unless overridden.
Example: Service parameters, in this case sysadmin expects these to be at their 
standard locations.
Current state: We still store file locations within database, this should be 
fixed.

4. Database connection information
Why: Egg and chicken.

5. Local protected data
Why: Database is exposed to anyone, even if we encrypt a field we need to 
access the local key, which resides outside of the database.  A simple solution 
for local resources is to use the operating system ACL.
Example: The password of the engine keystore which is local resource.

6. Values that are used by other technologies used for maintenance
Why: If product is using multiple technologies to handle a resource (shell 
scripts, python scripts and java), it is easier to have it in configuration 
than in database. Especially if these required during setup or maintenance, as 
database may be unavailable. These are usually local resources already covered 
by the above items.
Example: The location and password of the engine key which is used by log 
collector, database upgrade, setup, upgrade.

Regards,
Alon Bar-Lev.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Add traffic shaping parameters for a network interface.

2013-06-12 Thread Giuseppe Vallarelli
- Original Message -
| From: Giuseppe Vallarelli gvall...@redhat.com
| To: Doron Fediuck dfedi...@redhat.com
| Cc: engine-devel@ovirt.org
| Sent: Wednesday, June 12, 2013 9:28:19 AM
| Subject: Re: [Engine-devel] Add traffic shaping parameters for a network 
interface.
| 
| 
| 
| - Original Message -
| | From: Doron Fediuck dfedi...@redhat.com
| | To: Giuseppe Vallarelli gvall...@redhat.com
| | Cc: Livnat Peer lp...@redhat.com, engine-devel@ovirt.org, Dan
| | Kenigsberg dan...@redhat.com
| | Sent: Wednesday, June 12, 2013 9:05:35 AM
| | Subject: Re: [Engine-devel] Add traffic shaping parameters for a network
| | interface.
| | 
| | 
| | 
| | - Original Message -
| |  From: Giuseppe Vallarelli gvall...@redhat.com
| |  To: Livnat Peer lp...@redhat.com
| |  Cc: Doron Fediuck dfedi...@redhat.com, engine-devel@ovirt.org, Dan
| |  Kenigsberg dan...@redhat.com
| |  Sent: Tuesday, June 11, 2013 5:34:16 PM
| |  Subject: Re: [Engine-devel] Add traffic shaping parameters for a network
| |  interface.
| |  
| |  - Original Message -
| |  | From: Giuseppe Vallarelli gvall...@redhat.com
| |  | To: Livnat Peer lp...@redhat.com
| |  | Cc: Doron Fediuck dfedi...@redhat.com, engine-devel@ovirt.org, Dan
| |  | Kenigsberg dan...@redhat.com
| |  | Sent: Tuesday, June 11, 2013 3:07:32 PM
| |  | Subject: Re: [Engine-devel] Add traffic shaping parameters for a
| |  | network
| |  | interface.
| |  | 
| |  | Related to QoS parameters reporting to the engine. Looks like they're
| |  | already
| |  | available, I tried to use vdsClient with list verb and I've got the
| |  | devices
| |  | list where a 'specParams' is defined (it's empty because I didn't try
| |  | it
| |  | with
| |  | my last patch).
| |  | 
| |  | devices = [... {'nicModel': 'pv', 'macAddr': '00:1a:4a:22:3f:04',
| |  | 'network':
| |  | 'ovirtmgmt', 'alias': 'net0', 'specParams': {},
| |  | 'deviceId': '76173ffc-603a-496f-8ffc-31dc4d41cef6', 'address': {'slot':
| |  | '0x03', 'bus': '0x00', 'domain': '0x',
| |  | 'type': 'pci', 'function': '0x0'}, 'device': 'bridge', 'type':
| |  | 'interface'}
| |  | ...]
| |  | 
| |  | Perhaps I'm missing something, any ideas/hints ?
| |  | Thanks Giuseppe
| |  
| |  A few pastes:
| |  creating the vm: http://paste.fedoraproject.org/17953/37096041/
| |  dumping vm xml: http://paste.fedoraproject.org/17951/37096009/
| |  
| |  I've tried this out thanks to Toni, losing sanity with vdsClient..
| |  
| |  Giuseppe
| |  
| | 
| | Thanks Giuseppe.
| | Is this also reported by vdsm in getVmStats?
| | 
| 
| Unfortunately is not reported using getVmStats, I'm looking into it.
| Giuseppe

New paste: http://paste.fedoraproject.org/18161/71033933/

Giuseppe
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Add traffic shaping parameters for a network interface.

2013-06-12 Thread Doron Fediuck


- Original Message -
 From: Giuseppe Vallarelli gvall...@redhat.com
 To: Doron Fediuck dfedi...@redhat.com
 Cc: engine-devel@ovirt.org
 Sent: Wednesday, June 12, 2013 1:47:03 PM
 Subject: Re: [Engine-devel] Add traffic shaping parameters for a network 
 interface.
 
 - Original Message -
 | From: Giuseppe Vallarelli gvall...@redhat.com
 | To: Doron Fediuck dfedi...@redhat.com
 | Cc: engine-devel@ovirt.org
 | Sent: Wednesday, June 12, 2013 9:28:19 AM
 | Subject: Re: [Engine-devel] Add traffic shaping parameters for a network
 | interface.
 | 
 | 
 | 
 | - Original Message -
 | | From: Doron Fediuck dfedi...@redhat.com
 | | To: Giuseppe Vallarelli gvall...@redhat.com
 | | Cc: Livnat Peer lp...@redhat.com, engine-devel@ovirt.org, Dan
 | | Kenigsberg dan...@redhat.com
 | | Sent: Wednesday, June 12, 2013 9:05:35 AM
 | | Subject: Re: [Engine-devel] Add traffic shaping parameters for a network
 | | interface.
 | | 
 | | 
 | | 
 | | - Original Message -
 | |  From: Giuseppe Vallarelli gvall...@redhat.com
 | |  To: Livnat Peer lp...@redhat.com
 | |  Cc: Doron Fediuck dfedi...@redhat.com, engine-devel@ovirt.org, Dan
 | |  Kenigsberg dan...@redhat.com
 | |  Sent: Tuesday, June 11, 2013 5:34:16 PM
 | |  Subject: Re: [Engine-devel] Add traffic shaping parameters for a
 | |  network
 | |  interface.
 | |  
 | |  - Original Message -
 | |  | From: Giuseppe Vallarelli gvall...@redhat.com
 | |  | To: Livnat Peer lp...@redhat.com
 | |  | Cc: Doron Fediuck dfedi...@redhat.com, engine-devel@ovirt.org,
 | |  | Dan
 | |  | Kenigsberg dan...@redhat.com
 | |  | Sent: Tuesday, June 11, 2013 3:07:32 PM
 | |  | Subject: Re: [Engine-devel] Add traffic shaping parameters for a
 | |  | network
 | |  | interface.
 | |  | 
 | |  | Related to QoS parameters reporting to the engine. Looks like they're
 | |  | already
 | |  | available, I tried to use vdsClient with list verb and I've got the
 | |  | devices
 | |  | list where a 'specParams' is defined (it's empty because I didn't try
 | |  | it
 | |  | with
 | |  | my last patch).
 | |  | 
 | |  | devices = [... {'nicModel': 'pv', 'macAddr': '00:1a:4a:22:3f:04',
 | |  | 'network':
 | |  | 'ovirtmgmt', 'alias': 'net0', 'specParams': {},
 | |  | 'deviceId': '76173ffc-603a-496f-8ffc-31dc4d41cef6', 'address':
 | |  | {'slot':
 | |  | '0x03', 'bus': '0x00', 'domain': '0x',
 | |  | 'type': 'pci', 'function': '0x0'}, 'device': 'bridge', 'type':
 | |  | 'interface'}
 | |  | ...]
 | |  | 
 | |  | Perhaps I'm missing something, any ideas/hints ?
 | |  | Thanks Giuseppe
 | |  
 | |  A few pastes:
 | |  creating the vm: http://paste.fedoraproject.org/17953/37096041/
 | |  dumping vm xml: http://paste.fedoraproject.org/17951/37096009/
 | |  
 | |  I've tried this out thanks to Toni, losing sanity with vdsClient..
 | |  
 | |  Giuseppe
 | |  
 | | 
 | | Thanks Giuseppe.
 | | Is this also reported by vdsm in getVmStats?
 | | 
 | 
 | Unfortunately is not reported using getVmStats, I'm looking into it.
 | Giuseppe
 
 New paste: http://paste.fedoraproject.org/18161/71033933/
 
 Giuseppe

Brilliant, thanks!
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Add traffic shaping parameters for a network interface.

2013-06-12 Thread Dan Kenigsberg
On Wed, Jun 12, 2013 at 06:54:52AM -0400, Doron Fediuck wrote:
 
 
 - Original Message -
  From: Giuseppe Vallarelli gvall...@redhat.com
  To: Doron Fediuck dfedi...@redhat.com
  Cc: engine-devel@ovirt.org
  Sent: Wednesday, June 12, 2013 1:47:03 PM
  Subject: Re: [Engine-devel] Add traffic shaping parameters for a network 
  interface.
  
  - Original Message -
  | From: Giuseppe Vallarelli gvall...@redhat.com
  | To: Doron Fediuck dfedi...@redhat.com
  | Cc: engine-devel@ovirt.org
  | Sent: Wednesday, June 12, 2013 9:28:19 AM
  | Subject: Re: [Engine-devel] Add traffic shaping parameters for a network
  | interface.
  | 
  | 
  | 
  | - Original Message -
  | | From: Doron Fediuck dfedi...@redhat.com
  | | To: Giuseppe Vallarelli gvall...@redhat.com
  | | Cc: Livnat Peer lp...@redhat.com, engine-devel@ovirt.org, Dan
  | | Kenigsberg dan...@redhat.com
  | | Sent: Wednesday, June 12, 2013 9:05:35 AM
  | | Subject: Re: [Engine-devel] Add traffic shaping parameters for a network
  | | interface.
  | | 
  | | 
  | | 
  | | - Original Message -
  | |  From: Giuseppe Vallarelli gvall...@redhat.com
  | |  To: Livnat Peer lp...@redhat.com
  | |  Cc: Doron Fediuck dfedi...@redhat.com, engine-devel@ovirt.org, 
  Dan
  | |  Kenigsberg dan...@redhat.com
  | |  Sent: Tuesday, June 11, 2013 5:34:16 PM
  | |  Subject: Re: [Engine-devel] Add traffic shaping parameters for a
  | |  network
  | |  interface.
  | |  
  | |  - Original Message -
  | |  | From: Giuseppe Vallarelli gvall...@redhat.com
  | |  | To: Livnat Peer lp...@redhat.com
  | |  | Cc: Doron Fediuck dfedi...@redhat.com, engine-devel@ovirt.org,
  | |  | Dan
  | |  | Kenigsberg dan...@redhat.com
  | |  | Sent: Tuesday, June 11, 2013 3:07:32 PM
  | |  | Subject: Re: [Engine-devel] Add traffic shaping parameters for a
  | |  | network
  | |  | interface.
  | |  | 
  | |  | Related to QoS parameters reporting to the engine. Looks like 
  they're
  | |  | already
  | |  | available, I tried to use vdsClient with list verb and I've got the
  | |  | devices
  | |  | list where a 'specParams' is defined (it's empty because I didn't 
  try
  | |  | it
  | |  | with
  | |  | my last patch).
  | |  | 
  | |  | devices = [... {'nicModel': 'pv', 'macAddr': '00:1a:4a:22:3f:04',
  | |  | 'network':
  | |  | 'ovirtmgmt', 'alias': 'net0', 'specParams': {},
  | |  | 'deviceId': '76173ffc-603a-496f-8ffc-31dc4d41cef6', 'address':
  | |  | {'slot':
  | |  | '0x03', 'bus': '0x00', 'domain': '0x',
  | |  | 'type': 'pci', 'function': '0x0'}, 'device': 'bridge', 'type':
  | |  | 'interface'}
  | |  | ...]
  | |  | 
  | |  | Perhaps I'm missing something, any ideas/hints ?
  | |  | Thanks Giuseppe
  | |  
  | |  A few pastes:
  | |  creating the vm: http://paste.fedoraproject.org/17953/37096041/
  | |  dumping vm xml: http://paste.fedoraproject.org/17951/37096009/
  | |  
  | |  I've tried this out thanks to Toni, losing sanity with vdsClient..
  | |  
  | |  Giuseppe
  | |  
  | | 
  | | Thanks Giuseppe.
  | | Is this also reported by vdsm in getVmStats?
  | | 
  | 
  | Unfortunately is not reported using getVmStats, I'm looking into it.
  | Giuseppe
  
  New paste: http://paste.fedoraproject.org/18161/71033933/

(using hyperlinks instead of inlining the suggested API is unfriendly to
reviewers and to list archives)

network = {'vnet0': {'macAddr': 'D0:67:E5:F0:75:B4', 'outbound': {'average': 
'1024'}, 'rxDropped': '0', 'txDropped': '0', 'inbound': {'average': '1024'}, 
'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 
'unknown', 'speed': '1000', 'name': 'vnet0'}}

  
  Giuseppe
 
 Brilliant, thanks!

Why does it fit into getVmStats? getVmStats should report
dynamically-changing properties. Bandwidth limitations are more static
control entities, why should we report them every 10 seconds or so?

Dan
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo

2013-06-12 Thread Liran Zelkha
Hi all,

Current status of VdsUpdateRuntimeInfo (please note - patch is not yet 
committed).
Initial UpdateRuntimeInfo performance wasted roughly 57% of CPU time on DB 
activity. Of that, about 25% of CPU time wasted on update behavior. 
After modification (moving updates to batch update) UpdateRuntimeInfo 
performance wasted roughly 40% of CPU time on DB activity. And only 12% of that 
was wasted on update behavior.

My next task is to try and optimize the rest of overall database performance.


- Original Message -
From: Liran Zelkha liran.zel...@gmail.com
To: Barak Azulay bazu...@redhat.com
Cc: engine-devel engine-devel@ovirt.org
Sent: Wednesday, June 12, 2013 12:35:03 PM
Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo

Hi guys,

I'm working hard on this. I'll send summary mail and findings later today. 

On Jun 12, 2013, at 10:45 AM, Barak Azulay wrote:

 
 
 - Original Message -
 From: Itamar Heim ih...@redhat.com
 To: Barak Azulay bazu...@redhat.com
 Cc: engine-devel engine-devel@ovirt.org
 Sent: Wednesday, June 12, 2013 10:09:09 AM
 Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo
 
 On 06/12/2013 09:41 AM, Barak Azulay wrote:
 
 
 - Original Message -
 From: Itamar Heim ih...@redhat.com
 To: Liran Zelkha lzel...@redhat.com
 Cc: engine-devel engine-devel@ovirt.org
 Sent: Tuesday, June 11, 2013 4:00:21 PM
 Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo
 
 On 06/11/2013 03:26 PM, Liran Zelkha wrote:
 Hi all,
 
 I'm checking performance for VdsUpdateRunTimeInfo.
 Naturally, much of the performance surrounds database activity
 (getVmsRunningOnVds queries, updateDeviceRuntimeInfo, updateVmDynamic)
 
 Few questions:
 1. I have implemented batch updates for procedure
 UpdateVmDeviceRuntimeInfo
 for improved performance.
 2. Seems like the only parameters UpdateVmDeviceRuntimeInfo is getting
 are
 vm_id,vm_device_id,address and alias. Are those rapidly changing, or will
 it be beneficial to implement caching on those updates (to ensure
 not-required updates do not travel to the database).
 
 slowly changing, but how will you cover all flows changing these devices
 to invalidate the cache (iiuc, this table is modified by engine when
 adding devices to a VM as well?)
 
 
 I don't think that in the device run time info we need to invalidate once
 we add a device.
 This is a specific case where we actually get the information from the VDSM
 (addresses are received from libvirt)
 The commands IIRC are first send to VDSM and than update the runtime info
 only on changed info (we can also hash it),
 It may put the placeholder in the DB first but it still relies on the data
 received from VDSM.
 
 if this table is only updated from vdsm to save it, i agree.
 but isn't the engine also manipulating it?
 wasn't there a request to be able to maybe edit the addresses some day?
 
 Even if there was such a request, we still update when we receive the info 
 from libvirt.
 
 Anyway there are obviously a few improvements to be done before caching.
 - update only when info received from VDSM change (need to verify it is the 
 practice here)
 - do batch update for the new updated data received.
 - do the hashing (of vm device runtime info) on VDSM for the data , and 
 update only when hash changes
 - caching ...
 
 Barak
 
 
 
 
 ___
 Engine-devel mailing list
 Engine-devel@ovirt.org
 http://lists.ovirt.org/mailman/listinfo/engine-devel
 
 
 
 ___
 Engine-devel mailing list
 Engine-devel@ovirt.org
 http://lists.ovirt.org/mailman/listinfo/engine-devel

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo

2013-06-12 Thread Itamar Heim

On 06/12/2013 04:03 PM, Liran Zelkha wrote:

Hi all,

Current status of VdsUpdateRuntimeInfo (please note - patch is not yet 
committed).
Initial UpdateRuntimeInfo performance wasted roughly 57% of CPU time on DB 
activity. Of that, about 25% of CPU time wasted on update behavior.
After modification (moving updates to batch update) UpdateRuntimeInfo 
performance wasted roughly 40% of CPU time on DB activity. And only 12% of that 
was wasted on update behavior.


which changes are in place for above?


My next task is to try and optimize the rest of overall database performance.


- Original Message -
From: Liran Zelkha liran.zel...@gmail.com
To: Barak Azulay bazu...@redhat.com
Cc: engine-devel engine-devel@ovirt.org
Sent: Wednesday, June 12, 2013 12:35:03 PM
Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo

Hi guys,

I'm working hard on this. I'll send summary mail and findings later today.

On Jun 12, 2013, at 10:45 AM, Barak Azulay wrote:




- Original Message -

From: Itamar Heim ih...@redhat.com
To: Barak Azulay bazu...@redhat.com
Cc: engine-devel engine-devel@ovirt.org
Sent: Wednesday, June 12, 2013 10:09:09 AM
Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo

On 06/12/2013 09:41 AM, Barak Azulay wrote:



- Original Message -

From: Itamar Heim ih...@redhat.com
To: Liran Zelkha lzel...@redhat.com
Cc: engine-devel engine-devel@ovirt.org
Sent: Tuesday, June 11, 2013 4:00:21 PM
Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo

On 06/11/2013 03:26 PM, Liran Zelkha wrote:

Hi all,

I'm checking performance for VdsUpdateRunTimeInfo.
Naturally, much of the performance surrounds database activity
(getVmsRunningOnVds queries, updateDeviceRuntimeInfo, updateVmDynamic)

Few questions:
1. I have implemented batch updates for procedure
UpdateVmDeviceRuntimeInfo
for improved performance.
2. Seems like the only parameters UpdateVmDeviceRuntimeInfo is getting
are
vm_id,vm_device_id,address and alias. Are those rapidly changing, or will
it be beneficial to implement caching on those updates (to ensure
not-required updates do not travel to the database).


slowly changing, but how will you cover all flows changing these devices
to invalidate the cache (iiuc, this table is modified by engine when
adding devices to a VM as well?)



I don't think that in the device run time info we need to invalidate once
we add a device.
This is a specific case where we actually get the information from the VDSM
(addresses are received from libvirt)
The commands IIRC are first send to VDSM and than update the runtime info
only on changed info (we can also hash it),
It may put the placeholder in the DB first but it still relies on the data
received from VDSM.


if this table is only updated from vdsm to save it, i agree.
but isn't the engine also manipulating it?
wasn't there a request to be able to maybe edit the addresses some day?


Even if there was such a request, we still update when we receive the info from 
libvirt.

Anyway there are obviously a few improvements to be done before caching.
- update only when info received from VDSM change (need to verify it is the 
practice here)
- do batch update for the new updated data received.
- do the hashing (of vm device runtime info) on VDSM for the data , and update 
only when hash changes
- caching ...

Barak





___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel




___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel



___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo

2013-06-12 Thread Eli Mesika


- Original Message -
 From: Barak Azulay bazu...@redhat.com
 To: Itamar Heim ih...@redhat.com
 Cc: engine-devel engine-devel@ovirt.org
 Sent: Wednesday, June 12, 2013 10:45:25 AM
 Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo
 
 
 
 - Original Message -
  From: Itamar Heim ih...@redhat.com
  To: Barak Azulay bazu...@redhat.com
  Cc: engine-devel engine-devel@ovirt.org
  Sent: Wednesday, June 12, 2013 10:09:09 AM
  Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo
  
  On 06/12/2013 09:41 AM, Barak Azulay wrote:
  
  
   - Original Message -
   From: Itamar Heim ih...@redhat.com
   To: Liran Zelkha lzel...@redhat.com
   Cc: engine-devel engine-devel@ovirt.org
   Sent: Tuesday, June 11, 2013 4:00:21 PM
   Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo
  
   On 06/11/2013 03:26 PM, Liran Zelkha wrote:
   Hi all,
  
   I'm checking performance for VdsUpdateRunTimeInfo.
   Naturally, much of the performance surrounds database activity
   (getVmsRunningOnVds queries, updateDeviceRuntimeInfo, updateVmDynamic)
  
   Few questions:
   1. I have implemented batch updates for procedure
   UpdateVmDeviceRuntimeInfo
   for improved performance.
   2. Seems like the only parameters UpdateVmDeviceRuntimeInfo is getting
   are
   vm_id,vm_device_id,address and alias. Are those rapidly changing, or
   will
   it be beneficial to implement caching on those updates (to ensure
   not-required updates do not travel to the database).
  
   slowly changing, but how will you cover all flows changing these devices
   to invalidate the cache (iiuc, this table is modified by engine when
   adding devices to a VM as well?)
  
  
   I don't think that in the device run time info we need to invalidate once
   we add a device.
   This is a specific case where we actually get the information from the
   VDSM
   (addresses are received from libvirt)
   The commands IIRC are first send to VDSM and than update the runtime info
   only on changed info (we can also hash it),
   It may put the placeholder in the DB first but it still relies on the
   data
   received from VDSM.
  
  if this table is only updated from vdsm to save it, i agree.
  but isn't the engine also manipulating it?
  wasn't there a request to be able to maybe edit the addresses some day?
 
 Even if there was such a request, we still update when we receive the info
 from libvirt.
 
 Anyway there are obviously a few improvements to be done before caching.
 - update only when info received from VDSM change (need to verify it is the
 practice here)

This should be the behavior right now
There is a hash column in vm_dynamic , this hash is passed by VDSM
We are updating only VMs that change their hash value 

 - do batch update for the new updated data received.
 - do the hashing (of vm device runtime info) on VDSM for the data , and
 update only when hash changes
 - caching ...
 
 Barak
  
 
 
  
  ___
  Engine-devel mailing list
  Engine-devel@ovirt.org
  http://lists.ovirt.org/mailman/listinfo/engine-devel
  
  
  
 ___
 Engine-devel mailing list
 Engine-devel@ovirt.org
 http://lists.ovirt.org/mailman/listinfo/engine-devel
 
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo

2013-06-12 Thread Liran Zelkha
Batch updates for UpdateVmDeviceRuntimeInfo, UpdateVdsInterfaceStatistics and 
UpdateVdsDynamic.
All improved execution performance by about 60-80%.

On Jun 12, 2013, at 4:05 PM, Itamar Heim wrote:

 On 06/12/2013 04:03 PM, Liran Zelkha wrote:
 Hi all,
 
 Current status of VdsUpdateRuntimeInfo (please note - patch is not yet 
 committed).
 Initial UpdateRuntimeInfo performance wasted roughly 57% of CPU time on DB 
 activity. Of that, about 25% of CPU time wasted on update behavior.
 After modification (moving updates to batch update) UpdateRuntimeInfo 
 performance wasted roughly 40% of CPU time on DB activity. And only 12% of 
 that was wasted on update behavior.
 
 which changes are in place for above?
 
 My next task is to try and optimize the rest of overall database performance.
 
 
 - Original Message -
 From: Liran Zelkha liran.zel...@gmail.com
 To: Barak Azulay bazu...@redhat.com
 Cc: engine-devel engine-devel@ovirt.org
 Sent: Wednesday, June 12, 2013 12:35:03 PM
 Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo
 
 Hi guys,
 
 I'm working hard on this. I'll send summary mail and findings later today.
 
 On Jun 12, 2013, at 10:45 AM, Barak Azulay wrote:
 
 
 
 - Original Message -
 From: Itamar Heim ih...@redhat.com
 To: Barak Azulay bazu...@redhat.com
 Cc: engine-devel engine-devel@ovirt.org
 Sent: Wednesday, June 12, 2013 10:09:09 AM
 Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo
 
 On 06/12/2013 09:41 AM, Barak Azulay wrote:
 
 
 - Original Message -
 From: Itamar Heim ih...@redhat.com
 To: Liran Zelkha lzel...@redhat.com
 Cc: engine-devel engine-devel@ovirt.org
 Sent: Tuesday, June 11, 2013 4:00:21 PM
 Subject: Re: [Engine-devel] Updates in VdsUpdateRuntimeInfo
 
 On 06/11/2013 03:26 PM, Liran Zelkha wrote:
 Hi all,
 
 I'm checking performance for VdsUpdateRunTimeInfo.
 Naturally, much of the performance surrounds database activity
 (getVmsRunningOnVds queries, updateDeviceRuntimeInfo, updateVmDynamic)
 
 Few questions:
 1. I have implemented batch updates for procedure
 UpdateVmDeviceRuntimeInfo
 for improved performance.
 2. Seems like the only parameters UpdateVmDeviceRuntimeInfo is getting
 are
 vm_id,vm_device_id,address and alias. Are those rapidly changing, or 
 will
 it be beneficial to implement caching on those updates (to ensure
 not-required updates do not travel to the database).
 
 slowly changing, but how will you cover all flows changing these devices
 to invalidate the cache (iiuc, this table is modified by engine when
 adding devices to a VM as well?)
 
 
 I don't think that in the device run time info we need to invalidate once
 we add a device.
 This is a specific case where we actually get the information from the 
 VDSM
 (addresses are received from libvirt)
 The commands IIRC are first send to VDSM and than update the runtime info
 only on changed info (we can also hash it),
 It may put the placeholder in the DB first but it still relies on the data
 received from VDSM.
 
 if this table is only updated from vdsm to save it, i agree.
 but isn't the engine also manipulating it?
 wasn't there a request to be able to maybe edit the addresses some day?
 
 Even if there was such a request, we still update when we receive the info 
 from libvirt.
 
 Anyway there are obviously a few improvements to be done before caching.
 - update only when info received from VDSM change (need to verify it is the 
 practice here)
 - do batch update for the new updated data received.
 - do the hashing (of vm device runtime info) on VDSM for the data , and 
 update only when hash changes
 - caching ...
 
 Barak
 
 
 
 
 ___
 Engine-devel mailing list
 Engine-devel@ovirt.org
 http://lists.ovirt.org/mailman/listinfo/engine-devel
 
 
 
 ___
 Engine-devel mailing list
 Engine-devel@ovirt.org
 http://lists.ovirt.org/mailman/listinfo/engine-devel
 
 ___
 Engine-devel mailing list
 Engine-devel@ovirt.org
 http://lists.ovirt.org/mailman/listinfo/engine-devel
 ___
 Engine-devel mailing list
 Engine-devel@ovirt.org
 http://lists.ovirt.org/mailman/listinfo/engine-devel
 
 

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Add traffic shaping parameters for a network interface.

2013-06-12 Thread Doron Fediuck
- Original Message -
 From: Dan Kenigsberg dan...@redhat.com
 To: Doron Fediuck dfedi...@redhat.com
 Cc: Giuseppe Vallarelli gvall...@redhat.com, engine-devel@ovirt.org
 Sent: Wednesday, June 12, 2013 3:59:43 PM
 Subject: Re: [Engine-devel] Add traffic shaping parameters for a network 
 interface.
 
 On Wed, Jun 12, 2013 at 06:54:52AM -0400, Doron Fediuck wrote:
  
  
  - Original Message -
   From: Giuseppe Vallarelli gvall...@redhat.com
   To: Doron Fediuck dfedi...@redhat.com
   Cc: engine-devel@ovirt.org
   Sent: Wednesday, June 12, 2013 1:47:03 PM
   Subject: Re: [Engine-devel] Add traffic shaping parameters for a network
   interface.
   
   - Original Message -
   | From: Giuseppe Vallarelli gvall...@redhat.com
   | To: Doron Fediuck dfedi...@redhat.com
   | Cc: engine-devel@ovirt.org
   | Sent: Wednesday, June 12, 2013 9:28:19 AM
   | Subject: Re: [Engine-devel] Add traffic shaping parameters for a
   | network
   | interface.
   | 
   | 
   | 
   | - Original Message -
   | | From: Doron Fediuck dfedi...@redhat.com
   | | To: Giuseppe Vallarelli gvall...@redhat.com
   | | Cc: Livnat Peer lp...@redhat.com, engine-devel@ovirt.org, Dan
   | | Kenigsberg dan...@redhat.com
   | | Sent: Wednesday, June 12, 2013 9:05:35 AM
   | | Subject: Re: [Engine-devel] Add traffic shaping parameters for a
   | | network
   | | interface.
   | | 
   | | 
   | | 
   | | - Original Message -
   | |  From: Giuseppe Vallarelli gvall...@redhat.com
   | |  To: Livnat Peer lp...@redhat.com
   | |  Cc: Doron Fediuck dfedi...@redhat.com, engine-devel@ovirt.org,
   | |  Dan
   | |  Kenigsberg dan...@redhat.com
   | |  Sent: Tuesday, June 11, 2013 5:34:16 PM
   | |  Subject: Re: [Engine-devel] Add traffic shaping parameters for a
   | |  network
   | |  interface.
   | |  
   | |  - Original Message -
   | |  | From: Giuseppe Vallarelli gvall...@redhat.com
   | |  | To: Livnat Peer lp...@redhat.com
   | |  | Cc: Doron Fediuck dfedi...@redhat.com,
   | |  | engine-devel@ovirt.org,
   | |  | Dan
   | |  | Kenigsberg dan...@redhat.com
   | |  | Sent: Tuesday, June 11, 2013 3:07:32 PM
   | |  | Subject: Re: [Engine-devel] Add traffic shaping parameters for a
   | |  | network
   | |  | interface.
   | |  | 
   | |  | Related to QoS parameters reporting to the engine. Looks like
   | |  | they're
   | |  | already
   | |  | available, I tried to use vdsClient with list verb and I've got
   | |  | the
   | |  | devices
   | |  | list where a 'specParams' is defined (it's empty because I didn't
   | |  | try
   | |  | it
   | |  | with
   | |  | my last patch).
   | |  | 
   | |  | devices = [... {'nicModel': 'pv', 'macAddr': '00:1a:4a:22:3f:04',
   | |  | 'network':
   | |  | 'ovirtmgmt', 'alias': 'net0', 'specParams': {},
   | |  | 'deviceId': '76173ffc-603a-496f-8ffc-31dc4d41cef6', 'address':
   | |  | {'slot':
   | |  | '0x03', 'bus': '0x00', 'domain': '0x',
   | |  | 'type': 'pci', 'function': '0x0'}, 'device': 'bridge', 'type':
   | |  | 'interface'}
   | |  | ...]
   | |  | 
   | |  | Perhaps I'm missing something, any ideas/hints ?
   | |  | Thanks Giuseppe
   | |  
   | |  A few pastes:
   | |  creating the vm: http://paste.fedoraproject.org/17953/37096041/
   | |  dumping vm xml: http://paste.fedoraproject.org/17951/37096009/
   | |  
   | |  I've tried this out thanks to Toni, losing sanity with vdsClient..
   | |  
   | |  Giuseppe
   | |  
   | | 
   | | Thanks Giuseppe.
   | | Is this also reported by vdsm in getVmStats?
   | | 
   | 
   | Unfortunately is not reported using getVmStats, I'm looking into it.
   | Giuseppe
   
   New paste: http://paste.fedoraproject.org/18161/71033933/
 
 (using hyperlinks instead of inlining the suggested API is unfriendly to
 reviewers and to list archives)
 
 network = {'vnet0': {'macAddr': 'D0:67:E5:F0:75:B4', 'outbound': {'average':
 '1024'}, 'rxDropped': '0', 'txDropped': '0', 'inbound': {'average': '1024'},
 'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state':
 'unknown', 'speed': '1000', 'name': 'vnet0'}}
 
   
   Giuseppe
  
  Brilliant, thanks!
 
 Why does it fit into getVmStats? getVmStats should report
 dynamically-changing properties. Bandwidth limitations are more static
 control entities, why should we report them every 10 seconds or so?
 
 Dan
 

I have no problems with having it in something which runs less frequently.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


[Engine-devel] Cancelled: ovirt network

2013-06-12 Thread Livnat Peer
BEGIN:VCALENDAR
PRODID:Zimbra-Calendar-Provider
VERSION:2.0
METHOD:CANCEL
BEGIN:VTIMEZONE
TZID:Asia/Jerusalem
BEGIN:STANDARD
DTSTART:16010101T02
TZOFFSETTO:+0200
TZOFFSETFROM:+0300
RRULE:FREQ=YEARLY;WKST=MO;INTERVAL=1;BYMONTH=10;BYDAY=1SU
TZNAME:IST
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:16010101T02
TZOFFSETTO:+0300
TZOFFSETFROM:+0200
RRULE:FREQ=YEARLY;WKST=MO;INTERVAL=1;BYMONTH=3;BYDAY=-1FR
TZNAME:IDT
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VEVENT
UID:cfd70d5d-b663-41f3-b72a-15b532782b23
SUMMARY:Cancelled: ovirt network 
COMMENT:The following meeting has been cancelled:
ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:engine-
 de...@ovirt.org
ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:vdsm-de
 v...@lists.fedorahosted.org
ATTENDEE;CN=Tony Gargya/Germany/IBM;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTI
 ON;RSVP=TRUE:mailto:gar...@de.ibm.com
ATTENDEE;CN=Dan Yasny;ROLE=REQ-PARTICIPANT;PARTSTAT=TENTATIVE;RSVP=TRUE:mail
 to:dya...@redhat.com
ATTENDEE;CN=Simon Grinberg;ROLE=REQ-PARTICIPANT;PARTSTAT=ACCEPTED;RSVP=TRUE:
 mailto:si...@redhat.com
ATTENDEE;CN=Mike Kolesnik;PARTSTAT=ACCEPTED:mailto:mkole...@redhat.com
ATTENDEE;CN=Avi Tal;PARTSTAT=ACCEPTED:mailto:a...@redhat.com
ATTENDEE;CN=Igor Lvovsky;PARTSTAT=ACCEPTED:mailto:ilvov...@redhat.com
ATTENDEE;CN=Doron Fediuck;PARTSTAT=ACCEPTED:mailto:dfedi...@redhat.com
ATTENDEE;CN=Pradipta Kumar Banerjee;ROLE=REQ-PARTICIPANT;PARTSTAT=ACCEPTED:m
 ailto:pradipta.baner...@gmail.com
ATTENDEE;CN=Gary Kotton;ROLE=REQ-PARTICIPANT;PARTSTAT=ACCEPTED:mailto:gkotto
 n...@redhat.com
ATTENDEE;CN=Keith Robertson;PARTSTAT=ACCEPTED:mailto:krobe...@redhat.com
ATTENDEE;CN=Antoni Segura Puimedon;PARTSTAT=ACCEPTED:mailto:asegurap@redhat.
 com
ATTENDEE;CN=Thang Pham/Poughkeepsie/IBM;SENT-BY=mailto:thang.p...@us.ibm.co
 m;PARTSTAT=ACCEPTED:mailto:thang.p...@us.ibm.com
ATTENDEE;CN=Yaniv Kaul;PARTSTAT=TENTATIVE:mailto:yk...@redhat.com
ATTENDEE;CN=Ayal Baron;PARTSTAT=TENTATIVE:mailto:aba...@redhat.com
ORGANIZER;CN=Livnat Peer:mailto:lp...@redhat.com
DTSTART;TZID=Asia/Jerusalem:20120815T16
DTEND;TZID=Asia/Jerusalem:20120815T17
STATUS:CANCELLED
CLASS:PUBLIC
X-MICROSOFT-CDO-INTENDEDSTATUS:BUSY
TRANSP:OPAQUE
LAST-MODIFIED:20130612T174006Z
DTSTAMP:20130612T174006Z
SEQUENCE:2
DESCRIPTION:The following meeting has been cancelled:\n\nSubject: ovirt netw
 ork  \nOrganizer: Livnat Peer lp...@redhat.com \n\nTime: 4:00:00 PM - 5:
 00:00 PM GMT +02:00 Jerusalem\n Recurrence : Every 5 week(s) on Wednesday No
  end date Effective Aug 15\, 2012\n\nInvitees: engine-devel@ovirt.org\; vdsm
 -de...@lists.fedorahosted.org\; gar...@de.ibm.com\; dya...@redhat.com\; simo
 n...@redhat.com\; mkole...@redhat.com\; a...@redhat.com\; ilvov...@redhat.com\;
  dfedi...@redhat.com\; pradipta.baner...@gmail.com\; gkot...@redhat.com ... 
 \n\n\n*~*~*~*~*~*~*~*~*~*\n\nHi All\, \nAs discussed previously on the list\
 , I am adding a monthly discussion on Networking in oVirt. \nIn this meeting
  we'll discuss general status of networking and features that we're missing.
  \n\nThanks\, Livnat \n\n\nBridge ID: 972506565679 \nDial-in information: \n
 Reservationless-Plus Toll Free Dial-In Number (US  Canada): (800) 451-8679 
 \nReservationless-Plus International Dial-In Number: (212) 729-5016 \nConfer
 ence code: 8425973915 \n\nGlobal Access Numbers Local: \nAustralia\, Sydney 
 Dial-In #: 0289852326 \nAustria\, Vienna Dial-In #: 012534978196 \nBelgium\,
  Brussels Dial-In #: 027920405 \nChina Dial-In #: 4006205013 \nDenmark\, Cop
 enhagen Dial-In #: 32729215 \nFinland\, Helsinki Dial-In #: 0923194436 \nFra
 nce\, Paris Dial-In #: 0170377140 \nGermany\, Berlin Dial-In #: 030300190579
  \nIreland\, Dublin Dial-In #: 014367793 \nItaly\, Milan Dial-In #: 02362695
 29 \nNetherlands\, Amsterdam Dial-In #: 0207975872 \nNorway\, Oslo Dial-In #
 : 21033188 \nSingapore Dial-In #: 64840858 \nSpain\, Barcelona Dial-In #: 93
 5452328 \nSweden\, Stockholm Dial-In #: 0850513770 \nSwitzerland\, Geneva Di
 al-In #: 0225927881 \nUnited Kingdom Dial-In #: 02078970515 \nUnited Kingdom
  Dial-In #: 08445790676 \nUnited Kingdom\, LocalCall Dial-In #: 08445790678 
 \nUnited States Dial-In #: 2127295016 \n\n\nGlobal Access Numbers Tollfree: 
 \nArgentina Dial-In #: 8004441016 \nAustralia Dial-In #: 1800337169 \nAustri
 a Dial-In #: 085898 \nBahamas Dial-In #: 18002054776 \nBahrain Dial-In #
 : 80004377 \nBelgium Dial-In #: 080048325 \nBrazil Dial-In #: 08008921002 \n
 Bulgaria Dial-In #: 008001100236 \nChile Dial-In #: 800370228 \nColombia Dia
 l-In #: 018009134033 \nCosta Rica Dial-In #: 08000131048 \nCyprus Dial-In #:
  80095297 \nCzech Republic Dial-In #: 800700318 \nDenmark Dial-In #: 8088711
 4 \nDominican Republic Dial-In #: 18887512313 \nEstonia Dial-In #: 800010023
 2 \nFinland Dial-In #: 0800117116 \nFrance Dial-In #: 0805632867 \nGermany D
 ial-In #: 8006647541 \nGreece Dial-In #: 00800127562 \nHong Kong Dial-In #: 
 800930349 \nHungary Dial-In #: 0680016796 \nIceland Dial-In #: 8008967 \nInd
 ia