date:20160217

Re: [ovirt-devel] [VDSM] Profiling results - 144 cores, 144 vms

2016-02-17 Thread Francesco Romani

- Original Message -
> From: "Nir Soffer" 
> To: "devel" , "Francesco Romani" , 
> "Eldad Marciano" ,
> "Vinzenz Feenstra" , "Dan Kenigsberg" 
> , "Piotr Kliczewski"
> 
> Sent: Thursday, February 18, 2016 12:38:44 AM
> Subject: [VDSM] Profiling results - 144 cores, 144 vms
> 
> Hi all,
> 
> I noticed that Eldad posted profiling result from 144 cores / 144 vms run:
> See https://bugzilla.redhat.com/attachment.cgi?id=1128028
> 
> Looking at the profile bellow:
> 
> 2. Creating DomainDescriptor seems expensive - is this xml.minidom?

Let's look at DomainDescriptor's __init__


class DomainDescriptor(object):

def __init__(self, xmlStr):
self._xml = xmlStr
self._dom = xml.dom.minidom.parseString(xmlStr)
self._devices = self._first_element_by_tag_name('devices')
self._devices_hash = hash(self._devices.toxml()
  if self._devices is not None else '')

so we parse using the slow minidom _and_ we do toxml() on a subset of the xml 
tree.
Both operations are expensive (and quite memory expensive).

Milan is already working on moving from minidom to etree, I think we are on 
good track
here for 4.0.


-- 
Francesco Romani
RedHat Engineering Virtualization R & D
Phone: 8261328
IRC: fromani
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

[ovirt-devel] [VDSM] Profiling results - 144 cores, 144 vms

2016-02-17 Thread Nir Soffer

Hi all,

I noticed that Eldad posted profiling result from 144 cores / 144 vms run:
See https://bugzilla.redhat.com/attachment.cgi?id=1128028

Looking at the profile bellow:

1. Why _rtnl_link_alloc_cache is so expensive?
called only 389 times, but the most expensive call

2. Creating DomainDescriptor seems expensive - is this xml.minidom?

3. guestagent.py:84(filt) is very expensive - we probbaly want to backport
7091bde guestagent: Speed up xml character filtering, can shave about
17% of cpu time in this run.

4. pthread.py function will disapper from the profile if we replace
pthreading with cthreading:
https://github.com/nirs/cthreading

5. Why do we see xmlrpc at the top of the profile? Is this the cost
of mom calls?

Eldad, can you repeat the same test with current master?

Nir



Thu Feb 18 00:50:24 2016vdsm_144vms_1800sec_2.prof

 29858105 function calls (34034631 primitive calls) in 1049.574 seconds

   Ordered by: internal time
   List reduced from 2420 to 20 due to restriction <20>

  ncalls  tottime  percall  cumtime  percall filename:lineno(function)
 389   64.9370.167   64.9410.167
link.py:133(_rtnl_link_alloc_cache)
   598438   64.1760.000   87.8880.000
domain_descriptor.py:23(DomainDescriptor)
 1535528   38.9220.001  176.1770.000 guestagent.py:84(filt)
 57106   31.5450.001   31.5600.001
decoder.py:371(JSONDecoder.raw_decode)
   168463   25.9610.234  119.0040.001
xmlrpclib.py:721(Marshaller.dump_struct)
 4312452   25.0950.000   77.0120.000 utf_8.py:15(decode)
 7341795   21.9340.000   41.5070.000 string.py:511(replace)
 28881   18.0920.001   19.9210.001 pthread.py:133(Cond.timedwait)
   386282   17.5090.000   17.5090.000 pthread.py:95(Lock.lock)
 46277   16.7980.000   45.2800.001
guestagent.py:254(GuestAgent._handleMessage)
 48090   16.2980.000   37.8090.001
expatbuilder.py:743(ExpatBuilderNS.start_element_handler)
 2447266   14.1280.000   55.6350.000 httplib.py:331(HTTPResponse)
   339514   13.6890.014   20.6290.000 password.py:69(_walk)
 78549   12.4500.000   12.6350.000 minidom.py:344(Attr.__init__)
 20117   12.3810.001   20.9610.001 vm.py:181(SetLinkAndNetworkError)
 14470   12.2240.001   12.8020.001 xmlbuilder.py:1()
 21404   11.5270.001   36.2250.002 vmstats.py:240(disks)
 87612   11.4410.000   35.7010.000
sampling.py:59(InterfaceSample.readIfaceStat)
 39924   11.3230.000   23.0470.001
__init__.py:242(LogRecord.__init__)
 14509   10.8160.001   96.2360.007 xmlbuilder.py:12(Options)
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] IPv6 support in oVirt

2016-02-17 Thread Piotr Kliczewski

On Wed, Feb 17, 2016 at 5:47 PM, Yevgeny Zaspitsky  wrote:
> Piotr,
>
> According to our checks engine succeed connecting VDSM through ipv6 once the
> work you've reminded has been done. So it looks like jsonrpc is agnostic to
> IP layer it uses underneath.
>

I never tested it with ipv6 but looking at java internals that I used
and it accepts host as
specified in RFC 2732 or RFC 2373.
It looks that it should work out of the box.

> Could you please elaborate on why do you think that we break backward
> compatibility.

I suggested to add Address type which would hold both address types so we
could use it in many places. This change would break backward compatibility.

> If you compare the wiki to vdsmapi-schema.json file, looks like the last
> isn't updated for some time and needs to be updated according to the actual
> code.
>

Type extension would not break backward compatibility but it would introduce
bunch of optional parameters. I think we could do better.

> We add dhcpv6 in order to allow user to configure v6 stack independently
> from v4 one. That could be useful especially in a transition period, when a
> user have fully configured ipv4 and would like to try using ipv6 and he
> hasn't configured his DHCP server for v6 yet.
>

I would add that to new address type. What do you think?

> Regards,
> Yevgeny
>
> On Wed, Feb 17, 2016 at 10:26 AM, Piotr Kliczewski
>  wrote:
>>
>> Thank you for the effort.
>>
>> I have 2 comments:
>> - In [1] I do not see any information about using ipv6 in jsonrpc
>> code. I know that there was some work done on vdsm [2]
>>   side but nothing on the engine side.
>>
>> - We are about to break some backward compatibility so I suggest to
>> define new type which would hold ipv4 or ipv6
>>   information (vdsm api section) or if we want to keep the backward
>> compatibility we should reuse existing parameters
>>   and prepare the code to handle both formats.
>>   Why we are adding @dhcpv6? We do not have this setting for ipv4.
>>
>> Thanks,
>> Piotr
>>
>> [2] https://gerrit.ovirt.org/51319
>>
>>
>> On Tue, Feb 16, 2016 at 5:02 PM, Yevgeny Zaspitsky 
>> wrote:
>> > Hi All,
>> >
>> > IPv6 support is a long waited feature in oVirt and that was decided to
>> > include it in v4.0.
>> > I have updated the feature page [1] with the recent projected changes.
>> >
>> > Please take a look on the page and please send me your comments so those
>> > would be reflected in the feature implementation.
>> >
>> > Regards,
>> > Yevgeny
>> >
>> > [1] http://www.ovirt.org/Features/IPv6_support
>> >
>> > ___
>> > Devel mailing list
>> > Devel@ovirt.org
>> > http://lists.ovirt.org/mailman/listinfo/devel
>
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] cpu profile permission error while editing/creating vm

2016-02-17 Thread Tomer Saban

Hi everyone,

I've just uploaded a patch that fixes this problem.
You no longer suppose to experience any difficulty creating/editing VMs.

Sorry for the temporary inconvenience. 

FYI, 
Tomer 

- Original Message -
From: "Jakub Niedermertl" 
To: "Greg Sheremeta" 
Cc: "Tomer Saban" , "devel" 
Sent: Thursday, February 4, 2016 4:58:58 PM
Subject: Re: [ovirt-devel] cpu profile permission error while   
editing/creating vm

https://bugzilla.redhat.com/show_bug.cgi?id=1297920

- Original Message -
> From: "Greg Sheremeta" 
> To: "Tomer Saban" 
> Cc: "devel" 
> Sent: Tuesday, February 2, 2016 6:53:49 PM
> Subject: Re: [ovirt-devel] cpu profile permission error while 
> editing/creating vm
> 
> Do you have an open BZ on this?
> 
> If not, there should be one :) Lmk and I will open it.
> 
> Greg
> 
> 
> On Wed, Jan 13, 2016 at 8:55 AM, Tomer Saban  wrote:
> > Hi,
> >
> > If you get an error message like the following while editing/creating VM:
> > """
> > User doesn't have permissions to assign the cpu profile ASFDS with id
> > 3c990c63-8078-4916-bf18-f38596232a5a to VMs.
> > """
> >
> > Go to Clusters->choose your cluster-> cpu profiles in the subtab-> choose
> > the any cpu profile-> click add in the right frame and add permission for
> > your user with role "CpuProfileCreator".
> >
> > Thanks,
> > Tomer
> >
> > ___
> > Devel mailing list
> > Devel@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/devel
> 
> 
> 
> --
> Greg Sheremeta, MBA
> Red Hat, Inc.
> Sr. Software Engineer
> gsher...@redhat.com
> 919-741-4016
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] IPv6 support in oVirt

2016-02-17 Thread Yevgeny Zaspitsky

Piotr,

According to our checks engine succeed connecting VDSM through ipv6 once
the work you've reminded has been done. So it looks like jsonrpc is
agnostic to IP layer it uses underneath.

Could you please elaborate on why do you think that we break backward
compatibility.
If you compare the wiki to vdsmapi-schema.json file, looks like the last
isn't updated for some time and needs to be updated according to the actual
code.

We add dhcpv6 in order to allow user to configure v6 stack independently
from v4 one. That could be useful especially in a transition period, when a
user have fully configured ipv4 and would like to try using ipv6 and he
hasn't configured his DHCP server for v6 yet.

Regards,
Yevgeny

On Wed, Feb 17, 2016 at 10:26 AM, Piotr Kliczewski <
piotr.kliczew...@gmail.com> wrote:

> Thank you for the effort.
>
> I have 2 comments:
> - In [1] I do not see any information about using ipv6 in jsonrpc
> code. I know that there was some work done on vdsm [2]
>   side but nothing on the engine side.
>
> - We are about to break some backward compatibility so I suggest to
> define new type which would hold ipv4 or ipv6
>   information (vdsm api section) or if we want to keep the backward
> compatibility we should reuse existing parameters
>   and prepare the code to handle both formats.
>   Why we are adding @dhcpv6? We do not have this setting for ipv4.
>
> Thanks,
> Piotr
>
> [2] https://gerrit.ovirt.org/51319
>
>
> On Tue, Feb 16, 2016 at 5:02 PM, Yevgeny Zaspitsky 
> wrote:
> > Hi All,
> >
> > IPv6 support is a long waited feature in oVirt and that was decided to
> > include it in v4.0.
> > I have updated the feature page [1] with the recent projected changes.
> >
> > Please take a look on the page and please send me your comments so those
> > would be reflected in the feature implementation.
> >
> > Regards,
> > Yevgeny
> >
> > [1] http://www.ovirt.org/Features/IPv6_support
> >
> > ___
> > Devel mailing list
> > Devel@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/devel
>
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] IPv6 support in oVirt

2016-02-17 Thread Dan Kenigsberg

On Wed, Feb 17, 2016 at 08:59:20AM -0500, Martin Perina wrote:
> I think we have created a bug [1] for fencing related ipv6 support.
> 
> Martin
> 
> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1194055
> 
> - Original Message -
> > From: "Eli Mesika" 
> > To: "Yevgeny Zaspitsky" 
> > Cc: "devel" , "Michael Burman" , "Oved 
> > Ourfali" , "Martin
> > Perina" 
> > Sent: Wednesday, February 17, 2016 2:57:08 PM
> > Subject: Re: [ovirt-devel] IPv6 support in oVirt
> > 
> > This will require a change also in the fencing arguments
> > We will have to add inet4_only  inet6_only support from the UI/API as a
> > common field (like port/slot/secure)
> > 
> > See
> > https://fedorahosted.org/cluster/wiki/FenceArguments
> > 
> > Please add that as well to your doc.

The bug above was open in order to allow typing an ipv6 address for
power management.

I've added Eli's suggestion to comment 2 of that bug.

Thanks.
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] IPv6 support in oVirt

2016-02-17 Thread Martin Perina

I think we have created a bug [1] for fencing related ipv6 support.

Martin

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1194055

- Original Message -
> From: "Eli Mesika" 
> To: "Yevgeny Zaspitsky" 
> Cc: "devel" , "Michael Burman" , "Oved 
> Ourfali" , "Martin
> Perina" 
> Sent: Wednesday, February 17, 2016 2:57:08 PM
> Subject: Re: [ovirt-devel] IPv6 support in oVirt
> 
> This will require a change also in the fencing arguments
> We will have to add inet4_only  inet6_only support from the UI/API as a
> common field (like port/slot/secure)
> 
> See
> https://fedorahosted.org/cluster/wiki/FenceArguments
> 
> Please add that as well to your doc.
> 
> 
> - Original Message -
> > From: "Yevgeny Zaspitsky" 
> > To: "devel" 
> > Cc: "Michael Burman" 
> > Sent: Tuesday, February 16, 2016 6:02:03 PM
> > Subject: [ovirt-devel] IPv6 support in oVirt
> > 
> > Hi All,
> > 
> > IPv6 support is a long waited feature in oVirt and that was decided to
> > include it in v4.0.
> > I have updated the feature page [1] with the recent projected changes.
> > 
> > Please take a look on the page and please send me your comments so those
> > would be reflected in the feature implementation.
> > 
> > Regards,
> > Yevgeny
> > 
> > [1] http://www.ovirt.org/Features/IPv6_support
> > 
> > ___
> > Devel mailing list
> > Devel@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/devel
> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] IPv6 support in oVirt

2016-02-17 Thread Eli Mesika

This will require a change also in the fencing arguments 
We will have to add inet4_only  inet6_only support from the UI/API as a common 
field (like port/slot/secure)

See 
https://fedorahosted.org/cluster/wiki/FenceArguments

Please add that as well to your doc.


- Original Message -
> From: "Yevgeny Zaspitsky" 
> To: "devel" 
> Cc: "Michael Burman" 
> Sent: Tuesday, February 16, 2016 6:02:03 PM
> Subject: [ovirt-devel] IPv6 support in oVirt
> 
> Hi All,
> 
> IPv6 support is a long waited feature in oVirt and that was decided to
> include it in v4.0.
> I have updated the feature page [1] with the recent projected changes.
> 
> Please take a look on the page and please send me your comments so those
> would be reflected in the feature implementation.
> 
> Regards,
> Yevgeny
> 
> [1] http://www.ovirt.org/Features/IPv6_support
> 
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [vdsm][virt] what's new in virt (20160216)

2016-02-17 Thread Sven Kieske

On 16/02/16 15:35, Francesco Romani wrote:
> * Francesco: the `convirt' module mostly (support containers with
> libvirt API) plus cleanup

Hi,

I assume this is a self written module for ovirt/libvirt to manage
containers? Is convirt the real name of it?

Because there seems to be another open source software "convirt" for
kvm/xen management?

https://www.convirture.com/

https://www.convirture.com/products_opensource.php

maybe rename it, to avoid trademark conflicts?



-- 
Mit freundlichen Grüßen / Regards

Sven Kieske

Systemadministrator
Mittwald CM Service GmbH & Co. KG
Königsberger Straße 6
32339 Espelkamp
T: +495772 293100
F: +495772 29
https://www.mittwald.de
Geschäftsführer: Robert Meyer
St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen
Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad
Oeynhausen



signature.asc
Description: OpenPGP digital signature
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] IPv6 support in oVirt

2016-02-17 Thread Piotr Kliczewski

Thank you for the effort.

I have 2 comments:
- In [1] I do not see any information about using ipv6 in jsonrpc
code. I know that there was some work done on vdsm [2]
  side but nothing on the engine side.

- We are about to break some backward compatibility so I suggest to
define new type which would hold ipv4 or ipv6
  information (vdsm api section) or if we want to keep the backward
compatibility we should reuse existing parameters
  and prepare the code to handle both formats.
  Why we are adding @dhcpv6? We do not have this setting for ipv4.

Thanks,
Piotr

[2] https://gerrit.ovirt.org/51319


On Tue, Feb 16, 2016 at 5:02 PM, Yevgeny Zaspitsky  wrote:
> Hi All,
>
> IPv6 support is a long waited feature in oVirt and that was decided to
> include it in v4.0.
> I have updated the feature page [1] with the recent projected changes.
>
> Please take a look on the page and please send me your comments so those
> would be reflected in the feature implementation.
>
> Regards,
> Yevgeny
>
> [1] http://www.ovirt.org/Features/IPv6_support
>
> ___
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] [VDSM] Profiling results - 144 cores, 144 vms

[ovirt-devel] [VDSM] Profiling results - 144 cores, 144 vms

Re: [ovirt-devel] IPv6 support in oVirt

Re: [ovirt-devel] cpu profile permission error while editing/creating vm

Re: [ovirt-devel] IPv6 support in oVirt

Re: [ovirt-devel] IPv6 support in oVirt

Re: [ovirt-devel] IPv6 support in oVirt

Re: [ovirt-devel] IPv6 support in oVirt

Re: [ovirt-devel] [vdsm][virt] what's new in virt (20160216)

Re: [ovirt-devel] IPv6 support in oVirt

10 matches

Site Navigation

Mail list logo

Footer information