Re: [ovirt-devel] [VDSM] about improved libvirt-python bindings
- Original Message -
From: Dan Kenigsberg dan...@redhat.com
To: Francesco Romani from...@redhat.com
Cc: devel@ovirt.org
Sent: Tuesday, February 24, 2015 11:19:20 AM
Subject: Re: [ovirt-devel] [VDSM] about improved libvirt-python bindings
On Wed, Feb 18, 2015 at 07:53:17AM -0500, Francesco Romani wrote:
Hi,
during my sampling overhaul effort I faced various times a mismatch between
what the libvirt-python API offers for bulk stats and what VDSM needs.
The return value of bulk stats is a something like
[(dom1, {dict_of_bulk_stats1}), {dom2, {dict_of_bulk_stats2}), ...]
While VDSM almost all the times really wants to use
{ dom_uuid_1: dict_of_bulk_stats_1, dom_uuid_2: dict_of_bulk_stats_2 }
translation is trivial to be coded, but bulk stats retrieval is quite
always in the hot path. Moreover, this is just wasteful.
How big is the waste?
Will provide hard numbers (and possibly pretty graphs)
But in general, there will always be a not-perfect match from
libvirt-python
and VDSM needs. This is expected: libvirt-python needs to be more generic.
So, I coded a Proof of Concept extension module which *complements*
and not *replaces* the libvirt-python API.
Does it require Vdsm to hold two open connections to libvirtd? [...]
Nope, it is intended as transparent replacement, much like our pthreading
does for threading standard module.
[...]
It was a nice and fun hack, but now, the question is: is that a concept
worth developing further? Do we want to use and depend on this module?
I believe that with such a buffer we can gain some agility with respect
to libvirt plans and needs, and the maintainership cost _seems_ fairly low.
Thoughts welcome.
To me, adding this module seems quite expensive: new ovirt project,
inclusion in fedora/el/debian, versioning and release-engineering
hassle.
Right, but much of this cost should be bootstrap, not recurring, right?
Not trying to push things, just to assess the cons.
To support the idea, I'd prefer to see a measureable improvement
in architecture or performence, that still alludes me.
Right, will work on that direction.
Thanks and bests,
--
Francesco Romani
RedHat Engineering Virtualization R D
Phone: 8261328
IRC: fromani
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] [VDSM] about improved libvirt-python bindings
- Original Message -
From: Francesco Romani from...@redhat.com
To: devel@ovirt.org
Sent: Wednesday, February 18, 2015 2:53:17 PM
Subject: [ovirt-devel] [VDSM] about improved libvirt-python bindings
Hi,
during my sampling overhaul effort I faced various times a mismatch between
what the libvirt-python API offers for bulk stats and what VDSM needs.
The return value of bulk stats is a something like
[(dom1, {dict_of_bulk_stats1}), {dom2, {dict_of_bulk_stats2}), ...]
While VDSM almost all the times really wants to use
{ dom_uuid_1: dict_of_bulk_stats_1, dom_uuid_2: dict_of_bulk_stats_2 }
translation is trivial to be coded, but bulk stats retrieval is quite
always in the hot path. Moreover, this is just wasteful.
But in general, there will always be a not-perfect match from libvirt-python
and VDSM needs. This is expected: libvirt-python needs to be more generic.
So, I coded a Proof of Concept extension module which *complements*
and not *replaces* the libvirt-python API.
https://github.com/mojaves/libvirt-python-ovirt
It monkeys-patch some APIs and replace them with VDSM-crafted ones.
If we have enough of monkey patching, it is even easier to just add new APIs
instead of replacing them.
It was a nice and fun hack, but now, the question is: is that a concept worth
developing further? Do we want to use and depend on this module?
I believe that with such a buffer we can gain some agility with respect
to libvirt plans and needs, and the maintainership cost _seems_ fairly low.
Thoughts welcome.
Without looking in the details, this sounds like a bad idea.
Your effort should go to improving libvirt-python bindings, which all
libvirt users can benefit from, and not creating a special bindings for
ovirt.
Nir
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] [VDSM] about improved libvirt-python bindings
On Tue, Feb 24, 2015 at 05:40:07AM -0500, Francesco Romani wrote:
- Original Message -
From: Dan Kenigsberg dan...@redhat.com
To: Francesco Romani from...@redhat.com
Cc: devel@ovirt.org
Sent: Tuesday, February 24, 2015 11:19:20 AM
Subject: Re: [ovirt-devel] [VDSM] about improved libvirt-python bindings
On Wed, Feb 18, 2015 at 07:53:17AM -0500, Francesco Romani wrote:
Hi,
during my sampling overhaul effort I faced various times a mismatch
between
what the libvirt-python API offers for bulk stats and what VDSM needs.
The return value of bulk stats is a something like
[(dom1, {dict_of_bulk_stats1}), {dom2, {dict_of_bulk_stats2}), ...]
While VDSM almost all the times really wants to use
{ dom_uuid_1: dict_of_bulk_stats_1, dom_uuid_2: dict_of_bulk_stats_2 }
translation is trivial to be coded, but bulk stats retrieval is quite
always in the hot path. Moreover, this is just wasteful.
How big is the waste?
Will provide hard numbers (and possibly pretty graphs)
But in general, there will always be a not-perfect match from
libvirt-python
and VDSM needs. This is expected: libvirt-python needs to be more generic.
So, I coded a Proof of Concept extension module which *complements*
and not *replaces* the libvirt-python API.
Does it require Vdsm to hold two open connections to libvirtd? [...]
Nope, it is intended as transparent replacement, much like our pthreading
does for threading standard module.
ah, ok. please read again your I coded a Proof...API sentence to see
why I was confused ;-)
[...]
It was a nice and fun hack, but now, the question is: is that a concept
worth developing further? Do we want to use and depend on this module?
I believe that with such a buffer we can gain some agility with respect
to libvirt plans and needs, and the maintainership cost _seems_ fairly
low.
Thoughts welcome.
To me, adding this module seems quite expensive: new ovirt project,
inclusion in fedora/el/debian, versioning and release-engineering
hassle.
Right, but much of this cost should be bootstrap, not recurring, right?
Not trying to push things, just to assess the cons.
Well, maintaining another package per distribution has some recurrent
costs. It has to be rebuilt on each ovirt release (assuming no urgent
security bugs), and there might be unexpected needs to recompile (python
change, libvirt upgrade).
Regards,
Dan.
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] [VDSM] about improved libvirt-python bindings
- Original Message - From: Nir Soffer nsof...@redhat.com To: Francesco Romani from...@redhat.com Cc: devel@ovirt.org Sent: Tuesday, February 24, 2015 3:24:30 PM Subject: Re: [ovirt-devel] [VDSM] about improved libvirt-python bindings I believe that with such a buffer we can gain some agility with respect to libvirt plans and needs, and the maintainership cost _seems_ fairly low. Thoughts welcome. Without looking in the details, this sounds like a bad idea. Your effort should go to improving libvirt-python bindings, which all libvirt users can benefit from, and not creating a special bindings for ovirt. Well, my problem was just that the API was different enough to make me think that it was unsuitable for libvirt upstream. But worth trying to upstream first, definitely. -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] [VDSM] about improved libvirt-python bindings
Hi, - Original Message - From: Sven Kieske s.kie...@mittwald.de To: devel@ovirt.org Sent: Wednesday, February 18, 2015 2:12:15 PM Subject: Re: [ovirt-devel] [VDSM] about improved libvirt-python bindings Hi, as far as I understand this would make it easier to implement/use upstreams(libvirt) existing python-bindings in ovirt? in this case, I think this is a huge improvement. My point here is libvirt and oVirt have slightly misaligned path. This is absolutely expected and OK, since libvirt should and can not only serve the purposes of oVirt, but a much broader scope! Nowadays we (oVirt, VDSM in particular) just consume the libvirt python API and we fix the mismatches in our own application code. Most often is a no-brainer, sometimes is annoying. Especially also given the recent efforts to improve and enhance the scalability of VDSM, we may need (today or tomorrow) a different approach. The one I'm offering is *a* potential solution (IMVHO pretty fitting), hence my discussion here :) -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Re: [ovirt-devel] [VDSM] about improved libvirt-python bindings
Hi, as far as I understand this would make it easier to implement/use upstreams(libvirt) existing python-bindings in ovirt? in this case, I think this is a huge improvement. -- Mit freundlichen Grüßen / Regards Sven Kieske Systemadministrator Mittwald CM Service GmbH Co. KG Königsberger Straße 6 32339 Espelkamp T: +49-5772-293-100 F: +49-5772-293-333 https://www.mittwald.de Geschäftsführer: Robert Meyer St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen ___ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
