Re: [Engine-devel] Exit message: unsupported configuration: spice graphics are not supported with this QEMU.

[Dan Kenigsberg]

On Tue, Apr 01, 2014 at 04:21:24AM -0400, marco lin wrote:
> Exit message: unsupported configuration: spice graphics are not supported
> with this QEMU.

Could you provide more information about your platform, the version of
qemu-kvm, libvirt and spice-server?

The relevant chunk of /var/log/libvirt/libvirtd.log might prove useful.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] hosted engine in 3.4

[Dan Kenigsberg]

On Fri, Mar 14, 2014 at 10:46:00AM +0100, Sandro Bonazzola wrote:
> Il 14/03/2014 10:39, Jiri Moskovcak ha scritto:
> > On 03/14/2014 10:13 AM, Martin Sivak wrote:
> >> Hi,
> >>
> > 1. You need to have some MAC address with static ip and FQDN, otherwise
> > you have to change /etc/hosts at least for the first part of the setup
> >>
> >> I believe you were there when we were discussing this with pstehlik :)
> >>
> >> Btw in this case you have to change /etc/hosts on all hosts and inside the
> >> engine VM.
> >>
> > 2. When the VM install is complete I would expect the setup wizard to
> > install the engine to the VM automatically - which at least in my case -
> > doesn't happen
> >>
> >> I also proposed kickstart based setup to Sandro. You would give a repo to 
> >> setup
> >> and it would download the kernel/initrd from it [1] and create a kickstart 
> >> with
> >> the right root password, engine setup and so on..
> >>
> >> [1] 
> >> http://ftp.linux.cz/pub/linux/fedora/linux/releases/20/Fedora/x86_64/os/images/pxeboot/
> >>
> > 1. once I managed to install the engine to the vm it tried to add the
> > host it was running on to the engine and it failed with a message "Host
> > compatibility version doesn't match the cluster compatibility version",
> > and then it marked the host as non operational which killed the vm with
> > the engine, so the engine actually committed suicide…
> >>
> >> Check your VDSM version. Especially the content of 
> >> /usr/share/vdsm/dsaversion.py
> >> file. There are couple of lists specifying the cluster and engine versions 
> >> that
> >> the VDSM supports.
> >>
> >> If you use 3.4 engine it needs 3.4 VDSM as well.. and the prerelease repo 
> >> is not
> >> enabled by default.
> >>
> > 
> > vdsm version on the host:
> > vdsm-4.14.5-0.fc19.x86_64 @ovirt-3.4.0-prerelease
> > 
> > ovirt-engine installed in the VM:
> > ovirt-engine-3.4.0-0.13.rc.fc19.noarch
> > 
> > - and it dies with the error message: "Host hosted_engine_1 is compatible 
> > with versions (3.0,3.1,3.2,3.3) and cannot join Cluster Default which is set
> > to version 3.4"
> 
> Above is probably due to libvirt not updated to latest from 
> fedora-virt-preview repo.
> Dan, Yaniv, any reason for not explicitly requiring the updated version of 
> libvirt in VDSM package?
> If it's just because libvirt is not yet in official Fedora repository, maybe 
> we should ship it on ovirt repository until it's in and put some pressure
> on libvirt people for having them pushing the newest package into Fedora.
> Thoughts?

The problem is that F19's libvirt lacks VIR_MIGRATE_ABORT_ON_ERROR
(http://gerrit.ovirt.org/#/c/23273/) which is required by clusterLevel
3.4.

We found it quite rude for a vdsm upgrade in Fedora 19 to require a
version of libvirt that is not on that platform, particularly since the
existing version is perfectly usable for clusterLevel 3.3.

I do not believe we should ship libvirt within oVirt. It is a pain to
mainatain (and issue bugfixes, and worry about security hotfixes). A
prefereable approach would be to have ovirt-release.f19.rpm link to the
relevant virt-preview repository.



___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Asynchronous tasks for live merge

[Dan Kenigsberg]

On Mon, Mar 03, 2014 at 09:51:15AM -0500, Adam Litke wrote:
> On 03/03/14 14:28 +0000, Dan Kenigsberg wrote:
> >On Fri, Feb 28, 2014 at 09:30:16AM -0500, Adam Litke wrote:
> >>Hi all,
> >>
> >>As part of our plan to support live merging of VM disk snapshots it
> >>seems we will need a new form of asynchronous task in ovirt-engine.  I
> >>am aware of AsyncTaskManager but it seems to be limited to managing
> >>SPM tasks.  For live merge, we are going to need something called
> >>VmTasks since the async command can be run only on the host that
> >>currently runs the VM.
> >>
> >>The way I see this working from an engine perspective is:
> >>1. RemoveSnapshotCommand in bll is invoked as usual but since the VM is
> >>  found to be up, we activate an alternative live merge flow.
> >>2. We submit a LiveMerge VDS Command for each impacted disk.  This is
> >>  an asynchronous command which we need to monitor for completion.
> >>3. A VmJob is inserted into the DB so we'll remember to handle it.
> >>4. The VDS Broker monitors the operation via an extension to the
> >>  already collected VmStatistics data.  Vdsm will report active Block
> >>  Jobs only.  Once the job stops (in error or success) it will cease
> >>  to be reported by vdsm and engine will know to proceed.
> >
> >You describe a reasonable way for Vdsm to report whether an async
> >operation has finished. However, may we instead use the oportunity to
> >introduce generic "hsm" tasks?
> 
> Sure, I am happy to have that conversation :)  If I understand
> correctly, HSM tasks, while ideal, might be too complex to get right
> and would block the Live Merge feature for longer than we would like.
> Has anyone looked into what it would take to implement a HSM Tasks
> framework like this in vdsm?  Are there any WIP implementations?  If
> the scope of this is not too big, it can be completed relatively
> quickly, and the resulting implementation would cover all known use
> cases, then this could be worth it.  It's important to support Live
> Merge soon.
> 
> Regarding deprecation of the current tasks API:  Could your suggested
> HSM Tasks framework be extended to cover SPM/SDM tasks as well?  I
> would hope that a it could.  In that case, we could look forward to a
> unified async task architecture in vdsm.

The current task framework in Vdsm is outrageously complex, yet
unreliable. It meant to do all kinds of things, like having the new spm
take over a task that was orphaned by the former spm. This has never
worked properly.

I'm looking for a much simpler infrastructure, where rollback is done by
virtue of having an "except" clause, and spm-only verbs simply fail when
the host loses spm status for some reason.

> 
> >I suggest to have something loosely modeled on posix fork/wait.
> >
> >- Engine asks Vdsm to start an API verb asynchronously and supplies a
> > uuid. This is unlike fork(2), where the system chooses the pid, but
> > that's required so that Engine could tell if the command has reached
> > Vdsm in case of a network error.
> >
> >- Engine may monitor the task (a-la wait(WNOHANG))
> 
> Allon has communicated a desire to limit engine-side polling.  Perhaps
> the active tasks could be added to the host stats?

Engine is reluctant to add more polling, I understand that. I'd prefer a
standalone new getAllTaskStats2() verb, but if lumping it into
getVdsStats is going to convince everybody to have it, I'd put my
aesthetic taste in the fridge.

> >- When the task is finished, Engine may collect its result (a-la wait).
> > Until that happens, Vdsm must report the task forever; restart or
> > upgrade are no excuses. On reboot, though, all tasks are forgotten, so
> > Engine may stop monitoring tasks on a fenced host.
> 
> This could be a good comprimise.  I hate the idea of requiring engine
> to play janitor and clean up stale vdsm data, but there is not much
> better of a way to do it.  Allowing reboot to auto-clear tasks will at
> least provide some backstop to how long tasks could pile up if
> forgotten.
> 
> >This may be an over kill for your use case, but it would come useful for
> >other cases. In particular, setupNetwork returns before it is completely
> >done, since dhcp address acquisition may take too much time. Engine may
> >poll getVdsCaps to see when it's done (or timeout), but it would be
> >nicer to have a generic mechanism that can serve us all.
> 
> If we were to consider this, I would want to vet the architecture
> against all known use cases for tasks to make sure we don't need to
> create a ne

Re: [Engine-devel] Asynchronous tasks for live merge

[Dan Kenigsberg]

On Mon, Mar 03, 2014 at 09:56:56AM -0500, Adam Litke wrote:
> On 03/03/14 16:36 +0200, Itamar Heim wrote:
> >On 03/03/2014 04:28 PM, Dan Kenigsberg wrote:
> >>On Fri, Feb 28, 2014 at 09:30:16AM -0500, Adam Litke wrote:
> >>>Hi all,
> >>>
> >>>As part of our plan to support live merging of VM disk snapshots it
> >>>seems we will need a new form of asynchronous task in ovirt-engine.  I
> >>>am aware of AsyncTaskManager but it seems to be limited to managing
> >>>SPM tasks.  For live merge, we are going to need something called
> >>>VmTasks since the async command can be run only on the host that
> >>>currently runs the VM.
> >>>
> >>>The way I see this working from an engine perspective is:
> >>>1. RemoveSnapshotCommand in bll is invoked as usual but since the VM is
> >>>  found to be up, we activate an alternative live merge flow.
> >>>2. We submit a LiveMerge VDS Command for each impacted disk.  This is
> >>>  an asynchronous command which we need to monitor for completion.
> >>>3. A VmJob is inserted into the DB so we'll remember to handle it.
> >>>4. The VDS Broker monitors the operation via an extension to the
> >>>  already collected VmStatistics data.  Vdsm will report active Block
> >>>  Jobs only.  Once the job stops (in error or success) it will cease
> >>>  to be reported by vdsm and engine will know to proceed.
> >>
> >>You describe a reasonable way for Vdsm to report whether an async
> >>operation has finished. However, may we instead use the oportunity to
> >>introduce generic "hsm" tasks?
> >>
> >>I suggest to have something loosely modeled on posix fork/wait.
> >>
> >>- Engine asks Vdsm to start an API verb asynchronously and supplies a
> >>  uuid. This is unlike fork(2), where the system chooses the pid, but
> >>  that's required so that Engine could tell if the command has reached
> >>  Vdsm in case of a network error.
> >>
> >>- Engine may monitor the task (a-la wait(WNOHANG))
> >>
> >>- When the task is finished, Engine may collect its result (a-la wait).
> >>  Until that happens, Vdsm must report the task forever; restart or
> >>  upgrade are no excuses. On reboot, though, all tasks are forgotten, so
> >>  Engine may stop monitoring tasks on a fenced host.
> >>
> >>This may be an over kill for your use case, but it would come useful for
> >>other cases. In particular, setupNetwork returns before it is completely
> >>done, since dhcp address acquisition may take too much time. Engine may
> >>poll getVdsCaps to see when it's done (or timeout), but it would be
> >>nicer to have a generic mechanism that can serve us all.
> >>
> >>Note that I'm suggesting a completely new task framwork, at least on
> >>Vdsm side, as the current one (with its broken persistence, arcane
> >>states and never-reliable rollback) is beyond redemption, imho.
> >>
> >>>5. When the job has completed, VDS Broker raises an event up to bll.
> >>>  Maybe this could be done via VmJobDAO on the stored VmJob?
> >>>6. Bll receives the event and issues a series of VDS commands to
> >>>  complete the operation:
> >>>  a) Verify the new image chain matches our expectations (the snap is
> >>> no longer present in the chain).
> >>>  b) Delete the snapshot volume
> >>>  c) Remove the VmJob from the DB
> >>>
> >>>Could you guys review this proposed flow for sanity?  The main
> >>>conceptual gaps I am left with concern #5 and #6.  What is the
> >>>appropriate way for VDSBroker to communicate with BLL?  Is there an
> >>>event mechanism I can explore or should I use the database?  I am
> >>>leaning toward the database because it is persistent and will ensure
> >>>#6 gets completed even if engine is restarted somewhere in the middle.
> >>>For #6, is there an existing polling / event loop in bll that I can
> >>>plug into?
> >>>
> >>>Thanks in advance for taking the time to think about this flow and for
> >>>providing your insights!
> >>___
> >>Engine-devel mailing list
> >>Engine-devel@ovirt.org
> >>http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>
> >
> >the way i read Adam's proposal, there is no "task" entity at vdsm
> >side to monito

Re: [Engine-devel] Asynchronous tasks for live merge

[Dan Kenigsberg]

On Fri, Feb 28, 2014 at 09:30:16AM -0500, Adam Litke wrote:
> Hi all,
> 
> As part of our plan to support live merging of VM disk snapshots it
> seems we will need a new form of asynchronous task in ovirt-engine.  I
> am aware of AsyncTaskManager but it seems to be limited to managing
> SPM tasks.  For live merge, we are going to need something called
> VmTasks since the async command can be run only on the host that
> currently runs the VM.
> 
> The way I see this working from an engine perspective is:
> 1. RemoveSnapshotCommand in bll is invoked as usual but since the VM is
>   found to be up, we activate an alternative live merge flow.
> 2. We submit a LiveMerge VDS Command for each impacted disk.  This is
>   an asynchronous command which we need to monitor for completion.
> 3. A VmJob is inserted into the DB so we'll remember to handle it.
> 4. The VDS Broker monitors the operation via an extension to the
>   already collected VmStatistics data.  Vdsm will report active Block
>   Jobs only.  Once the job stops (in error or success) it will cease
>   to be reported by vdsm and engine will know to proceed.

You describe a reasonable way for Vdsm to report whether an async
operation has finished. However, may we instead use the oportunity to
introduce generic "hsm" tasks?

I suggest to have something loosely modeled on posix fork/wait.

- Engine asks Vdsm to start an API verb asynchronously and supplies a
  uuid. This is unlike fork(2), where the system chooses the pid, but
  that's required so that Engine could tell if the command has reached
  Vdsm in case of a network error.

- Engine may monitor the task (a-la wait(WNOHANG))

- When the task is finished, Engine may collect its result (a-la wait).
  Until that happens, Vdsm must report the task forever; restart or
  upgrade are no excuses. On reboot, though, all tasks are forgotten, so
  Engine may stop monitoring tasks on a fenced host.

This may be an over kill for your use case, but it would come useful for
other cases. In particular, setupNetwork returns before it is completely
done, since dhcp address acquisition may take too much time. Engine may
poll getVdsCaps to see when it's done (or timeout), but it would be
nicer to have a generic mechanism that can serve us all.

Note that I'm suggesting a completely new task framwork, at least on
Vdsm side, as the current one (with its broken persistence, arcane
states and never-reliable rollback) is beyond redemption, imho.

> 5. When the job has completed, VDS Broker raises an event up to bll.
>   Maybe this could be done via VmJobDAO on the stored VmJob?
> 6. Bll receives the event and issues a series of VDS commands to
>   complete the operation:
>   a) Verify the new image chain matches our expectations (the snap is
>  no longer present in the chain).
>   b) Delete the snapshot volume
>   c) Remove the VmJob from the DB
> 
> Could you guys review this proposed flow for sanity?  The main
> conceptual gaps I am left with concern #5 and #6.  What is the
> appropriate way for VDSBroker to communicate with BLL?  Is there an
> event mechanism I can explore or should I use the database?  I am
> leaning toward the database because it is persistent and will ensure
> #6 gets completed even if engine is restarted somewhere in the middle.
> For #6, is there an existing polling / event loop in bll that I can
> plug into?
> 
> Thanks in advance for taking the time to think about this flow and for
> providing your insights!
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [ovirt-testday 2] Fake PPC Support - Feedback

[Dan Kenigsberg]

On Tue, Feb 11, 2014 at 04:16:27PM +, Vitor de Lima wrote:
> Hi Vinzenz,
> 
> This bug is fixed in more recent QEMU versions (1.6.x, for example). I have 
> opened a bug for this issue:
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=1063799

Thanks! Would you consider solving it, by adding a ppc-specific
requirement of qemu >= 1.6?

What do you do in el6, where such a version is not provided by the
platform?
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] mom RPMs for 3.4

[Dan Kenigsberg]

On Fri, Jan 31, 2014 at 04:56:12PM -0500, Adam Litke wrote:
> On 31/01/14 08:36 +0100, Sandro Bonazzola wrote:
> >Il 30/01/2014 19:30, Adam Litke ha scritto:
> >>On 30/01/14 18:13 +0000, Dan Kenigsberg wrote:
> >>>On Thu, Jan 30, 2014 at 11:49:42AM -0500, Adam Litke wrote:
> >>>>Hi Sandro,
> >>>>
> >>>>After updating the MOM project's build system, I have used jenkins to
> >>>>produce a set of RPMs that I would like to tag into the oVirt 3.4
> >>>>release.  Please see the jenkins job [1] for the relevant artifacts
> >>>>for EL6[2], F19[3], and F20[4].
> >>>>
> >>>>Dan, should I submit a patch to vdsm to make it require mom >= 0.4.0?
> >>>>I want to be careful to not break people's environments this late in
> >>>>the 3.4 release cycle.  What is the best way to minimize that damage?
> >>>
> >>>Hey, we're during beta. I prefer making this requirement explicit now
> >>>over having users with supervdsmd.log retate due to log spam.
> >>
> >>In that case, Sandro, can you let me know when those RPMs hit the
> >>ovirt repos (for master and 3.4) and then I will submit a patch to
> >>vdsm to require the new version.
> >
> >
> >mom 0.4.0 has been built in last night nightly job [1] and published to 
> >nightly by publisher job [2]
> >so it's already available on nightly [3]
> >
> >For 3.4.0, it has been planned [4] a beta 2 release on 2014-02-06 so we'll 
> >include your builds in that release.
> 
> I presume the scripting for 3.4 release rpms will produce a version
> without the git-rev based suffix: ie. mom-0.4.0-1.rpm?
> 
> I need to figure out how to handle a problem that might be a bit
> unique to mom.  MOM is used by non-oVirt users who install it from the
> main Fedora repository.  I think it's fine that we are producing our
> own rpms in oVirt (that may have additional patches applied and may
> resync to upstream mom code more frequently than would be desired for
> the main Fedora repository).  Given this, I think it makes sense to
> tag the oVirt RPMs with a special version suffix to indicate that
> these are oVirt produced and not upstream Fedora.
> 
> For example:
> The next Fedora update will be mom-0.4.0-1.f20.rpm.
> The next oVirt update will be mom-0.4.0-1ovirt.f20.rpm.
> 
> Is this the best practice for accomplishing my goals?  One other thing
> I'd like to have the option of doing is to make vdsm depend on an
> ovirt distribution of mom so that the upstream Fedora version will not
> satisfy the dependency for vdsm.

What is the motivation for this? You would not like to bother Fedora
users with updates that are required only for oVirt?

Vdsm itself is built, signed, and distributed via Fedora.
It is also copied into the ovirt repo, for completeness sake. Could MoM
do the same?

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] mom RPMs for 3.4

[Dan Kenigsberg]

On Thu, Jan 30, 2014 at 11:49:42AM -0500, Adam Litke wrote:
> Hi Sandro,
> 
> After updating the MOM project's build system, I have used jenkins to
> produce a set of RPMs that I would like to tag into the oVirt 3.4
> release.  Please see the jenkins job [1] for the relevant artifacts
> for EL6[2], F19[3], and F20[4].
> 
> Dan, should I submit a patch to vdsm to make it require mom >= 0.4.0?
> I want to be careful to not break people's environments this late in
> the 3.4 release cycle.  What is the best way to minimize that damage?

Hey, we're during beta. I prefer making this requirement explicit now
over having users with supervdsmd.log retate due to log spam.

> 
> [1] http://jenkins.ovirt.org/view/All/job/manual-build-tarball/179/
> [2] 
> http://jenkins.ovirt.org/view/All/job/manual-build-tarball/179/label=centos6-host/artifact/exported-artifacts/mom-0.4.0-1.el6.noarch.rpm
> [3] 
> http://jenkins.ovirt.org/view/All/job/manual-build-tarball/179/label=fedora19-host/artifact/exported-artifacts/mom-0.4.0-1.fc19.noarch.rpm
> [4] 
> http://jenkins.ovirt.org/view/All/job/manual-build-tarball/179/label=fedora20-host/artifact/exported-artifacts/mom-0.4.0-1.fc20.noarch.rpm
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Predictable_vNIC_Order

[Dan Kenigsberg]

On Wed, Jan 29, 2014 at 02:07:00PM +0100, Michal Skrivanek wrote:
> 
> On Jan 29, 2014, at 08:05 , Meni Yakove  wrote:
> 
> > Hi,
> > 
> > On oVirt 3.4 there is new feature:
> > http://liver.englab.brq.redhat.com/redirect/?url=http://wiki.ovirt.org/Feature/Predictable_vNIC_Order
> > 
> > Right now the feature make sure that the VNIC ordering for VM is the same 
> > after create VM from template/snapshot/import.
> > For example:
> > 
> > On original VM VNIC1 attached to network blue and VNIC2 attached to network 
> > red and on the VM we have two interfaces ETH0 and ETH1, after creating 
> > template from this VM and create VM from the template the order of the 
> > VNICs are the same for the new VM.
> > On the VM from template the first VNIC is attached to blue network and the 
> > second VNIC is attached to red network but on the VM we have ETH2 and ETH3
> > 
> > I think that the feature should also include Sealing_Linux_VM:
> > http://www.ovirt.org/Sealing_Linux_VM
> 
> we do have a tentative feature to seal/reset Linux VM via virt-sysprep. Not 
> targeted to a release yet...
> 
> > Whit this on the VM from template the interface naming will be the same as 
> > the original VM. (ETH0 and ETH1)
> 
> Where do you see the connection? (I'm not opposing/arguing, just curious what 
> you have in mind how it should work)

In certain guest OS, most notably EL6, when a VM is first run, it
creates /etc/udev/rules.d/70-persistent-net.rules which persists the
mapping between mac addresses and interface names. If the VM is booted
again, this rule makes sure that the eth0 has the same mac address as
before.

This method breaks when the mac is changed while the VM was offline, or
when a new VM is cloned out of the original one (and is allocated with a
completely new set of mac addresses). Then, in a further run of the VM,
the old "eth0" would point no where, and a new eth4 would point to a
device with the new mac address. Fedora 20, with
http://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/,
no longer has this issue. But other OSes do.

To avoid this, any reference to the mac address has to be stripped
before the cloned VM is run. /etc/udev/rules.d/70-persistent-net.rules
needs to be tweaked/deleted, and HWADDR lines need to be fixed/dropped
from /etc/sysconfig/network-scripts/ifcfg-*.

http://www.ovirt.org/Sealing_Linux_VM would do this (and more) before a
template is created.

A more delicate question is what needs to be done before when we clone
from a template, or import a VM. In these conditions, we usually cannot
change the original VM before cloning, and it is less clear if we really
want to remove every bit of identity from the image.

IMO it would be great to call something like

virt-sysprep --enable=net-hwaddr /path/to/cloned/image

just after a VM is cloned. Ideally, this could be done in an HSM async
task (one we have them). Until then, an SPM task would do.


Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [vdsm] RES: oVirt 3.4 test day - PPC support

[Dan Kenigsberg]

On Sun, Jan 26, 2014 at 04:53:08AM -0500, Yaniv Bronheim wrote:
> Yes, sounds like the overriding of vdsm.conf is an issue that we need to avoid
> what I wonder is that it is not a regression, old versions of vdsm also 
> overwrited the vdsm.conf file during deploy, so how is it harming us only now?
> 
> please open a bug on the overwriting issue

I do not think a bug is warrented.
We wanted that a remote-controlled installation of a host would get it
into a known good state.
Needing to faking the hardware is a special case, not the rule, and
for it,
https://github.com/oVirt/ovirt-host-deploy/blob/master/README#L52
VDSM/configOverride=bool:False
was devised.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Copy reviewer scores on trivial rebase/commit msg changes

[Dan Kenigsberg]

On Sat, Jan 18, 2014 at 01:48:52AM +0200, Itamar Heim wrote:
> I'd like to enable these - comments welcome:
> 
> 1. label.Label-Name.copyAllScoresOnTrivialRebase
> 
> If true, all scores for the label are copied forward when a new
> patch set is uploaded that is a trivial rebase. A new patch set is
> considered as trivial rebase if the commit message is the same as in
> the previous patch set and if it has the same code delta as the
> previous patch set. This is the case if the change was rebased onto
> a different parent. This can be used to enable sticky approvals,
> reducing turn-around for trivial rebases prior to submitting a
> change. Defaults to false.
> 
> 
> 2. label.Label-Name.copyAllScoresIfNoCodeChange
> 
> If true, all scores for the label are copied forward when a new
> patch set is uploaded that has the same parent commit as the
> previous patch set and the same code delta as the previous patch
> set. This means only the commit message is different. This can be
> used to enable sticky approvals on labels that only depend on the
> code, reducing turn-around if only the commit message is changed
> prior to submitting a change. Defaults to false.
> 
> 
> https://gerrit-review.googlesource.com/Documentation/config-labels.html

I think that the time saved by these copying is worth the dangers.

But is there a way to tell a human ack from an ack auto-copied by these
options? It's not so fair to blame X for "X approved this patch" when he
only approved a very similar version thereof.

Assuming that a clean rebase can do no wrong is sometimes wrong
(a recent example is detailed by Nir's http://gerrit.ovirt.org/21649/ )

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [vdsm] Gerrit NEW Change Screen

[Dan Kenigsberg]

On Thu, Jan 16, 2014 at 08:16:17AM -0500, Adam Litke wrote:
> On 16/01/14 00:08 +0200, Itamar Heim wrote:
> >with gerrit 2.8, there is a new change screen.
> >its not enabled by default (yet), please use and see what you think.
> >
> >to enable, go to settings (click the top-right arrow next to your
> >name, and choose settings).
> >select preferences and set "Change View:" to "New Screen".
> 
> Thanks Itamar.  For me, this new screen is much better.  Things are
> more compact and for most patches all of the important information
> fits easily on one screen.  I also like the easy to find gitweb link
> and the ability to edit the commit message right from the interface.

On my narrow lcd, though, the new look makes comments in diff to be seen
only partially. I failed to find a configurable to make the comment
windows narrower, so my only solution is to reduce the font (which is
bad for my poor old eyes).
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [Users] [QE] 3.4.0 Release tracker

[Dan Kenigsberg]

On Mon, Jan 13, 2014 at 11:05:05AM +0100, Sander Grendelman wrote:
> On Mon, Jan 13, 2014 at 12:32 AM, Dan Kenigsberg  wrote:
> > On Fri, Jan 10, 2014 at 04:55:18PM +0200, Itamar Heim wrote:
> >> On 01/10/2014 01:52 PM, Sander Grendelman wrote:
> >> >Can I propose BZ#1035314 for 3.3.3 or 3.4.0, simple, trivial fix to a 
> >> >hook.
> >>
> >> Hi Sander,
> >>
> >> please use bug summary so folks won't have to go and look what the
> >> number means just to see if relevant to them.
> >>
> >> this is about:
> >> Bug 1035314 - vdsm-hook-nestedvt uses kvm_intel-only syntax
> 
> OK, I'll keep that in mind!
> 
> >> i see you posted a patch in the bug - can you post the patch to
> >> gerrit as well?
> >
> > I've done that alraedy: http://gerrit.ovirt.org/#/c/23164/
> >
> > Sander (and others), could you review and formally verify it?
> 
> Looks fine to me, should I also do/confirm something in gerrit?

Yes, it would be great if you tick the patch as verified, as I did not
try it myself.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [Users] [QE] 3.4.0 Release tracker

[Dan Kenigsberg]

On Fri, Jan 10, 2014 at 04:55:18PM +0200, Itamar Heim wrote:
> On 01/10/2014 01:52 PM, Sander Grendelman wrote:
> >Can I propose BZ#1035314 for 3.3.3 or 3.4.0, simple, trivial fix to a hook.
> 
> Hi Sander,
> 
> please use bug summary so folks won't have to go and look what the
> number means just to see if relevant to them.
> 
> this is about:
> Bug 1035314 - vdsm-hook-nestedvt uses kvm_intel-only syntax
> 
> i see you posted a patch in the bug - can you post the patch to
> gerrit as well?

I've done that alraedy: http://gerrit.ovirt.org/#/c/23164/

Sander (and others), could you review and formally verify it?
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] oVirt 3.4.0 alpha repository closure failure

[Dan Kenigsberg]

On Fri, Jan 10, 2014 at 08:48:52AM +0100, Sandro Bonazzola wrote:
> Hi,
> oVirt 3.4.0 alpha repository has been composed but alpha has not been 
> announced due to repository closure failures:
> 
> on CentOS 6.5:
> 
> # repoclosure -r ovirt-3.4.0-alpha -l ovirt-3.3.2 -l base -l epel -l 
> glusterfs-epel -l updates -l extra -l glusterfs-noarch-epel -l ovirt-stable -n
> Reading in repository metadata - please wait
> Checking Dependencies
> Repos looked at: 8
>base
>epel
>glusterfs-epel
>glusterfs-noarch-epel
>ovirt-3.3.2
>ovirt-3.4.0-alpha
>ovirt-stable
>updates
> Num Packages in Repos: 16581
> package: mom-0.3.2-20140101.git2691f25.el6.noarch from ovirt-3.4.0-alpha
>   unresolved deps:
>  procps-ng

Adam, this seems like a real bug in http://gerrit.ovirt.org/#/c/22087/ :
el6 still carries the older "procps" (which is, btw, provided by
procps-ng).


> package: vdsm-hook-vhostmd-4.14.0-1.git6fdd55f.el6.noarch from 
> ovirt-3.4.0-alpha
>   unresolved deps:
>  vhostmd

Douglas, could you add a with_vhostmd option to the spec, and have it
default to 0 on el*, and to 1 on fedoras?

Thanks,
Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Request for power-user permissions

[Dan Kenigsberg]

On Thu, Dec 26, 2013 at 03:10:27AM -0500, Ayal Baron wrote:
> 
> 
> - Original Message -
> > Hi all,
> > 
> > I'm Nir from storage team. I work mostly on vdsm.
> > 
> > I want power-user permissions for jenkins.ovirt.org so I would be able to
> > create and modify jobs related to vdsm.
> 
> +1 from me.

+1
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [Users] oVirt Weekly Meeting Minutes -- 2013-11-27

[Dan Kenigsberg]

On Fri, Nov 29, 2013 at 01:18:53PM +0100, Sandro Bonazzola wrote:
> Il 29/11/2013 13:13, Dan Kenigsberg ha scritto:
> > On Fri, Nov 29, 2013 at 11:49:59AM +0100, Sandro Bonazzola wrote:
> >> Il 29/11/2013 09:43, Gianluca Cecchi ha scritto:
> >>> On Fri, Nov 29, 2013 at 8:57 AM, Sandro Bonazzola  wrote:
> >>>
> >>>>>
> >>>>> Meeting summary
> >>>>> ---
> >>>>> * Agenda and roll Call  (doron, 15:02:42)
> >>>>>   * 3.3 update releases  (doron, 15:04:23)
> >>>>>   * 3.4 planning  (doron, 15:04:24)
> >>>>>   * conferences and workshops  (doron, 15:04:26)
> >>>>>   * infra update  (doron, 15:04:27)
> >>>>>   * other topics  (doron, 15:04:29)
> >>>>>   * LINK: http://gerrit.ovirt.org/#/admin/projects/ovirt-release ~
> >>>>> (danken, 15:12:58)
> >>>>>   * LINK: http://gerrit.ovirt.org/21794   (mburns, 15:15:04)
> >>>>>   * LINK: http://jenkins.ovirt.org/job/ovirt-release/16800/   (mburns,
> >>>>> 15:15:48)
> >>>>>   * mburns to add sbonazzo as a maintainer to support ovirt-release
> >>>>> project  (doron, 15:18:17)
> >>>>
> >>>> ovirt-release-9 released yesterday
> >>>
> >>> BTW: I see that this package contains
> >>> /etc/yum.repos.d/fedora-virt-preview.repo
> >>> (and ovirt-release-fedora-8-1.noarch already did so)
> >>> By default all lines are disabled in it.
> >>> When and how this repo should be enabled? Only when using nightly or
> >>> only under developers/maintainers indications?
> >>
> >> I think that fedora-virt-preview should be used with nightly, stable 
> >> shouldn't need it.
> >> However, since fedora-virt-preview contains vdsm - related packages not 
> >> needed if you run only ovirt-engine (without using the same host as
> >> hypervisor) I think it's better to wait for VDSM guys answer.
> > 
> > Vdsm is not in
> > http://fedorapeople.org/groups/virt/virt-preview/fedora-20/x86_64/
> > 
> > virt-preview is not needed for ovirt-3.3, and frankly, I think it should
> > be dropped from ovirt-release.
> > 
> > It used to be needed on the nodes when vdsm required a version of
> > libvirt that was not yet in Fedora. Now that we have el6 as a
> > fully-supported platform, and given that el6 is missing from
> > virt-preview, virt-preview is much less helpful to us.
> > 
> > Dan.
> > 
> 
> So, any objection in removing virt-preview from ovirt-release?
> What about nightly? Will it be needed there?

Should be removed from both, since it is currently unused. We could
reintroduce it if the need arises.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] failed to add host into cluster

[Dan Kenigsberg]

On Tue, Nov 26, 2013 at 01:08:00PM +0800, ouyang guohua wrote:
> On 11/22/2013 08:38 PM, ouyang guohua wrote:
> >On 11/22/2013 07:52 PM, Dan Kenigsberg wrote:
> >>On Thu, Nov 21, 2013 at 05:22:35PM +0100, Michal Skrivanek wrote:
> >>>>>ok, that looks ok. Might be text parsing issue.
> >>>>>And "virsh capabilities" output?
> >>>>>what's the vdsm you're using?
> >>>>virsh capabilities" output is a bit long, see attachment
> >>>>
> >>>># rpm -q vdsm
> >>>>vdsm-4.10.3-18.fc19.x86_64

> 
> 
> so, the problem maybe exists on the old vdsm package but fixed on the new 
> package.

Thank Mark Wu's http://gerrit.ovirt.org/#/c/8208/!
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] failed to add host into cluster

[Dan Kenigsberg]

On Thu, Nov 21, 2013 at 05:22:35PM +0100, Michal Skrivanek wrote:
> 
> >> ok, that looks ok. Might be text parsing issue.
> >> And "virsh capabilities" output?
> >> what's the vdsm you're using?
> > 
> > virsh capabilities" output is a bit long, see attachment
> > 
> > # rpm -q vdsm
> > vdsm-4.10.3-18.fc19.x86_64
> 
> hmm, works well on my box
> try master, though this code didn't change since 2012
> 
> the code in vdsm/caps.py is pretty straightforward:
> caps = minidom.parseString(capabilities)
> for archTag in caps.getElementsByTagName('arch'):
> if archTag.getAttribute('name') == 'x86_64':
> return [m.firstChild.data for m in archTag.childNodes
> if m.nodeName == 'machine']
> 
> 
> try to uninstall qemu Alpha support if it changes a thing or not...

As Michal noted, Vdsm parses the attached capabilities as it should.
Maybe it receives something else from libvirt. Could you add some
logging and report?

diff --git a/vdsm/caps.py b/vdsm/caps.py
index 5599522..4daca46 100644
--- a/vdsm/caps.py
+++ b/vdsm/caps.py
@@ -136,6 +136,7 @@ def _getCpuTopology(capabilities):
 def _getEmulatedMachines(capabilities=None):
 if capabilities is None:
 capabilities = _getCapsXMLStr()
+logging.debug('caps: %s', capabilities)
 caps = minidom.parseString(capabilities)
 for archTag in caps.getElementsByTagName('arch'):
 if archTag.getAttribute('name') == 'x86_64':

Maybe you can add some logging
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] oVirt 3.3.2 beta status

[Dan Kenigsberg]

On Mon, Nov 18, 2013 at 10:12:02AM +0100, Sandro Bonazzola wrote:
> Hi,
> 
> we're going to branch and build oVirt 3.3.2 beta on Nov 27th.
> A bug tracker is available at [1] and it shows only 2 bugs blocking the 
> release:
> 
> Bug 1029792 - VDSM does not report the qemu version in capabilities, if 
> qemu-kvm-rhev is used

Backported
http://gerrit.ovirt.org/21363
http://gerrit.ovirt.org/21364
to ovirt-3.3 branch to address this request.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [UX] how to design a bar/line chart?

[Dan Kenigsberg]

On Tue, Nov 12, 2013 at 11:49:12AM +0100, Michal Skrivanek wrote:
> 
> On Nov 12, 2013, at 11:18 , Tomas Jelinek  wrote:
> 
> > So we have looked into the resource usage sampling with mbetak and also 
> > with Michal and it seems that
> > 
> > for the CPU usage:
> > - VDSM polls libvirt to get the runtime statistics of the VM regularly. The 
> > pooling interval is configured in 
> >  vdsm.conf as vm_sample_cpu_interval and by default it is 15 seconds
> > - libvirt returns something we than use to calculate the average CPU usage 
> > since the last poll
> > - engine polls VDSM once in 15 seconds to get the current statistics (the 
> > same 15 seconds is just a coincidence and we can not count on this)
> > - than the frontend polls the engine each 5-60 seconds (depends on the 
> > refresh rate) and gets the current value from the engine
> > - the user can press the refresh button anytime to poll the engine again
> > 
> > For network usage:
> > - it should be pretty much the same as the CPU just the VDSM poll interval 
> > is configured as vm_sample_net_interval and by default it is 5 seconds
> 
> Dan, since we poll only every 15s and cpu info is 15s wouldn't it make
> sense to change the default for network monitoring to 15s as well?
> it's 2 libvirt rounds trip for nothing really. Or does it serve some
> other purpose?

I see no motivation for Vdsm to poll libvirt faster than Engine polls
Vdsm.

Maybe Federico, who's improved scalability back in
https://bugzilla.redhat.com/show_bug.cgi?id=661321 [vdsm] [scale] vdsm CPU 
consumption goes between 180-400 when running 100 vms
remembers a reason.

I'm for changing the default to 15.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [vdsm] daily oVirt 3.3.1 blocker status

[Dan Kenigsberg]

On Mon, Nov 11, 2013 at 10:26:19AM +0100, Sandro Bonazzola wrote:
> Il 08/11/2013 01:28, Dan Kenigsberg ha scritto:
> > On Wed, Nov 06, 2013 at 02:36:08AM -0500, Eduardo Warszawski wrote:
> >>> Hi,
> >>> only one bloker remains to be fixed:
> >>> Bug 1022961 - Running a VM from a gluster domain uses mount instead of
> >>> gluster URI
> >>> I don't see any activity about it.
> >>> Can anybody join Eduardo for fixing it ASAP?
> >>>
> >> I'm working on it. 
> >> Hope soon we will have a fix.
> >> Don't panic! :)
> > 
> > It's not the /real thing/ that Eduardo wanted, but it's something and
> > it's working. Please review and verify
> > 
> > http://gerrit.ovirt.org/#/c/21059/
> > gluster prepareImage: return gluster-sepecific information
> > 
> 
> Hi,
> Bug 1022961 - Running a VM from a gluster domain uses mount instead of 
> gluster URI
> is the only bug blocking 3.3.1 and is on POST.
> the related patch http://gerrit.ovirt.org/21059 is marked as verified and has 
> two +1.
> Can you merge the patch and build VDSM for having it under testing so we can 
> try to release 3.3.1 this week?

I'd love to merge this in, but given this being my own code, I prefer
another core developer ack it, even though no one really likes it.

Eduardo dislikes the fact that this patch maintains the awkward API of
prepareImage returning this 'info' dictionary on top of the simple
'path' it used to return before the glusterSD effort.

Federico dislikes the fact that this patch adds gluster-specific
code into our top-level class HSM. To relieve this, I am now suggesting
three other cleanup patches
   http://gerrit.ovirt.org/21122
   http://gerrit.ovirt.org/21123
   http://gerrit.ovirt.org/21124
but I would like to take the simple fix first. Unbreaking a major
feature in master and ovirt-3.3.1 should have precedence over
architectural debates.

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [vdsm] daily oVirt 3.3.1 blocker status

[Dan Kenigsberg]

On Wed, Nov 06, 2013 at 02:36:08AM -0500, Eduardo Warszawski wrote:
> > Hi,
> > only one bloker remains to be fixed:
> > Bug 1022961 - Running a VM from a gluster domain uses mount instead of
> > gluster URI
> > I don't see any activity about it.
> > Can anybody join Eduardo for fixing it ASAP?
> > 
> I'm working on it. 
> Hope soon we will have a fix.
> Don't panic! :)

It's not the /real thing/ that Eduardo wanted, but it's something and
it's working. Please review and verify

http://gerrit.ovirt.org/#/c/21059/
gluster prepareImage: return gluster-sepecific information
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] daily oVirt 3.3.1 blocker status

[Dan Kenigsberg]

On Mon, Nov 04, 2013 at 09:05:19AM +0100, Sandro Bonazzola wrote:
> Il 31/10/2013 08:49, Sandro Bonazzola ha scritto:
> > Hi,
> > The following blockers are still not fixed:
> > VDSM:
> > Bug 1022961 - Running a VM from a gluster domain uses mount instead of 
> > gluster URI
> > Bug 1022975 - [vdsm] storage domain upgrade fails with attributeError
> > 
> > Federico, Eduardo, can you provide an ETA for those?
> 
> Still not fixed with no ETA.

WWIW, Bug 1022975 is solved by http://gerrit.ovirt.org/20721; waiting
only for a formal verification.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] ovirt live 1.1: 'local_vm' stuck in Wait For Launch

[Dan Kenigsberg]

On Sun, Nov 03, 2013 at 12:01:28PM -0500, Einav Cohen wrote:
> Hi All,
> 
> In my ovirt live 1.1 system (ran as-is, done no alterations to the system 
> once "installed"), 
> after attempting to run the default 'local_vm' VM, it is getting stuck on 
> 'Wait For Launch'.
> I tried running the ovirt live 1.1 system for ~5 times, only in one of these 
> times the VM 
> has actually moved to the Powering Up / Up state when attempting to run it.
> Any ideas?
> 
> [attached engine.log, vdsm.log]
> 
> Many thanks in advance!
> 
> 
> Regards, 
> Einav

snip

> 
> libvirtEventLoop::DEBUG::2013-11-03 
> 11:52:26,383::vm::4724::vm.Vm::(_onLibvirtLifecycleEvent) 
> vmId=`1f0a2763-5c40-4fa3-a51c-ed98276b8e6d`::event Started detail 0 opaque 
> None
> Thread-126::DEBUG::2013-11-03 11:52:26,465::sampling::285::vm.Vm::(start) 
> vmId=`1f0a2763-5c40-4fa3-a51c-ed98276b8e6d`::Start statistics collection
> Thread-128::DEBUG::2013-11-03 11:52:26,479::sampling::314::vm.Vm::(run) 
> vmId=`1f0a2763-5c40-4fa3-a51c-ed98276b8e6d`::Stats thread started
> Thread-128::DEBUG::2013-11-03 
> 11:52:26,480::task::579::TaskManager.Task::(_updateState) 
> Task=`06de0c6d-28e6-4c20-9139-90c3f440dc6b`::moving from state init -> state 
> preparing
> Thread-128::INFO::2013-11-03 
> 11:52:26,480::logUtils::44::dispatcher::(wrapper) Run and protect: 
> getVolumeSize(sdUUID='5b5e5322-df22-4516-860b-361f00c46233', 
> spUUID='9b476274-7c1c-4762-8cb4-699c2729b152', 
> imgUUID='a544c2c3-cf3a-42c7-8e20-371344ad22da', 
> volUUID='bcabc2ba-b7e1-47f9-9098-bbe19b096ad5', options=None)
> Thread-128::DEBUG::2013-11-03 
> 11:52:26,483::fileVolume::520::Storage.Volume::(validateVolumePath) validate 
> path for bcabc2ba-b7e1-47f9-9098-bbe19b096ad5
> Thread-128::INFO::2013-11-03 
> 11:52:26,485::logUtils::47::dispatcher::(wrapper) Run and protect: 
> getVolumeSize, Return response: {'truesize': '139264', 'apparentsize': 
> '262144'}
> Thread-128::DEBUG::2013-11-03 
> 11:52:26,485::task::1168::TaskManager.Task::(prepare) 
> Task=`06de0c6d-28e6-4c20-9139-90c3f440dc6b`::finished: {'truesize': '139264', 
> 'apparentsize': '262144'}
> Thread-128::DEBUG::2013-11-03 
> 11:52:26,485::task::579::TaskManager.Task::(_updateState) 
> Task=`06de0c6d-28e6-4c20-9139-90c3f440dc6b`::moving from state preparing -> 
> state finished
> Thread-128::DEBUG::2013-11-03 
> 11:52:26,485::resourceManager::939::ResourceManager.Owner::(releaseAll) 
> Owner.releaseAll requests {} resources {}
> Thread-128::DEBUG::2013-11-03 
> 11:52:26,485::resourceManager::976::ResourceManager.Owner::(cancelAll) 
> Owner.cancelAll requests {}
> Thread-128::DEBUG::2013-11-03 
> 11:52:26,491::task::974::TaskManager.Task::(_decref) 
> Task=`06de0c6d-28e6-4c20-9139-90c3f440dc6b`::ref 0 aborting False
> Thread-126::DEBUG::2013-11-03 11:52:26,507::vmChannels::194::vds::(register) 
> Add fileno 45 to listener's channels.
> Thread-126::WARNING::2013-11-03 
> 11:52:26,522::vm::3326::vm.Vm::(_readPauseCode) 
> vmId=`1f0a2763-5c40-4fa3-a51c-ed98276b8e6d`::_readPauseCode unsupported by 
> libvirt vm
> Thread-126::DEBUG::2013-11-03 
> 11:52:26,552::vm::2036::vm.Vm::(_startUnderlyingVm) 
> vmId=`1f0a2763-5c40-4fa3-a51c-ed98276b8e6d`::_ongoingCreations released

Is this a log of a successful or of a failed attempt? "_ongoingCreations
released" (should) mean that the Vm is "Up" as much as Vdsm concerns.

Could you verify that Vdsm reports "Wait For Launch" via vdsClient?
Could you attach libvirtd.log from a failed attempt?
Which kernel and qemu-kvm are you using?

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] 3.3.0.1 Release branch and tracker

[Dan Kenigsberg]

On Wed, Sep 25, 2013 at 03:49:40AM -0400, Alon Bar-Lev wrote:
> 
> 
> - Original Message -
> > From: "Ofer Schreiber" 
> > To: "engine-devel" 
> > Sent: Wednesday, September 25, 2013 10:40:33 AM
> > Subject: [Engine-devel] 3.3.0.1 Release branch and tracker
> > 
> > Hey,
> > 
> > As you may know, we're planning to release oVirt 3.3.0.1 soon.
> > I've created a tracker bug
> > (https://bugzilla.redhat.com/show_bug.cgi?id=1011800) and a git branch
> > (ovirt-engine-3.3.0.1, based on 3.3.0) for this release.
> > 
> 
> Once again, I do not understand why go into 4 digit version and not release 
> 3.3.1 as z-stream, deferring remaining queue to 3.3.2.
> 
> The argument of small/large change is irrelevant in z-stream as something 
> small for one can be important for other.
> 
> The number should not be important, whenever z-stream is released you take 
> last+1.

hear hear. Z is the last letter of the English alphabet. We should not
go past it, unless 3.3.1 was already in beta and we have to ship a quick
3.3.0.1.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] fake VDSM as oVirt project?

[Dan Kenigsberg]

On Sun, Sep 15, 2013 at 12:57:29PM +0300, Itamar Heim wrote:
> On 09/13/2013 09:52 AM, Liran Zelkha wrote:
> >+1 I use it constantly.
> 
> +1
> adding infra, where new git repo's are usually requested.
> (we also ask board if a new project scope, but this seems like just
> a repo for a help/test program)
> 
> if more +1's and no objections, ping next week to create  repo.

It would have been much cooler to fix the real vdsm so it can be run in
a crippled mode with no root-related privileges. But I admit it would
require more work, and would miss some of the comforts of a Java-only
stub. So +1 from me.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [Users] [ATN] oVirt 3.3 release, moving to RC [3.3 branching, blockers review]

[Dan Kenigsberg]

On Tue, Jul 30, 2013 at 06:05:46PM +0300, Moran Goldboim wrote:
> We would like to go a head on oVirt 3.3 release process and issue an RC
> 
> for tomorrow meeting:
> 
> Maintainers/ Package owners
> ==
> -branch your git repo (master) with 3.3 branch (making master ready for 3.4)
> -for tomorrow meeting, overview [1], see if you have any changes to it
> -if you don't have any blockers under your component, please branch
> 3.3 with RC1 branch
> 
> |
> -master
> |
> -engine_3.3
> |
> -RC1
> |
> -engine_3.2
> ...
> 
> Users
> 
> with your experience from test day and with nightlies, if you feel
> there are additional candidates to block this version for please add
> it to the tracker bug [1].
> 
> Suggested Schedule
> 
> Wed Jul 31st - review of blockers for the version and component readiness
> Mon Aug 5th - RC1 release
> Wed Aug 7th - Release readiness review (in case of blockers an RC2
> will be issued)
> 
> Thanks.
> 
> [1]*Bug 918494* 
> -Tracker: oVirt 3.3 release

I've just tagged vdsm-4.12.0 and branched ovirt-3.3 branch for the vdsm
repository starting at git has 620343d6317c849fc985a5083cebb68c995f7c15.

Expect a deluge of non-ovirt-3.3 merges to the master branch soon.
Future ovirt-3.3 fixes would have to be backported and cherry-picked.

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [Users] [Feedback required][host-deploy] Fedora-19 misses tar at minimal setup

[Dan Kenigsberg]

On Tue, Jul 30, 2013 at 10:09:47AM -0400, Antoni Segura Puimedon wrote:
> I would advocate for  option 2.
> 
> - Original Message -
> > From: "Michal Skrivanek" 
> > To: "Alon Bar-Lev" 
> > Cc: "Juan Hernandez" , "engine-devel" 
> > , "arch" , "users"
> > 
> > Sent: Tuesday, July 30, 2013 3:25:24 PM
> > Subject: Re: [Engine-devel] [Users] [Feedback required][host-deploy] 
> > Fedora-19  misses tar at minimal setup
> > 
> > 
> > On Jul 30, 2013, at 15:12 , Alon Bar-Lev  wrote:
> > 
> > > Hello All,
> > > 
> > > Starting the discussion again...
> > > 
> > > I would like to receive feedback regarding how we should cope with a state
> > > presented to use by Fedora.
> > > 
> > > Fedora-19 minimal setup does not install tar utility which is required to
> > > deploy files during the host-deploy process (Hosts->Add Host).
> > > 
> > > I guess because of 2.8M in size (including translations) -- a standard
> > > commonly used utility was removed.
> > 
> > How about filing bug on that? This is such a basic utility I can't imagine
> > anyone removing it.
> > 
> > > 
> > > There are three alternatives :
> > > 
> > > 1. Instruct users who are using minimal installations to manually install
> > > tar utility just like they configure repository, dns, etc..
> > > 
> > > Benefit: simplicity.
> > > Benefit: use standard tools.
> > > Benefit: lower payload to transmit.
> > > Drawback: require tar at destination machine.
> > > 
> > > 2. Do not use tar but self extracting python script, a patch is ready[1].
> > > 
> > > Benefit: ability to deploy environment in which tar is missing.
> > > Drawback: non standard tool at destination machine.
> > > Drawback: complexity within our code.

How about option 2.1: convince Fedora to reintroduce tar? It is ironic
that Gnome is shipped by default, but not such a staple utility.

Where in Fedora did this decision take place? Can it be undone?
Is it commonplace these days among other distros to boycot tar?

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [Users] [ATN] oVirt 3.3 release, moving to RC [3.3 branching, blockers review]

[Dan Kenigsberg]

On Tue, Jul 30, 2013 at 06:05:46PM +0300, Moran Goldboim wrote:
> We would like to go a head on oVirt 3.3 release process and issue an RC
> 
> for tomorrow meeting:
> 
> Maintainers/ Package owners
> ==
> -branch your git repo (master) with 3.3 branch (making master ready for 3.4)
> -for tomorrow meeting, overview [1], see if you have any changes to it
> -if you don't have any blockers under your component, please branch
> 3.3 with RC1 branch
> 
> |
> -master
> |
> -engine_3.3
> |
> -RC1
> |
> -engine_3.2
> ...
> 
> Users
> 
> with your experience from test day and with nightlies, if you feel
> there are additional candidates to block this version for please add
> it to the tracker bug [1].
> 
> Suggested Schedule
> 
> Wed Jul 31st - review of blockers for the version and component readiness
> Mon Aug 5th - RC1 release
> Wed Aug 7th - Release readiness review (in case of blockers an RC2
> will be issued)
> 
> Thanks.
> 
> [1]*Bug 918494* 
> -Tracker: oVirt 3.3 release

I've just tagged vdsm-4.12.0 and branched ovirt-3.3 branch for the vdsm
repository starting at git has 620343d6317c849fc985a5083cebb68c995f7c15.

Expect a deluge of non-ovirt-3.3 merges to the master branch soon.
Future ovirt-3.3 fixes would have to be backported and cherry-picked.

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Proposal for new commit msg design for engine commits

[Dan Kenigsberg]

On Tue, Jul 09, 2013 at 05:41:45AM -0400, Antoni Segura Puimedon wrote:
> I like the idea of having a label in the bottom part of the commit that is:
> 
> METADATA: network
> 

This makes sense, but I find "METADATA" as an overly general term. Even
something like

Region_of_Interest: network

is more humanly comprehensible.

Another idea would be to add a gerrit-only set of flags, where the
poster or reviewers have to tick which group of tests are most important
to be run.

> which would be your second proposal.
> 
> - Original Message -
> > From: "Eyal Edri" 
> > To: "engine-devel" 
> > Cc: "infra" 
> > Sent: Tuesday, July 9, 2013 11:38:51 AM
> > Subject: [Engine-devel] Proposal for new commit msg design for engine 
> > commits
> > 
> > Hi,
> > 
> > You all probably know and familiar with 'ovirt-engine' git hook for commit
> > msg template [1].
> > this helps understand the general area of the patch in the project but it
> > lacks additional info that might
> > be valuable for scaling automatic tests in Jenkins CI.
> > 
> > Let me explain:
> > 
> > Infra team is working hard on expanding oVirt CI infrastructure and adding
> > more tests in jenkins (per commit/patch).
> > Adding important meta-data per patch can significatly improve the ability to
> > run specific tests for each patch/commit,
> > and not waste valuable resources on Jenkins jobs that are not relevant to 
> > the
> > code in the patch.
> > 
> > So the idea is to add/expand current metadata per patch, in the form of:
> > (either)
> >  1. expanding current header template to include more data like 'network' ,
> >  'setup', 'tools', 'virt'
> >  2. adding a new label with relevant tags for the patch, called e.g
> >  'METADATA: network, rest, virt'
> > 
> > Jenkins jobs will then be able to parse that data and trigger only relevant
> > jobs for it.
> > this can also allow us to add more jobs per patch, an option that is very
> > problematic today considering the amount of
> > patches coming in to engine.
> > 
> > Once agreed on a format, we'll be able to add a git hook to verify the
> > validity of the commit msg. (similar to bug-url).
> > 
> > if we're not 100% sure that the tags will cover all corner cases and we feel
> > like we need to run the code on all jobs,
> > we can a nightly job to run all the remaining jobs (but at least it won't 
> > run
> > on every patch/commit).
> > 
> > [1] :
> > 
> > 
> > thoughts?
> > 
> > Eyal Edri.
> > ___
> > Engine-devel mailing list
> > Engine-devel@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > 
> ___
> Engine-devel mailing list
> Engine-devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] SSH Soft Fencing

[Dan Kenigsberg]

On Thu, Jun 27, 2013 at 08:48:39AM -0400, Eli Mesika wrote:
> 
> 
> - Original Message -
> > From: "Martin Perina" 
> > To: engine-devel@ovirt.org
> > Cc: "Yair Zaslavsky" , "Barak Azulay" 
> > , "Eli Mesika" 
> > Sent: Thursday, June 27, 2013 1:51:06 PM
> > Subject: SSH Soft Fencing
> > 
> > Hi,
> > 
> > SSH Soft Fencing is a new feature for 3.3 and it tries to restart VDSM
> > using SSH connection on non responsive hosts prior to real fencing.
> > More info can be found at
> > 
> > http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3
> > 
> > In current SSH Soft Fencing implementation the restart VDSM using SSH
> > command is part of standard fencing implementation in
> > VdsNotRespondingTreatmentCommand. But this command is executed only
> > if a host has a valid PM configuration. If host doesn't have a valid
> > PM configuration, the execution of the command is disabled and host
> > state is change to Non Responsive.
> > 
> > So my question are:
> > 
> > 1) Should SSH Soft Fencing be executed on hosts without valid PM
> >configuration?
> 
> I think that the answer should be yes. The vdsm restart will solve most of 
> problems

Would you enumerate the problems that would be solved by a vdsm restart
(on list, but on the feature page, too)?
I am aware of two issues, both are vdsm bugs:
- If libvirtd crashes, vdsm not is not restarted unless there are
  running VMs
- Vdsm had several bugs in its soft prepareForShutdown process, getting
  itself stuck there in case of various background storage processes.

I think that solving these two issues would be safer and cleaner than
introducing `ssh host service vdsmd restart` flow.

The first issue is only a matter of untangling some vdsm internal
ugliness: whenever a libvirtconnection is produced, it should be wrapped
so that it cathces libvirt crashes. Unlike now, where only VM-related
libvirtconnection undergo this treatment.

The second issue can be avoiding by vdsm resorting to kill-9-ing itself.
After all, this is what `service vdsmd restart` ends up doing after a
VERY short timeout (2-3 seconds, iicr).

I suppose that there are other reasoning for a remote restart, but in
general, I think that it's better to have Vdsm "do the right thing" than
expecting Engine to control that remotely.

Regards,
Dan.
> , so why not using it whether a PM agent is defined or not.
> 
> > 
> > 2) Should VDSM restart using SSH command be reimplemented
> >as standalone command to be usable also in other parts of engine?
> >If 1) is true, I think it will have to be done anyway.
> 
> +1 
> 
> > 
> > 
> > Martin Perina
> > 
> ___
> Engine-devel mailing list
> Engine-devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Introduce mail

[Dan Kenigsberg]

On Sun, Jun 30, 2013 at 09:16:00AM +0300, Livnat Peer wrote:
> Hi Fellipe,
> 
> We'll be happy to use your help, there is always a lot of work to do :)
> Is there any specific area in virtualization you would like to focus on?
> Network | storage | VMs life-cycle | host life-cycle other?
> 
> To start setting your development environment, take a look at -
> www.ovirt.org/Vdsm_Developers
> 
> 
> Also adding Dan to point any low hanging fruits, or pointers for a
> potentially first task :)

I see that we currently only have 4 EasyFix bugs
https://bugzilla.redhat.com/buglist.cgi?bug_status=NEW&component=vdsm&f0=OP&f1=OP&f3=CP&f4=CP&j1=OR&keywords=EasyFix%2C%20&keywords_type=allwords&list_id=1490222&o2=notsubstring&query_format=advanced&v2=rhel-6.2.0

I would very much like to get help testing vdsm on Fedora 19 and fixing
issues that might arrise there. E.g. my http://gerrit.ovirt.org/16059

I find running the tests (unit+functional) and extending them, as a good
starting poing.

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Cloud-Init integration

[Dan Kenigsberg]

On Sun, Jun 16, 2013 at 05:50:38PM +0300, Greg Padgett wrote:
> On 06/16/2013 05:08 PM, Mike Kolesnik wrote:
> >- Original Message -
> >>On Thu, Mar 28, 2013 at 07:35:23PM -0400, Greg Padgett wrote:
> >>>Hi Everyone,
> >>>
> >>>I'd like to propose a feature we've been doing some investigation
> >>>into, which is to integrate cloud-init support into oVirt.
> >>>
> >>>Cloud-init is used to help provision new Linux systems by setting
> >>>the hostname, ip, ssh keys, timezone, injecting files, and more.
> >>>It's used by OpenStack (amongst others) now, and has a lot of
> >>>features that may be helpful to our users.
> >>>
> >>>Details are still evolving, but for more info please see the wiki page:
> >>>
> >>>http://www.ovirt.org/Features/Cloud-Init_Integration
> >>>
> >>>All feedback is welcome!
> >>
> >>All feedback? Even if it's 3 months too late?
> >>
> 
> I'll have to be more careful next time to specify a time limit :)
> But really, thank you both for the comments.  Feel free to add them
> to a "nice to have" list on the wiki page, or I can add it.  Replies
> inline...
> 
> >>If so, then here's a few comments:
> >>
> >>1. oVirt-3.2 completely supports IPv6 within guests. It would be nice to
> >>carry this on to cloud-init, by allowing to set the IPv6 address of
> >>the guest. (or are we happy with the auto configured ipv6 addresses?)
> >>
> 
> It would, but unfortunately cloud-init doesn't yet translate ipv6 fields
> from /etc/network/interfaces (its chosen networking input format) to
> ifcfg files.  Now that you mention it, it doesn't add IPV6INIT=yes,
> either.

ifcfg files? What's that? Those easily-edited text files that are being
deprecated by NetworkManager? Does cloud-init play well with the latter?
(we found a couple of pitfalls, the hard way).

> 
> These are things we can add--to both oVirt and cloud-init, I guess at a
> later time.
> 
> >>2. I think that the GUI used for setting IP addresses should be
> >>immitated here. It allows Static/DHCP/None, and disables the
> >>irrelvant fields when DHCP/None is selected.
> >
> 
> It's close: there is a checkbox for dhcp, and if selected it will
> hide the non-relevant fields.

I hate surprises, so I'm in favor of having the same thing, as well as
the "keep recent config in memory when bootproto is moved to None"
semantics.

This applies more strongly to the REST api. Host level configuration
http://www.ovirt.org/Features/Design/Network/SetupNetworks#Scheme has
   
 em1
 
 dhcp
   

And for reporting guest configuration
http://www.ovirt.org/Feature/ReportingVnicImplementation#API_Changes has
  
 p1p2
 guest reported data
 
 
 
 
  

which we should strive to maintain with this feature. Even if cloud-init is not
currently capable of setting multiple addresses, let the API allow for it.

> 
> >Additionally, you should consider showing the vNIC name and not "eth0" etc.
> >IIUC udev rules are rather unpredictable in this regard and could give your
> >vNIC a different name on different VM instances (probably by MAC address 
> >and/or
> >PCI address).
> >Either way, I think it's less confusing to refer to the vNIC itself.
> >
> 
> Similarly to the above, I'm not sure how much cloud-init supports
> with respect to vNICs.  The name is used as the
> /etc/network/interfaces adapter name and the ifcfg-* suffix (per
> distro), so I guess if you plumbed everything around that in the
> image and could finish the setup with just the interfaces/ifcfg
> config file then it would work as-is.  If that isn't adequate (I
> haven't looked or tested) then we may need to consider submitting a
> patch to add this to cloud-init.

Either way, RHEV should not expose in its own interface the "eth0" names
that we cannot enforce within the guest. E.g., my Fedora has funny
interface names such as em1 and wlp3s0, nothing like the good old eth*.

> 
> >>3. Is "Support custom volume label for vm payloads" still on the TODO
> >>list? Note: F19's dosfstools has renamed mkfs.msdos to mkfs.fat.
> >>This means that creating a virtual floppy with a payload currently
> >>does not work there (it's a tiny fix, for sure).
> >>
> 
> It's implemented.  Of course, it still uses mkfs.msdos.

Implemented but not yet posted, I presume? Because upstream vdsm's does
not use mkfs.msdos's -n.

> 
> >>4. I do not see ovirt-guest-agent mentioned in the feaure page. It is
> >>the obvious channel to report that cloud-init is Done to Engine.
> >>
> 
> You're right, and this is one part of the feature that hasn't been
> done. It may also require some work on cloud-init, or for us to use
> a different input format (i.e. a mime-formatted sequence instead of
> vanilla config-drive-2).  It would be great to add this, though my
> time to do that now is a bit limited.

What is the target oVirt version for this feature, by the way? It is not listed
in http://www.ovirt.org/OVirt_3.3_release-management, so I presume it's for
3.4?

> 
> >>5

Re: [Engine-devel] Cloud-Init integration

[Dan Kenigsberg]

On Thu, Mar 28, 2013 at 07:35:23PM -0400, Greg Padgett wrote:
> Hi Everyone,
> 
> I'd like to propose a feature we've been doing some investigation
> into, which is to integrate cloud-init support into oVirt.
> 
> Cloud-init is used to help provision new Linux systems by setting
> the hostname, ip, ssh keys, timezone, injecting files, and more.
> It's used by OpenStack (amongst others) now, and has a lot of
> features that may be helpful to our users.
> 
> Details are still evolving, but for more info please see the wiki page:
> 
> http://www.ovirt.org/Features/Cloud-Init_Integration
> 
> All feedback is welcome!

All feedback? Even if it's 3 months too late?

If so, then here's a few comments:

1. oVirt-3.2 completely supports IPv6 within guests. It would be nice to
   carry this on to cloud-init, by allowing to set the IPv6 address of
   the guest. (or are we happy with the auto configured ipv6 addresses?)

2. I think that the GUI used for setting IP addresses should be
   immitated here. It allows Static/DHCP/None, and disables the
   irrelvant fields when DHCP/None is selected.

3. Is "Support custom volume label for vm payloads" still on the TODO
   list? Note: F19's dosfstools has renamed mkfs.msdos to mkfs.fat.
   This means that creating a virtual floppy with a payload currently
   does not work there (it's a tiny fix, for sure).

4. I do not see ovirt-guest-agent mentioned in the feaure page. It is
   the obvious channel to report that cloud-init is Done to Engine.

5. When we come to implement auto-generate of system ssh key, we may
   want to install a virtio-rng device in our VMs, to ensure that the
   keys are not too easy to guess. (or create the key in the host and
   inject it to the guest)

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Add traffic shaping parameters for a network interface.

[Dan Kenigsberg]

On Wed, Jun 12, 2013 at 06:54:52AM -0400, Doron Fediuck wrote:
> 
> 
> - Original Message -
> > From: "Giuseppe Vallarelli" 
> > To: "Doron Fediuck" 
> > Cc: engine-devel@ovirt.org
> > Sent: Wednesday, June 12, 2013 1:47:03 PM
> > Subject: Re: [Engine-devel] Add traffic shaping parameters for a network 
> > interface.
> > 
> > - Original Message -
> > | From: "Giuseppe Vallarelli" 
> > | To: "Doron Fediuck" 
> > | Cc: engine-devel@ovirt.org
> > | Sent: Wednesday, June 12, 2013 9:28:19 AM
> > | Subject: Re: [Engine-devel] Add traffic shaping parameters for a network
> > | interface.
> > | 
> > | 
> > | 
> > | ----- Original Message -
> > | | From: "Doron Fediuck" 
> > | | To: "Giuseppe Vallarelli" 
> > | | Cc: "Livnat Peer" , engine-devel@ovirt.org, "Dan
> > | | Kenigsberg" 
> > | | Sent: Wednesday, June 12, 2013 9:05:35 AM
> > | | Subject: Re: [Engine-devel] Add traffic shaping parameters for a network
> > | | interface.
> > | | 
> > | | 
> > | | 
> > | | - Original Message -
> > | | > From: "Giuseppe Vallarelli" 
> > | | > To: "Livnat Peer" 
> > | | > Cc: "Doron Fediuck" , engine-devel@ovirt.org, 
> > "Dan
> > | | > Kenigsberg" 
> > | | > Sent: Tuesday, June 11, 2013 5:34:16 PM
> > | | > Subject: Re: [Engine-devel] Add traffic shaping parameters for a
> > | | > network
> > | | > interface.
> > | | > 
> > | | > - Original Message -
> > | | > | From: "Giuseppe Vallarelli" 
> > | | > | To: "Livnat Peer" 
> > | | > | Cc: "Doron Fediuck" , engine-devel@ovirt.org,
> > | | > | "Dan
> > | | > | Kenigsberg" 
> > | | > | Sent: Tuesday, June 11, 2013 3:07:32 PM
> > | | > | Subject: Re: [Engine-devel] Add traffic shaping parameters for a
> > | | > | network
> > | | > | interface.
> > | | > | 
> > | | > | Related to QoS parameters reporting to the engine. Looks like 
> > they're
> > | | > | already
> > | | > | available, I tried to use vdsClient with list verb and I've got the
> > | | > | devices
> > | | > | list where a 'specParams' is defined (it's empty because I didn't 
> > try
> > | | > | it
> > | | > | with
> > | | > | my last patch).
> > | | > | 
> > | | > | devices = [... {'nicModel': 'pv', 'macAddr': '00:1a:4a:22:3f:04',
> > | | > | 'network':
> > | | > | 'ovirtmgmt', 'alias': 'net0', 'specParams': {},
> > | | > | 'deviceId': '76173ffc-603a-496f-8ffc-31dc4d41cef6', 'address':
> > | | > | {'slot':
> > | | > | '0x03', 'bus': '0x00', 'domain': '0x',
> > | | > | 'type': 'pci', 'function': '0x0'}, 'device': 'bridge', 'type':
> > | | > | 'interface'}
> > | | > | ...]
> > | | > | 
> > | | > | Perhaps I'm missing something, any ideas/hints ?
> > | | > | Thanks Giuseppe
> > | | > 
> > | | > A few pastes:
> > | | > creating the vm: http://paste.fedoraproject.org/17953/37096041/
> > | | > dumping vm xml: http://paste.fedoraproject.org/17951/37096009/
> > | | > 
> > | | > I've tried this out thanks to Toni, losing sanity with vdsClient..
> > | | > 
> > | | > Giuseppe
> > | | > 
> > | | 
> > | | Thanks Giuseppe.
> > | | Is this also reported by vdsm in getVmStats?
> > | | 
> > | 
> > | Unfortunately is not reported using getVmStats, I'm looking into it.
> > | Giuseppe
> > 
> > New paste: http://paste.fedoraproject.org/18161/71033933/

(using hyperlinks instead of inlining the suggested API is unfriendly to
reviewers and to list archives)

network = {'vnet0': {'macAddr': 'D0:67:E5:F0:75:B4', 'outbound': {'average': 
'1024'}, 'rxDropped': '0', 'txDropped': '0', 'inbound': {'average': '1024'}, 
'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 
'unknown', 'speed': '1000', 'name': 'vnet0'}}

> > 
> > Giuseppe
> 
> Brilliant, thanks!

Why does it fit into getVmStats? getVmStats should report
dynamically-changing properties. Bandwidth limitations are more static
control entities, why should we report them every 10 seconds or so?

Dan
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Add traffic shaping parameters for a network interface.

[Dan Kenigsberg]

On Tue, Jun 11, 2013 at 10:34:16AM -0400, Giuseppe Vallarelli wrote:
> - Original Message -
> | From: "Giuseppe Vallarelli" 
> | To: "Livnat Peer" 
> | Cc: "Doron Fediuck" , engine-devel@ovirt.org, "Dan 
> Kenigsberg" 
> | Sent: Tuesday, June 11, 2013 3:07:32 PM
> | Subject: Re: [Engine-devel] Add traffic shaping parameters for a network 
> interface.
> | 
> | Related to QoS parameters reporting to the engine. Looks like they're 
> already
> | available, I tried to use vdsClient with list verb and I've got the devices
> | list where a 'specParams' is defined (it's empty because I didn't try it 
> with
> | my last patch).
> | 
> | devices = [... {'nicModel': 'pv', 'macAddr': '00:1a:4a:22:3f:04', 'network':
> | 'ovirtmgmt', 'alias': 'net0', 'specParams': {},
> | 'deviceId': '76173ffc-603a-496f-8ffc-31dc4d41cef6', 'address': {'slot':
> | '0x03', 'bus': '0x00', 'domain': '0x',
> | 'type': 'pci', 'function': '0x0'}, 'device': 'bridge', 'type': 'interface'}
> | ...]
> | 
> | Perhaps I'm missing something, any ideas/hints ?

I believe this is enough.

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Add traffic shaping parameters for a network interface.

[Dan Kenigsberg]

On Mon, Jun 10, 2013 at 08:56:21AM -0400, Giuseppe Vallarelli wrote:
> Hi Guys, I've recently submitted a patch to support traffic shaping for a 
> network interface (http://gerrit.ovirt.org/#/c/15445/).
> This work is needed in order to support 
> http://www.ovirt.org/Features/Network_QoS .
> Given:
> 
> 'specParams': {'inbound': {'average': '1000', 'peak': '5000', 'burst': 
> '1024'},
>'outbound': {'average': '128', 'burst': '256'}}}
> 
> Generated xml is the following one:
> 
> 
> 
> 
> 
> 
> As you can see I tried to keep the data structure as flat as possible having 
> the bandwidth element not carrying any useful information.
> Feedback is highly appreciated.
> 

The issue has not been mentioned on the wiki page, but may need a means
to report the currently-configured QoS of each vNIC from Vdsm to Engine.
For example, when a VM is de-hibernated, we may want to tell whether its
QoS needs to be set according to a recently-tweaked policy.

I suggest that we use the "getVmList" verb of Vdsm, which is intended to
report "static" properties of one Vm (or all of them).

On the other hand, Engine would want to blindly set new values whenever
in doubt. In such a case, I think that reporting of QoS can be avoided.

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] cannot sign in to gerrit.ovirt.org with google ID?

[Dan Kenigsberg]

On Thu, May 02, 2013 at 01:00:36PM -0400, Einav Cohen wrote:
> thanks for all of your replies, guys - I really appreciate it!
> 
> unfortunately, I still cannot sign into gerrit.
> This is definitely a server-side problem, since I tried multiple browsers 
> from multiple client machines, and the behavior is the same in all of them.
> This is probably not a google openid problem, as I am able to use my google 
> openid to sign into other web-sites/applications (e.g. stackoverflow.com).
> 
> so my guess is that the problem is with gerrit, however since other users 
> don't seem to experience the same problem, I assume that it is something in 
> particular in my gerrit user (again - on the server-side, e.g. my record in 
> the gerrit DB got messed up for some reason or something similar).
> 
> if anyone has additional ideas and/or able to help me troubleshoot/solve the 
> problem - it would be greatly appreciated!
> 
> Many thanks in advance.

I can only say that yesterday I had a similar issue - I failed to login
with my fedora id. Instead of researching the problem I switched to my
google id, which worked. My fedora id worked fine several hours later.

So I do not have better ideas than asking your favorite gerrit admin to
link another identetity (yahoo, fedora) to your gerrit account, and hope
that the problem solves itself later on.

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Dropping encryption of database password

[Dan Kenigsberg]

On Tue, Apr 30, 2013 at 03:41:20PM -0400, Alon Bar-Lev wrote:
> Hello,
> 
> Currently we store database password encrypted using 
> org.picketbox.datasource.security.SecureIdentityLoginModule.
> 
> This is reverse encryption with common knowledge shared secret.
> 
> Using encryption with common knowledge shared secret is close to void 
> protection.
> 
> So far we also stored the password as plain text at 
> /etc/ovirt-engine/.pgpass, this is going to be removed as no component 
> actually uses the .pgpass, however we do need to store non-java specific 
> password in for utilities.
> 
> In master (aiming to 3.3), we store the database connection details in own 
> file /etc/ovirt-engine/engine.conf.d/50-setup-database.conf owned by ovirt 
> user and not world readable.
> 
> I would like to use the same 50-setup-database.conf to store plain text 
> password and remove the java specific reversible encrypted password usage.
> 
> Bottom line...
> 1. We drop the .pgpass file.
> 2. We store database connection information in 
> /etc/ovirt-engine/engine.conf.d/ that is readable only by ovirt usage.
> 3. We drop the java specific reversible encryption in favor of plain text.
> 

+1.
Obfuscating passwords only gives a false sense of security.

However, many applications, such Firefox in its signons.sqlite, do that
to avoid revealing the password during a casual browse of the
filesystem.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [vdsm] Snapshots with RAM feature

[Dan Kenigsberg]

On Fri, Apr 19, 2013 at 10:30:19AM +0200, Michal Skrivanek wrote:
> 
> On 11 Apr 2013, at 14:01, Dan Kenigsberg wrote:
> 
> > On Wed, Apr 10, 2013 at 11:43:12AM +0300, Dan Kenigsberg wrote:
> >> On Wed, Apr 10, 2013 at 02:12:20AM -0400, Michal Skrivanek wrote:
> >>> 
> >>> 
> >>> On 9 Apr 2013, at 10:01, Arik Hadas  wrote:
> >>> 
> >>>> Hi All,
> >>>> 
> >>>> The proposed feature will make it possible to run a VM which was 
> >>>> reverted to live snapshot
> >>>> or created from live snapshot with the same memory state as it was at 
> >>>> the moment the live
> >>>> snapshot was taken.
> >>>> 
> >>>> http://www.ovirt.org/Features/RAM_Snapshots
> >>>> 
> >>>> All feedback is welcome!
> >>> Nice!
> >> 
> >> (I prefer to inline the document when discussing it)
> >> 
> >>> VDSM changes
> >>> 
> >>> Default parameter will be added to vmSnapshot verb that maps string to 
> >>> string.
> >>> The map will include two keys for now:
> >>> 'mode' that can be mapped to 'disks_only' or 'disks_memory' to
> >>>indicate if memory state should be saved.
> >>> 'memVol' that will be mapped to a string that represent the two
> >>>  volums that will be used to save the memory state and the
> >>>  VM configuration. The default map will include the
> >>>  mapping of 'mode':'disks_only' only.
> >>> 
> >>>  If the 'mode' value in the map decribed above is
> >>>  'disks_memory' the first volume in 'memVol' will be
> >>>  passed to libvirt in order to dump the memory to
> >>>  it, and the second volume in 'memVol' will be used
> >>>  to save the VM configuration (the same way it is
> >>>  done for hibernate operation).
> >> 
> >> This definition of 'memVol' would not allow saving the state to another
> >> storage domain, or a direct lun or whatever.
> >> 
> >> I suggest that you have two independet arguments, say memVol and
> >> vmConfVol. Both may have the standard pool-domain-image-volume quartet,
> >> or a lun specification, or a local path.
> > 
> > On a second thought - why should we even store vmConf on vmConfVol? Why
> > not keep it in Engine's db?
> you trust the engine db? I don't:-) RAM snapshot absolutely needs to 
> correspond to the actual VM configuration otherwise it can crash the VM or 
> corrupt 
> 
> > Sure, we do this for hibernation. But it creates a lot of inconsistency 
> > pains.
> > Engine sends one vmconf to vdsm, but vdsm ignores it and starts the VM
> > with an old vmconf that was stored besides the hibernation volume. On
> > top of this, it wastes some GiB of disk space for each snapshot.
> Pardon my lack of knowledge in this area, how/where is it wasting that much?

If I recall correctly, Engine has a 1GiB minimum for the size of the LV
that it can create, for one odd reason or another. vmConf takes less
than 1% of that.

> 
> > I think it is time to have Engine keep a snapshot of its vm config
> > whenever it takes a snapshot - live or offline.
> I'm really not sure about the impacts when it goes out of sync. I'd
> rather have the VM resumed and configuration in engine
> updated/overwritten afterwards in first stats update than screwing up
> the VM

I do not know why you trust the Engine DB less than the content of an
LV. Sure, keeping a copy of vmConf near the RAM snapshot, may be useful
in case of a total DB crash, or a manual vm-export. But it duplicates
data, and the fact that it may be possible to do it this way, is not a
good reason to avoid making Engine aware of that vmConf is a snapshot
property, not a vm property.

It is a shame that we cannot add/remove a disk/nic/video card to a VM
without affecting history retroactively.

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [vdsm] ovirt-host-deploy and multible bridges

[Dan Kenigsberg]

On Wed, Apr 10, 2013 at 02:23:14PM +0530, Sahina Bose wrote:
> 
> On 04/10/2013 02:19 PM, Balamurugan Arumugam wrote:
> >On 04/10/2013 11:18 AM, Sahina Bose wrote:
> >>
> >>On 04/10/2013 12:25 AM, Dan Kenigsberg wrote:
> >>>On Tue, Apr 09, 2013 at 07:28:17PM +0530, Balamurugan Arumugam wrote:
> >>>>On 04/09/2013 06:37 PM, Sahina Bose wrote:
> >>>>>Decoding "correct address"  - glusterHostsList should return any
> >>>>>ipAddress that engine knows as being associated with host.
> >>>>>It could be either ipAddress used while adding host
> >>>>>(stored as hostname
> >>>>>in vds_static) or any of the ipAddresses populated in vds_interface
> >>>>>table (addr column) .
> >>>>>I do not have enough knowledge about this bit of code to say what
> >>>>>entries are made in vds_interface table. I know there's an entry for
> >>>>>ovirtmgmt here but not sure if this gets added as part of
> >>>>>addHost flow
> >>>>>or not.
> >>>>>
> >>>>I guess, vds_interface table is populated by ips given by vdsm
> >>>>through getVdsCaps.
> >>>>
> >>>>Current glusterHostsList provides one of ipaddress of the local host
> >>>>(other than 127.*.*.*).   If virbr0 is enabled, it picks up
> >>>>192.168.122.1 ip address of the bridge and sends to the engine, but
> >>>>this entry is missing in the table.
> >>>>
> >>>>The requirement is that we need a ip of the local host which is also
> >>>>stored in the database.
> >>>>
> >>>>The database has entries of ips of a host those are from physical
> >>>>nics and/or bridges who has slaves to nics.
> >>>It's not something I've tested, or want to encourage, but currently,
> >>>outside of gluster, Vdsm may run behind a fancy NAT as a
> >>>virtual server.
> >>>I.e., its local undress may be utterly different from the address used
> >>>by Engine.
> >>>
> >>>I'd like to keep having this flexibility, and not to assume otherwise.
> >>>
> >>>Why does glusterHostsList need to return the ip of the management
> >>>network? The client that issued this verb has to know that IP in the
> >>>first place.
> >>>
> >>>I notice that the idiom "_getLocalIpAddress() or _getGlusterHostName()"
> >>>is used all too often in vdsm/gluster/cli.py.


And what about these ^^^ ?
Is there any reason to keep these guesses elsewhere in the code?

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [vdsm] Snapshots with RAM feature

[Dan Kenigsberg]

On Wed, Apr 10, 2013 at 11:43:12AM +0300, Dan Kenigsberg wrote:
> On Wed, Apr 10, 2013 at 02:12:20AM -0400, Michal Skrivanek wrote:
> > 
> > 
> > On 9 Apr 2013, at 10:01, Arik Hadas  wrote:
> > 
> > > Hi All,
> > > 
> > > The proposed feature will make it possible to run a VM which was reverted 
> > > to live snapshot
> > > or created from live snapshot with the same memory state as it was at the 
> > > moment the live
> > > snapshot was taken.
> > > 
> > > http://www.ovirt.org/Features/RAM_Snapshots
> > > 
> > > All feedback is welcome!
> > Nice!
> 
> (I prefer to inline the document when discussing it)
> 
> > VDSM changes
> >
> >Default parameter will be added to vmSnapshot verb that maps string to 
> > string.
> >The map will include two keys for now:
> >'mode' that can be mapped to 'disks_only' or 'disks_memory' to
> >   indicate if memory state should be saved.
> >'memVol' that will be mapped to a string that represent the two
> > volums that will be used to save the memory state and 
> > the
> > VM configuration. The default map will include the
> > mapping of 'mode':'disks_only' only.
> >
> > If the 'mode' value in the map decribed above is
> > 'disks_memory' the first volume in 'memVol' will be
> > passed to libvirt in order to dump the memory to
> > it, and the second volume in 'memVol' will be used
> > to save the VM configuration (the same way it is
> > done for hibernate operation).
> 
> This definition of 'memVol' would not allow saving the state to another
> storage domain, or a direct lun or whatever.
> 
> I suggest that you have two independet arguments, say memVol and
> vmConfVol. Both may have the standard pool-domain-image-volume quartet,
> or a lun specification, or a local path.

On a second thought - why should we even store vmConf on vmConfVol? Why
not keep it in Engine's db?

Sure, we do this for hibernation. But it creates a lot of inconsistency pains.
Engine sends one vmconf to vdsm, but vdsm ignores it and starts the VM
with an old vmconf that was stored besides the hibernation volume. On
top of this, it wastes some GiB of disk space for each snapshot.

I think it is time to have Engine keep a snapshot of its vm config
whenever it takes a snapshot - live or offline.

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [vdsm] ovirt-host-deploy and multible bridges

[Dan Kenigsberg]

On Wed, Apr 10, 2013 at 02:27:55PM +0530, Balamurugan Arumugam wrote:
> On 04/10/2013 02:23 PM, Sahina Bose wrote:
> >
> >On 04/10/2013 02:19 PM, Balamurugan Arumugam wrote:
> >>On 04/10/2013 11:18 AM, Sahina Bose wrote:
> >>>
> >>>On 04/10/2013 12:25 AM, Dan Kenigsberg wrote:
> >>>>On Tue, Apr 09, 2013 at 07:28:17PM +0530, Balamurugan Arumugam wrote:
> >>>>>On 04/09/2013 06:37 PM, Sahina Bose wrote:
> >>>>>>Decoding "correct address"  - glusterHostsList should return any
> >>>>>>ipAddress that engine knows as being associated with host.
> >>>>>>It could be either ipAddress used while adding host (stored as
> >>>>>>hostname
> >>>>>>in vds_static) or any of the ipAddresses populated in vds_interface
> >>>>>>table (addr column) .
> >>>>>>I do not have enough knowledge about this bit of code to say what
> >>>>>>entries are made in vds_interface table. I know there's an entry for
> >>>>>>ovirtmgmt here but not sure if this gets added as part of addHost
> >>>>>>flow
> >>>>>>or not.
> >>>>>>
> >>>>>I guess, vds_interface table is populated by ips given by vdsm
> >>>>>through getVdsCaps.
> >>>>>
> >>>>>Current glusterHostsList provides one of ipaddress of the local host
> >>>>>(other than 127.*.*.*).   If virbr0 is enabled, it picks up
> >>>>>192.168.122.1 ip address of the bridge and sends to the engine, but
> >>>>>this entry is missing in the table.
> >>>>>
> >>>>>The requirement is that we need a ip of the local host which is also
> >>>>>stored in the database.
> >>>>>
> >>>>>The database has entries of ips of a host those are from physical
> >>>>>nics and/or bridges who has slaves to nics.
> >>>>It's not something I've tested, or want to encourage, but currently,
> >>>>outside of gluster, Vdsm may run behind a fancy NAT as a virtual
> >>>>server.
> >>>>I.e., its local undress may be utterly different from the address used
> >>>>by Engine.
> >>>>
> >>>>I'd like to keep having this flexibility, and not to assume otherwise.
> >>>>
> >>>>Why does glusterHostsList need to return the ip of the management
> >>>>network? The client that issued this verb has to know that IP in the
> >>>>first place.
> >>>>
> >>>>I notice that the idiom "_getLocalIpAddress() or _getGlusterHostName()"
> >>>>is used all too often in vdsm/gluster/cli.py.
> >>>>
> >>>>How about changing the Vdsm/Engine API so that the string
> >>>>"localhost" is
> >>>>returned instead? Then, Engine can replace it with whatever it seems
> >>>>fit.
> >>>>
> >>>>Dan.
> >>>Dan,
> >>>
> >>>Thanks for clarifying. Looks like relying on the IpAddress to determine
> >>>the host will be prone to errors going forward.
> >>>We will change the approach and start using the UUID that gluster peer
> >>>status returns to identify host - will create a new verb glusterPeerList
> >>>that does this.
> >>>
> >>
> >>Current glusterHostsList provides list of
> >>{'hostname': HOSTNAME, 'uuid': UUID, 'status': STATE} including local
> >>host.
> >>
> >>What will be the difference between new glusterPeerList and existing
> >>glusterHostsList?
> >>
> >If this is the case, we just need to make sure at engine we use UUID and
> >not IP address to identify host. We would still need a vdsm verb that
> >will return the current host gluster UUID, to store in engine in case of
> >Add Host flow.
> 
> 
> I think, getVdsCaps can include this.  Dan, is it good idea?

I do not mind adding glusterUUID to this "bag of things".
(preferably impleneting it within the vdsm-gluster subpackage)

I hope Saggi or Adam do not mind making getVdsCaps a little bit more
dirty - they may sagguest that you add a getGlusterUUID verb.

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [vdsm] Snapshots with RAM feature

[Dan Kenigsberg]

On Wed, Apr 10, 2013 at 02:12:20AM -0400, Michal Skrivanek wrote:
> 
> 
> On 9 Apr 2013, at 10:01, Arik Hadas  wrote:
> 
> > Hi All,
> > 
> > The proposed feature will make it possible to run a VM which was reverted 
> > to live snapshot
> > or created from live snapshot with the same memory state as it was at the 
> > moment the live
> > snapshot was taken.
> > 
> > http://www.ovirt.org/Features/RAM_Snapshots
> > 
> > All feedback is welcome!
> Nice!

(I prefer to inline the document when discussing it)

> VDSM changes
>
>Default parameter will be added to vmSnapshot verb that maps string to 
> string.
>The map will include two keys for now:
>'mode' that can be mapped to 'disks_only' or 'disks_memory' to
>   indicate if memory state should be saved.
>'memVol' that will be mapped to a string that represent the two
> volums that will be used to save the memory state and the
> VM configuration. The default map will include the
> mapping of 'mode':'disks_only' only.
>
> If the 'mode' value in the map decribed above is
> 'disks_memory' the first volume in 'memVol' will be
> passed to libvirt in order to dump the memory to
> it, and the second volume in 'memVol' will be used
> to save the VM configuration (the same way it is
> done for hibernate operation).

This definition of 'memVol' would not allow saving the state to another
storage domain, or a direct lun or whatever.

I suggest that you have two independet arguments, say memVol and
vmConfVol. Both may have the standard pool-domain-image-volume quartet,
or a lun specification, or a local path.

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [vdsm] ovirt-host-deploy and multible bridges

[Dan Kenigsberg]

On Tue, Apr 09, 2013 at 07:28:17PM +0530, Balamurugan Arumugam wrote:
> On 04/09/2013 06:37 PM, Sahina Bose wrote:
> >Decoding "correct address"  - glusterHostsList should return any
> >ipAddress that engine knows as being associated with host.
> >It could be either ipAddress used while adding host (stored as hostname
> >in vds_static) or any of the ipAddresses populated in vds_interface
> >table (addr column) .
> >I do not have enough knowledge about this bit of code to say what
> >entries are made in vds_interface table. I know there's an entry for
> >ovirtmgmt here but not sure if this gets added as part of addHost flow
> >or not.
> >
> 
> I guess, vds_interface table is populated by ips given by vdsm
> through getVdsCaps.
> 
> Current glusterHostsList provides one of ipaddress of the local host
> (other than 127.*.*.*).   If virbr0 is enabled, it picks up
> 192.168.122.1 ip address of the bridge and sends to the engine, but
> this entry is missing in the table.
> 
> The requirement is that we need a ip of the local host which is also
> stored in the database.
> 
> The database has entries of ips of a host those are from physical
> nics and/or bridges who has slaves to nics.

It's not something I've tested, or want to encourage, but currently,
outside of gluster, Vdsm may run behind a fancy NAT as a virtual server.
I.e., its local undress may be utterly different from the address used
by Engine.

I'd like to keep having this flexibility, and not to assume otherwise.

Why does glusterHostsList need to return the ip of the management
network? The client that issued this verb has to know that IP in the
first place.

I notice that the idiom "_getLocalIpAddress() or _getGlusterHostName()"
is used all too often in vdsm/gluster/cli.py.

How about changing the Vdsm/Engine API so that the string "localhost" is
returned instead? Then, Engine can replace it with whatever it seems
fit.

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] ovirt-host-deploy and multible bridges

[Dan Kenigsberg]

On Tue, Apr 09, 2013 at 03:55:25PM +0530, Sahina Bose wrote:
> [Adding vdsm-devel]
> 
> On 04/09/2013 03:40 PM, Sahina Bose wrote:
> >Hi all,
> >
> >I'm testing the bootstrapping of host without reboot on Fedora 18. After
> >host's bootstrap,
> >Ifconfig output returns this:
> >
> >ovirtmgmt: flags=4163  mtu 1500
> >  inet 10.70.37.219  netmask 255.255.254.0  broadcast 10.70.37.255
> >
> >
> >virbr0: flags=4099  mtu 1500
> >  inet 192.168.122.1  netmask 255.255.255.0  broadcast
> >192.168.122.255
> > 
> >
> >Running*glusterHostsList*  vdsm verb, returns the ip address
> >192.168.122.1, whereas my host has been added with ip address 10.70.37.219
> >
> >If I reboot the host, the virbr0 bridge is removed, and there's no issue.
> >
> >The vdsm verb glusterHostsList - returns ipAddress of host +
> >output of gluster peer probe. This is needed because a periodic
> >sync job needs to make sure that the hosts added in engine are in
> >sync with the gluster cli (hosts could also be added/removed from
> >gluster cli).
> >
> >How can we make sure glusterHostsList picks the correct ipAddress?

Can you define (in plain English) what is the "correct" address?
The host may have multiple valid addresses (storage, migration, display,
whatnot).

Only when it's clear to us, we can start expressing this in Python.

> >Reading the inetinfo based on bridge has been vetoed as we are
> >doing away with bridges.
> >
> >It would also work if virbr0 was updated in vds_interfaces table.
> >Since this is not happening either - we have an issue.

It might be a valid hack to drop this default virbr0 on vdsm start - not
only the libvirt definition thereof, but also the running kernel device.

However, as expressed above, this would not solve your problem when you
have a currently-running host with multiple addresses.

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [RFC] new power management protocol "libvirtp"

[Dan Kenigsberg]

On Tue, Apr 02, 2013 at 10:15:44AM +0800, Shu Ming wrote:
> Hi,
> 
> In oVirt environment, power manager should be enabled for the host to
> support VM high availability in the cluster. Various kinds of
> physical power management protocol are supported, ie., ALOM, ILO&etc.
> However, when the oVirt is running on a nested KVM
> environment, there is no feasible way to do the power management of the
> VDSM host(also a KVM virtual machine). A new protocol
> will be based on libvirt to achieve the power management of a virtual
> host. The new protocol can be named as "libvirtp".
> 
> In oVirt engine, a new type will be added to
> 
> power management---> type libvirtp
> power management--->address it will be the IP of the physical host where
> the virtual VDSM host is on when "libvirtp" is selected
> power management--->user name it will be the user name to the libvirtd
> service
> power management--->password it will be the password to the libvirtd service
> power management--->port it will be the port to the libvirtd service

Have you looked into fence_virsh or fence_virt ? Don't they provide
what you want?
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [vdsm] Proposal VDSM <=> Engine Data Statistics Retrieval Optimization

[Dan Kenigsberg]

On Sun, Mar 17, 2013 at 10:28:15AM -0400, Ayal Baron wrote:
> 
> 
> - Original Message -
> > On 17/03/13 15:13, Ayal Baron wrote:
> > >
> > > - Original Message -
> > >> On 03/13/2013 11:55 PM, Ayal Baron wrote:
> > >> ...
> > >> The only reason we have this problem is because there is this
> > >> thing against making multiple calls.
> > >>
> > >> Just split it up.
> > >> getVmRuntimeStats() - transient things like mem and cpu%
> > >> getVmInformation() - (semi)static things like disk\networking
> > >> layout
> > >> etc.
> > >> Each updated at different intervals.
> > > +1 on splitting the data up into 2 separate API calls.
> > > You could potentially add a checksum (md5, or any other way) of
> > > the
> > > "static" data to getVmRuntimeStats and not bother even with
> > > polling
> > > the VmInformation if this hasn't changed.  Then you could poll
> > > as
> > > often as you'd like the stats and immediately see if you also
> > > need
> > > to retrieve VmInfo or not (you rarely would).
> >  +1 To Ayal's suggestion
> >  except that instead of the engine hashing the data VDSM sends
> >  the
> >  key which is opaque to the engine.
> >  This can be a local timestap or a generation number.
> > >>> Of course vdsm does the hash, otherwise you'd need to pass all
> > >>> the
> > >>> data to engine which would beat the purpose.
> > >> I thought you meant engine will be sending the hash of previous
> > >> requests
> > >> per VM to vdsm, then vdsm will reply back with vm's removed, vm's
> > >> added,
> > >> and the details for vm's that changed (i.e., engine would be doing
> > >> something like if-modified-since-checksum per vm).
> > >> benefit is reducing a round trip.
> > >> but first would need to split to calls of stats (always changing)
> > >> and
> > >> slowly/never changing data.
> > > If vdms accepts the hash then in your method engine would have to
> > > periodically call getVmInfo(hash).
> > > What I was suggesting is that getVmStats would return vmInfo hash
> > > so that we could avoid calling getVmInfo altogether.
> > > The stats *always* change so there is no need for checking if that
> > > info has changed.
> > > What we could do is avoid the split into 2 verbs by calling
> > > getVmStats(hash) and then have getVmStats return everything if the
> > > hash has changed or only the stats if it hasn't.  This would be
> > > the least number of roundtrips and avoid the split.  If you don't
> > > pass a hash it would return everything so this way it's also fully
> > > backward compatible.
> > 
> > For the 'static' data, why is there a need for a hash?
> > If VDSM sends in each update a timestamp, can't RHEVM just use
> > if-modified-since with the last timestamp it got from VDSM?
> > Is it cheaper for VDSM to calculate the hash, than update the
> > timestamp
> > per change in any of the fields? It doesn't really need to update the
> > timestamp per change, only for the first change since last update
> > sent
> > actually (so 'dirty' flag in a way, to signify data that RHEVM hasn't
> > seen yet).
> > Y.
> 
> As Saggi mentioned: "VDSM sends the key which is opaque to the engine. This 
> can be a local timestap or a generation number."
> 
> The content doesn't matter, what matters is that it has changed.
> timestamp assumes that vdsm will track changes and send only delta.
> Although possible this would be an overkill (for every value in the
> dict you'd have to hold a timestamp of last change and send only those
> which have changed since the timestamp which was passed by the user).

If we're in the spirit of quoting Saggi, this suggestion is not
compatible with "...mak[ing] the return value differ according to input
... is a big no no when talking about type safe APIs.".

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] VmDynamic kvm_enable

[Dan Kenigsberg]

On Wed, Mar 13, 2013 at 12:58:44AM +0200, Itamar Heim wrote:
> On 03/12/2013 05:07 PM, Dan Kenigsberg wrote:
> >On Tue, Mar 12, 2013 at 10:37:41AM -0400, Laszlo Hornyak wrote:
> >>Hi,
> >>
> >>I came across a VmDynamic property 'kvm_enable'. It sounds strange for me, 
> >>because ovirt is very colsely integrated with kvm. So a short dig into this 
> >>flag...
> >>There is a similar thing for the VDS, it is set to true by vdsm is the host 
> >>CPU has a VT flag. It is actually used to check if the host is OK to run 
> >>VMS.
> >>
> >>But the one for Vm it looks like a distributed logical loop in vdsm: it is 
> >>set when constructing a VM object (vdsm/vm.py:~343) from the data sent by 
> >>the client (engine) and then reported back in vm stats, so it is just a 
> >>roundtrip between vdsm and it's client.
> >>In the engine side, it is just keeps sending it between the frontend DB and 
> >>vdsm, never part of a decision.
> >>
> >>Is this still needed here? Can I remove?
> >>VDSM guys?
> >
> >I have a vague memeory that once upon at time, qemu occasionally failed
> >to enable kvm support - even though it was asked to. I then silently
> >switched to emulated mode, which was grindingly slow. Engine wanted to
> >know about such occasions.
> >
> >I believe that a better technical approach would have been to kill the
> >violating process, and not let it run at all (unless qemu emulation was
> >strictly requested by management).
> >
> >Anyway, as you have noted, this has rotten away throuh the years, and
> >unless older Engine versions are expecting this value in any way, I am
> >all for dropping it.
> 
> I think this was about something else - the kvmEnable flag was used
> to launch installations of guests not correctly supported by kvm
> that required real mode or something like that.

You are now speaking about kvmEnable sent by Engine to Vdsm. Laszlo was
speaking about a supposedly-dynamically-changing kvmEnable that is
reported back, in getVmStats.

> hopefully not relevant any more, but need to validate won't break
> older engines indeed.

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] VmDynamic kvm_enable

[Dan Kenigsberg]

On Tue, Mar 12, 2013 at 10:37:41AM -0400, Laszlo Hornyak wrote:
> Hi,
> 
> I came across a VmDynamic property 'kvm_enable'. It sounds strange for me, 
> because ovirt is very colsely integrated with kvm. So a short dig into this 
> flag...
> There is a similar thing for the VDS, it is set to true by vdsm is the host 
> CPU has a VT flag. It is actually used to check if the host is OK to run VMS.
> 
> But the one for Vm it looks like a distributed logical loop in vdsm: it is 
> set when constructing a VM object (vdsm/vm.py:~343) from the data sent by the 
> client (engine) and then reported back in vm stats, so it is just a roundtrip 
> between vdsm and it's client.
> In the engine side, it is just keeps sending it between the frontend DB and 
> vdsm, never part of a decision.
> 
> Is this still needed here? Can I remove?
> VDSM guys?

I have a vague memeory that once upon at time, qemu occasionally failed
to enable kvm support - even though it was asked to. I then silently
switched to emulated mode, which was grindingly slow. Engine wanted to
know about such occasions.

I believe that a better technical approach would have been to kill the
violating process, and not let it run at all (unless qemu emulation was
strictly requested by management).

Anyway, as you have noted, this has rotten away throuh the years, and
unless older Engine versions are expecting this value in any way, I am
all for dropping it.

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [vdsm] Proposal VDSM <=> Engine Data Statistics Retrieval Optimization

[Dan Kenigsberg]

On Fri, Mar 08, 2013 at 10:21:27AM +0800, Mark Wu wrote:
> On 03/07/2013 07:25 PM, Vinzenz Feenstra wrote:
> >Please find the prettier version on the wiki: 
> >http://www.ovirt.org/Proposal_VDSM_-_Engine_Data_Statistics_Retrieval
> >
> >
> >  Proposal VDSM - Engine Data Statistics Retrieval
> >
> >
> >VDSM <=> Engine data retrieval optimization
> >
> >
> >  Motivation:
> >
> >Currently the RHEVM engine is polling the a lot of data from VDSM
> >every 15 seconds. This should be optimized and the amount of data
> >requested should be more specific.
> >
> If the data size really matters,  we could also consider to pack the
> information into binary.  I am not sure if it's suitable in the
> transmission of  XMLRPC.

I do not think we should embed binary in XMLRPC. I'd consider
compressing the data at the transport layer - but that would be a
completely deferent feature.

> >
> >For each VM the data currently contains much more information than
> >actually needed which blows up the size of the XML content quite
> >big. We could optimize this by splitting the reply on the
> >getVmStats based on the request of the engine into sections. For
> >this reason Omer Frenkel and me have split up the data into parts
> >based on their usage.
> >
> >This data can and usually does change during the lifetime of the VM.
> >
> >
> >Rarely Changed:
> >
> >This data is change not very frequent and it should be enough to
> >update this only once in a while. Most commonly this data changes
> >after changes made in the UI or after a migration of the VM to
> >another Host.
> >
> >*Status*  = Running
> >*acpiEnable*  = true
> >*vmType*  = kvm
> >*guestName*  = W864GUESTAGENTT
> >*displayType*  = qxl
> >*guestOs*  = Win 8
> >*kvmEnable*  = true #/*this should be constant and never changed*/
> Then it should be removed from vm stats. In my opinion, any
> information belongs to vm's static configuration, it shouldn't be
> included in vm stats. For the fields above, except 'Status',  engine
> can get the information without querying the vdsm host. It could not
> be changed by vdsm itself, right?

actually, guestName and guestOs may change - for example by installing
Linux on that Windows guest.

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [vdsm] Proposal VDSM <=> Engine Data Statistics Retrieval Optimization

[Dan Kenigsberg]

On Fri, Mar 08, 2013 at 09:37:03AM +0100, Vinzenz Feenstra wrote:

> >>>
> >>>
> >>>  Improvement of the Guest Agent:
> >>>
> >>>As part of the proposed solution it is necessary to improve the
> >>>guest agent as well.
> >>Improving the agent may be a good idea, but I do not see the necessity
> >>in it.
> The guest agent is doing 'expensive' queries (e.g.
> "application_list") way too often. And things like network
> interfaces, disk usage and installed applications won't usually
> change every n minutes.
> Those queries could be much more reactive then proactive.

Of course, but as I said:

> >>It's also important to improve the horrible multithreaded
> >>vdsm/libvirt statistics acquisition, but just as unrelated to the core
> >>of this feature.

I think it is misleading to include this in the discussion about
vdsm/Engine interface change.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Proposal VDSM <=> Engine Data Statistics Retrieval Optimization

[Dan Kenigsberg]

On Thu, Mar 07, 2013 at 12:25:54PM +0100, Vinzenz Feenstra wrote:
> Please find the prettier version on the wiki:
> http://www.ovirt.org/Proposal_VDSM_-_Engine_Data_Statistics_Retrieval
> 
> 
>  Proposal VDSM - Engine Data Statistics Retrieval
> 
> 
>VDSM <=> Engine data retrieval optimization
> 
> 
>  Motivation:
> 
> Currently the RHEVM engine is polling the a lot of data from VDSM
> every 15 seconds. This should be optimized and the amount of data
> requested should be more specific.

It feels like a good idea, but do you have numbers? How much traffic
would be saved? Remember the added computation incurred on each host -
there's always a price to pay.

> 
> For each VM the data currently contains much more information than
> actually needed which blows up the size of the XML content quite
> big. We could optimize this by splitting the reply on the getVmStats
> based on the request of the engine into sections. For this reason
> Omer Frenkel and me have split up the data into parts based on their
> usage.
> 
> This data can and usually does change during the lifetime of the VM.
> 
> 
>Rarely Changed:
> 
> This data is change not very frequent and it should be enough to
> update this only once in a while. Most commonly this data changes
> after changes made in the UI or after a migration of the VM to
> another Host.
> 
>*Status*  = Running

Status does not change much, but when it does, it is important to report
that quickly.

>*acpiEnable*  = true
>*vmType*  = kvm
>*guestName*  = W864GUESTAGENTT
>*displayType*  = qxl
>*guestOs*  = Win 8
>*kvmEnable*  = true #/*this should be constant and never changed*/
>*pauseCode*  = NOERR
>*monitorResponse*  = 0
>*session*  = Locked # unused
>*netIfaces*  = [{'name': 'Realtek RTL8139C+ Fast Ethernet NIC', 'inet6':  
> ['fe80::490c:92bb:bbcc:9f87'], 'inet': ['10.34.60.148'], 'hw': 
> '00:1a:4a:22:3c:db'}]
>*appsList*  = ['RHEV-Tools 3.2.4', 'RHEV-Agent64 3.2.3', 'RHEV-Serial64 
> 3.2.3', 'RHEV-Network64 3.2.2', 'RHEV-Network64 3.2.3', 'RHEV-Block64 3.2.3', 
> 'RHEV-Balloon64 3.2.3', 'RHEV-Balloon64 3.2.2', 'RHEV-Agent64 3.2.2', 
> 'RHEV-USB 3.2.3', 'RHEV-Block64 3.2.2', 'RHEV-Serial64 3.2.2']
>*pid*  = 11314
>*guestIPs*  = 10.34.60.148 # duplicated info
>*displayIp*  = 0
>*displayPort*  = 5902
>*displaySecurePort*  = 5903
>*username*  = user@W864GUESTAGENTT
>*clientIp*  =
>*lastLogin*  = 1361976900.67
> 
> 
>Often Changed:
> 
> This data is changed quite often however it is not necessary to
> update this data every 15 seconds. As this is cumulative data and
> reflects the current status, and it does not need to be snapshotted
> every 15 seconds to retrieve statistics. The data can be retrieved
> in much more generous time slices. (e.g. Every 5 minutes)
> 
>*network*  = {'vnet1': {'macAddr': '00:1a:4a:22:3c:db', 'rxDropped': '0', 
> 'txDropped': '0', 'rxErrors': '0', 'txRate': '0.0', 'rxRate': '0.0', 
> 'txErrors': '0', 'state': 'unknown', 'speed': '100', 'name': 'vnet1'}}
>*disksUsage*  = [{'path': 'c:\\', 'total': '64055406592', 'fs': 'NTFS', 
> 'used': '19223846912'}, {'path': 'd:\\', 'total': '3490912256', 'fs': 'UDF', 
> 'used': '3490912256'}]
>*timeOffset*  = 14422
>*elapsedTime*  = 68591
>*hash*  = 2335461227228498964
>*statsAge*  = 0.09 # unused
> 
> 
>Often Changed but unused
> 
> This data does not seem to be used in the engine at all. It is *not*
> even used in the data warehouse.
> 
>*memoryStats*  = {'swap_out': '0', 'majflt': '0', 'mem_free': '1466884', 
> 'swap_in': '0', 'pageflt': '0', 'mem_total': '2096736', 'mem_unused': 
> '1466884'}
>*balloonInfo*  = {'balloon_max': 2097152, 'balloon_cur': 2097152}
>*disks*  = {'vda': {'readLatency': '0', 'apparentsize': '64424509440', 
> 'writeLatency': '1754496',  'imageID': 
> '28abb923-7b89-4638-84f8-1700f0b76482', 'flushLatency': '156549',  
> 'readRate': '0.00', 'truesize': '18855059456', 'writeRate': '952.05'}, 'hdc': 
> {'readLatency': '0', 'apparentsize': '0', 'writeLatency': '0', 
> 'flushLatency': '0', 'readRate': '0.00', 'truesize': '0', 'writeRate': 
> '0.00'}}

I am pretty sure that {read,write,flush}Latency is collected and
reported by Engine. `git grep writeLatency` reinforces my vague memory.
> 
> 
>Very frequent uppdates needed by webadmin portal:
> 
> This data is mostly needed for the webadmin portal and might be
> required to be updated quite often. An exception here is the
> statsAge field, which seems to be unused by the Engine. This data
> could be requested every 15 seconds to keep things as they are now.
> 
>*cpuSys*  = 2.32
>*cpuUser*  = 1.34
>*memUsage*  = 30
> 
> 
>Proposed Solution for VDSM & Engine:
> 
> We will introduce new optional parameters to getVmStats,
> getAllVmStats and list to allow a finer grained specification of
> data which should be included.
> 
> *Parameter:* *statsType*=/**/ (getVmStats, getAllVmStat

Re: [Engine-devel] fedora openid authentication for gerrit is broken

[Dan Kenigsberg]

On Wed, Mar 06, 2013 at 09:55:45AM +0100, Alexander Rydekull wrote:
> Itamar, I think Vered summarize it quite perfectly in a parallell thread:
> http://lists.ovirt.org/pipermail/infra/2013-March/002314.html
> 
> He was also kind enough to open a ticket on the issue. Could you look into
> it?

I wonder if our friendly gerrit.ovirt.org dba could add the new url
 https://danken.id.fedoraproject.org/
for every user with the old one, so that people lacking the new one can
keep on working? (/me not included, I have both urls)

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Changing Gerrit -1 message

[Dan Kenigsberg]

On Tue, Feb 19, 2013 at 11:54:56AM +0200, Yaniv Kaul wrote:
> On 19/02/13 11:51, Ofer Schreiber wrote:
> >I feel that the current "-1 I would prefer that you didn't submit this" 
> >message in Gerrit is pretty rude, as usually those -1 reviews are just small 
> >fix-ups in the code itself.
> 
> I think 'I would prefer that you didn't' is fine, I'm not sure why
> 'submit this' and not 'merge this'.

Gerrit uses the "submit" term for "taking a change into a branch"
elsewhere, too. Maybe because "merge" is something else in gittish.

> Y.
> 
> >
> >Any thoughts about a more suitable "-1" message?
> >I thought about "-1 Please fix your code" or something similar.

I don't care much about the text, but many people told me that "I would
prefer that you didn't submit this" sounds condensding. So a change is
in place, and I do not mind your suggestion, Ofer.

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] vdsm/zombiereaper

[Dan Kenigsberg]

On Mon, Feb 18, 2013 at 04:21:16PM -0600, Dead Horse wrote:
> Any movement or thoughts yet on this one: http://gerrit.ovirt.org/#/c/11492/
> 
> It still has vdsm master broken.

Guys, I think that DHC is right - we cannot keep deliberating forever.
Having a process leak is bad, but keeping master broken for so long is
even worse.

So unless anyone violently objects, I'll revert the offending patch.

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Current oVirt 3.2 blockers

[Dan Kenigsberg]

On Wed, Feb 06, 2013 at 02:17:04PM +0200, Moran Goldboim wrote:
> attached is the list of current blockers as appears on version
> tracker bug [1]
> maintainers please see that the below issues are fixed (unless not
> really blockers).
> users, if there are any additional blockers you are facing - let's
> discuss it in the meeting today.
> 
> network
> ===
> Bug IDProductWhiteboardComponentAssignee Status
> ResolutionSummary
> 906289oVirtnetworkovirt-engine-webadmin
> alkap...@redhat.comPOST---[oVirt-webadmin] [network]
> Non-VM networks shown as VM networks on cluster attachment dialog
> 906291oVirtnetworkovirt-engine-webadmin lp...@redhat.com
> POST---[oVirt-webadmin] [network] Non-VM networks not being
> detached from cluster

These two bugs have patches, needing only merge to master (and later to
ovirt-3.2). Alona, could you take care of them asap?

> 906383oVirtnetworkvdsmdan...@redhat.comPOST ---
> [vdsm] [setupNetworks] Error while attaching non-VM network to
> interface on Fedora 18

Already pushed to ovirt-3.2 branch, would be in next vdsm build.

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [vdsm] CPU Overcommit Feature

[Dan Kenigsberg]

On Thu, Dec 20, 2012 at 10:17:50AM -0500, Doron Fediuck wrote:
> 
> 
> - Original Message -
> > From: "Simon Grinberg" 
> > To: "Doron Fediuck" 
> > Cc: "engine-devel" , vdsm-de...@fedorahosted.org
> > Sent: Thursday, December 20, 2012 4:56:14 PM
> > Subject: Re: [Engine-devel] [vdsm]  CPU Overcommit Feature
> > 
> > 
> > 
> > - Original Message -
> > > From: "Doron Fediuck" 
> > > To: "Dan Kenigsberg" 
> > > Cc: "engine-devel" ,
> > > vdsm-de...@fedorahosted.org
> > > Sent: Thursday, December 20, 2012 2:14:45 PM
> > > Subject: Re: [Engine-devel] [vdsm]  CPU Overcommit Feature
> > > 
> > > 
> > > 
> > > - Original Message -
> > > > From: "Dan Kenigsberg" 
> > > > To: "Itamar Heim" 
> > > > Cc: "engine-devel" ,
> > > > vdsm-de...@fedorahosted.org
> > > > Sent: Thursday, December 20, 2012 11:55:10 AM
> > > > Subject: Re: [Engine-devel] [vdsm]  CPU Overcommit Feature
> > > > 
> > > > On Thu, Dec 20, 2012 at 10:23:55AM +0200, Itamar Heim wrote:
> > > > > On 12/20/2012 09:43 AM, Dan Kenigsberg wrote:
> > > > > >On Wed, Dec 19, 2012 at 09:53:15AM -0500, Doron Fediuck wrote:
> > > > > >>
> > > > > >>- Original Message -
> > > > > >>>From: "Dan Kenigsberg" 
> > > > > >>>To: "Greg Padgett" 
> > > > > >>>Cc: "engine-devel" ,
> > > > > >>>vdsm-de...@fedorahosted.org
> > > > > >>>Sent: Wednesday, December 19, 2012 3:59:11 PM
> > > > > >>>Subject: Re: [Engine-devel] CPU Overcommit Feature
> > > > > >>>
> > > > > >>>On Mon, Dec 17, 2012 at 09:37:57AM -0500, Greg Padgett
> > > > > >>>wrote:
> > > > > >>>>Hi,
> > > > > >>>>
> > > > > >>>>I've been working on a feature to allow CPU Overcommitment
> > > > > >>>>of
> > > > > >>>>hosts
> > > > > >>>>in a cluster.  This first stage allows the engine to
> > > > > >>>>consider
> > > > > >>>>host
> > > > > >>>>cpu threads as cores for the purposes of VM resource
> > > > > >>>>allocation.
> > > > > >>>>
> > > > > >>>>This wiki page has further details, your comments are
> > > > > >>>>welcome!
> > > > > >>>>http://www.ovirt.org/Features/cpu_overcommit
> > > > > >>>
> > > > > >>>I've commented about the vdsm/engine API on
> > > > > >>>http://gerrit.ovirt.org/#/c/10144/ but it is probably better
> > > > > >>>to
> > > > > >>>reiterate it here.
> > > > > >>>
> > > > > >>>The suggested API is tightly coupled with an ugly hack we
> > > > > >>>pushed
> > > > > >>>to
> > > > > >>>vdsm
> > > > > >>>in order not to solve the issue properly on the first
> > > > > >>>strike.
> > > > > >>>
> > > > > >>>If we had not have report_host_threads_as_cores, I think
> > > > > >>>we'd
> > > > > >>>have a
> > > > > >>>simpler API reporting only cpuThreads and cpuCores; with no
> > > > > >>>funny
> > > > > >>>boolean flags.
> > > > > >>>
> > > > > >>>Let us strive to that position as much as we can.
> > > > > >>>
> > > > > >>>How about asking whoever used report_host_threads_as_cores
> > > > > >>>to
> > > > > >>>unset
> > > > > >>>it
> > > > > >>>once they install Engine 3.2 ? I think that these are very
> > > > > >>>few
> > > > > >>>people,
> > > > > >>>that would not mind this very much.
> > > > > >>>
> > > > > >>>If this is impossible, I'd add a cpuCores2, always 

Re: [Engine-devel] [vdsm] CPU Overcommit Feature

[Dan Kenigsberg]

On Thu, Dec 20, 2012 at 10:23:55AM +0200, Itamar Heim wrote:
> On 12/20/2012 09:43 AM, Dan Kenigsberg wrote:
> >On Wed, Dec 19, 2012 at 09:53:15AM -0500, Doron Fediuck wrote:
> >>
> >>- Original Message -
> >>>From: "Dan Kenigsberg" 
> >>>To: "Greg Padgett" 
> >>>Cc: "engine-devel" , vdsm-de...@fedorahosted.org
> >>>Sent: Wednesday, December 19, 2012 3:59:11 PM
> >>>Subject: Re: [Engine-devel] CPU Overcommit Feature
> >>>
> >>>On Mon, Dec 17, 2012 at 09:37:57AM -0500, Greg Padgett wrote:
> >>>>Hi,
> >>>>
> >>>>I've been working on a feature to allow CPU Overcommitment of hosts
> >>>>in a cluster.  This first stage allows the engine to consider host
> >>>>cpu threads as cores for the purposes of VM resource allocation.
> >>>>
> >>>>This wiki page has further details, your comments are welcome!
> >>>>http://www.ovirt.org/Features/cpu_overcommit
> >>>
> >>>I've commented about the vdsm/engine API on
> >>>http://gerrit.ovirt.org/#/c/10144/ but it is probably better to
> >>>reiterate it here.
> >>>
> >>>The suggested API is tightly coupled with an ugly hack we pushed to
> >>>vdsm
> >>>in order not to solve the issue properly on the first strike.
> >>>
> >>>If we had not have report_host_threads_as_cores, I think we'd have a
> >>>simpler API reporting only cpuThreads and cpuCores; with no funny
> >>>boolean flags.
> >>>
> >>>Let us strive to that position as much as we can.
> >>>
> >>>How about asking whoever used report_host_threads_as_cores to unset
> >>>it
> >>>once they install Engine 3.2 ? I think that these are very few
> >>>people,
> >>>that would not mind this very much.
> >>>
> >>>If this is impossible, I'd add a cpuCores2, always reporting the true
> >>>number, to be used by new Engines. We may even report it only on the
> >>>very few cases of report_host_threads_as_cores being set.
> >>>
> >>>Dan.
> >>
> >>Hi Dan,
> >>Thanks for the review.
> >>
> >>I agree simply reporting cores and threads would be the right solution.
> >>However, when you have hyperthreading turned off you get cores=threads.
> >>This is the same situation you have when hyperthreading turned on, and
> >>someone used the vdsm configuration of reporting threads as cores.
> >>
> >>So the engine won't know the real status of the host.
> >
> >This is not surprising, as report_host_threads_as_cores means in blunt
> >English "lie to Engine about the number of cores". The newly suggested
> >flag says "don't believe what I said in cpuCores, since I'm lying". Next
> >thing we'd have is another flag saying that the former flag was a lie,
> >and cpuCores is actually trustworthy.
> >
> >Instead of dancing this dance, I suggest to stop lying.
> >
> >report_host_threads_as_cores was a hack to assist a older Engine
> >versions. Engine users that needed it had to set it out-of-band on their
> >hosts. Now if they upgrade their Engine, they can -- as easily -- reset
> >that value.
> >
> >If they forget, nothing devastating happens beyond Engine assuming that
> >hyperthreading is off.
> >
> >Please consider this suggestion. I find it the simplest for all involved
> >parties.
> 
> the only problem is the new vdsm doesn't know which engine may be using it.
> if engine would say "getVdsCaps engineVersion=3.2", then vdsm could
> know engine no longer needs lying to and ignore the flag, re-using
> same field.

Note that I do not suggest to drop report_host_threads_as_cores now.
I am suggesting to keep on lying even to new Engine.
If someone thinks that lying is bad, he should reset
report_host_threads_as_cores.

It seems to me that the suggested API is being coerced by a very limited
use case, that is not going to be really harmed by a straight-forward API.

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] CPU Overcommit Feature

[Dan Kenigsberg]

On Wed, Dec 19, 2012 at 09:53:15AM -0500, Doron Fediuck wrote:
> 
> - Original Message -
> > From: "Dan Kenigsberg" 
> > To: "Greg Padgett" 
> > Cc: "engine-devel" , vdsm-de...@fedorahosted.org
> > Sent: Wednesday, December 19, 2012 3:59:11 PM
> > Subject: Re: [Engine-devel] CPU Overcommit Feature
> > 
> > On Mon, Dec 17, 2012 at 09:37:57AM -0500, Greg Padgett wrote:
> > > Hi,
> > > 
> > > I've been working on a feature to allow CPU Overcommitment of hosts
> > > in a cluster.  This first stage allows the engine to consider host
> > > cpu threads as cores for the purposes of VM resource allocation.
> > > 
> > > This wiki page has further details, your comments are welcome!
> > > http://www.ovirt.org/Features/cpu_overcommit
> > 
> > I've commented about the vdsm/engine API on
> > http://gerrit.ovirt.org/#/c/10144/ but it is probably better to
> > reiterate it here.
> > 
> > The suggested API is tightly coupled with an ugly hack we pushed to
> > vdsm
> > in order not to solve the issue properly on the first strike.
> > 
> > If we had not have report_host_threads_as_cores, I think we'd have a
> > simpler API reporting only cpuThreads and cpuCores; with no funny
> > boolean flags.
> > 
> > Let us strive to that position as much as we can.
> > 
> > How about asking whoever used report_host_threads_as_cores to unset
> > it
> > once they install Engine 3.2 ? I think that these are very few
> > people,
> > that would not mind this very much.
> > 
> > If this is impossible, I'd add a cpuCores2, always reporting the true
> > number, to be used by new Engines. We may even report it only on the
> > very few cases of report_host_threads_as_cores being set.
> > 
> > Dan.
> 
> Hi Dan,
> Thanks for the review.
> 
> I agree simply reporting cores and threads would be the right solution.
> However, when you have hyperthreading turned off you get cores=threads.
> This is the same situation you have when hyperthreading turned on, and
> someone used the vdsm configuration of reporting threads as cores.
> 
> So the engine won't know the real status of the host.

This is not surprising, as report_host_threads_as_cores means in blunt
English "lie to Engine about the number of cores". The newly suggested
flag says "don't believe what I said in cpuCores, since I'm lying". Next
thing we'd have is another flag saying that the former flag was a lie,
and cpuCores is actually trustworthy.

Instead of dancing this dance, I suggest to stop lying.

report_host_threads_as_cores was a hack to assist a older Engine
versions. Engine users that needed it had to set it out-of-band on their
hosts. Now if they upgrade their Engine, they can -- as easily -- reset
that value.

If they forget, nothing devastating happens beyond Engine assuming that
hyperthreading is off.

Please consider this suggestion. I find it the simplest for all involved
parties.

Dan.

> We need to be
> able to tell the difference. So this moves us to cpuCores2 suggestion.
> This is one possibility (cpuRealCores?), and the alternative is an
> indication of vdsm config (true/false) which may be removed in the future.
> I suspect over time cpu and cpu2 will confuse people.
> 
> So I'd suggest having the boolean and removing it along with the vdsm 
> configuration in the next ovirt version.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] CPU Overcommit Feature

[Dan Kenigsberg]

On Mon, Dec 17, 2012 at 09:37:57AM -0500, Greg Padgett wrote:
> Hi,
> 
> I've been working on a feature to allow CPU Overcommitment of hosts
> in a cluster.  This first stage allows the engine to consider host
> cpu threads as cores for the purposes of VM resource allocation.
> 
> This wiki page has further details, your comments are welcome!
> http://www.ovirt.org/Features/cpu_overcommit

I've commented about the vdsm/engine API on
http://gerrit.ovirt.org/#/c/10144/ but it is probably better to
reiterate it here.

The suggested API is tightly coupled with an ugly hack we pushed to vdsm
in order not to solve the issue properly on the first strike.

If we had not have report_host_threads_as_cores, I think we'd have a
simpler API reporting only cpuThreads and cpuCores; with no funny
boolean flags.

Let us strive to that position as much as we can.

How about asking whoever used report_host_threads_as_cores to unset it
once they install Engine 3.2 ? I think that these are very few people,
that would not mind this very much.

If this is impossible, I'd add a cpuCores2, always reporting the true
number, to be used by new Engines. We may even report it only on the
very few cases of report_host_threads_as_cores being set.

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [vdsm] VDSM tasks, the future

[Dan Kenigsberg]

On Wed, Dec 05, 2012 at 09:45:48AM -0500, Saggi Mizrahi wrote:
> I'm sorry but your email client messed up the formatting and I can't figure 
> out what are you comments.
> Could you please use text only emails.

It's funny to hear this form someone that uses top-posting with long
lines.

But that's not the reason of my confusion. I know very well that the
current task framework in vdsm is an over complicated mess. I believe
that there is only one living person that understands
http://wiki.ovirt.org/File:Vdsmtasks.jpg, and it is not me. However, I
do not know what you want to achieve in the future.

You detail what tasks are not going to give me: no persistence, no
cleanup, no synchronization. Do they give me a fake progress bar
percentage?

When Engine requests to run a new VM, it calls the vmCreate verb, which
returns immediately. Engine then polls the VM until it leaves its
initial waitForLaunch state. I've daydreamed about changing this to
something more generic, using the task framework - but I do not see how
your new vision helps to achieve this.

Could you provide some guidance on your vision, with this use case in
mind?

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Request power-user status for mom project in jenkins

[Dan Kenigsberg]

On Tue, Dec 11, 2012 at 11:24:07AM -0500, Eyal Edri wrote:
> [adding engine-devel]
> 
> usually there is an official vote on each new power user for jenkins,
> and as long as there are no objections it is approved.
> 
> since i know you're a mom maintainer and familiar with jenkins, i vote +1.

I trust Adam (who is mom's maintainer), and trust that he is capable of
becoming familiar with jenkins without causing pain to other users.

+1
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] host cpu feature

[Dan Kenigsberg]

On Wed, Dec 05, 2012 at 04:05:00PM +0200, Yaniv Kaul wrote:
> On 12/05/2012 03:55 PM, Laszlo Hornyak wrote:
> >
> 
> The nice thing about hostModel (unlike hostPassthrough) is that
> once
> we
> created the VM we can migrate it to stronger hosts, and back to
> the
> original host. I suppose that it complicates the scheduler.
> >>>Yes with host-model you get the features that libvirt handles. In
> >>>such cases the engine could decide, if you want this
> >>>functionality. Well the scheduler architecture is just being
> >>>reinvented.
> >>>
> >>>For the host-passthrough, I think the AllowMigrateCPUHost
> >>>configuration option would be a simple decision for the
> >>>administrator: set it to true if all hosts are uniform.

If it is THAT simple, Engine could take this decision without human
intervension.

> >>>If it is
> >>>not set to true, then we will not allow migration of such VMs.
> >>That's not what I understood from libvirt's documentation. I
> >You may be right, could you send an URL to that point of the documentation 
> >or copy-paste?
> 
> The link I followed from your feature page:
> http://libvirt.org/formatdomain.html#elementsCPU :
> 
> host-model
> The host-model mode is essentially a shortcut to copying host CPU
> definition from capabilities XML into domain XML. Since the CPU
> definition is copied just before starting a domain, exactly the same
> XML can be used on different hosts while still providing the best
> guest CPU each host supports. Neither match attribute nor any
> feature elements can be used in this mode. Specifying CPU model is
> not supported either, but model's fallback attribute may still be
> used. Libvirt does not model every aspect of each CPU so the guest
> CPU will not match the host CPU exactly. On the other hand, the ABI
> provided to the guest is reproducible. During migration, complete
> CPU model definition is transferred to the destination host so the
> migrated guest will see exactly the same CPU model even if the
> destination host contains more capable CPUs for the running instance
> of the guest; but shutting down and restarting the guest may present
> different hardware to the guest according to the capabilities of the
> new host.
> host-passthrough
> With this mode, the CPU visible to the guest should be exactly the
> same as the host CPU even in the aspects that libvirt does not
> understand. Though the downside of this mode is that the guest
> environment cannot be reproduced on different hardware. Thus, if you
> hit any bugs, you are on your own.

That's exactly where AllowMigrateCPUHost fits well: when a user ticks
this for a cluster he says "yeah, I like to be on my own."
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] host cpu feature

[Dan Kenigsberg]

On Wed, Dec 05, 2012 at 08:39:52AM -0500, Laszlo Hornyak wrote:
> 
> 
> - Original Message -
> > From: "Dan Kenigsberg" 
> > To: "Laszlo Hornyak" 
> > Cc: "Yaniv Kaul" , "engine-devel" 
> > Sent: Wednesday, December 5, 2012 1:55:19 PM
> > Subject: Re: [Engine-devel] host cpu feature
> > 
> > On Wed, Dec 05, 2012 at 06:46:09AM -0500, Laszlo Hornyak wrote:
> > > 
> > > 
> > > - Original Message -
> > > > From: "Yaniv Kaul" 
> > > > To: "Laszlo Hornyak" 
> > > > Cc: "engine-devel" 
> > > > Sent: Wednesday, December 5, 2012 12:23:47 PM
> > > > Subject: Re: [Engine-devel] host cpu feature
> > > > 
> > > > On 12/05/2012 12:32 PM, Laszlo Hornyak wrote:
> > > > > Hi,
> > > > >
> > > > > CPU-Host support allows the virtual machines to see and utilize
> > > > > the
> > > > > host's CPU flags, this enables better performance in VM's, at
> > > > > the
> > > > > price of worse portablity.
> > > > >
> > > > > http://www.ovirt.org/Features/Cpu-host_Support
> > > > >
> > > > > Your feedback is welcome!
> > > > >
> > > > > Thank you,
> > > > > Laszlo
> > > > > ___
> > > > > Engine-devel mailing list
> > > > > Engine-devel@ovirt.org
> > > > > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > > > 
> > > > - I assume that when you allow migration, you'd use host-model?
> > > > This
> > > > is
> > > > not clear from the design. It seems like we VDSM developers can
> > > > choose
> > > > to use either this or passthrough, while in practice we should
> > > > support both.
> > 
> > I join Kaul's question: it is an ovirt-level question whether
> > hostPassthrough or hostModel or both should be supported. It should
> > not
> > be a unilateral vdsm decision.
> 
> Ah, possibly misunderstanding, I did not mean that VDSM should decide whether 
> to use host-passthrough or host-model. The engine should decide.
> I meant _you_ should decide which version of vdsm api modification do you 
> want :)
> 
> > 
> > > 
> > > If AllowMigrateCPUHost is set to true (in case you have the same
> > > cpu model everywhere in your DC) migration of such hsots will be
> > > enabled. Otherwise it will not be enabled.
> > 
> > What is the breadth of AllowMigrateCPUHost? Engine wide? Per DC? Per
> > cluster?
> 
> I thought of eninge-wide. The of course you can have different models in two 
> different DC, but they should be unique in one.
> We can add this to DC or cluster level, imho it would be just another 
> checkbox on the UI that most users would not use.

Most users are not going to use hostcpu. This feature is intended to
people who have performance-critical VMs, and a set of hosts that can
run them. These very same people may have less critical desktops that
are to be allowed to run on any host.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] host cpu feature

[Dan Kenigsberg]

On Wed, Dec 05, 2012 at 06:46:09AM -0500, Laszlo Hornyak wrote:
> 
> 
> - Original Message -
> > From: "Yaniv Kaul" 
> > To: "Laszlo Hornyak" 
> > Cc: "engine-devel" 
> > Sent: Wednesday, December 5, 2012 12:23:47 PM
> > Subject: Re: [Engine-devel] host cpu feature
> > 
> > On 12/05/2012 12:32 PM, Laszlo Hornyak wrote:
> > > Hi,
> > >
> > > CPU-Host support allows the virtual machines to see and utilize the
> > > host's CPU flags, this enables better performance in VM's, at the
> > > price of worse portablity.
> > >
> > > http://www.ovirt.org/Features/Cpu-host_Support
> > >
> > > Your feedback is welcome!
> > >
> > > Thank you,
> > > Laszlo
> > > ___
> > > Engine-devel mailing list
> > > Engine-devel@ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > 
> > - I assume that when you allow migration, you'd use host-model? This
> > is
> > not clear from the design. It seems like we VDSM developers can
> > choose
> > to use either this or passthrough, while in practice we should
> > support both.

I join Kaul's question: it is an ovirt-level question whether
hostPassthrough or hostModel or both should be supported. It should not
be a unilateral vdsm decision.

> 
> If AllowMigrateCPUHost is set to true (in case you have the same cpu model 
> everywhere in your DC) migration of such hsots will be enabled. Otherwise it 
> will not be enabled.

What is the breadth of AllowMigrateCPUHost? Engine wide? Per DC? Per cluster?
I favor the latter; a user may have a cluster of exact-same hosts, where
hostcpu migration is allowed, and other cluster where it is forbiden.

The nice thing about hostModel (unlike hostPassthrough) is that once we
created the VM we can migrate it to stronger hosts, and back to the
original host. I suppose that it complicates the scheduler.

> 
> > 
> > - I'm still convinced and commented on both relevat oVirt and libvirt
> > BZs that we need to add x2apic support to the CPU, regardless of what
> > the host CPU exposes.
> > AFAIK, the KVM developers agree with me.
> 
> Not quite sure how is this related... could you send some URL's for the 
> bugreports?
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Task cancelation feature

[Dan Kenigsberg]

On Mon, Dec 03, 2012 at 12:15:07PM -0500, Saggi Mizrahi wrote:
> VDSM tasks are changing to something completely different.
> It's still under discussion but the general direction is that:
> - TaskIDs will be decided by the caller.
> - VDSM can start tasks on it's own
> - There will be no distinction between async tasks and sync tasks. Everything 
> is always async.
> - There will be no cleanTask() when tasks are done they return result to the 
> caller and disappear immediately.

I'm not sure I understand the motivation for the latter change. I kinda
like the unix process semantics, were the return code of a process is
kept with its id after the process ends, until the process parent calls
wait(2). Otherwise, how can the caller tell why its task has failed?

For example, I'd like to see vmCreate using async tasks like that.
vmCreate returns immediately, and a vdsm task is tracking the vm
creation. If something bad happens, the information about the failure
can be polled by the Engine that created the vm (or a new Engine
instance, after an Engine crash).

Similarily, we may need to make setupNetwork asynchronous, since we
depend on dhclient, which may take a lot of time to finish.

Have these future use cases been debated?

Dan.
> 
> Also, some stuff you consider tasks will no longer be tasks any more.
> For instance, copying and image will finish successfully once VDSM registers 
> the operation for with the storage subsystem and creates the image handle.
> After that the status of the copy is bound to the status of the new image and 
> is tracked that way.
> This means that the thing you track when you do copyImage() is actually the 
> creation of the image handle and the metadata to make it usable.
> After that is done any host can query the state of the new image by using the 
> image ID and not the task Id which was deprecated.
> This will be true for all storage operations.
> 
> 
> - Original Message -
> > From: "Michael Kublin" 
> > To: "engine-devel" 
> > Sent: Monday, December 3, 2012 4:19:48 AM
> > Subject: [Engine-devel] Task cancelation feature
> > 
> > Hi, I created a wiki page with design of task cancellation feature.
> > The url is : http://www.ovirt.org/Features/TaskManagerCancelTask
> > I can not call these design, I have not any requirements , except a
> > name of the feature,
> > so my wiki doesn't contains anything except open questions.
> > Also, I think that it is impossible to make a good feature based on
> > very problematic infrastructure,
> > I think before we should fix all our infrastructure problems, and
> > after that to add any cancellation task
> > feature will be a meter of couple hours of work
> > 
> > Regards Michael
> > ___
> > Engine-devel mailing list
> > Engine-devel@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > 
> ___
> Engine-devel mailing list
> Engine-devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [vdsm] [ATTENTION] vdsm-bootstrap/host deployment (pre-3.2)

[Dan Kenigsberg]

On Wed, Nov 28, 2012 at 03:29:35PM -0600, Adam Litke wrote:
> On Wed, Nov 28, 2012 at 03:45:28PM -0500, Alon Bar-Lev wrote:
> > 
> > 
> > - Original Message -----
> > > From: "Dan Kenigsberg" 
> > > To: "Alon Bar-Lev" 
> > > Cc: "VDSM Project Development" , 
> > > "engine-devel" , "users"
> > > 
> > > Sent: Wednesday, November 28, 2012 10:39:42 PM
> > > Subject: Re: [vdsm] [ATTENTION] vdsm-bootstrap/host deployment (pre-3.2)
> > > 
> > > On Wed, Nov 28, 2012 at 02:57:17PM -0500, Alon Bar-Lev wrote:
> > > > 
> > > > > > No... we need it as compatibility with older engines...
> > > > > > We keep minimum changes there for legacy, until end-of-life.
> > > > > 
> > > > > Is there an EoL statement for oVirt-3.1?
> > > > > We can make sure that oVirt-3.2's vdsm installs properly with
> > > > > ovirt-3.1's vdsm-bootstrap, or even require that Engine must be
> > > > > upgraded
> > > > > to ovirt-3.2 before upgrading any of the hosts. Is it too harsh
> > > > > to
> > > > > our
> > > > > vast install base?  us...@ovirt.org, please chime in!
> > > > >
> > > > 
> > > > I tried to find such, but the more I dig I find that we need to
> > > > support old legacy.
> > > 
> > > Why, exactly? Fedora gives no such guarntees (heck, I'm stuck with an
> > > unupgradable F16). Should we be any better than our (currently
> > > single)
> > > platform?
> > 
> > We should start and detach from specific distro procedures.
> > 
> > > 
> > > > > > > > 
> > > > > > > >  * legacy-removed: change machine width core file
> > > > > > > >   # echo /var/lib/vdsm/core > /proc/sys/kernel/core_pattern
> > > > > > > 
> > > > > > > Yeah, qemu-kvm and libvirtd are much more stable than in the
> > > > > > > old
> > > > > > > days,
> > > > > > > but wouldn't we want to keep a means to collect the corpses
> > > > > > > of
> > > > > > > dead
> > > > > > > processes from hypervisors? It has helped us nail down nasty
> > > > > > > bugs,
> > > > > > > even
> > > > > > > in Python.
> > > > > > 
> > > > > > It does not mean it should be at /var/lib/vdsm ... :)
> > > > > 
> > > > > I don't get the joke :-(. If you mind the location, we can think
> > > > > of
> > > > > somewhere else to put the core dumps. Would it be hard to
> > > > > reinstate a
> > > > > parallel feature in otopi?
> > > > 
> > > > I usually do not make any jokes...
> > > > A global system setting should not go into package specific
> > > > location.
> > > > Usually core dumps are off by default, I like this approach as
> > > > unattended system may fast consume all disk space because of
> > > > dumps.
> > > 
> > > If a host fills up with dumps so quickly, it's a sign that it should
> > > not
> > > be used for production, and that someone should look into the cores.
> > > (P.S. we have a logrotate rule for them in vdsm)
> > 
> > There should be a vdsm-debug-aids (or similar) to perform such changes.
> > Again, I don't think vdsm should (by default) modify any system width 
> > parameter such as this.
> > But I will happy to hear more views.
> 
> I agree with your statement above that a single package should not override a
> global system setting.  We should really work to remove as many of these from
> vdsm as we possibly can.  It will help to make vdsm a much safer/well-behaved
> package.

I'm fine with dropping these from vdsm, but I think they are good for
ovirt - we would like to (be able to) enfornce policy on our nodes.

If configuring core dumps is removed from vdsm, it should go somewhere
else, or our log-collector users would miss their beloved dumps.

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [vdsm] [ATTENTION] vdsm-bootstrap/host deployment (pre-3.2)

[Dan Kenigsberg]

On Wed, Nov 28, 2012 at 02:57:17PM -0500, Alon Bar-Lev wrote:
> 
> > > No... we need it as compatibility with older engines...
> > > We keep minimum changes there for legacy, until end-of-life.
> > 
> > Is there an EoL statement for oVirt-3.1?
> > We can make sure that oVirt-3.2's vdsm installs properly with
> > ovirt-3.1's vdsm-bootstrap, or even require that Engine must be
> > upgraded
> > to ovirt-3.2 before upgrading any of the hosts. Is it too harsh to
> > our
> > vast install base?  us...@ovirt.org, please chime in!
> >
> 
> I tried to find such, but the more I dig I find that we need to support old 
> legacy.

Why, exactly? Fedora gives no such guarntees (heck, I'm stuck with an
unupgradable F16). Should we be any better than our (currently single)
platform?

> > > > > 
> > > > >  * legacy-removed: change machine width core file
> > > > >   # echo /var/lib/vdsm/core > /proc/sys/kernel/core_pattern
> > > > 
> > > > Yeah, qemu-kvm and libvirtd are much more stable than in the old
> > > > days,
> > > > but wouldn't we want to keep a means to collect the corpses of
> > > > dead
> > > > processes from hypervisors? It has helped us nail down nasty
> > > > bugs,
> > > > even
> > > > in Python.
> > > 
> > > It does not mean it should be at /var/lib/vdsm ... :)
> > 
> > I don't get the joke :-(. If you mind the location, we can think of
> > somewhere else to put the core dumps. Would it be hard to reinstate a
> > parallel feature in otopi?
> 
> I usually do not make any jokes...
> A global system setting should not go into package specific location.
> Usually core dumps are off by default, I like this approach as unattended 
> system may fast consume all disk space because of dumps.

If a host fills up with dumps so quickly, it's a sign that it should not
be used for production, and that someone should look into the cores.
(P.S. we have a logrotate rule for them in vdsm)

> If sysadmin manually enables dumps, he may do this at a location of his own 
> choice.

Note that we've just swapped hats: you're arguing for letting a local
admin log in and mess with system configuration, and I'm for keeping a
centralized feature for storing and collecting core dumps.

> If we want to automatically enable dumps I guess it should go to 
> /var/lib/core or similar.


___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [vdsm] [ATTENTION] vdsm-bootstrap/host deployment (pre-3.2)

[Dan Kenigsberg]

On Wed, Nov 28, 2012 at 09:45:17AM -0500, Alon Bar-Lev wrote:
> 
> > On Wed, Nov 28, 2012 at 08:59:10AM -0500, Alon Bar-Lev wrote:
> > > Hello All,
> > > 
> > > Preparing to ovirt-engine 3.2 the entire "vdsm-bootstrap" bootstrap
> > > was re-written from scratch into more pluggable and flexible
> > > implementation, available at git master and nightly snapshots.
> > > 
> > > As far as packaging is concerned there are now two more
> > > dependencies to ovirt-engine:
> > > 
> > >  * otopi -- oVirt Task Oriented Pluggable Installer/Implementation
> > >  * ovirt-host-deploy -- oVirt host deploy tool
> > > 
> > > These packages replace the legacy vdsm-bootstrap package that was
> > > distributed with vdsm.
> > 
> > Hurray!
> > 
> > I suspect that a `git-rm vds_bootstrap/*` is pending?
> 
> No... we need it as compatibility with older engines...
> We keep minimum changes there for legacy, until end-of-life.

Is there an EoL statement for oVirt-3.1?
We can make sure that oVirt-3.2's vdsm installs properly with
ovirt-3.1's vdsm-bootstrap, or even require that Engine must be upgraded
to ovirt-3.2 before upgrading any of the hosts. Is it too harsh to our
vast install base?  us...@ovirt.org, please chime in!

> 
> > 
> > > 
> > > Git repositories are available at at[1][2].
> > > Documentation is available at Git repositories - README*.
> > > Builds are available at usual place[3].
> > > Bugzilla components will be available shortly.
> > 
> > Are there requests to add the components to Fedora (18, EPEL6)?
> > I think we should add these requests as blockers for Bug 881006 -
> > Tracker: oVirt 3.2 release.
> 
> Yes, I am on this one.
> 
> > 
> > > Change log is attached.
> > > 
> > > There is no change in the way the engine is performing the host
> > > deployment process in term of user experience, other than event log
> > > messages during deployment were improved.
> > > 
> > > The log of the deployment is fetched from host and stored at engine
> > > machine at /var/log/ovirt-engine/host-deploy, on host it is at
> > > /tmp/ovirt-host-deploy*.log and deleted when fetched to engine.
> > > 
> > > Among other features, the ovir-host-deploy package can be installed
> > > manually on host and executed to prepare host for installation, in
> > > future we may be able to add host to engine without performing the
> > > deployment process, for now it will be usable for integration
> > > tests.
> > > 
> > > The internals are completely different, instead of having 3
> > > different
> > > bootstrap sequences:
> > >  1. host install
> > >  2. ovirt-node install
> > >  3. ovirt-node approve
> > > 
> > > We now have single sequence which is common to host and node
> > > installation or re-installation, end result is much simpler
> > > implementation.
> > > 
> > > Please report any issues even minor issues, so we can stabilize it
> > > for
> > > 3.2 release.
> > > 
> > > Best Regards,
> > > Alon Bar-Lev.
> > > 
> > > [1] http://gerrit.ovirt.org/gitweb?p=otopi.git;a=tree
> > > [2] http://gerrit.ovirt.org/gitweb?p=ovirt-host-deploy.git;a=tree
> > > [3] http://www.ovirt.org/releases/nightly/rpm/Fedora/17/noarch/
> > > 
> > > ---
> > > 
> > > Change Log
> > > 
> > >  * offline packager feature.
> > > 
> > >  * tuned is installed with virtual-host profile.
> > 
> > I never understood why this is an installer step, and not part of
> > vdsmd
> > start up
> 
> There may be several method to tune a machine.
> Why VDSM should depend on specific one?

Maybe because I tend to install vdsm using `yum`, and would like it to
do The Right Thing to make the host an oVirt node. I suspect that if 
ovirt-host-deploy
proves to be easy to use, I could follow my `yum install vdsm` with
`ovirt-host-deploy`.

> 
> > 
> > >  * initial implementation based on otpoi.
> > > 
> > >  * implementation is based on legacy vdsm-bootstrap pacakge
> > >  functionality.
> > > 
> > >  * legacy-removed: legacy VDSM (<3.0) config upgrade.
> > > 
> > >  * legacy-removed: change machine width core file
> > >   # echo /var/lib/vdsm/core > /proc/sys/kernel/core_pattern
> > 
> > Yeah, qemu-kvm and libvirtd are much more stable than in the old
> > days,
> > but wouldn't we want to keep a means to collect the corpses of dead
> > processes from hypervisors? It has helped us nail down nasty bugs,
> > even
> > in Python.
> 
> It does not mean it should be at /var/lib/vdsm ... :)

I don't get the joke :-(. If you mind the location, we can think of
somewhere else to put the core dumps. Would it be hard to reinstate a
parallel feature in otopi?


> > 
> > 
> > Alon, thanks for your tremendous work on this. I cannot wait to have
> > it
> > up and running in the release.
> 
> Thank you!
> I truly hope that from this point we can only make it better.

Do you mean that we've reached rock bottom? ;-)
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [vdsm] [ATTENTION] vdsm-bootstrap/host deployment (pre-3.2)

[Dan Kenigsberg]

On Wed, Nov 28, 2012 at 08:59:10AM -0500, Alon Bar-Lev wrote:
> Hello All,
> 
> Preparing to ovirt-engine 3.2 the entire "vdsm-bootstrap" bootstrap
> was re-written from scratch into more pluggable and flexible
> implementation, available at git master and nightly snapshots.
> 
> As far as packaging is concerned there are now two more dependencies to 
> ovirt-engine:
> 
>  * otopi -- oVirt Task Oriented Pluggable Installer/Implementation
>  * ovirt-host-deploy -- oVirt host deploy tool
> 
> These packages replace the legacy vdsm-bootstrap package that was
> distributed with vdsm.

Hurray!

I suspect that a `git-rm vds_bootstrap/*` is pending?

> 
> Git repositories are available at at[1][2].
> Documentation is available at Git repositories - README*.
> Builds are available at usual place[3].
> Bugzilla components will be available shortly.

Are there requests to add the components to Fedora (18, EPEL6)?
I think we should add these requests as blockers for Bug 881006 -
Tracker: oVirt 3.2 release.

> Change log is attached.
> 
> There is no change in the way the engine is performing the host
> deployment process in term of user experience, other than event log
> messages during deployment were improved.
> 
> The log of the deployment is fetched from host and stored at engine
> machine at /var/log/ovirt-engine/host-deploy, on host it is at
> /tmp/ovirt-host-deploy*.log and deleted when fetched to engine.
> 
> Among other features, the ovir-host-deploy package can be installed
> manually on host and executed to prepare host for installation, in
> future we may be able to add host to engine without performing the
> deployment process, for now it will be usable for integration tests.
> 
> The internals are completely different, instead of having 3 different
> bootstrap sequences:
>  1. host install
>  2. ovirt-node install
>  3. ovirt-node approve
> 
> We now have single sequence which is common to host and node
> installation or re-installation, end result is much simpler
> implementation.
> 
> Please report any issues even minor issues, so we can stabilize it for
> 3.2 release.
> 
> Best Regards,
> Alon Bar-Lev.
> 
> [1] http://gerrit.ovirt.org/gitweb?p=otopi.git;a=tree
> [2] http://gerrit.ovirt.org/gitweb?p=ovirt-host-deploy.git;a=tree
> [3] http://www.ovirt.org/releases/nightly/rpm/Fedora/17/noarch/
> 
> ---
> 
> Change Log
> 
>  * offline packager feature.
> 
>  * tuned is installed with virtual-host profile.

I never understood why this is an installer step, and not part of vdsmd
start up

>  * initial implementation based on otpoi.
> 
>  * implementation is based on legacy vdsm-bootstrap pacakge functionality.
> 
>  * legacy-removed: legacy VDSM (<3.0) config upgrade.
> 
>  * legacy-removed: change machine width core file
>   # echo /var/lib/vdsm/core > /proc/sys/kernel/core_pattern

Yeah, qemu-kvm and libvirtd are much more stable than in the old days,
but wouldn't we want to keep a means to collect the corpses of dead
processes from hypervisors? It has helped us nail down nasty bugs, even
in Python.

> 
>  * legacy-removed: kernel version test, package dependency is sufficient.
> 
>  * legacy-removed: do not add kernel parameter processor.max_cstate=1
>warn if not have constant_tsc
>https://bugzilla.redhat.com/show_bug.cgi?id=770153
> 
>  * legacy-change: io elevator scheduler set in kernel command-line
>use either udev rule in vdsm package or tuned.
> 
>  * legacy-change: vdsm libvirt reconfigure
>vdsm is reconfigured with file based trigger instead unsupported systemd
>init.d parameter.
> 
>  * legacy-change: distribution checks are simpler based on Python platform,
>minimum:
>- rhel-6.2
>- fedora-17
> 
>  * legacy-change: minimum vdsm version is taken from engine not hard coded.
> 
>  * legacy-change: pki is now using m2crypto to generate certificate request
>and parse certificates.
> 
>  * legacy-change: use iproute2 instead of python ethtool to avoid another
>dependency for host name validation.
> 
>  * legacy-change: use iproute2 instead of reading /proc/net/route for route
>information and interface information.
> 
>  * legacy-change: do not use vdsm.netinfo for vlan and bonding as it requires
>/usr/share/vdsm modules, and it is trivial anyway.
> 
>  * legacy-change: use vdsm-store-net-config script to commit network config
>instead of internal duplicate implementation.
> 
>  * legacy-change: /etc/vdsm/vdsm.conf is overridden unless VDSM/configOverride
>environment is set to True

I'm a bit confused by the negation: I'd expect VDSM/configOverride=True
to mean "override /etc/vdsm/vdsm.conf".

> 
>  * legacy-change: /etc/vdsm/vdsm.conf is not read of fake_qemu.
>set VDSM/checkVirtHardware environment to False to avoid hardware 
> detection.
> 
>  * legacy-change: following gluster packages not installed:
>- glusterfs-rdma
>- glusterfs-geo-replication


Alon, thanks for your tremendous work on this. I cannot wait to have it
up and r

Re: [Engine-devel] Report vNic Implementation Details

[Dan Kenigsberg]

On Sun, Nov 25, 2012 at 03:21:28PM +0200, Livnat Peer wrote:
> On 25/11/12 15:00, Itamar Heim wrote:
> > On 11/25/2012 02:56 PM, Livnat Peer wrote:
> >> On 22/11/12 23:18, Itamar Heim wrote:
> >>> On 11/22/2012 08:40 PM, Simon Grinberg wrote:
>  Back to the original question:
> 
>  What is most inconvenient for many users is:
>  1. The name we provide while creating the VNIC differs from the one in
>  the guest
>  2. No correlation between IP and NIC
> 
>  The current page covers for this but indeed as raised here does not
>  cover what happens if this information is not easy to retrieve due to
>  non strait forward configurations.
> 
>  What I suggest is,
> 
>  For the short term:
>  - Agent to report the MACs, IPs and interface names if can be found,
>  engine to match these to the existing and show
>  Name In Engine| Name in guest | MAC | IP  etc like the current feature
>  page, but...
> 
>  - If a match could not be found then just report Name in Engine N/A
>  and then the rest and keep it in dynamic only table.
>  This is useful for NICs created by hooks, advanced topology that we
>  can't support ATM etc.
> 
>  *The above does require the agent to at least match MAC to IP.
> 
> 
>  Long term: The agent to report a topology the same as vdsm does (use
>  same code at least for Linux guests?) and present it similar to what
>  we present in the host tab. In most cases this will collapse to the
>  short term.
> 
>  MTU, is good to have in any of the two if we can get it.
> 
>  More?
> >>>
> >>> I don't think the guest agent ip information should be correlated to the
> >>> vnic engine information at rest api level.
> >>> the vm (and vnic) api provides the authoritative configuration
> >>> information of the guest as defined in the engine.
> >>> I don't think we should 'taint' it with untrusted data from the guest.
> >>> it would make sense to place there IPs configured/allocated by engine
> >>> when we deal with ip allocation though.
> >>>
> >>
> >>
> >> I was too quick to say we have an agreement...
> >>
> >> The comment above seems to give more emphasis on the segregation between
> >> data collected via the GA and data configured via the engine.
> >>
> >> In the API we have today the following modeling: per VM entity we hold
> >> GuestInfo entity and there we hold a list of IP addresses.
> >>
> >> Are you suggesting to keep this approach and not report anything on the
> >> vNIC level at this point (until we'll be able to configure IP addresses
> >> via the engine)?
> > 
> > yes.
> > 
> >> Or add GuestInfo element under /api/vms/{vm:id}/nics/{nic:id}/ which
> >> should be additional to the one on the VM level (as we discussed before
> >> the correlation between VNIC and GA reported data is not always possible)
> > 
> > i didn't think about reporting guest info at vnic level, only at vm
> > level. it could be a valid option, but since some network information
> > doesn't correlate to vnic's, i think a more natural modeling at vm level
> > may be easier.
> > 
> >>
> >> Also what you have in mind for the UI?
> > 
> > at ui level i do think/agree it would make sense to show the ip per vnic
> > if the correlation between the two is clean and direct (based on mac
> > address i assume).
> > you do need to make sure "bad data" won't break the ui though.
> 
> I'm not sure I agree with the differentiation you are doing between UI
> and API in this case.
> 
> If we think the IP address field per vNIC should display info configured
> via the engine and not untrusted data (reported via the guest agent)
> then we should keep this approach in the UI as well.
> 
> I agree that in some cases the API can model data differently, or enable
> more complicated actions, but this is not the case, here we have the
> same data modeled differently which can cause confusion with users.

Yeah, it's a bit unfair to the UI users: it's like saying "we do not
want to get API users confused by an evil guest agent, but we allow this
to happen to those in the UI".
> 
> I think that we should keep the separation in the UI as well and add GA
> sub-tab in VM and there have a field network devices with the data given
> by the GA. no correlation (between engine configured vNICs and GA
> report) at this point, when we'll have the ip address configuration via
> the engine we'll present per vNIC the address.

Still, the reasonable UI user is less likely to correlate
Engine-configured mac addresses to GA-reported ones. So presenting the
naive correlation between the two has a benefit.

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Report vNic Implementation Details

[Dan Kenigsberg]

On Fri, Nov 23, 2012 at 09:02:09AM +0200, Livnat Peer wrote:
> On 22/11/12 13:59, Dan Kenigsberg wrote:
> > On Thu, Nov 22, 2012 at 09:55:43AM +0200, Livnat Peer wrote:
> >> On 22/11/12 00:02, Itamar Heim wrote:
> >>> On 11/21/2012 10:53 PM, Andrew Cathrow wrote:
> >>>>
> >>>>
> >>>> - Original Message -
> >>>>> From: "Moti Asayag" 
> >>>>> To: engine-devel@ovirt.org
> >>>>> Sent: Wednesday, November 21, 2012 12:36:58 PM
> >>>>> Subject: [Engine-devel] Report  vNic Implementation Details
> >>>>>
> >>>>> Hi all,
> >>>>>
> >>>>> This is a proposal for reporting the vNic implementation details as
> >>>>> reported by the guest agent per each vNic.
> >>>>>
> >>>>> Please review the wiki and add your comments.
> >>>>
> >>>>
> >>>> While we're making the change is there anything else we'd want to
> >>>> report - MTU, MAC (since a user might try to override), etc ?
> >>>>>
> >>>>> http://wiki.ovirt.org/wiki/Feature/ReportingVnicImplementation
> >>>
> >>> iirc, the fact ip addresses appear under guest info in the api and not
> >>> under the vnic was a modeling decision.
> >>> for example, what if guest is using a bridge, or a bond (yes, sounds
> >>> unlikely, but the point is it may be incorrect to assume ip-vnic relation.
> >>> michael - do you remember the details of that discussion?
> >>
> >> I'd love to know what drove this modeling decision.
> >> The use case above is not a typical use case.
> >> We know we won't be able to present the guest internal network
> >> configuration accurately in some scenarios but if we cover 90% of the
> >> use cases correctly I think we should not let perfect be the enemy of
> >> (very) good ;)
> > 
> > We do not support this yet, but once we have nested virtualization, it
> > won't be that odd to have a bridge or bond in the guest. I know that we
> > already have users with in-guest vlan devices.
> > 
> 
> The fact that it's not odd does not mean it is common.., which was the
> point I was trying to make.
> I agree that we should be able to accommodate such info, not sure that
> it is required at this point.
> 
> 
> > I think that the api should accomodate these configurations - even if we
> > do not report them right away. The guest agent already reports (some) of
> > the information:
> > 
> 
> Which API are you referring to? if you are talking about VDSM-Engine API
> we do not change it, only use what is already reported by the GA. I
> don't think we should change the API for future support...

I was refering to the Engine API. Now that we are designing how guest IP
addresses are to be reported, we should think how to accomodate IP
addresses on non-nic devices.

I believe that the Engine API should replicate the guest agent API: give
a list of guest network devices, each one with its ethernet/ipv4/6
addresses.

I am less sure how we should correlate this information with the VNICs
defined on the VM. If we do not want to "taint" the VNIC with this dubious 
information, we can leave this correlation (based on MAC address) to UI or user 
script.

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Report vNic Implementation Details

[Dan Kenigsberg]

On Thu, Nov 22, 2012 at 09:48:19AM +0200, Livnat Peer wrote:
> On 21/11/12 22:53, Andrew Cathrow wrote:
> > 
> > 
> > - Original Message -
> >> From: "Moti Asayag" 
> >> To: engine-devel@ovirt.org
> >> Sent: Wednesday, November 21, 2012 12:36:58 PM
> >> Subject: [Engine-devel] Report  vNic Implementation Details
> >>
> >> Hi all,
> >>
> >> This is a proposal for reporting the vNic implementation details as
> >> reported by the guest agent per each vNic.
> >>
> >> Please review the wiki and add your comments.
> > 
> > 
> > While we're making the change is there anything else we'd want to report - 
> > MTU, MAC (since a user might try to override), etc ?
> > 
> 
> About the MAC address - the engine uses the MAC address to correlate
> between the guest-agent report and the VNICs defined in the engine. If
> the GA reports a MAC that the engine does not recognize the engine can
> not associate it with a VNIC.
> What the engine can do is to issue a warning to the audit log in case
> such a mismatch is recognized, Is that good enough?

Oops, I've been scooped...
The guest may also report devices with unknown MAC (bridge, vlan, dummy,
etc...), which is not a reason for alarm.
If the configured MAC is completely missing, a warning seems valid.

>
> About MTU - It is not being reported by the guest agent ATM (while IPv4
> IPv6 and MAC are), If we'd like to have it I can open RFE for the GA to
> report additional fields.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Report vNic Implementation Details

[Dan Kenigsberg]

On Thu, Nov 22, 2012 at 09:55:43AM +0200, Livnat Peer wrote:
> On 22/11/12 00:02, Itamar Heim wrote:
> > On 11/21/2012 10:53 PM, Andrew Cathrow wrote:
> >>
> >>
> >> - Original Message -
> >>> From: "Moti Asayag" 
> >>> To: engine-devel@ovirt.org
> >>> Sent: Wednesday, November 21, 2012 12:36:58 PM
> >>> Subject: [Engine-devel] Report  vNic Implementation Details
> >>>
> >>> Hi all,
> >>>
> >>> This is a proposal for reporting the vNic implementation details as
> >>> reported by the guest agent per each vNic.
> >>>
> >>> Please review the wiki and add your comments.
> >>
> >>
> >> While we're making the change is there anything else we'd want to
> >> report - MTU, MAC (since a user might try to override), etc ?
> >>>
> >>> http://wiki.ovirt.org/wiki/Feature/ReportingVnicImplementation
> > 
> > iirc, the fact ip addresses appear under guest info in the api and not
> > under the vnic was a modeling decision.
> > for example, what if guest is using a bridge, or a bond (yes, sounds
> > unlikely, but the point is it may be incorrect to assume ip-vnic relation.
> > michael - do you remember the details of that discussion?
> 
> I'd love to know what drove this modeling decision.
> The use case above is not a typical use case.
> We know we won't be able to present the guest internal network
> configuration accurately in some scenarios but if we cover 90% of the
> use cases correctly I think we should not let perfect be the enemy of
> (very) good ;)

We do not support this yet, but once we have nested virtualization, it
won't be that odd to have a bridge or bond in the guest. I know that we
already have users with in-guest vlan devices.

I think that the api should accomodate these configurations - even if we
do not report them right away. The guest agent already reports (some) of
the information:

> The Guest Agent reports the vNic details:
> 
> IP addresses (both IPv4 and IPv6).
> vNic internal name 

Actually, the guest agent reports all in-guest network device. vNics (and bonds
and vlans) included.
> 
> The retrieved information by the Guest Agent should be reported to the 
> ovirt-engine and to be viewed by its client
> Today only the IPv4 addresses are reported to the User, kept on VM level. 
> This feature will maintain the information on vNic level.

I think we should report the information on the vNic level only when we
can. If we have a vlan device, which we do not know how tie to a
specific vNic, we'd better report its IP address on the VM level.

It might be worthwhile to note that we should (try to) correlate Engine
idea of a vNic with the guest agent report, according to the mac address.
The guest can try to fool us by changing the mac address, but that's
true for every bit of info coming from the agent.

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Network Wiring

[Dan Kenigsberg]

On Sun, Nov 18, 2012 at 05:01:30AM -0500, Alona Kaplan wrote:
> 



> > > purge a network while it is connected to VMs: Link-Down on all nics
> > > and connect to the empty/no network. (Yes I know, it's not par of
> > > the
> > > feature, but you know someone will ask for it soon :))
> > 
> > It should not be hard to implement; In
> > http://wiki.ovirt.org/wiki/Feature/DetailedNetworkWiring#New_API I
> > suggest passing
> > no 'network' element to mean "connected to nothing".
> >
> I don't really understand why changing the link state to down is not enough?
> What is the added value of connecting "unwired" nic to a none network?

It is not a big deal of a difference, but the semantics of having no
network is clear: you can run the VM if networks are missing, you can
remove a network when the VM is running. When a VM is associated to a
network, but its link state is down, the "right" semantics is more
vague.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Network Wiring

[Dan Kenigsberg]

On Thu, Nov 15, 2012 at 03:04:45PM -0500, Simon Grinberg wrote:
> 
> > 
> > NX users may confuse between connected to vm or connected to the
> > switch (despite of ),
> > as they used to =up|down
> > 
> 
> Yap, no convergence on terminology yet, right?
> 
> I think we need to keep the plugged/unplugged not to confuse libvirt users

Yes, with real iron nics (and hard drives), the term
"hotplug" is widely understood as "shove a new device into a running
machine". So I see no reason to change anything.
But I do not see how this is related to libvirt (which does not have an
API call named *plug*). 

> however let's do wired/un-wired = link-up / link-down it's understood by 
> everyone and Michael pointed out

I'm fine with that.

> 
> What about allowing a nic with the no-network? Did not see discussion on this.

> It is useful as an interim state when changing networks or if the nic
> is there to be use by a hook. This may also be useful when allowing to
> purge a network while it is connected to VMs: Link-Down on all nics
> and connect to the empty/no network. (Yes I know, it's not par of the
> feature, but you know someone will ask for it soon :))

It should not be hard to implement; In
http://wiki.ovirt.org/wiki/Feature/DetailedNetworkWiring#New_API I suggest 
passing
no 'network' element to mean "connected to nothing".

> 
> The coupling between vNIC and LN should break - for this feature (that I hate 
> to call it wired) and for future to come.  
> 
> Simon.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Network Wiring

[Dan Kenigsberg]

On Thu, Nov 15, 2012 at 02:43:23PM +0200, Yaniv Kaul wrote:
> On 11/15/2012 02:33 PM, Itamar Heim wrote:
> >On 11/15/2012 02:29 PM, Dan Kenigsberg wrote:
> >>On Wed, Nov 14, 2012 at 02:12:07PM -0500, Simon Grinberg wrote:
> >>>>
> >>>>The intention is to use the new API VDSM.libvirtVm.updateVmInteface
> >>>>for
> >>>>performing the network rewire in a single command.
> >>>
> >>>What does it do? I could not find updateVmInteface in vdsm git.
> >>>Where is this defined?
> >>>
> >>>It's critical.
> >>>
> >>>- You can change the interface directly by moving the VM from
> >>>one network to another
> >>>- You can do that but toggle the ling state so the VM will be aware.
> >>>
> >>>Which if these two?
> >>>If you do only the first then it's not the common use case. In
> >>>most cases you must toggle the link status to the VM.
> >>>This will cause:
> >>>1. Speed negotiation + arp request that also informs the
> >>>switched about the change
> >>>2. In case it's DHCP (which most likely the case for guests)
> >>>it will trigger new DHCP request.
> >>>
> >>>If you don't baaad things will happen :)
> >>
> >>I think that bad things are going to happen anyway. In "bad
> >>things", I mean "stuff that require guest intervension".
> >>
> >>The guest may be moved from one subnet into another one, maybe on
> >>different VLAN or physical LAN. We can not expect that the applications
> >>running on it will be happy about these changes. A similar case appears
> >>if we rewire the network while the VM is down (or hibernated). When the
> >>VM is restarted, it is going to use mismatching IP addresses.
> >>
> >>You are right that it may make sense to request an new IP address after
> >>the rewiring, however, a little test I just did on my desktop
> >>showed that
> >>dhclient does not initiate a new request just because the carrier
> >>dropped for few seconds. So we should try harder if we want to refresh
> >>dhcp after rewiring: I think that it would be cool to have a
> >>guest agent verb
> >>that does it.
> 
> Blame your OS if it doesn't do media sensing at all (or correctly).

Media is sensed:

Nov 15 14:15:46 kernel: [3379655.213183] e1000e: eth0 NIC Link is Down
Nov 15 14:15:52 kernel: [3379661.265946] e1000e: eth0 NIC Link is Up 100 Mbps 
Full Duplex, Flow Control: None
Nov 15 14:15:52 kernel: [3379661.265951] e1000e :00:19.0: eth0: 10/100 
speed: disabling TSO

but dhcp does not cancel its leases due to this. And I would not expect it to:
my dhcp server could change without carrier loss (e.g. vlan change on my
nearest switch, or dhcp reconfiguration)

> 
> >
> >shouldn't this simulate a link disconnect/connect event to the OS?
> 
> I sincerely hope it does.

Itamar, what is "this"? Setting link state to down does just that.

I was suggesting a guest agent verb that clears all pending dhcp leases after
the guest is connected again.

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Network Wiring

[Dan Kenigsberg]

On Wed, Nov 14, 2012 at 02:12:07PM -0500, Simon Grinberg wrote:
> > 
> > The intention is to use the new API VDSM.libvirtVm.updateVmInteface
> > for
> > performing the network rewire in a single command.
> 
> What does it do? I could not find updateVmInteface in vdsm git. 
> Where is this defined? 
> 
> It's critical. 
> 
> - You can change the interface directly by moving the VM from one network to 
> another 
> - You can do that but toggle the ling state so the VM will be aware. 
> 
> Which if these two?
> If you do only the first then it's not the common use case. In most cases you 
> must toggle the link status to the VM.
> This will cause:
> 1. Speed negotiation + arp request that also informs the switched about the 
> change  
> 2. In case it's DHCP (which most likely the case for guests) it will trigger 
> new DHCP request. 
> 
> If you don't baaad things will happen :) 

I think that bad things are going to happen anyway. In "bad
things", I mean "stuff that require guest intervension".

The guest may be moved from one subnet into another one, maybe on
different VLAN or physical LAN. We can not expect that the applications
running on it will be happy about these changes. A similar case appears
if we rewire the network while the VM is down (or hibernated). When the
VM is restarted, it is going to use mismatching IP addresses.

You are right that it may make sense to request an new IP address after
the rewiring, however, a little test I just did on my desktop showed that
dhclient does not initiate a new request just because the carrier
dropped for few seconds. So we should try harder if we want to refresh
dhcp after rewiring: I think that it would be cool to have a guest agent verb
that does it.

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [Design for 3.2 RFE] Improving proxy selection algorithm for Power Management operations

[Dan Kenigsberg]

On Sun, Nov 11, 2012 at 06:18:53AM -0500, Eli Mesika wrote:
> 
> 
> - Original Message -
> > From: "Eli Mesika" 
> > To: "Itamar Heim" 
> > Cc: "engine-devel" 
> > Sent: Friday, November 9, 2012 12:06:05 PM
> > Subject: Re: [Engine-devel] [Design for 3.2 RFE] Improving proxy selection 
> > algorithm for Power Management operations
> > 
> > 
> > 
> > - Original Message -
> > > From: "Itamar Heim" 
> > > To: "Eli Mesika" 
> > > Cc: "engine-devel" , "Michael Pasternak"
> > > , "Simon Grinberg"
> > > , "Dan Kenigsberg" 
> > > Sent: Friday, November 9, 2012 12:02:37 PM
> > > Subject: Re: [Engine-devel] [Design for 3.2 RFE] Improving proxy
> > > selection algorithm for Power Management operations
> > > 
> > > On 11/09/2012 10:52 AM, Eli Mesika wrote:
> > > 
> > > >> >
> > > >> >  > FenceWrapper
> > > >> >
> > > >> >i understand danken suggested going this way, rather than than
> > > >> >another
> > > >> >instance of vdsm.
> > > >> >is vdsm only calling these scripts today and all logic is in
> > > >> >engine,
> > > >> >or
> > > >> >does vdsm has any logic in wrapping these scripts (not a
> > > >> >blocker
> > > >> >to
> > > >> >doing FenceWrapper, just worth extracting that logic from vdsm
> > > >> >to
> > > >> >such a
> > > >> >script, then using it in both. i hope answer is 'no logic'...)
> > > > vdsm has some logic that maps between the call passed to it from
> > > > engine and the actual parameters generated for the script.
> > > > AFAIK, this logic only "builds" the correct arguments for the
> > > > command according to the agent type
> > > >
> > > 
> > > can we extract it to an external wrapper?
> > > I'd hate to fix bugs/changes twice for this.
> > 
> > I'll check it with danken on SUN
> 
> Well, looked at it a bit , the VDSM code is in fenceNote function in API.py
> What I think is that we can exclude the fenceNote implementation to a 
> separate fence.py file and call it from the API.py
> Then we can use one of the following in Java to call the method from fence.py
> 1) jython
> 2) org.python.util.PythonInterpreter
> 
> See http://stackoverflow.com/questions/8898765/calling-python-in-java
> 
> danken, what do you think ?

BTW, no one has promised the the fence script is implemented in Python

$ file `which fence_ipmilan `
/usr/sbin/fence_ipmilan: ELF 64-bit LSB executable...
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [Design for 3.2 RFE] Improving proxy selection algorithm for Power Management operations

[Dan Kenigsberg]

On Sun, Nov 11, 2012 at 01:44:50PM +0200, Itamar Heim wrote:
> On 11/11/2012 01:27 PM, Dan Kenigsberg wrote:
> >On Fri, Nov 09, 2012 at 05:06:05AM -0500, Eli Mesika wrote:
> >>
> >>
> >>- Original Message -
> >>>From: "Itamar Heim" 
> >>>To: "Eli Mesika" 
> >>>Cc: "engine-devel" , "Michael Pasternak" 
> >>>, "Simon Grinberg"
> >>>, "Dan Kenigsberg" 
> >>>Sent: Friday, November 9, 2012 12:02:37 PM
> >>>Subject: Re: [Engine-devel] [Design for 3.2 RFE] Improving proxy selection 
> >>>algorithm for Power Management operations
> >>>
> >>>On 11/09/2012 10:52 AM, Eli Mesika wrote:
> >>>
> >>>>>>
> >>>>>>  > FenceWrapper
> >>>>>>
> >>>>>>i understand danken suggested going this way, rather than than
> >>>>>>another
> >>>>>>instance of vdsm.
> >>>>>>is vdsm only calling these scripts today and all logic is in
> >>>>>>engine,
> >>>>>>or
> >>>>>>does vdsm has any logic in wrapping these scripts (not a blocker
> >>>>>>to
> >>>>>>doing FenceWrapper, just worth extracting that logic from vdsm to
> >>>>>>such a
> >>>>>>script, then using it in both. i hope answer is 'no logic'...)
> >>>>vdsm has some logic that maps between the call passed to it from
> >>>>engine and the actual parameters generated for the script.
> >>>>AFAIK, this logic only "builds" the correct arguments for the
> >>>>command according to the agent type
> >>>>
> >>>
> >>>can we extract it to an external wrapper?
> >>>I'd hate to fix bugs/changes twice for this.
> >>
> >>I'll check it with danken on SUN
> >
> >Saggi has had a nascent attempt to factor the little logic we have out
> >http://gerrit.ovirt.org/#/c/7190/7/vdsm/API.py
> >AFAIR there's nothing there beyond:
> >- log everything but passwords,
> >- build the input stream,
> >- run the script
> >- convert its return code
> >and there's also killing dormant scripts on vdsm exist (which I find not
> >important at all).
> 
> if the wrapping isn't doing anything but calling the scripts, then
> doing it again from java isn't an issue.
> it's only an issue if there is any business logic in the wrapping.

I believe there's no issue. The only so-called-reason for this verb
existing in Vdsm is the pre-historical platform of oVirt-Engine's
predecessor, which did not support[*] running the fence-agent scripts.
That's why I'm advocating to cut the middle man (even though he is I).

[*] "no support" meaning: "possible, but no one's gonna help you if
there's trouble."

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [Design for 3.2 RFE] Improving proxy selection algorithm for Power Management operations

[Dan Kenigsberg]

On Fri, Nov 09, 2012 at 05:06:05AM -0500, Eli Mesika wrote:
> 
> 
> - Original Message -
> > From: "Itamar Heim" 
> > To: "Eli Mesika" 
> > Cc: "engine-devel" , "Michael Pasternak" 
> > , "Simon Grinberg"
> > , "Dan Kenigsberg" 
> > Sent: Friday, November 9, 2012 12:02:37 PM
> > Subject: Re: [Engine-devel] [Design for 3.2 RFE] Improving proxy selection 
> > algorithm for Power Management operations
> > 
> > On 11/09/2012 10:52 AM, Eli Mesika wrote:
> > 
> > >> >
> > >> >  > FenceWrapper
> > >> >
> > >> >i understand danken suggested going this way, rather than than
> > >> >another
> > >> >instance of vdsm.
> > >> >is vdsm only calling these scripts today and all logic is in
> > >> >engine,
> > >> >or
> > >> >does vdsm has any logic in wrapping these scripts (not a blocker
> > >> >to
> > >> >doing FenceWrapper, just worth extracting that logic from vdsm to
> > >> >such a
> > >> >script, then using it in both. i hope answer is 'no logic'...)
> > > vdsm has some logic that maps between the call passed to it from
> > > engine and the actual parameters generated for the script.
> > > AFAIK, this logic only "builds" the correct arguments for the
> > > command according to the agent type
> > >
> > 
> > can we extract it to an external wrapper?
> > I'd hate to fix bugs/changes twice for this.
> 
> I'll check it with danken on SUN

Saggi has had a nascent attempt to factor the little logic we have out
http://gerrit.ovirt.org/#/c/7190/7/vdsm/API.py
AFAIR there's nothing there beyond:
- log everything but passwords,
- build the input stream,
- run the script
- convert its return code
and there's also killing dormant scripts on vdsm exist (which I find not
important at all).

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [vdsm] unmanaged devices thrown into 'custom' feature

[Dan Kenigsberg]

On Tue, Oct 23, 2012 at 05:53:20AM -0400, Doron Fediuck wrote:
> 
> 
> - Original Message -
> > From: "Livnat Peer" 
> > To: "Dan Kenigsberg" 
> > Cc: engine-devel@ovirt.org, "Genadi Chereshnya" , 
> > vdsm-de...@fedorahosted.org
> > Sent: Monday, October 22, 2012 8:29:20 AM
> > Subject: Re: [Engine-devel] [vdsm] unmanaged devices thrown into 'custom' 
> > feature
> > 
> > On 21/10/12 23:49, Dan Kenigsberg wrote:
> > > On Sun, Oct 21, 2012 at 11:57:10AM -0400, Eli Mesika wrote:
> > >>
> > >>
> > >> - Original Message -
> > >>> From: "Yair Zaslavsky" 
> > >>> To: "Livnat Peer" 
> > >>> Cc: "Genadi Chereshnya" ,
> > >>> engine-devel@ovirt.org, vdsm-de...@fedorahosted.org
> > >>> Sent: Sunday, October 21, 2012 5:38:54 PM
> > >>> Subject: Re: [Engine-devel] unmanaged devices thrown into
> > >>> 'custom' feature
> > >>>
> > >>>
> > >>>
> > >>> - Original Message -----
> > >>>> From: "Livnat Peer" 
> > >>>> To: "Dan Kenigsberg" 
> > >>>> Cc: "Genadi Chereshnya" ,
> > >>>> engine-devel@ovirt.org, vdsm-de...@fedorahosted.org
> > >>>> Sent: Sunday, October 21, 2012 5:18:31 PM
> > >>>> Subject: Re: [Engine-devel] unmanaged devices thrown into
> > >>>> 'custom'
> > >>>> feature
> > >>>>
> > >>>> On 21/10/12 16:42, Dan Kenigsberg wrote:
> > >>>>> I have just noticed that when a VM is started for the second
> > >>>>> time,
> > >>>>> Engine
> > >>>>> issues the "create" vdsm verb with some information regarding
> > >>>>> "unmanaged" devices (an example is shown below[1]) in the
> > >>>>> 'custom'
> > >>>>> propery bag.
> > >>>>>
> > >>>>> I'm surprised about this, as I was not aware of this usage of
> > >>>>> the
> > >>>>> 'custom' dictionary, and Vdsm is not doing anything with the
> > >>>>> data.
> > >>>>>
> > >>>>
> > >>>> If I recall correctly the idea of passing the unmanaged devices
> > >>>> data
> > >>>> in
> > >>>> the custom property was to enable managing stable device
> > >>>> addresses
> > >>>> in
> > >>>> the hooks (to devices that were added to the VM via hooks from
> > >>>> the
> > >>>> first
> > >>>> place), so this info is there not for VDSM use.
> > >>>> For example if you add a device in a hook it will be kept in the
> > >>>> engine
> > >>>> as a non managed device. later when starting the VM again you
> > >>>> would
> > >>>> like
> > >>>> to assign the same device address to your device, and you can do
> > >>>> so
> > >>>> because you have access to the original address in the custom
> > >>>> properties
> > >>>> of the VM.
> > >>>
> > >>> This is exactly what Eli has explained Gendai and Dan today.
> > > 
> > > (I was asking here because I did not understand the verbal
> > > explanation.)
> > > 
> > >>
> > >> This is taken from the Stable Device Address design in
> > >> http://wiki.ovirt.org/wiki/Features/Design/StableDeviceAddresses
> > >>
> > >> Unmanaged Device
> > >> -
> > >> Unmanaged Device will be supported in the new format and will
> > >> include all unhandled devices as sound/controller/etc and future
> > >> devices. Those devices will be persistent and will have Type ,
> > >> SubType (device specific) and an Address. For 3.1 an unmanaged
> > >> Device is not exposed to any GUI/REST API. Unmanaged devices are
> > >> passed to vdsm inside a Custom property. VDSM in it turn is
> > >> passing this as is for possible hook processing.
> > > 
> > > Thanks for the elaboration. Too bad that I've missed this issue
> > > before.
> > > 
> > > Are you aware of any hook making use of this?  I hope that hook
> > > writers
> > > are not using APIs that are not documented in vdsmd(8).
> > > 
> > > It seems as a classic case where a generic bag interface is coerced
> > > into
> > > an awkward partially-documented interface.
> > > 
> > > I think that a better approach would have been to pass all devices
> > > (managed and unmanaged alike) in the 'devices' property, and let
> > > vdsm
> > > expose whatever is needed to the before_vm_start hook.
> > > 
> > > Maybe we can still do this.
> > 
> > That was the original idea but Ayal objected and I think Igor did not
> > like it as well...
> > 
> 
> +2.
> The original design had an 'unmanaged' (or generic) device type, and all
> devices should have been normalized. But as explained, this was strongly
> rejected in the VDSM side, causing Eli write some special handling for this 
> anomaly.

Can someone (Ayal?) explain the rejection on Vdsm side?
Hiding part of the API in the custom propery bag requires strong
reasoning indeed.

Dan.

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [vdsm] Strange input from oVirt-engine for create (VM) API

[Dan Kenigsberg]

On Mon, Oct 22, 2012 at 02:22:18PM -0500, Adam Litke wrote:
> Hi all,
> 
> Today I was watching the vdsm log as ovirt-engine started a VM and I saw
> something peculiar with how VM device addresses were specified.  Here is a
> sample of the python dictionary for the VM from vdsm.log (I reformatted it 
> with
> pprint for readability):
> 
>  {'address': {' bus': '1',
>   ' controller': '0',
>   ' target': '0',
>   ' type': 'drive',
>   'unit': '0'},
> 
> Notice the whitespace in the 'controller', 'target', and 'type' keys.  Could
> someone explain why this is happening?  Is it deliberate or a bug?

I'm ashamed to see that I've taken a patch stripping these whitespaces
without discussion
http://gerrit.ovirt.org/#/c/3107/1/vdsm/libvirtvm.py

Igor, do you remember why this was not fixed properly, at the Engine
side?

Vdsm's schema should not be as lenient as the current code.

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] unmanaged devices thrown into 'custom' feature

[Dan Kenigsberg]

On Sun, Oct 21, 2012 at 11:57:10AM -0400, Eli Mesika wrote:
> 
> 
> - Original Message -
> > From: "Yair Zaslavsky" 
> > To: "Livnat Peer" 
> > Cc: "Genadi Chereshnya" , engine-devel@ovirt.org, 
> > vdsm-de...@fedorahosted.org
> > Sent: Sunday, October 21, 2012 5:38:54 PM
> > Subject: Re: [Engine-devel] unmanaged devices thrown into 'custom' feature
> > 
> > 
> > 
> > - Original Message -
> > > From: "Livnat Peer" 
> > > To: "Dan Kenigsberg" 
> > > Cc: "Genadi Chereshnya" ,
> > > engine-devel@ovirt.org, vdsm-de...@fedorahosted.org
> > > Sent: Sunday, October 21, 2012 5:18:31 PM
> > > Subject: Re: [Engine-devel] unmanaged devices thrown into 'custom'
> > > feature
> > > 
> > > On 21/10/12 16:42, Dan Kenigsberg wrote:
> > > > I have just noticed that when a VM is started for the second
> > > > time,
> > > > Engine
> > > > issues the "create" vdsm verb with some information regarding
> > > > "unmanaged" devices (an example is shown below[1]) in the
> > > > 'custom'
> > > > propery bag.
> > > > 
> > > > I'm surprised about this, as I was not aware of this usage of the
> > > > 'custom' dictionary, and Vdsm is not doing anything with the
> > > > data.
> > > > 
> > > 
> > > If I recall correctly the idea of passing the unmanaged devices
> > > data
> > > in
> > > the custom property was to enable managing stable device addresses
> > > in
> > > the hooks (to devices that were added to the VM via hooks from the
> > > first
> > > place), so this info is there not for VDSM use.
> > > For example if you add a device in a hook it will be kept in the
> > > engine
> > > as a non managed device. later when starting the VM again you would
> > > like
> > > to assign the same device address to your device, and you can do so
> > > because you have access to the original address in the custom
> > > properties
> > > of the VM.
> > 
> > This is exactly what Eli has explained Gendai and Dan today.

(I was asking here because I did not understand the verbal explanation.)

> 
> This is taken from the Stable Device Address design in
> http://wiki.ovirt.org/wiki/Features/Design/StableDeviceAddresses
> 
> Unmanaged Device
> -
> Unmanaged Device will be supported in the new format and will include all 
> unhandled devices as sound/controller/etc and future devices. Those devices 
> will be persistent and will have Type , SubType (device specific) and an 
> Address. For 3.1 an unmanaged Device is not exposed to any GUI/REST API. 
> Unmanaged devices are passed to vdsm inside a Custom property. VDSM in it 
> turn is passing this as is for possible hook processing. 

Thanks for the elaboration. Too bad that I've missed this issue before.

Are you aware of any hook making use of this?  I hope that hook writers
are not using APIs that are not documented in vdsmd(8).

It seems as a classic case where a generic bag interface is coerced into
an awkward partially-documented interface.

I think that a better approach would have been to pass all devices
(managed and unmanaged alike) in the 'devices' property, and let vdsm
expose whatever is needed to the before_vm_start hook.

Maybe we can still do this.

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

[Engine-devel] unmanaged devices thrown into 'custom' feature

[Dan Kenigsberg]

I have just noticed that when a VM is started for the second time, Engine
issues the "create" vdsm verb with some information regarding
"unmanaged" devices (an example is shown below[1]) in the 'custom'
propery bag.

I'm surprised about this, as I was not aware of this usage of the
'custom' dictionary, and Vdsm is not doing anything with the data.

Would anyone elaborate about it? On the face of it, it does not seem
like a pleasant API. If Engine wants to tell Vdsm about the location of
various devices, we should probably be using the 'devices' property, not
the bag of 'custom' property made for user-defined hooks.

I hope this API pecularity can be avoided, and very much hope that no
one is depending on it.

Dan.


[1]
'custom': {
'device_e97a9759-1c1b-45ed-9ed9-7136ef538315': 'VmDevice 
{vmId=068d4914-4191-400d-a220-17a7f2d8e80c, 
deviceId=e97a9759-1c1b-45ed-9ed9-7136ef538315, device=ide, type=controller, 
bootOrder=0, specParams={}, address={bus=0x00, domain=0x, type=pci, 
slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, 
alias=ide0}',

'device_e97a9759-1c1b-45ed-9ed9-7136ef538315device_133d9bfa-c531-414e-ad20-208d67d5a5e6device_7bfffa34-2e27-4b01-b499-6ac79c997709':
 'VmDevice {vmId=068d4914-4191-400d-a220-17a7f2d8e80c, 
deviceId=7bfffa34-2e27-4b01-b499-6ac79c997709, device=unix, type=channel, 
bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, 
type=virtio-serial}, managed=false, plugged=true, readOnly=false, 
alias=channel0}',

'device_e97a9759-1c1b-45ed-9ed9-7136ef538315device_133d9bfa-c531-414e-ad20-208d67d5a5e6':
 'VmDevice {vmId=068d4914-4191-400d-a220-17a7f2d8e80c, 
deviceId=133d9bfa-c531-414e-ad20-208d67d5a5e6, device=virtio-serial, 
type=controller, bootOrder=0, specParams={}, address={bus=0x00, domain=0x, 
type=pci, slot=0x04, function=0x0}, managed=false, plugged=true, 
readOnly=false, alias=virtio-serial0}',

'device_e97a9759-1c1b-45ed-9ed9-7136ef538315device_133d9bfa-c531-414e-ad20-208d67d5a5e6device_7bfffa34-2e27-4b01-b499-6ac79c997709device_7c107c63-605f-4b21-9893-c052ec211424device_48007de9-467d-46a1-aa84-cc1a6419b5fb':
 'VmDevice {vmId=068d4914-4191-400d-a220-17a7f2d8e80c, 
deviceId=48007de9-467d-46a1-aa84-cc1a6419b5fb, device=spicevmc, type=channel, 
bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, 
type=virtio-serial}, managed=false, plugged=true, readOnly=false, 
alias=channel2}',

'device_e97a9759-1c1b-45ed-9ed9-7136ef538315device_133d9bfa-c531-414e-ad20-208d67d5a5e6device_7bfffa34-2e27-4b01-b499-6ac79c997709device_7c107c63-605f-4b21-9893-c052ec211424':
 'VmDevice {vmId=068d4914-4191-400d-a220-17a7f2d8e80c, 
deviceId=7c107c63-605f-4b21-9893-c052ec211424, device=unix, type=channel, 
bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, 
type=virtio-serial}, managed=false, plugged=true, readOnly=false, 
alias=channel1}'
}
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Network related hooks in vdsm

[Dan Kenigsberg]

On Wed, Oct 10, 2012 at 03:44:24PM -0400, Andrew Cathrow wrote:
> 
> 
> - Original Message -
> > From: "Yaniv Kaul" 
> > To: "Igor Lvovsky" 
> > Cc: "Dan Yasny" , "engine-devel" 
> > , "vdsm-devel"
> > 
> > Sent: Wednesday, October 10, 2012 10:53:19 AM
> > Subject: Re: [Engine-devel] Network related hooks in vdsm
> > 
> > On 10/10/2012 04:47 PM, Igor Lvovsky wrote:
> > >Hi everyone,
> > > As you know vdsm has hooks mechanism and we already support dozen
> > > of hooks for different needs.
> > > Now it's a network's time.
> > > We would like to get your comments regarding our proposition for
> > > network related hooks.
> > >
> > > In general we are planning to prepare framework for future support
> > > of bunch network related hooks.
> > > Some of them already proposed by Itzik Brown [1] and Dan Yasny [2].
> > >
> > > Below you can find the additional hooks list that we propose:
> > >
> > > Note: In the first stage we can implement these hooks without any
> > > parameters, just to provide an entry point
> > >   for simple hooks.
> > >
> > > Networks manipulation:
> > > - before_add_network(conf={}, customProperty={})
> > > - after_add_network(conf={}, customProperty={})
> > > - before_del_network(conf={}, customProperty={})
> > > - after_del_network(conf={}, customProperty={})
> > > - before_edit_network(conf={}, customProperty={})
> > > - after_edit_network(conf={}, customProperty={})
> > > - TBD
> > >
> > > Bondings manipulations:
> > 
> > Bond might be interesting because it may require switch configuration
> > -
> > but so will VLAN changes, so perhaps triggers in VLAN changes are
> > worthwhile as well.
> > Y.
> > 
> > > - before_add_bond(conf={}, customProperty={})
> > > - after_add_bond(conf={}, customProperty={})
> > > - before_del_bond(conf={}, customProperty={})
> > > - after_del_bond(conf={}, customProperty={})
> > > - before_edit_bond(conf={}, customProperty={})
> > > - after_edit_bond(conf={}, customProperty={})
> > > - TBD
> > >
> > > General purpose:
> > > - before_persist_network
> > > - after_persist_network
> 
> What about some way of doing a push that's not tied to an event - if we want 
> to "push" something

Would you elaborate on that? I don't understand the English.

Are you looking for a hook that is not tied to a specific network,
rather a before_setupnetworks/after_setupnetworks ?

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] RFE: Netwrok Main Tab

[Dan Kenigsberg]

On Tue, Oct 09, 2012 at 05:34:56AM -0400, Alona Kaplan wrote:
> The wiki was updated with the final design.
> http://wiki.ovirt.org/wiki/Feature/NetworkMainTab
> 
> Thanks for the comments,

One other comment for consideration: it may be interesting to show in
the VM sub-tab the current bandwidth eaten by each VM. For exapmle, if a
network is congensted, such a column would show the cultprit VM quicker.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] what does engine with cpuIdle?

[Dan Kenigsberg]

On Tue, Sep 11, 2012 at 08:52:51AM -0400, Laszlo Hornyak wrote:
> 
> 
> - Original Message -
> > From: "Dan Kenigsberg" 
> > To: "Laszlo Hornyak" 
> > Cc: "engine-devel" , "Omer Frenkel" 
> > 
> > Sent: Tuesday, September 11, 2012 2:34:13 PM
> > Subject: Re: [Engine-devel] what does engine with cpuIdle?
> > 
> > On Tue, Sep 11, 2012 at 07:38:11AM -0400, Laszlo Hornyak wrote:
> > > 
> > > - Original Message -
> > > > From: "Dan Kenigsberg" 
> > > > To: "Omer Frenkel" 
> > > > Cc: "Laszlo Hornyak" , "engine-devel"
> > > > 
> > > > Sent: Tuesday, September 11, 2012 9:22:15 AM
> > > > Subject: Re: [Engine-devel] what does engine with cpuIdle?
> > > > 
> > > > On Tue, Sep 11, 2012 at 01:55:01AM -0400, Omer Frenkel wrote:
> > > > > 
> > > > > 
> > > > > - Original Message -
> > > > > > From: "Laszlo Hornyak" 
> > > > > > To: "engine-devel" 
> > > > > > Sent: Monday, September 10, 2012 3:51:59 PM
> > > > > > Subject: [Engine-devel] what does engine with cpuIdle?
> > > > > > 
> > > > > > hi,
> > > > > > 
> > > > > > I am trying to change a behavior in vdsm. When you pass 100%
> > > > > > load
> > > > > > on
> > > > > > a VM, it will stop reporting further load and will keep
> > > > > > telling
> > > > > > 100%
> > > > > > until the load drops under 100% again in it's cpuIdle
> > > > > > information.
> > > > > > This is totally correct if you have only single-cpu VM's, but
> > > > > > it
> > > > > > is
> > > > > > false when you have multiple vcpu's, I think the cpuIdle
> > > > > > information
> > > > > > should not be on a 0-100 scale, but on a 0-100*vcpus scale.
> > > > > > 
> > > > > > So I submitted this patch to vdsm:
> > > > > > http://gerrit.ovirt.org/#/c/7892/2
> > > > > > and Dan pointed out that some functionality may depend on the
> > > > > > value
> > > > > > in the 0-100 interval. For me it seems it is ignored and the
> > > > > > load
> > > > > > is
> > > > > > calculated only from sysCpu + userCpu. Does anyone build on
> > > > > > the
> > > > > > cpuIdle value?
> > > > > > 
> > > > > > Thanks,
> > > > > > Laszlo
> > > > > >
> > > > > 
> > > > > you are right, engine doesn't save cpuIdle for vm,
> > > > > so it's not in use in the engine.
> > > > 
> > > > Laszlo, in this case, I think it would be best to drop this bogus
> > > > piece
> > > > of information.
> > > 
> > > Ok.
> > > 
> > > However, before I abandon this patch:
> > 
> > Why abandon? I've suggested you to keep it, just make it even
> > simpler.
> 
> Ok, it is only burocracy, but the new patch will do something
> completely different than the original, so it does not seem to make
> sense to continue this patch. It is more simple to make another one.

Actually, if the patch has the same intentions, I prefer to keep it as
another version - this way it is easier to see if former comments have
been addressed. But it's your code and your call.

> 
> > 
> > > we have a requirement to report cpuSys and cpuUser separately.
> > > Afaik
> > > in libvirt cpuUser and cpuSys does not include the actual guest
> > > time
> > > (at least not with KVM), and in this way if we only report cpuSys
> > > and
> > > cpuUser, the sum does not give the actual load, only a relatively
> > > little percentage of it.
> > 
> > I am not sure I understand what you are saying, but afaik, libvirt's
> > relatively-new
> > http://libvirt.org/html/libvirt-libvirt.html#virDomainGetCPUStats
> > reports the cpu time spent by the entire qemu process - in guest and
> > host modes.
> 
> It seems like sysCpu + userCpu < cpuTime, therefore something is
> missing. I will give it another try, maybe something wrong with my
> hosts.

I may well be wrong.
http://libvirt.org/html/libvirt-libvirt.html#VIR_DOMAIN_CPU_STATS_CPUTIME

someone would have to compare your report requirement with what libvirt
provides.

> 
> > 
> > > If we have the cpuIdle information in engine,
> > > we can calculate the guest time. Therefore, should I - include the
> > > guest time in cpuSys or cpuUser?
> > > - add another exported field?
> > > 
> > > And in both case, we will still have to calculate from cpuIdle
> > > because
> > > libvirt does not tell the guest cpu time :-(
> > 
> > Now I'm completely at loss. Why should we calculate cpuIdle per VM?
> > Haven't we agreed that it is useless?
> 
> Well, if libvirt exports the guest time in sysCpu, then we do not have
> to. But it seems it does not.

Even we should jump throguh hoops to get the data you require, I do not
see how cpuIdle is related (or actually, what it means).

Dan.

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] what does engine with cpuIdle?

[Dan Kenigsberg]

On Tue, Sep 11, 2012 at 07:38:11AM -0400, Laszlo Hornyak wrote:
> 
> - Original Message -
> > From: "Dan Kenigsberg" 
> > To: "Omer Frenkel" 
> > Cc: "Laszlo Hornyak" , "engine-devel" 
> > 
> > Sent: Tuesday, September 11, 2012 9:22:15 AM
> > Subject: Re: [Engine-devel] what does engine with cpuIdle?
> > 
> > On Tue, Sep 11, 2012 at 01:55:01AM -0400, Omer Frenkel wrote:
> > > 
> > > 
> > > - Original Message -
> > > > From: "Laszlo Hornyak" 
> > > > To: "engine-devel" 
> > > > Sent: Monday, September 10, 2012 3:51:59 PM
> > > > Subject: [Engine-devel] what does engine with cpuIdle?
> > > > 
> > > > hi,
> > > > 
> > > > I am trying to change a behavior in vdsm. When you pass 100% load
> > > > on
> > > > a VM, it will stop reporting further load and will keep telling
> > > > 100%
> > > > until the load drops under 100% again in it's cpuIdle
> > > > information.
> > > > This is totally correct if you have only single-cpu VM's, but it
> > > > is
> > > > false when you have multiple vcpu's, I think the cpuIdle
> > > > information
> > > > should not be on a 0-100 scale, but on a 0-100*vcpus scale.
> > > > 
> > > > So I submitted this patch to vdsm:
> > > > http://gerrit.ovirt.org/#/c/7892/2
> > > > and Dan pointed out that some functionality may depend on the
> > > > value
> > > > in the 0-100 interval. For me it seems it is ignored and the load
> > > > is
> > > > calculated only from sysCpu + userCpu. Does anyone build on the
> > > > cpuIdle value?
> > > > 
> > > > Thanks,
> > > > Laszlo
> > > >
> > > 
> > > you are right, engine doesn't save cpuIdle for vm,
> > > so it's not in use in the engine.
> > 
> > Laszlo, in this case, I think it would be best to drop this bogus
> > piece
> > of information.
> 
> Ok.
> 
> However, before I abandon this patch:

Why abandon? I've suggested you to keep it, just make it even simpler.

> we have a requirement to report cpuSys and cpuUser separately. Afaik
> in libvirt cpuUser and cpuSys does not include the actual guest time
> (at least not with KVM), and in this way if we only report cpuSys and
> cpuUser, the sum does not give the actual load, only a relatively
> little percentage of it.

I am not sure I understand what you are saying, but afaik, libvirt's
relatively-new
http://libvirt.org/html/libvirt-libvirt.html#virDomainGetCPUStats
reports the cpu time spent by the entire qemu process - in guest and
host modes.

> If we have the cpuIdle information in engine,
> we can calculate the guest time. Therefore, should I - include the
> guest time in cpuSys or cpuUser?
> - add another exported field?
> 
> And in both case, we will still have to calculate from cpuIdle because
> libvirt does not tell the guest cpu time :-(

Now I'm completely at loss. Why should we calculate cpuIdle per VM?
Haven't we agreed that it is useless?

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] what does engine with cpuIdle?

[Dan Kenigsberg]

On Tue, Sep 11, 2012 at 01:55:01AM -0400, Omer Frenkel wrote:
> 
> 
> - Original Message -
> > From: "Laszlo Hornyak" 
> > To: "engine-devel" 
> > Sent: Monday, September 10, 2012 3:51:59 PM
> > Subject: [Engine-devel] what does engine with cpuIdle?
> > 
> > hi,
> > 
> > I am trying to change a behavior in vdsm. When you pass 100% load on
> > a VM, it will stop reporting further load and will keep telling 100%
> > until the load drops under 100% again in it's cpuIdle information.
> > This is totally correct if you have only single-cpu VM's, but it is
> > false when you have multiple vcpu's, I think the cpuIdle information
> > should not be on a 0-100 scale, but on a 0-100*vcpus scale.
> > 
> > So I submitted this patch to vdsm: http://gerrit.ovirt.org/#/c/7892/2
> > and Dan pointed out that some functionality may depend on the value
> > in the 0-100 interval. For me it seems it is ignored and the load is
> > calculated only from sysCpu + userCpu. Does anyone build on the
> > cpuIdle value?
> > 
> > Thanks,
> > Laszlo
> >
> 
> you are right, engine doesn't save cpuIdle for vm,
> so it's not in use in the engine.

Laszlo, in this case, I think it would be best to drop this bogus piece
of information.

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Minor change in the per patch process.

[Dan Kenigsberg]

On Sun, Aug 19, 2012 at 04:06:09AM -0400, Robert Middleswarth wrote:
> On 08/19/2012 03:21 AM, Dan Kenigsberg wrote:
> >On Thu, Aug 16, 2012 at 09:32:20PM -0400, Robert Middleswarth wrote:
> >>Part of the process now include creating rpm packages.
> >>http://jenkins.ovirt.info/view/patches/job/patch_engine_create_rpms/
> >>This allows people to download and test packages based on the change
> >>if they want to.
> >I wonder if this is needed by people. For me it duplicates the number of
> >emails per change...
> >
> >BTW, I see that a failed job 
> >http://jenkins.ovirt.info/job/patch_vdsm_unit_tests/486/console
> >does not set V-1 on the change. Could this be fixed? I'd like the poster
> >and human reviewer to be perfectly aware that a change breaks unit
> >tests.
> >
> >Regrads,
> >Dan.
> This was detailed in an earlier email.

I'm sorry that I've missed it.

> There are limits related to
> the current plugin and how it processes patches.  The issue is
> simple and effects job when aborted because someone isn't in the
> whitelist.  I have spent a lot of time testing diff options and the
> best I could come up with is that we abort the process.  The biggest
> problem is the gerrit-trigger plugin treats aborts as if they are a
> failure instead of as a non event.  I tried several diff ways to
> make it a non event but couldn't find one.  The choice I went with
> was to add text that they failed and leave them at zero the other
> options was to mark all failure / aborts (Including people not in
> the whitelist) as a -1 that wasn't really acceptable.  There are 2
> diff bug reports and if either gets fixed we will be able to -1
> failures but until the are done I am very limited on the options.

We'll make with what we've got. Thanks.

Oh, and a tiny "security" comment: we may trust infalli...@engineer.org
but not her falli...@engineer.org colleague. So grep'ing the
whitelist should be done with something like

  grep -q ^${email}$
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Minor change in the per patch process.

[Dan Kenigsberg]

On Thu, Aug 16, 2012 at 09:32:20PM -0400, Robert Middleswarth wrote:
> Part of the process now include creating rpm packages.
> http://jenkins.ovirt.info/view/patches/job/patch_engine_create_rpms/
> This allows people to download and test packages based on the change
> if they want to.

I wonder if this is needed by people. For me it duplicates the number of
emails per change...

BTW, I see that a failed job 
http://jenkins.ovirt.info/job/patch_vdsm_unit_tests/486/console
does not set V-1 on the change. Could this be fixed? I'd like the poster
and human reviewer to be perfectly aware that a change breaks unit
tests.

Regrads,
Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] [vdsm] Jenkins testing.

[Dan Kenigsberg]

On Tue, Aug 14, 2012 at 01:43:06AM -0400, Robert Middleswarth wrote:
> After a few false starts it looks like we have per patch testing
> working on VDSM, oVirt-engine, oVirt-engine-sdk and
> oVirt-engine-cli.  There are 3 status a patch can get.  1) Success -
> Means the patch ran though the tests without issue.  2) Failure -
> Means the tests failed.  3) Aborted - Generally means the submitter
> is not in the whitelist and the tests were never run.  If you have
> any questions please feel free to ask.

Thanks Robert, for this great improvement.

However, it seems to me that the script is pulling the wrong git hash.
For example, my patch http://gerrit.ovirt.org/#/c/7097/ with git has
539ccfbf02f0ca9605149885ae6b3e6feb4f1976 reports of success. However the
console output
http://jenkins.ovirt.info/job/patch_vdsm_unit_tests/417/console
show that the git hash that was actually built was
8af050b205994746198e5fb257652cd2fb8bfbc1 (vdsm/master)

Something is fishy here, and I may be getting false positive results.

Regards,
Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Separating engine-setup from ovirt-engine

[Dan Kenigsberg]

On Wed, Jul 04, 2012 at 03:16:17AM -0400, Ofer Schreiber wrote:
> 
> 
> - Original Message -
> > On Tue, Jul 03, 2012 at 06:20:43PM +0300, Michal Skrivanek wrote:
> > > On Jul 3, 2012, at 16:53 , Juan Hernandez wrote:
> > >
> > > > On 07/03/2012 03:43 PM, Ofer Schreiber wrote:
> > > >> In our days, ovirt-engine-setup is a part of the big
> > > >> ovirt-engine rpm.
> > > >> This means that each time you need to build yourself a new
> > > >> ovirt-engine-setup rpm, you need to compile all the engine.
> >
> > Could this possibly be avoided by an optional flag to the build
> > script?
> 
> It's problematic, as ovirt-engine-setup is a sub rpm of ovirt-engine.
> I have no idea how can we just build the setup without the engine, which is 
> compiled in a temporary directory (and removed straight after the build)
> 
> >
> > > >>
> > > >> I've started to think about separating it into another git
> > > >> (similar to ovirt-iso-uploader), so we will be able to build
> > > >> this rpm separately.
> > > >>
> > > >> This change is really easy to implement (actually, I have
> > > >> already done it locally), and sounds to me like it's the right
> > > >> thing to do.
> > > >>
> > > >> Thought?
> > > >> Ofer.
> > > >
> > > > I agree that is the right thing to do. Take into account that
> > > > this also
> > > > means that ovirt-engine-setup will no longer be a subpackage of
> > > > ovirt-engine, so you will have to submit a new package request to
> > > > have
> > > > it included in Fedora.
> > > not quite sure having 10+ packages is a win…
> > > - why do you have to have a separate git?
> > > - why do you have to recompile when there's a change elsewhere?
> > > isn't that a matter of compilation scripts only? (though
> > > understand size of the rpm might be an issue…)
> > > I personally do not see a point in separating of something
> > > inseparable…but that's just me perhaps:)
> > >
> > > in other words, if you would kindly explain me the benefits please,
> > > I'll shut up:-)
> >
> > indeed - having another package, with its own release cycle and
> > versioning scheme is quite costy. and isn't ovirt-engine-setup quite
> > tightly coupled with Engine's db scheme? (I really do not know, I
> > should
> > probably shut up, too).
> 
> Quite costly? why?

It is another package to release, that requires its own errata process
and inter-package dependencies.

If you envisage a user that would like to use ovirt-engine-setup of one
version, with an ovirt-setup of another one, then go ahead. I simply do
not see the use case for this, only the complications.
> 
> engine-setup is not tightly coupled with the db-scripts, we just execute the 
> createDB script.
> 
> >
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] Separating engine-setup from ovirt-engine

[Dan Kenigsberg]

On Tue, Jul 03, 2012 at 06:20:43PM +0300, Michal Skrivanek wrote:
> On Jul 3, 2012, at 16:53 , Juan Hernandez wrote:
> 
> > On 07/03/2012 03:43 PM, Ofer Schreiber wrote:
> >> In our days, ovirt-engine-setup is a part of the big ovirt-engine rpm.
> >> This means that each time you need to build yourself a new 
> >> ovirt-engine-setup rpm, you need to compile all the engine.

Could this possibly be avoided by an optional flag to the build script?

> >> 
> >> I've started to think about separating it into another git (similar to 
> >> ovirt-iso-uploader), so we will be able to build this rpm separately.
> >> 
> >> This change is really easy to implement (actually, I have already done it 
> >> locally), and sounds to me like it's the right thing to do.
> >> 
> >> Thought?
> >> Ofer.
> > 
> > I agree that is the right thing to do. Take into account that this also
> > means that ovirt-engine-setup will no longer be a subpackage of
> > ovirt-engine, so you will have to submit a new package request to have
> > it included in Fedora.
> not quite sure having 10+ packages is a win…
> - why do you have to have a separate git?
> - why do you have to recompile when there's a change elsewhere? isn't that a 
> matter of compilation scripts only? (though understand size of the rpm might 
> be an issue…)
> I personally do not see a point in separating of something inseparable…but 
> that's just me perhaps:)
> 
> in other words, if you would kindly explain me the benefits please, I'll shut 
> up:-) 

indeed - having another package, with its own release cycle and
versioning scheme is quite costy. and isn't ovirt-engine-setup quite
tightly coupled with Engine's db scheme? (I really do not know, I should
probably shut up, too).
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] New Fedora VDSM Build?

[Dan Kenigsberg]

On Tue, Jun 26, 2012 at 06:57:45AM -0400, Ofer Schreiber wrote:
> 
> 
> - Original Message -
> > On Tue, Jun 26, 2012 at 05:53:56AM -0400, Ofer Schreiber wrote:
> > > Any updates about $topic?
> > 
> > Federico is herding sheep in Scotland right now.
> > Saggi is his second-in-command.
> > 
> > Currently, there ore nly 2 patches on top of Fedrico latest build
> > https://koji.fedoraproject.org/koji/buildinfo?buildID=327015
> 
> Did you fixed the bootstrap?
> If yes, I'd like to get a new build, so people will be able to install VDSM.

bootstrap fixes are in the above-mentioned build -2.  (But there
should be another one regarding kernel version. I could use some help
understanding why douglas cannot verify
http://gerrit.ovirt.org/#/c/5637/ )

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] New Fedora VDSM Build?

[Dan Kenigsberg]

On Tue, Jun 26, 2012 at 05:53:56AM -0400, Ofer Schreiber wrote:
> Any updates about $topic?

Federico is herding sheep in Scotland right now.
Saggi is his second-in-command.

Currently, there ore nly 2 patches on top of Fedrico latest build
https://koji.fedoraproject.org/koji/buildinfo?buildID=327015

But setupNetworks is still rather broken, so it is not our last one :-(

Dan.
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

Re: [Engine-devel] live migration and different technologies

[Dan Kenigsberg]

On Mon, Jun 11, 2012 at 10:47:16AM +0100, Daniel P. Berrange wrote:
> On Sat, Jun 09, 2012 at 03:57:40PM +0300, Itamar Heim wrote:
> > On 06/08/2012 06:54 PM, Daniel P. Berrange wrote:
> > >On Wed, Jun 06, 2012 at 05:15:53PM +0300, Itamar Heim wrote:
> > >>Hi Daniel,
> > >>
> > >>on the quantum-ovirt call today the question of live migration
> > >>between multiple technologies was raised.
> > >>
> > >>iirc, you implemented the abstraction in libvirt between what the
> > >>guest sees and the actual host networking implementation for live
> > >>migration.
> > >>
> > >>can you please share if there are any considerations around live
> > >>migrations across different network implementations (bridge, sr-iov,
> > >>ovs, qbg, openflow, etc.)
> > >
> > >Yes, we added the ability to use libvirt's 'virtual network' APIs
> > >(virNetworkX) to define host networks using bridging, macvtap,
> > >etc, etc. A guest's NICs can then be configured solely using
> > >. This means that the guest XML will
> > >not have any host-specific data in it, as you see when using
> > >  or
> > >
> > >This means you can migrate between machines where the bridges have
> > >different names (eg br0 on host A and br7 on host B), without any
> > >limitations.
> > >
> > >You can also migrate between different impls of the same technology
> > >(eg traditional software bridging vs macvtap bridging without
> > >limitations.
> > >
> > >Finally, you can migrate between completely different technologies
> > >(eg bridging vs vepa), but you will likely loose connectivity in
> > >the guests, since the technologies are not compatible at the ethernet
> > >layer.
> > 
> > can you please explain this point - how would packet going out of
> > the host or arriving to the guest would be different between a
> > bridged and a vepa implemtnaiton?
> 
> I'm not the expert on VEPA - I'm just relaying what I have been told
> wrt VEPA modes in the past.
> 
> IIUC, with VEPA modes there is quite alot of extra traffic due to a
> handshake negotiation between the host & switch, before any guest
> traffic can pass, and there needs to be a special synchronization
> done with VEPA during migration to maintain state in the switch.

But at least on the libvirt level,  tags are ignored if a
domain is migrated from Qbsomething to a bridge?

I suppose that a default  tag cannot be generated, so
migration from bridge to Qb* is impossible without the source tweaking
the destinationxml?
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel