Re: [Xen-devel] [Qemu-devel] [PATCH v3] xen-disk: use an IOThread per instance

[Daniel P. Berrange]

On Wed, Nov 08, 2017 at 05:42:27PM +, Stefan Hajnoczi wrote:
> On Tue, Nov 07, 2017 at 05:46:53AM -0500, Paul Durrant wrote:
> > This patch allocates an IOThread object for each xen_disk instance and
> > sets the AIO context appropriately on connect. This allows processing
> > of I/O to proceed in parallel.
> > 
> > The patch also adds tracepoints into xen_disk to make it possible to
> > follow the state transtions of an instance in the log.
> 
> virtio-blk and virtio-scsi allow the user to specify an IOThread object.
> This allows users to configure the device<->IOThread mapping any way
> they like (e.g. 1:1, M:N).  Are you sure you want to hard-code the
> IOThread mapping?

I certainly think it'd be better for mgmt apps if all disks had the same
approach to IOThread mapping.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [Qemu-devel] [PATCH v2] hw/display/xenfb: Simulate auto-repeat key events

[Daniel P. Berrange]

On Thu, Nov 02, 2017 at 05:26:50PM +, Peter Maydell wrote:
> On 2 November 2017 at 17:18, Liang Yan  wrote:
> > New tigervnc changes the way to send long pressed key,
> > from "down up down up ..." to "down down ... up", it only
> > affects xen pv console mode. I send a patch to latest
> > kernel side, but it may have a fix in qemu backend for
> > back compatible becase guest VMs may use very old kernel.
> > This patch inserts an up event after each regular key down
> > event to simulate an auto-repeat key event for xen keyboard
> > frontend driver.
> >
> > Signed-off-by: Liang Yan 
> > ---
> > v2:
> > - exclude extended key
> > - change log comment
> >
> >  hw/display/xenfb.c | 5 +
> >  1 file changed, 5 insertions(+)
> >
> > diff --git a/hw/display/xenfb.c b/hw/display/xenfb.c
> > index 8e2547ac05..1bc5b41ab7 100644
> > --- a/hw/display/xenfb.c
> > +++ b/hw/display/xenfb.c
> > @@ -292,6 +292,11 @@ static void xenfb_key_event(void *opaque, int scancode)
> >  }
> >  trace_xenfb_key_event(opaque, scancode2linux[scancode], down);
> >  xenfb_send_key(xenfb, down, scancode2linux[scancode]);
> > +
> > +/* insert an up event for regular down key event */
> > +if (down && !xenfb->extended) {
> > +xenfb_send_key(xenfb, 0, scancode2linux[scancode]);
> > +}
> >  }
> 
> This doesn't look to me like the right place to fix this bug.
> The xenfb key event handler is just one QEMU keyboard backend
> (in a setup where there are many possible sources of keyboard
> events: vnc, gtk, SDL, cocoa UI frontends; and many possible
> sinks: xenfb's key handling, ps2 keyboard emulator, etc etc).
> 
> We need to be clear in our definition of generic QEMU key
> events how key repeat is supposed to be handled, and then
> every consumer and every producer needs to follow that.
> In the specific case of the vnc UI frontend, we need to
> also look at what the VNC protocol specifies for key repeat.
> That then tells us whether the bug to be fixed is in QEMU,
> or in a particular VNC client.

I'm somewhat inclined to say this is a Tigervnc bug. We fixed this
same issue in GTK-VNC ~10 years ago. While X11 would send a sequence
of press,release,press,release,  GTK would turn this into
press,press,press,press,release which broke some VNC servers.
So GTK-VNC undoes this optimization from GTK to ensure a full set
of press,release,press,release pairs is always sent.

The official RFC for VNC does not specify any auto-repeat behaviour

  https://tools.ietf.org/html/rfc6143#section-7.5.4

The unofficial community authored extension to the RFC suggests
taking the press,press,press,release approach to allow servers to
distinguish auto-repeat from manual repeat, but I'm not really
convinced by that justification really

  http://vncdotool.readthedocs.io/en/latest/rfbproto.html#keyevent

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [Qemu-devel] [PATCH 7/8] os-posix: Provide new -runasid option

[Daniel P. Berrange]

On Mon, Oct 09, 2017 at 04:05:10PM +0100, Ian Jackson wrote:
> Markus Armbruster writes ("Re: [Qemu-devel] [PATCH 7/8] os-posix: Provide new 
> -runasid option"):
> > The last thing the QEMU command line needs is more exotic options.  Are
> > you sure we need a new one here?  Can we make existing -runas serve?
> > Precedence: Coreutils[*].  Pseudo-code:
> > 
> > if argument is a decimal number starting with '+':
> > user ID
> > else if argument is a valid user name:
> > user name
> > else if argument is a valid user ID:
> > user ID
> > else:
> > error
> 
> I can do this.  So -runas . then.  I don't think it makes
> sense to try to -runas  because: you wouldn't have a username
> to pass to initgroups: not calling initgroups would be a bear trap;
> and otherwise we wouldn't know what gid to use.

Just use  getpwuid() to get the "struct passwd *", then change_process_uid()
doesn't need any changes at all AFAICT.


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [Qemu-devel] [PATCH 0/3 v4] xenfb: Enablement for Windows PV HID frontend

[Daniel P. Berrange]

On Fri, Sep 29, 2017 at 12:26:52PM +0200, Gerd Hoffmann wrote:
> On Tue, 2017-09-26 at 14:43 +, Owen Smith wrote:
> > Improve the input device model in xenfb, by updating the
> > Qemu input handlers and adding a feature to allow for
> > raw (unscaled) absolute coordinates to be represented.
> > 
> > By using a reverse mapping call, the Linux input.h #defines
> > are not pulled into xenfb, and so should remove the compiler
> > warnings reported.
> 
> Reviewed-by: Gerd Hoffmann 
> 
> What is the plan with this?  Merge through xen queue?  I can merge it
> via ui queue too, but in that case I'd like to get an review from the
> xen guys.
> 
> Note: conflicts with keycodemapdb patch series, might need a rebase
> depending on merge order.

I don't mind rebasing against this if it merges first.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [Qemu-devel] [PATCH] xen-disk: use g_malloc0 to fix build

[Daniel P. Berrange]

On Fri, Jul 28, 2017 at 07:43:59AM -0500, Eric Blake wrote:
> On 07/28/2017 07:31 AM, Olaf Hering wrote:
> > g_malloc0_n is available since glib-2.24. To allow build with older glib
> > versions use the generic g_malloc0, which is already used in many other
> > places in the code.
> > 
> > Fixes commit 3284fad728 ("xen-disk: add support for multi-page shared 
> > rings")
> > 
> > Signed-off-by: Olaf Hering 
> > ---
> >  hw/block/xen_disk.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/hw/block/xen_disk.c b/hw/block/xen_disk.c
> > index d42ed7070d..71deec17b0 100644
> > --- a/hw/block/xen_disk.c
> > +++ b/hw/block/xen_disk.c
> > @@ -1232,7 +1232,7 @@ static int blk_connect(struct XenDevice *xendev)
> >  return -1;
> >  }
> >  
> > -domids = g_malloc0_n(blkdev->nr_ring_ref, sizeof(uint32_t));
> > +domids = g_malloc0(blkdev->nr_ring_ref * sizeof(uint32_t));
> 
> This version is prone to multiplication overflow (well, maybe not, but
> you have to audit for that).  Wouldn't it be better to use:
> 
> domids = g_new0(blkdev->nr_ring_ref, uint32_t)

You mean   g_new0(uint32_t, blkdev->nr_ring_ref)  but yeah, g_new0 is
better than g_malloc0 pretty much every time.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [Qemu-devel] [PATCH v2] event: Add source information to SHUTDOWN

[Daniel P. Berrange]

On Thu, Apr 20, 2017 at 01:59:29PM +0200, Markus Armbruster wrote:
> Eric Blake  writes:
> 
> > Libvirt would like to be able to distinguish between a SHUTDOWN
> > event triggered solely by guest request and one triggered by a
> > SIGTERM or other action on the host.  qemu_kill_report() is
> > already able to tell whether a shutdown was triggered by a host
> > signal (but NOT by a host UI event, such as clicking the X on
> > the window), but that information was then lost after being
> > printed to stderr.
> >
> > Enhance the shutdown request path to take a parameter of which
> > way it is being triggered, and update ALL callers.  It would have
> > been less churn to keep the common case with no arguments as
> > meaning guest-triggered, and only modified the host-triggered
> > code paths, via a wrapper function, but then we'd still have to
> > audit that I didn't miss any host-triggered spots; changing the
> > signature forces us to double-check that I correctly categorized
> > all callers.
> >
> > Here is output from 'virsh qemu-monitor-event --loop' with the
> > patch installed:
> >
> > event SHUTDOWN at 1492639680.731251 for domain fedora_13: {"guest":true}
> > event STOP at 1492639680.732116 for domain fedora_13: 
> > event SHUTDOWN at 1492639680.732830 for domain fedora_13: {"guest":false}
> >
> > Note that libvirt runs qemu with -no-quit: the first SHUTDOWN event
> > was triggered by an action I took directly in the guest (shutdown -h),
> > at which point qemu stops the vcpus and waits for libvirt to do any
> > final cleanups; the second SHUTDOWN event is the result of libvirt
> > sending SIGTERM now that it has completed cleanup.
> >
> > See also https://bugzilla.redhat.com/1384007
> >
> > Signed-off-by: Eric Blake 
> >
> > ---
> >
> > I did not wire up the RESET event to report guest-triggered, although
> > I had to plumb that through all the guests since qemu has options that
> > allow remapping reset to shutdown.  It's easy to add that if we want
> > it in v3.
> >
> > The replay driver needs a followup patch if we want to be able to
> > faithfully replay the difference between a host- and guest-initiated
> > shutdown (for now, the replayed event is always attributed to host).
> >
> >
> > v2: retitle (was "event: Add signal information to SHUTDOWN"),
> > completely rework to post bool based on whether it is guest-initiated
> > v1: initial submission, exposing just Unix signals from host
> > ---
> >  qapi/event.json |  8 ++--
> >  include/sysemu/sysemu.h |  4 ++--
> >  vl.c| 19 +++
> >  hw/acpi/core.c  |  4 ++--
> >  hw/arm/highbank.c   |  4 ++--
> >  hw/arm/integratorcp.c   |  2 +-
> >  hw/arm/musicpal.c   |  2 +-
> >  hw/arm/omap1.c  |  6 +++---
> >  hw/arm/omap2.c  |  2 +-
> >  hw/arm/spitz.c  |  2 +-
> >  hw/arm/stellaris.c  |  2 +-
> >  hw/arm/tosa.c   |  2 +-
> >  hw/i386/pc.c|  2 +-
> >  hw/input/pckbd.c|  4 ++--
> >  hw/ipmi/ipmi.c  |  4 ++--
> >  hw/isa/lpc_ich9.c   |  2 +-
> >  hw/mips/boston.c|  2 +-
> >  hw/mips/mips_malta.c|  2 +-
> >  hw/mips/mips_r4k.c  |  4 ++--
> >  hw/misc/arm_sysctl.c|  8 
> >  hw/misc/cbus.c  |  2 +-
> >  hw/misc/macio/cuda.c|  4 ++--
> >  hw/misc/slavio_misc.c   |  4 ++--
> >  hw/misc/zynq_slcr.c |  2 +-
> >  hw/pci-host/apb.c   |  4 ++--
> >  hw/pci-host/bonito.c|  2 +-
> >  hw/pci-host/piix.c  |  2 +-
> >  hw/ppc/e500.c   |  2 +-
> >  hw/ppc/mpc8544_guts.c   |  2 +-
> >  hw/ppc/ppc.c|  2 +-
> >  hw/ppc/ppc405_uc.c  |  2 +-
> >  hw/ppc/spapr_hcall.c|  2 +-
> >  hw/ppc/spapr_rtas.c |  4 ++--
> >  hw/s390x/ipl.c  |  2 +-
> >  hw/sh4/r2d.c|  2 +-
> >  hw/timer/etraxfs_timer.c|  2 +-
> >  hw/timer/m48t59.c   |  4 ++--
> >  hw/timer/milkymist-sysctl.c |  4 ++--
> >  hw/timer/pxa2xx_timer.c |  2 +-
> >  hw/watchdog/watchdog.c  |  2 +-
> >  hw/xenpv/xen_domainbuild.c  |  2 +-
> >  hw/xtensa/xtfpga.c  |  2 +-
> >  kvm-all.c   |  6 +++---
> >  os-win32.c  |  2 +-
> >  qmp.c   |  4 ++--
> >  replay/replay.c |  5 -
> >  target/alpha/sys_helper.c   |  4 ++--
> >  target/arm/psci.c   |  4 ++--
> >  target/i386/excp_helper.c   |  2 +-
> >  target/i386/hax-all.c   |  6 +++---
> >  target/i386/helper.c|  2 +-
> >  target/i386/kvm.c   |  2 +-
> >  target/s390x/helper.c   |  2 +-
> >  target/s390x/kvm.c  |  4 ++--
> >  target/s390x/misc_helper.c  |  4 ++--
> >  target/sparc/int32_helper.c |  2 +-
> >  ui/sdl.c|  2 +-
> >  ui/sdl2.c   |  4 ++--
> >  xen-hvm.c   |  2 +-
> >  

Re: [Xen-devel] [libvirt] Question about hypervisor that are not tristate

[Daniel P. Berrange]

On Wed, Jan 11, 2017 at 10:49:48AM -0700, Jim Fehlig wrote:
> On 01/06/2017 05:31 PM, Jim Fehlig wrote:
> 
> Adding xen-devel for a question below...
> 
> > Happy new year!
> > 
> > Nearly a year ago I reported an issue with the  hypervisor feature on 
> > Xen
> > [1] and am now seeing a similar issue with the  feature. Setting the
> > default value of pae changed between xend and libxl. When not specified, 
> > xend
> > would enable pae for HVM domains. Clients such as xm and the old libvirt 
> > driver
> > did not have to explicitly enable it. In libxl, the pae field within
> > libxl_domain_build_info is initialized to 0. Clients must enable pae, and 
> > indeed
> > xl will do so if pae=1 is not specified in the xl.cfg.
> > 
> > The xend behavior prevents libvirt from disabling pae, whereas the libxl 
> > behvior
> > causes a guest ABI change (config that worked with libvirt+xend doesn't with
> > libvirt+libxl). The libxl behavior also forces management software (e.g.
> > OpenStack nova) to add  where it wasn't needed before.
> > 
> > To solve this problem for , it was changed it to a tristate [2], 
> > allowing
> > it to be turned off with explicit , and on if not 
> > specified or
> >  or . Should  (and the remaining hypervisor 
> > features
> > that are not tristate) be converted to tristate similar to ? 
> > Alternatively,
> > I could simply set pae=1 for all HVM domains in the libxl driver. Like the 
> > old
> > libvirt+xend behavior it couldn't be turned off, but I don't think there is 
> > a
> > practical use-case to do so. At least no one has complained over all the 
> > years
> > of libvirt+xend use.
> 
> Xen folks, what is your opinion of always enabling pae for HVM domains in
> the libvirt libxl driver? Is there a need these days to disable it?
> 
> Jan had mentioned that some old, buggy guest OS's (Win 9x) might need it
> disabled, and perhaps some cases where it may be desirable to suppress a
> guest OS entering 64-bit mode. But in practice do you think it is necessary
> to expose this knob to users?

If the guest arch is declared as x86_64, then you should unconditionally
force 'pae' to be present, otherwise you'd be giving the guest an i686
environment.  The app should request arch == i686 explicitly if they
want an i686 environment. Allowing pae to be toggled for i686 guests
is ok, since that indicates whethre the i686 CPU can address > 2GB
of RAM.



Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://entangle-photo.org   -o-http://search.cpan.org/~danberr/ :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [libvirt] Opinions on removing the old, legacy libvirt Xen driver

[Daniel P. Berrange]

On Fri, Nov 18, 2016 at 02:25:18PM -0700, Jim Fehlig wrote:
> Hi All,
> 
> I briefly mentioned this at an evening event during the KVM Forum / Xen Dev
> Summit, but the list is certainly a better place to discuss such a topic.
> What do folks think about finally removing the old, legacy, xend-based
> driver from the libvirt sources?
> 
> The Xen community made xl/libxl the primary toolstack in Xen 4.1. In Xen
> 4.2, it was made the default toolstack. In Xen 4.5, xm/xend was completely
> removed from the Xen source tree. According to the Xen release support
> matrix [0], upstream maintenance of Xen 4.1-4.3 has been dropped for some
> time, including "long term" security support. Xen 4.4-4.5 no longer receive
> regular maintenance support, with security support ending in March for 4.4
> and January 2018 for 4.5. In short, the fully maintained upstream Xen
> releases don't even contain xm/xend :-).
> 
> As for downstreams, I doubt anyone is interested in running the last several
> libvirt releases on an old Xen installition with xm/xend, let alone
> libvirt.git master. SUSE, which still supports Xen, has no interest in using
> a new libvirt on older (but still supported) SLES that uses the xm/xend
> toolstack. I struggle to find a good reason to keep any of the old cruft
> under src/xen/. I do think we should keep the xm/sexpr config
> parsing/formatting code src/xenconfig/ since it is useful for converting old
> xm and sexpr config to libvirt domXML.
> 
> Thanks for opinions and comments!

As a point of reference, for the QEMU/KVM driver we took the decision to drop
support for distros whose age is older than RHEL-6 (Nov 2010 release date).
Xen 4.1 came out in early 2011 IIUC, so dropping support for Xen < 4.1 is
a reasonable thing todo and consistent with what we've done for QEMU. So
ACK to that from a project support POV.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://entangle-photo.org   -o-http://search.cpan.org/~danberr/ :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [Qemu-devel] [PATCH 2/2] xen: add qemu device for each pvusb backend

[Daniel P. Berrange]

On Mon, Sep 26, 2016 at 02:43:57PM +0200, Juergen Gross wrote:
> In order to be able to specify to which pvusb controller a new pvusb
> device should be added we need a qemu device for each pvusb controller
> with an associated id.
> 
> Add such a device when a new controller is requested and attach the
> usb bus of that controller to the new device. Any device connected to
> that controller can now specify the bus and port directly via its
> properties.
> 
> Signed-off-by: Juergen Gross 
> ---
>  hw/usb/xen-usb.c | 81 
> +++-
>  1 file changed, 68 insertions(+), 13 deletions(-)
> 
> @@ -733,10 +740,10 @@ static void usbback_portid_add(struct usbback_info 
> *usbif, unsigned port,
>  {
>  unsigned speed;
>  char *portname;
> -USBPort *p;
>  Error *local_err = NULL;
>  QDict *qdict;
>  QemuOpts *opts;
> +char tmp[32];
>  
>  if (usbif->ports[port - 1].dev) {
>  return;
> @@ -749,11 +756,14 @@ static void usbback_portid_add(struct usbback_info 
> *usbif, unsigned port,
>  return;
>  }
>  portname++;
> -p = &(usbif->ports[port - 1].port);
> -snprintf(p->path, sizeof(p->path), "%s", portname);
>  
>  qdict = qdict_new();
>  qdict_put(qdict, "driver", qstring_from_str("usb-host"));
> +snprintf(tmp, sizeof(tmp), "%s.0", usbif->id);

Don't snprintf into fixed length buffers. g_strdup_printf() does the
right thing

> +qdict_put(qdict, "bus", qstring_from_str(tmp));
> +snprintf(tmp, sizeof(tmp), "%s-%u", usbif->id, port);
> +qdict_put(qdict, "id", qstring_from_str(tmp));


Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [libvirt] Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?

[Daniel P. Berrange]

On Mon, Jun 27, 2016 at 05:12:52PM +0100, Ian Jackson wrote:
> Daniel P. Berrange writes ("Re: [libvirt] [Xen-devel] Fixing libvirt's libxl 
> driver breakage -- where to define LIBXL_API_VERSION?"):
> > On Mon, Jun 27, 2016 at 04:54:35PM +0100, Ian Jackson wrote:
> > > Created the following branch refs on xenbits in the toplevel
> > > libvirt.git:
> > > 
> > >  osstest/frozen/xen-4.3-testing 9a0c7f5f834185db9017c34aabc03ad99cf37bed
> > >  osstest/frozen/xen-4.4-testing 33fb8ff185846a7b4974105d2c9400690a6f95cf
> > >  osstest/frozen/xen-4.5-testing cda1cc170f07b45911b3dad03e42c8ebfc210fa1
> > >  osstest/frozen/xen-4.6-testing eac167e2610d3e59b32f7ec7ba78cbc8c420a425
> > >  osstest/frozen/xen-4.7-testing 1a41ed5af5e1704dd9b0bdca40df5c9cacbdabfc
> > 
> > How did you pick those hashes ? Would it make more sense to pick the
> > nearest libvirt release tag ? eg v1.3.2 instead of 33fb8ff18584 ?
> > 
> > > These were those tested by the following `tolerable' osstest push gate
> > > flights for the corresponding Xen tree:
> > > 
> > >  xen-4.3-testing 9a0c7f5f8341 86673
> > >  xen-4.4-testing 33fb8ff18584 85031
> > >  xen-4.5-testing cda1cc170f07 83135
> > >  xen-4.6-testing eac167e2610d 96031
> > >  xen-4.7-testing 1a41ed5af5e1 95728
> 
> I picked them by searching my mail archives for osstest `tolerable'
> push gate flights - ie, passes in our CI system.
> 
> That minimises the risk that the selected versions are themselves
> troublesome for some reason, needing another round of adjustment.
> 
> It might indeed be better to convert them to nearby release tags.
> However:
> 
> mariner:libvirt> git-describe 9a0c7f5f834185db9017c34aabc03ad99cf37bed
> v1.3.2-202-g9a0c7f5
> mariner:libvirt> git-describe 33fb8ff185846a7b4974105d2c9400690a6f95cf
> v1.3.2-rc2-1-g33fb8ff
> mariner:libvirt> git-describe cda1cc170f07b45911b3dad03e42c8ebfc210fa1
> v1.3.1-262-gcda1cc1
> mariner:libvirt> git-describe eac167e2610d3e59b32f7ec7ba78cbc8c420a425
> v1.3.5-318-geac167e
> mariner:libvirt> git-describe 1a41ed5af5e1704dd9b0bdca40df5c9cacbdabfc
> v1.3.5-129-g1a41ed5
> mariner:libvirt> 
> 
> So in most cases these hashes are well away from a release tag.
> 
> Does libvirt have stable release branches ?  One approach would be to
> have osstest track a suitable libvirt stable release branche for each
> Xen stable release branch.

Yep, there is a vN.N.N-maint branch for every release

NB, with our new numbering that'll be changing nto vN.N-maint instead.

> That would involve setting up a push gate for each of the chosen
> libvirt stable branches.  That would be worthwhile if we expect those
> stable branches to acquire commits which break Xen, and which we could
> like to be told about.  But I'm not sure that's the case.

Stuff goes onto the stable branches on an as-needed basis - mostly coming
from the distro maintainers response to bug fixes from their users. There's
probably not a whole lot that's touching xen on a regular basis and we're
quite strict in what we accept for stable.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [libvirt] Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?

[Daniel P. Berrange]

On Mon, Jun 27, 2016 at 04:54:35PM +0100, Ian Jackson wrote:
> (Adding Jan Beulich)
> 
> Ian Jackson writes ("Re: [libvirt] [Xen-devel] Fixing libvirt's libxl driver 
> breakage -- where to define LIBXL_API_VERSION?"):
> > It seems that the libvirt LIBXL_API_VERSION is now rather higher, at
> > 0x040400, since libvirt#fccf27253ced
> >   libxl: switch to using libxl_domain_create_restore from v4.4 API
> > 
> > One unfortunate effect of this is to break the osstest tests of the
> > Xen 4.3 stable branch, which for the moment is still allegedly in
> > security support.
> > 
> > I can't really see a way that this kind of problem could be avoided
> > in principle, without
> >   - providing a more sophisticated way for libxl callers to set the
> > requested version
> >   - providing more compatibility code in libvirt, too, and retaining
> > it for some time
> > 
> > I think instead that it would probably be better for osstest to
> > "freeze" the version of libvirt that it is using, every time we branch
> > Xen.
> > 
> > So Xen 4.4 would be tested with whatever libvirt we were using when
> > the stable branch for Xen 4.4 was made, and so on.
> > 
> > Does that sound sensible ?
> 
> In the assumption that it is, I have:
> 
> Created the following branch refs on xenbits in the toplevel
> libvirt.git:
> 
>  osstest/frozen/xen-4.3-testing 9a0c7f5f834185db9017c34aabc03ad99cf37bed
>  osstest/frozen/xen-4.4-testing 33fb8ff185846a7b4974105d2c9400690a6f95cf
>  osstest/frozen/xen-4.5-testing cda1cc170f07b45911b3dad03e42c8ebfc210fa1
>  osstest/frozen/xen-4.6-testing eac167e2610d3e59b32f7ec7ba78cbc8c420a425
>  osstest/frozen/xen-4.7-testing 1a41ed5af5e1704dd9b0bdca40df5c9cacbdabfc

How did you pick those hashes ? Would it make more sense to pick the
nearest libvirt release tag ? eg v1.3.2 instead of 33fb8ff18584 ?

> 
> These were those tested by the following `tolerable' osstest push gate
> flights for the corresponding Xen tree:
> 
>  xen-4.3-testing 9a0c7f5f8341 86673
>  xen-4.4-testing 33fb8ff18584 85031
>  xen-4.5-testing cda1cc170f07 83135
>  xen-4.6-testing eac167e2610d 96031
>  xen-4.7-testing 1a41ed5af5e1 95728
> 
> And I have prepared the patch below, which (together with a
> prerequisite, in my tree) will implement this in osstest.
> 
> Ian.
> 
> From 5d1c91d3c53b580305e96d62f8ca84f85f8d3011 Mon Sep 17 00:00:00 2001
> From: Ian Jackson 
> Date: Mon, 27 Jun 2016 16:49:52 +0100
> Subject: [OSSTEST PATCH] cr-daily-branch: libvirt: use frozen version on
>  stable branches
> 
> libvirt master might increase its LIBXL_API_VERSION.  When this feeds
> through osstest it can cause the push gates of Xen stable branches to
> break.
> 
> So for stable Xen branches do not track libvirt upstream.  Instead,
> use a frozen revision.  (Only for main push gate tests of stable Xen
> branches.)
> 
> The frozen branch is never going to be updated so it is not suitable
> for other kinds of uses.  In particular it won't get security fixes.
> So we call the refs   osstest/frozen/xen-K.L-testing  to discourage
> users from using them.
> 
> Deployment note: The Xen release checklist needs a new item "add this
> frozen libvirt branch".
> 
> Signed-off-by: Ian Jackson 
> ---
>  cr-daily-branch | 6 ++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/cr-daily-branch b/cr-daily-branch
> index 8b7c789..21780b8 100755
> --- a/cr-daily-branch
> +++ b/cr-daily-branch
> @@ -186,6 +186,12 @@ if [ "x$REVISION_OVMF" = x ]; then
>  fi
>  fi
>  if [ "x$REVISION_LIBVIRT" = x ]; then
> + case "$xenbranch" in
> + xen-[0-9]*-testing)
> + BASE_TAG_LIBVIRT=osstest/frozen/$xenbranch
> + export BASE_TAG_LIBVIRT
> + ;;
> + esac
>   determine_version REVISION_LIBVIRT libvirt LIBVIRT
>   export REVISION_LIBVIRT
>  fi

Overall I think your approach makes sense.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [libvirt] Questions about virtlogd

[Daniel P. Berrange]

On Wed, Jun 08, 2016 at 11:57:45AM +0100, George Dunlap wrote:
> 
> Well we definitely want to make it possible for people to use xl while
> still avoiding DoSes.  But at the simplest level this could be done by
> having qemu's stderr/stdout piped to /dev/null by default, and allowing
> an option for the admin to enable piping it to a file on a per-guest
> basis when necessary.
> 
> This would effectively be declaring a "proper solution" out-of-scope,
> while not opening up our users to security issues.

I hadn't thought of /dev/null as an option, that's a nice idea.


BTW, I want to raise another item as more general food for thought
which is somewhat related to this topic, albeit not useful as an
solution to use today.

If you consider the risk to be a compromised QEMU inflicting DoS
on the host filesystem, then the stderr/out logging and serial
port file writing is not the only attack vector. If you give QEMU
any kind of file based disk, then (unless you have quotas in place)
it can expand those disk files to arbitrary size - this is by design
of course, since QEMU needs to save snapshots inside qcow2, and has
the ability to resize virtual disks, etc.

At the same time, it would be nice to be able to have a possibility
too have a locked down solution. Conceptually it would be nice to be
able to place size limits on individual files that were open. It turns
out Linux introduced just such a facility under the concept "file sealing"

The motivation for this was kDBus which wanted a way to share tmpfs backed
file handles between processes with certain policy rules. This concept was
implemented using a new fcntl() call F_ADD_SEAL which allows a number of
flags to be set against a file descriptor:

  - SHRINK: If SEAL_SHRINK is set, the file in question cannot be reduced
in size. This affects ftruncate() and open(O_TRUNC).
  - GROW: If SEAL_GROW is set, the file in question cannot be increased
  in size. This affects ftruncate(), fallocate() and write().
  - WRITE: If SEAL_WRITE is set, no write operations (besides resizing)
   are possible. This affects fallocate(PUNCH_HOLE), mmap() and
   write().
  - SEAL: If SEAL_SEAL is set, no further seals can be added to a file.
  This basically prevents the F_ADD_SEAL operation on a file and
  can be set to prevent others from adding further seals that you
  don't want.

The SEAL_GROW flag seems like exactly the kind of thing QEMU could make
use of. First of all we would have to assume that the QEMU disk image
is fully pre-allocated, ie a non-sparse raw file, or a qcow2 file which
has had its extents fully allocated. This is not unreasonable for many
usage scenarios. I could imagine -drive gaining a new parameter
'growable=yes|no'. eg

  $QEMU  -drive file=/some/image/vm.qcow2,growable=no

would cause QEMU to set SEAL_GROW when it open the disk image. After
that point it is not possible for a compromised QEMU to cause further
disk allocation on the /some/image filesystem. This of course relies
on SELinux/AppArmour/other-MAC to prevent QEMU opening/creating other
arbitrary files.

Hotplug is not so simple though. At the time we hotplug the disk we
have to assume QEMU is already hostile, so we can't really rely on
QEMU to honour the request to set SEAL_GROW. To deal with this we
would have to deny QEMU any "open" permission using MAC. Instead
libvirt (or equiv) would have to be responsible for opening disk
images, setting SEAL_GROW and passing the file descriptor onto the
running QEMU to use.

This all feels remarkably simple and useful, with one huge glaring
issue

the kernel only implemented support for F_ADD_SEAL on the tmpfs
filesystem, since that's all kDBus needs it for :-) To be useful for
QEMU/libxl/libvirt/etc we'd at least need it supported on ext4 and xfs,
and preferrably NFS too. I've no clue how hard this would be since I'm
not at all familiar with the kernel code.

So clearly this isn't something we can use at time in the forseeable
future, but if people are interested in attacking the more general
problem, it might be an approach worth investigating to see if it
really is viable for the future.

How it would relate to the bug we're talking about here, is that I
could imagine pre-allocating the logfile at say 128 KB in size, opening
it, setting the SEAL_GROW flag and then attaching it to QEMU stdout/err.
This would let QEMU write straight to the file as normal, but not let
it exceed 128 KB.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [libvirt] Questions about virtlogd

[Daniel P. Berrange]

On Wed, Jun 08, 2016 at 10:50:24AM +0100, George Dunlap wrote:
> On 07/06/16 16:57, Wei Liu wrote:
> >> I must admit I'm not familiar with the division of responsibility
> >> for managing QEMU between the Xen provided libxl library(s) and
> >> the libvirt libxl driver code. Naively I would expect the libvirt
> >> libxl driver code to deal with virtlogd and then configure the
> >> Xen libxl library / QEMU accordingly. Your request seems to imply
> >> that you will need the Xen libxl library to directly talk to
> >> virtlogd instead.
> >>
> >> Is there any way in which it would be practical for the libvirt
> >> libxl driver to talk to virtlogd to acquire the file descriptors
> >> to use and pass those file descriptors down to the libxl library ?
> >>
> > 
> > There are two classes of configurations.
> > 
> > For libvirt + libxl, There is currently no API for passing in a fd to be
> > used as QEMU logging fd. But I'm thinking about having one. It wouldn't
> > be too hard.
> > 
> > The other class is  configurations that don't have libvirt. We need some
> > sort of mechanism to handle QEMU logs. My intent of this email is mainly
> > for this class of configurations.
> 
> Just to be clear -- internally we're investigating options for dealing
> with the "qemu logging" problem* for XenProject for people not running
> libvirt -- people who use the xl toolstack, or people who build their
> own toolstack on top of libxl.
> 
> (We *also* need to figure out how to deal with  the libxl+libvirt
> situation, but that's just a matter of plumbing I think.)
> 
> The options we've come up with, broadly, are as follows:
> 
> 1. Try to use the existing syslog facilities
> 
> 2. Re-purpose one of our existing daemons to perform a role similar to
> virtlogd
> 
> 3. "Steal" virtlogd and import it into our tree (yay GPL!)
> 
> 4. Work with the libvirt community to make virtlogd an independent
> project which can be used by both libvirt and libxl directly

For completeness I'd also suggest

5. Declare it out of scope for xl toolstack to solve the whole
   problem. Merely provide the minimal hooks to enable the layer
   above libxl to solve it. This is effectively QEMU's approach.

Of course, this would mean that any non-libvirt layer using libxl
stil faces the same problem you're facing, so I understand if thats
not desirable from your POV.

> None of the options are really that great.  I'm sure you guys explored
> #1 yourselves before deciding to write your own tools, so I won't cover
> its deficiencies.  #2 and #3 both involve a lot of duplicate effort and
> duplicate code.
> 
> From a global perspective, #4 would seem to make sense, since it allows
> the virtlogd functionality to be more generally used (and potentially
> may be useful in non-virtualization scenarios as well). But it also has
> the cost of working cross-community, maintaining a clean separate
> codebase,And we realize for the libvirt project it's extra work
> for no obvious immediate benefit.

As you say there's not really any pre-existing tools that can easily
fit with the requirements, which is why we ended up creating virtlogd
ourselves - it was either that or OpenStack was going to invent their
own daemon which does what virtlogd does to solve it at the even higher
layer (though they could only fix serial port file based output, not
stderr outout)

So, we wanted a standard solution that libvirt and all apps using
libvirt could rely up unconditionally. From our existing libvirtd,
and codebase in general, we have alot of infrastructure pieces that
made creating virtlogd a pretty easy task. In particular our formal
RPC protocol and handling code for libvirtd and virtlockd, was
able to serve as the basis for virtlogd with little need for extra
code.

This in turn means that having virtlogd as a separate project would
be a major undertaking - it relies on so much libvirt infrastructure
code that to make it into a separate project, we'd have to pull out
a huge pile of libvirt internal code and turn it into a more formal
library that could be shared between an external virtlogd and libvirt.
We've never considered any of this code to be API/ABI stable, so don't
really want to go down that route.  This would also make your option #3
a surprisingly large effort - there's a load of libvirt code you would
have to pull into Xen, or alternatively re-write in a Xen friendly
manner.

Less problematic, though still relevant, is that virtlogd is intended
to align with libvirtd/virtlockd designs, to give a consistent model.
By this I mean the config files are in common locations, the way we
auto-spawn the daemons works the same way - eg we have one libvirtd
running privileged, and one per-user account, and auto-spawn a
corresponding virtlogd for each.

> As Wei said, we're still exploring options; even a negative response
> narrows down the search space. :-)
> 
> Let us know what you think.

I don't have a great answer for you I'm afraid, but I don't think
that #4 is 

Re: [Xen-devel] [libvirt] Questions about virtlogd

[Daniel P. Berrange]

On Tue, Jun 07, 2016 at 01:11:53PM +0100, Wei Liu wrote:
> Hello libvirt maintainers,
> 
> Libvirt implements virtlogd in version 1.3 which now handles logging
> for QEMU process. I am wondering if it is possible to make it a
> separate package and maintain stable interfaces for external users?

Ok, so you're essentially asking for us to create a libvirt-logd.so
library for talking to virtlogd, which would basically contain the
code currently in src/logging/log_manager.c

That's certainly possible from a technical POV, but the real question
is whether we want to do that from a policy POV, given the greater
support implications that has.

> This is related to XSA-180 / CVE-2014-3672 (unrestricted QEMU
> logging). We are evaluating using virtlogd vs writing our own
> solution. I believe there are still some open questions on how exactly
> the integration could be done but let's worry about that later.

I must admit I'm not familiar with the division of responsibility
for managing QEMU between the Xen provided libxl library(s) and
the libvirt libxl driver code. Naively I would expect the libvirt
libxl driver code to deal with virtlogd and then configure the
Xen libxl library / QEMU accordingly. Your request seems to imply
that you will need the Xen libxl library to directly talk to
virtlogd instead.

Is there any way in which it would be practical for the libvirt
libxl driver to talk to virtlogd to acquire the file descriptors
to use and pass those file descriptors down to the libxl library ?

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [libvirt] Request for XEN source code

[Daniel P. Berrange]

On Thu, Apr 14, 2016 at 03:58:55PM +0800, Zakirasafi wrote:
> Dear all I need XEN hyper-visor source code. Kindly send me

This mailing list is for the libvirt project, not the Xen project - libvirt
merely uses Xen. Please use the Xen project website http://www.xenproject.org/
to find what you're looking for.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [libvirt] Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?

[Daniel P. Berrange]

On Wed, Apr 13, 2016 at 10:09:16AM +0100, George Dunlap wrote:
> On Tue, Apr 12, 2016 at 10:31 PM, Jim Fehlig  wrote:
> > Wei Liu wrote:
> >> Hi libvirt maintainers,
> >
> > Sorry for the delay. Slowly catching up on mail after vacation...
> >
> >>
> >> Xen's control library libxenlight (libxl) requires application
> >> (libvirt in this case) to explictily define LIBXL_API_VERSION.
> >
> > Where is this requirement written? :-)
> >
> >> This is
> >> lacking at the moment so libvirt's libxl driver always gets the latest
> >> APIs.
> >
> > IMO, that is what we want for upstream libvirt. Downstreams can choose a
> > specific version if they want.
> 
> I think one of us isn't understanding the situation properly. Is it
> not the case that currently, all releases of libvirt *will not
> compile* against Xen 4.7 once it's released?  So people downloading
> and building libvirt will have to either 1) root around and try to
> figure out what version of Xen it will build against, 2) manually add
> in a #define *with the correct API version* to a header somewhere to
> make it build properly, or 3) update to a version of libvirt that
> supports the new api (which at the moment hasn't even been released
> yet)?
> 
> All of those options are completely unacceptable.  Older versions of
> libvirt should Just Work when compiled against newer versions of Xen.
> 
> I think it does make sense to have the libvirt development branch not
> specify an API version; but when it branches for release, it should
> set LIBXL_API_VERSION to whatever the current version is at the time
> of the branch.

FYI, libvirt doesn't do branching for releases - we always just cut the
release straight from the master branch. We only actually create branches
on demand, when we find we want to backport fixes to a previous release.

Does libvirt master really need to always use the latest API version ?

It feels like libvirt could just set LIBXL_API_VERSION to the lowest
version it requires in order to get the functionality we know we are
able to currently build against. IOW, we'd only need to update the
define for LIBXL_API_VERSION when we merge patches that actually need
the newer functionality.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [libvirt] [PATCH] libxl: libxl_domain_create_restore has an extra argument

[Daniel P. Berrange]

On Wed, Apr 06, 2016 at 04:43:07PM -0500, Doug Goldstein wrote:
> On 4/5/16 9:20 AM, Wei Liu wrote:
> > In the latest libxenlight code, libxl_domain_create_restore accepts a
> > new argument. Update libvirt's libxl driver for that. Use the macro
> > provided by libxenlight to detect which version should be used.
> > 
> > The new parameter (send_back_fd) is set to -1 because libvirt provides
> > no such fd.
> > 
> > Signed-off-by: Wei Liu 
> > ---
> > Build test with Xen 4.6.1 (old API) and Xen unstable (new API).
> > ---
> >  src/libxl/libxl_domain.c | 7 ++-
> >  1 file changed, 6 insertions(+), 1 deletion(-)
> > 
> > diff --git a/src/libxl/libxl_domain.c b/src/libxl/libxl_domain.c
> > index 04962a0..aed904b 100644
> > --- a/src/libxl/libxl_domain.c
> > +++ b/src/libxl/libxl_domain.c
> > @@ -1070,7 +1070,12 @@ libxlDomainStart(libxlDriverPrivatePtr driver, 
> > virDomainObjPtr vm,
> >  ret = libxl_domain_create_new(cfg->ctx, _config,
> >, NULL, _console_how);
> >  } else {
> > -#ifdef LIBXL_HAVE_DOMAIN_CREATE_RESTORE_PARAMS
> > +#if defined(LIBXL_HAVE_DOMAIN_CREATE_RESTORE_SEND_BACK_FD)
> > +params.checkpointed_stream = 0;
> > +ret = libxl_domain_create_restore(cfg->ctx, _config, ,
> > +  restore_fd, -1, , NULL,
> > +  _console_how);
> > +#elif defined(LIBXL_HAVE_DOMAIN_CREATE_RESTORE_PARAMS)
> >  params.checkpointed_stream = 0;
> >  ret = libxl_domain_create_restore(cfg->ctx, _config, ,
> >restore_fd, , NULL,
> > 
> 
> ACK
> 
> This fixes integration testing that Xen Project does with libvirt and
> Xen so it would be nice to get in for 1.3.3.

It missed the boat for 1.3.3, but I've pushed it now



Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [libvirt] [PATCH 4/4] libxl: add support for rbd qdisk

[Daniel P. Berrange]

On Tue, Feb 16, 2016 at 02:45:59PM -0700, Jim Fehlig wrote:
> xl/libxl already supports qemu's network-based block backends
> such as nbd and rbd. libvirt has supported configuring such
> s for long time too. This patch adds support for rbd
> disks in the libxl driver by generating a rbd device URL from
> the virDomainDiskDef object. The URL is passed to libxl via the
> pdev_path field of libxl_device_disk struct. libxl then passes
> the URL to qemu for cosumption by the rbd backend.
> 
> Signed-off-by: Jim Fehlig 
> ---
>  src/libxl/libxl_conf.c | 192 
> -
>  1 file changed, 191 insertions(+), 1 deletion(-)

ACK


Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [libvirt] [PATCH 3/4] xenconfig: support xl<->xml conversion of rbd disk devices

[Daniel P. Berrange]

On Tue, Feb 16, 2016 at 02:45:58PM -0700, Jim Fehlig wrote:
> The target= setting in xl disk configuration can be used to encode
> meta info that is meaningful to a backend. Leverage this fact to
> support qdisk network disk types such as rbd. E.g. xl disk config
> containing
> 
>   disk = [ "format=raw,vdev=hdb,access=rw,backendtype=qdisk,
> target=rbd:pool/image:auth_supported=none:\
> mon_host=mon1.example.org\:6321\;mon2.example.org\:6322\;\
> mon3.example.org\:6322" ]
> 
> can be converted to the following XML (and vice versa)
> 
>
>  
>  
>
>
>
>  
>  
>  
>
> 
> Note that conversion of  config is not handled in this patch,
> but can be done in a follow-up patch.
> 
> Also add a test for the conversions.
> 
> Signed-off-by: Jim Fehlig 
> ---
>  src/xenconfig/xen_xl.c   | 145 
> +--
>  tests/xlconfigdata/test-rbd-multihost-noauth.cfg |  26 
>  tests/xlconfigdata/test-rbd-multihost-noauth.xml |  51 
>  tests/xlconfigtest.c |   1 +
>  4 files changed, 216 insertions(+), 7 deletions(-)

ACK,


Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [libvirt] [PATCH 2/4] xenconfig: produce key=value disk config syntax in xl formatter

[Daniel P. Berrange]

On Tue, Feb 16, 2016 at 02:45:57PM -0700, Jim Fehlig wrote:
> The most formal form of xl disk configuration uses key=value
> syntax to define each configuration item, e.g.
> 
> format=raw, vdev=xvda, access=rw, backendtype=phy, target=disksrc
> 
> Change the xl disk formatter to produce this syntax, which allows
> target= to contain meta info needed to setup a network-based
> disksrc (e.g. rbd, nbd, iscsi). For details on xl disk config
> format, see  $xen-src/docs/misc/xl-disk-configuration.txt
> 
> Update the disk config in the tests to use the formal syntax.
> But add tests to ensure disks specified with the positional
> parameter syntax are correctly converted to  XML.
> 
> Signed-off-by: Jim Fehlig 
> ---
>  src/xenconfig/xen_xl.c | 27 ++-
>  .../test-disk-positional-parms-full.cfg| 26 +++
>  .../test-disk-positional-parms-full.xml| 54 
> ++
>  .../test-disk-positional-parms-partial.cfg | 26 +++
>  .../test-disk-positional-parms-partial.xml | 54 
> ++
>  .../test-fullvirt-direct-kernel-boot.cfg   |  2 +-
>  tests/xlconfigdata/test-fullvirt-multiusb.cfg  |  2 +-
>  tests/xlconfigdata/test-new-disk.cfg   |  2 +-
>  tests/xlconfigdata/test-paravirt-cmdline.cfg   |  2 +-
>  tests/xlconfigdata/test-paravirt-maxvcpus.cfg  |  2 +-
>  tests/xlconfigdata/test-spice-features.cfg |  2 +-
>  tests/xlconfigdata/test-spice.cfg  |  2 +-
>  tests/xlconfigdata/test-vif-rate.cfg   |  2 +-
>  tests/xlconfigtest.c   |  2 +
>  14 files changed, 186 insertions(+), 19 deletions(-)

ACK


Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [libvirt] [PATCH 1/4] xenconfig: replace text 'xm' with 'xl' in xlconfigtest

[Daniel P. Berrange]

On Tue, Feb 16, 2016 at 02:45:56PM -0700, Jim Fehlig wrote:
> While at it, improve a few comments. No functional change.
> 
> Signed-off-by: Jim Fehlig 
> ---
>  tests/xlconfigtest.c | 34 +++---
>  1 file changed, 19 insertions(+), 15 deletions(-)

ACK


Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [Qemu-devel] [PULL v2 44/45] fix MSI injection on Xen

[Daniel P. Berrange]

On Sat, Feb 06, 2016 at 09:15:13PM +0200, Michael S. Tsirkin wrote:
> From: Stefano Stabellini 
> 
> On Xen MSIs can be remapped into pirqs, which are a type of event
> channels. It's mostly for the benefit of PCI passthrough devices, to
> avoid the overhead of interacting with the emulated lapic.
> 
> However remapping interrupts and MSIs is also supported for emulated
> devices, such as the e1000 and virtio-net.
> 
> When an interrupt or an MSI is remapped into a pirq, masking and
> unmasking is done by masking and unmasking the event channel. The
> masking bit on the PCI config space or MSI-X table should be ignored,
> but it isn't at the moment.
> 
> As a consequence emulated devices which use MSI or MSI-X, such as
> virtio-net, don't work properly (the guest doesn't receive any
> notifications). The mechanism was working properly when xen_apic was
> introduced, but I haven't narrowed down which commit in particular is
> causing the regression.
> 
> Fix the issue by ignoring the masking bit for MSI and MSI-X which have
> been remapped into pirqs.
> 
> Signed-off-by: Stefano Stabellini 
> Reviewed-by: Michael S. Tsirkin 
> Signed-off-by: Michael S. Tsirkin 
> ---
>  include/hw/xen/xen.h |  1 +
>  hw/pci/msi.c |  9 -
>  hw/pci/msix.c| 12 ++--
>  hw/xen/xen_pt_msi.c  |  4 +---
>  xen-hvm-stub.c   |  5 +
>  xen-hvm.c|  9 +
>  6 files changed, 34 insertions(+), 6 deletions(-)

According to my git bisect, this change appears to have broken
the ivshmem test suite:

$ QTEST_QEMU_BINARY='x86_64-softmmu/qemu-system-x86_64' QTEST_QEMU_IMG=qemu-img 
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$((RANDOM % 255 + 1))}  gtester -k --verbose 
-m slow tests/ivshmem-test
TEST: tests/ivshmem-test... (pid=17294)
  /x86_64/ivshmem/single:  OK
  /x86_64/ivshmem/hotplug: OK
  /x86_64/ivshmem/memdev:  OK
  /x86_64/ivshmem/pair:OK
  /x86_64/ivshmem/server-msi:  **
ERROR:tests/ivshmem-test.c:358:test_ivshmem_server: assertion failed (ret != 
0): (0 != 0)
FAIL
GTester: last random seed: R02S00888b8a4102476be4f9fec733a717ac
(pid=17328)
  /x86_64/ivshmem/server-irq:  OK
FAIL: tests/ivshmem-test

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [libvirt] [PATCH LIBVIRT] libxl: Use libxentoollog in preference to libxenctrl if available.

[Daniel P. Berrange]

On Thu, Dec 10, 2015 at 11:38:36AM +, Ian Campbell wrote:
> Upstream Xen is in the process of splitting the (stable API) xtl_*
> interfaces out from the (unstable API) libxenctrl library and into a
> new (stable API) libxentoollog.
> 
> In order to be compatible with Xen both before and after this
> transition check for xtl_createlogger_stdiostream in a libxentoollog
> library and use it if present. If it is not present assume it is in
> libxenctrl.

Ok, so there's no API changes, just move stuf from one to the other.

> It might be nice to get this into 1.3.0 so that supports Xen 4.7 out
> of the box? Not sure what the libvirt stable backport policy is but it
> might also be good to eventually consider it for that?

We've missed 1.3.0 release, but I'd be ok with adding it to the
stable branch if that's going to be useful.

> diff --git a/configure.ac b/configure.ac
> index 98cf210..b641cc7 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -883,7 +883,6 @@ if test "$with_libxl" != "no" ; then
>  PKG_CHECK_MODULES([LIBXL], [xenlight], [
>   LIBXL_FIRMWARE_DIR=`$PKG_CONFIG --variable xenfirmwaredir xenlight`
>   LIBXL_EXECBIN_DIR=`$PKG_CONFIG --variable libexec_bin xenlight`
> - LIBXL_LIBS="$LIBXL_LIBS -lxenctrl"
>   with_libxl=yes
>  ], [LIBXL_FOUND=no])
>  if test "$LIBXL_FOUND" = "no"; then
> @@ -896,7 +895,7 @@ if test "$with_libxl" != "no" ; then
>  LIBS="$LIBS $LIBXL_LIBS"
>  AC_CHECK_LIB([xenlight], [libxl_ctx_alloc], [
>  with_libxl=yes
> -LIBXL_LIBS="$LIBXL_LIBS -lxenlight -lxenctrl"
> +LIBXL_LIBS="$LIBXL_LIBS -lxenlight"
>  ],[
>  if test "$with_libxl" = "yes"; then
>  fail=1
> @@ -924,6 +923,14 @@ if test "$with_libxl" = "yes"; then
>  if test "x$LIBXL_EXECBIN_DIR" != "x"; then
>  AC_DEFINE_UNQUOTED([LIBXL_EXECBIN_DIR], ["$LIBXL_EXECBIN_DIR"], 
> [directory containing Xen libexec binaries])
>  fi
> +dnl Check if the xtl_* infrastructure is in libxentoollog
> +dnl (since Xen 4.7) if not then assume it is in libxenctrl
> +dnl (as it was for 4.6 and earler)
> +AC_CHECK_LIB([xentoollog], [xtl_createlogger_stdiostream], [
> +LIBXL_LIBS="$LIBXL_LIBS -lxentoollog"
> +],[
> +LIBXL_LIBS="$LIBXL_LIBS -lxenctrl"
> +])
>  fi
>  AM_CONDITIONAL([WITH_LIBXL], [test "$with_libxl" = "yes"])

Looks, fine from me but will let Jim push it.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [libvirt] [FOR 1.3.0 PATCH] conf: add net device prefix for Xen

[Daniel P. Berrange]

On Mon, Dec 07, 2015 at 09:42:21AM -0700, Jim Fehlig wrote:
> In commit d2e5538b1, the libxl driver was changed to copy interface
> names autogenerated by libxl to the corresponding network def in the
> domain's virDomainDef object. The copied name is freed when the domain
> transitions to the shutoff state. But when migrating a domain, the
> autogenerated name is included in the XML sent to the destination host.
> It is possible an interface with the same name already exists on the
> destination host, causing migration to fail. Indeed the Xen project's
> OSSTEST CI already encountered such a failure.
> 
> This patch defines another VIR_NET_GENERATED_PREFIX for Xen, allowing
> the autogenerated names to be excluded when parsing and formatting
> inactive config.
> 
> Signed-off-by: Jim Fehlig 
> ---
> 
> This is an alternative approach to Joao's fix for this regression
> 
> https://www.redhat.com/archives/libvir-list/2015-December/msg00197.html
> 
> I think it is the same approach used by the qemu driver. My only
> reservation is that it expands the potential for clashes with
> user-defined names. I.e. with this change both 'vnet' and 'vif' are
> reserved prefixes.

Hmm, yes, tricky one.

If we only care about XML parsing, then you could register a post
parse callback instead to do this.

I'm not clear why we also have it in the virDomainNetDefFormat
method - and we can't solve that with a post-parse callback.


The other option would be to make the reserved prefix be a
capability that the parser/formatter could read.

> 
>  src/conf/domain_conf.c   | 6 --
>  src/conf/domain_conf.h   | 4 
>  src/libxl/libxl_domain.c | 5 +++--
>  3 files changed, 11 insertions(+), 4 deletions(-)
> 
> diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
> index 2f5c0ed..debcf4e 100644
> --- a/src/conf/domain_conf.c
> +++ b/src/conf/domain_conf.c
> @@ -8428,7 +8428,8 @@ virDomainNetDefParseXML(virDomainXMLOptionPtr xmlopt,
>  ifname = virXMLPropString(cur, "dev");
>  if (ifname &&
>  (flags & VIR_DOMAIN_DEF_PARSE_INACTIVE) &&
> -STRPREFIX(ifname, VIR_NET_GENERATED_PREFIX)) {
> +(STRPREFIX(ifname, VIR_NET_GENERATED_PREFIX) ||
> + STRPREFIX(ifname, VIR_NET_GENERATED_PREFIX_XEN))) {
>  /* An auto-generated target name, blank it out */
>  VIR_FREE(ifname);
>  }
> @@ -19790,7 +19791,8 @@ virDomainNetDefFormat(virBufferPtr buf,
>  virBufferEscapeString(buf, "\n", 
> def->domain_name);
>  if (def->ifname &&
>  !((flags & VIR_DOMAIN_DEF_FORMAT_INACTIVE) &&
> -  (STRPREFIX(def->ifname, VIR_NET_GENERATED_PREFIX {
> +  (STRPREFIX(def->ifname, VIR_NET_GENERATED_PREFIX) ||
> +   STRPREFIX(def->ifname, VIR_NET_GENERATED_PREFIX_XEN {
>  /* Skip auto-generated target names for inactive config. */
>  virBufferEscapeString(buf, "\n", def->ifname);
>  }
> diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h
> index 90d8e13..d2cc26f 100644
> --- a/src/conf/domain_conf.h
> +++ b/src/conf/domain_conf.h
> @@ -1085,6 +1085,10 @@ struct _virDomainNetDef {
>   * by libvirt, and cannot be used for a persistent network name.  */
>  # define VIR_NET_GENERATED_PREFIX "vnet"
>  
> +/* Used for prefix of ifname of any network name generated dynamically
> + * by libvirt for Xen, and cannot be used for a persistent network name.  */
> +# define VIR_NET_GENERATED_PREFIX_XEN "vif"
> +
>  typedef enum {
>  VIR_DOMAIN_CHR_DEVICE_STATE_DEFAULT = 0,
>  VIR_DOMAIN_CHR_DEVICE_STATE_CONNECTED,
> diff --git a/src/libxl/libxl_domain.c b/src/libxl/libxl_domain.c
> index ef92974..c5d84a4 100644
> --- a/src/libxl/libxl_domain.c
> +++ b/src/libxl/libxl_domain.c
> @@ -734,7 +734,7 @@ libxlDomainCleanup(libxlDriverPrivatePtr driver,
>  for (i = 0; i < vm->def->nnets; i++) {
>  virDomainNetDefPtr net = vm->def->nets[i];
>  
> -if (STRPREFIX(net->ifname, "vif"))
> +if (STRPREFIX(net->ifname, VIR_NET_GENERATED_PREFIX_XEN))
>  VIR_FREE(net->ifname);
>  }
>  }
> @@ -918,7 +918,8 @@ libxlDomainCreateIfaceNames(virDomainDefPtr def, 
> libxl_domain_config *d_config)
>  if (net->ifname)
>  continue;
>  
> -ignore_value(virAsprintf(>ifname, "vif%d.%d%s",
> +ignore_value(virAsprintf(>ifname,
> + VIR_NET_GENERATED_PREFIX_XEN "%d.%d%s",
>   def->id, x_nic->devid, suffix));
>  }
>  }
> -- 
> 2.6.1
> 
> --
> libvir-list mailing list
> libvir-l...@redhat.com
> https://www.redhat.com/mailman/listinfo/libvir-list

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- 

Re: [Xen-devel] [PATCH LIBVIRT v1 1/2] libxl: Correct value for xendConfigVersion to xen{Parse, Format}ConfigCommon

[Daniel P. Berrange]

On Thu, Dec 03, 2015 at 11:13:06PM -0700, Jim Fehlig wrote:
> On 11/26/2015 09:59 AM, Ian Campbell wrote:
> > libxlConnectDomainXMLFromNative calls both xenParseXM and xenParseXL
> > with cfg->verInfo->xen_version_major, however AFAICT they both (either
> > inherently, or through there use of xenParseConfigCommon expect a
> > value from xenConfigVersionEnum (which does not correspond to
> > xen_version_major).
> 
> I recall being a little apprehensive about using xen_version_major when 
> writing
> the code, and apparently that was justified. But now that we are revisiting
> this, I wonder if we care about these old xend config versions at all. Is 
> anyone
> even running Xen 3.0.x, or 3.1.x, particularly with a newish libvirt? IMO we
> could drop all of this xend config nonsense and go with the code associated 
> with
> the last xend config version, i.e. XEND_CONFIG_VERSION_3_1_0.
> 
> And while mentioning dropping support for these old xend config versions, do I
> dare mention dropping the xend driver altogether? :-) It will certainly need 
> to
> be done some day. Question is whether that's a bit premature now.

We just decided to drop QEMU driver code supporting for RHEL-5 vintage
distros, requiring RHEL-6 as minimum. So I think it is entirely reasonable
to drop Xen driver code supporting similar vintage XenD.  That would
certainly simplify or even eliminate the current crazy xend_config_version
code we have

I think we need to continue suppoorting XenD driver for a while, but at
least you can simplify the code shared with libxl.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH LIBVIRT v1 1/2] libxl: Correct value for xendConfigVersion to xen{Parse, Format}ConfigCommon

[Daniel P. Berrange]

On Fri, Dec 04, 2015 at 02:19:33PM +, Ian Campbell wrote:
> On Fri, 2015-12-04 at 10:01 +0000, Daniel P. Berrange wrote:
> > On Thu, Dec 03, 2015 at 11:13:06PM -0700, Jim Fehlig wrote:
> > > On 11/26/2015 09:59 AM, Ian Campbell wrote:
> > > > libxlConnectDomainXMLFromNative calls both xenParseXM and xenParseXL
> > > > with cfg->verInfo->xen_version_major, however AFAICT they both
> > > > (either
> > > > inherently, or through there use of xenParseConfigCommon expect a
> > > > value from xenConfigVersionEnum (which does not correspond to
> > > > xen_version_major).
> > > 
> > > I recall being a little apprehensive about using xen_version_major when
> > > writing
> > > the code, and apparently that was justified. But now that we are
> > > revisiting
> > > this, I wonder if we care about these old xend config versions at all.
> > > Is anyone
> > > even running Xen 3.0.x, or 3.1.x, particularly with a newish libvirt?
> > > IMO we
> > > could drop all of this xend config nonsense and go with the code
> > > associated with
> > > the last xend config version, i.e. XEND_CONFIG_VERSION_3_1_0.
> > > 
> > > And while mentioning dropping support for these old xend config
> > > versions, do I
> > > dare mention dropping the xend driver altogether? :-) It will certainly
> > > need to
> > > be done some day. Question is whether that's a bit premature now.
> > 
> > We just decided to drop QEMU driver code supporting for RHEL-5 vintage
> > distros, requiring RHEL-6 as minimum. So I think it is entirely reasonable
> > to drop Xen driver code supporting similar vintage XenD.  That would
> > certainly simplify or even eliminate the current crazy xend_config_version
> > code we have
> 
> RHEL 6.0 looks[0] to have been release on 2010-11-09, which was in the
> middle of Xen 4.0 and 4.1[1]. 4.0 seems like a decent enough cut off point
> (and is what is in Debian oldstable AKA Wheezy, FWIW) plus it is after the
> currently latest XEND_CONFIG_VERSION, so all that code could be removed.
> 
> Shall I respin this series with that as a precursor?

Yeah, that sounds reasonable. Would let us drop all Xen 3.x support
essentially.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [libvirt] [PATCH] libxl: open libxl log stream with libvirtd log_level

[Daniel P. Berrange]

On Wed, Nov 11, 2015 at 04:57:59PM +0100, Marek Marczykowski-Górecki wrote:
> On Tue, Sep 15, 2015 at 11:21:00AM -0600, Jim Fehlig wrote:
> > Daniel P. Berrange wrote:
> > > On Tue, Sep 15, 2015 at 10:57:50AM -0600, Jim Fehlig wrote:
> > >   
> > >> Daniel P. Berrange wrote:
> > >> 
> > >>> On Tue, Sep 15, 2015 at 09:26:23AM -0600, Jim Fehlig wrote:
> > >>>   
> > >>>   
> > >>>> Instead of a hardcoded DEBUG log level, use the overall
> > >>>> daemon log level specified in libvirtd.conf when opening
> > >>>> a log stream with libxl. libxl is very verbose when DEBUG
> > >>>> log level is set, resulting in huge log files that can
> > >>>> potentially fill a disk. Control of libxl verbosity should
> > >>>> be placed in the administrator's hands.
> > >>>>
> > >>>> Signed-off-by: Jim Fehlig <jfeh...@suse.com>
> > >>>> ---
> > >>>>  src/libxl/libxl_conf.c | 18 +-
> > >>>>  1 file changed, 17 insertions(+), 1 deletion(-)
> > >>>> 
> > >>>> 
> > >>> ACK, this makes sense as default behaviour. As a future enhancement
> > >>> you might also consider supporting a config setting in 
> > >>> /etc/libvirt/libxl.conf
> > >>> to explicitly control the libxl library logging behaviour independantly.
> > >>>   
> > >>>   
> > >> I had actually thought of adding it there first, but then took this
> > >> approach assuming it would be more receptive upstream :-). Personally,
> > >> I'm on the fence. I like the idea of a single knob to control log level
> > >> throughout the daemon, making it a bit easier on admins. On the other
> > >> hand, individual knobs are more friendly to those pouring through logs.
> > >> I can add a knob in libxl.conf if preferred.
> > >> 
> > >
> > > After thinking about it some more, I could actually see value in
> > > create a dedicated virLogSource() instance, solely for libxl
> > > library messages. If we then created a virLogSourceGetPriority()
> > > method, you could query that to see if to turn on logging for
> > > the libxl library. That would ultimately allow you to turn on
> > > debug for just the libxl library if desired, eg
> > >
> > >  static virLogSource virLogLibXL = {
> > >  .name = "libxl.libxl_library",
> > >  
> > >  }
> > >
> > > LIBVIRT_LOG_FILTERS="1:libxl_library"
> > >   
> > 
> > Ah, good idea. I'll look into it.
> 
> Is it done anywhere? If not, how can I help?
> 
> This the above change (setting libxl log level to libvirtd global one),
> makes almost impossible to get libxl debug, because the rest of libvirtd
> trashes the logs (hundreds virEvent* and virObject* messages, at
> "info"(!) level).

Presumably that's because you are using LIBVIRT_DEBUG=1 or setting
log_debug=1 in /etc/libvirt/libvirtd.log. That is generally discouraged
because it turns on all logging, which is essentially unusably verbose.

It is also better to use more specific log settings, eg when I debug
the QEMU driver I usually use

  log_filters="1:qemu 1:security 3:event 3:object 1:util"
  log_outputs="1:file:/var/log/libvirt/libvirtd.log"

This generates log messages from every source file in qemu and
security directories, and also log messages from every file
in the util/ directory *except* for event & object files.
So it is much more managable to read.

The log filter strings just do a wildcard match against the
VIR_LOG_INIT(""); lines in each source file, so you can
make things more or less specific as you desire.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [libvirt] [PATCH] libxl: open libxl log stream with libvirtd log_level

[Daniel P. Berrange]

On Tue, Sep 15, 2015 at 09:26:23AM -0600, Jim Fehlig wrote:
> Instead of a hardcoded DEBUG log level, use the overall
> daemon log level specified in libvirtd.conf when opening
> a log stream with libxl. libxl is very verbose when DEBUG
> log level is set, resulting in huge log files that can
> potentially fill a disk. Control of libxl verbosity should
> be placed in the administrator's hands.
> 
> Signed-off-by: Jim Fehlig 
> ---
>  src/libxl/libxl_conf.c | 18 +-
>  1 file changed, 17 insertions(+), 1 deletion(-)

ACK, this makes sense as default behaviour. As a future enhancement
you might also consider supporting a config setting in /etc/libvirt/libxl.conf
to explicitly control the libxl library logging behaviour independantly.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [libvirt] [PATCH] libxl: open libxl log stream with libvirtd log_level

[Daniel P. Berrange]

On Tue, Sep 15, 2015 at 10:57:50AM -0600, Jim Fehlig wrote:
> Daniel P. Berrange wrote:
> > On Tue, Sep 15, 2015 at 09:26:23AM -0600, Jim Fehlig wrote:
> >   
> >> Instead of a hardcoded DEBUG log level, use the overall
> >> daemon log level specified in libvirtd.conf when opening
> >> a log stream with libxl. libxl is very verbose when DEBUG
> >> log level is set, resulting in huge log files that can
> >> potentially fill a disk. Control of libxl verbosity should
> >> be placed in the administrator's hands.
> >>
> >> Signed-off-by: Jim Fehlig <jfeh...@suse.com>
> >> ---
> >>  src/libxl/libxl_conf.c | 18 +-
> >>  1 file changed, 17 insertions(+), 1 deletion(-)
> >> 
> >
> > ACK, this makes sense as default behaviour. As a future enhancement
> > you might also consider supporting a config setting in 
> > /etc/libvirt/libxl.conf
> > to explicitly control the libxl library logging behaviour independantly.
> >   
> 
> I had actually thought of adding it there first, but then took this
> approach assuming it would be more receptive upstream :-). Personally,
> I'm on the fence. I like the idea of a single knob to control log level
> throughout the daemon, making it a bit easier on admins. On the other
> hand, individual knobs are more friendly to those pouring through logs.
> I can add a knob in libxl.conf if preferred.

After thinking about it some more, I could actually see value in
create a dedicated virLogSource() instance, solely for libxl
library messages. If we then created a virLogSourceGetPriority()
method, you could query that to see if to turn on logging for
the libxl library. That would ultimately allow you to turn on
debug for just the libxl library if desired, eg

 static virLogSource virLogLibXL = {
 .name = "libxl.libxl_library",
 
 }

LIBVIRT_LOG_FILTERS="1:libxl_library"

Regardless we should just merge your current patch right now.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [Qemu-devel] [edk2] EDK II & GPL - Re: OVMF BoF @ KVM Forum 2015

[Daniel P. Berrange]

On Wed, Sep 09, 2015 at 05:41:59PM -0700, Jordan Justen wrote:
> On 2015-09-09 16:05:20, Andrew Fish wrote:
> > 
> > > On Sep 9, 2015, at 3:24 PM, Jordan Justen  
> > > wrot> > > FWIW, I don't mind if the consensus is that GplDriverPkg must 
> > > live in
> > > a separate repo. But, it would be nice to hear a good reason why it
> > > must live elsewhere.
> > 
> > Because GPL is not a permissive license. An accidental git grep and
> > copying some code can change the license of the code that gets the
> > GPL code pasted into it.
> 
> I like this argument. It is slightly tempered by the fact that git
> grep always shows the source path, and thus 'GplDriverPkg' would be
> obviously visible.

Plenty of projects have a scenario in which different parts of their
codebase are under different licenses, without there being undue
problems. If you make it clear by having a separate directory, then
I think you can ultimately credit the developers with having enough
intelligence to do the right thing here. If not, then I'd probably
question whether you can trust them to submit any code at all, as
they could equally have blindly copied it from a 3rd party project
under an incompatible license.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [Qemu-devel] [PATCH] Do not emulate a floppy drive when -nodefaults

[Daniel P. Berrange]

On Thu, May 14, 2015 at 02:45:30PM +0200, Markus Armbruster wrote:
 Paolo Bonzini pbonz...@redhat.com writes:
 
  On 14/05/2015 14:02, Markus Armbruster wrote:
It should certainly be off for pc-q35-2.4 and newer.  Real Q35 boards
commonly don't have an FDC (depends on the Super I/O chip used).
  
We may want to keep it off for pc-i440fx-2.4 and newer.  I doubt
there's a real i440FX without an FDC, but our virtual i440FX is quite
unlike a real one in other ways already.
 
  That would break libvirt for people upgrading from 2.3 to 2.4.  So it's
  more like pc-i440fx-3.0 and pc-q35-3.0.
 
 What exactly breaks when?

[quote]
  * Create the FDC only if the option is on.

  * Optional: make -drive if=floppy,... auto-enable it

I wouldn't bother doing the same for -global isa-fdc.driveA=... and
such.
[/quote]

Libvirt uses -global when enabling floppy devices. So since current libvirt
does not know about the new (to be created) machine type property to turn
on FDC, it will get an error using -global isa-fdc.driveA=

I'm not too bothered about this, as long as libvirt has enough advance
notice to add support for the new machine type property to enable FDC
before we change its default value to be off. Perhaps one QEMU major
release cycle before toggling the default, to give time for new libvirt
to penetrate to distros

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [Qemu-devel] [PATCH] Do not emulate a floppy drive when -nodefaults

[Daniel P. Berrange]

On Wed, May 13, 2015 at 06:29:46PM +0100, Stefano Stabellini wrote:
 Do not emulate a floppy drive if no drives are supposed to be present.
 
 This fixes the behavior of -nodefaults, that should remove the floppy
 drive (see docs/qdev-device-use.txt:Default Devices), but actually
 doesn't.

Technically that doc is just refering to the disablement of the
primary floppy drive, as opposed to the floppy controller. The
floppy controller itself is really a built-in device that is
defined as part of the machine type, along with the various other
platform devices hanging off the PIIX and ISA brige.

 diff --git a/hw/i386/pc.c b/hw/i386/pc.c
 index a8e6be1..c9f50b3 100644
 --- a/hw/i386/pc.c
 +++ b/hw/i386/pc.c
 @@ -1410,6 +1410,7 @@ void pc_basic_device_init(ISABus *isa_bus, qemu_irq 
 *gsi,
  qemu_irq *cpu_exit_irq;
  MemoryRegion *ioport80_io = g_new(MemoryRegion, 1);
  MemoryRegion *ioportF0_io = g_new(MemoryRegion, 1);
 +bool floppy_exist;
  
  memory_region_init_io(ioport80_io, NULL, ioport80_io_ops, NULL, 
 ioport80, 1);
  memory_region_add_subregion(isa_bus-address_space_io, 0x80, 
 ioport80_io);
 @@ -1487,10 +1488,17 @@ void pc_basic_device_init(ISABus *isa_bus, qemu_irq 
 *gsi,
  cpu_exit_irq = qemu_allocate_irqs(cpu_request_exit, NULL, 1);
  DMA_init(0, cpu_exit_irq);
  
 +*floppy = NULL;
 +floppy_exist = false;
  for(i = 0; i  MAX_FD; i++) {
  fd[i] = drive_get(IF_FLOPPY, 0, i);
 +if (fd[i] != NULL) {
 +floppy_exist = true;
 +}
 +}
 +if (floppy_exist) {
 +*floppy = fdctrl_init_isa(isa_bus, fd);
  }
 -*floppy = fdctrl_init_isa(isa_bus, fd);
  }

This results in a guest ABI change when updating QEMU, because the floppy
controller will disappear for existing guests. This is liable to break
guests upon migration.

If we want to be able to disable the floppy controller itself, in addition
to the floppy drives, then we'd likely need to define a property against
the machine type to allow its existence to be toggled on/off. We'd then
need to at least make sure the existing machine types defaulted to on,
if we decided that new machine types should default to off.

Libvirt would also need to gain the ability to represent the existance
of the floppy controller and allow it to be turned on / off explicitly.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [libvirt] [libvirt test] 50401: regressions - FAIL

[Daniel P. Berrange]

On Wed, Apr 15, 2015 at 11:17:29AM +0100, Ian Campbell wrote:
 On Tue, 2015-04-14 at 10:37 +0100, Daniel P. Berrange wrote:
  On Tue, Apr 14, 2015 at 10:33:45AM +0100, Ian Campbell wrote:
   On Tue, 2015-04-14 at 02:27 +, osstest service user wrote:
flight 50401 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/50401/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-armhf-libvirt   5 libvirt-build fail REGR. 
vs. 50368
   [...]
   Per
   http://logs.test-lab.xenproject.org/osstest/logs/50401/build-armhf-libvirt/5.ts-libvirt-build.log
this is:
   
   qemu/qemu_driver.c: In function 'qemuDomainAddCgroupForThread':
   qemu/qemu_driver.c:4641:34: error: declaration of 'index' shadows a 
   global declaration [-Werror=shadow]
   qemu/qemu_driver.c: In function 'qemuDomainHotplugAddPin':
   qemu/qemu_driver.c:4674:29: error: declaration of 'index' shadows a 
   global declaration [-Werror=shadow]
   qemu/qemu_driver.c: In function 'qemuDomainHotplugPinThread':
   qemu/qemu_driver.c:4702:32: error: declaration of 'index' shadows a 
   global declaration [-Werror=shadow]
   qemu/qemu_driver.c: In function 'qemuDomainDelCgroupForThread':
   qemu/qemu_driver.c:4733:34: error: declaration of 'index' shadows a 
   global declaration [-Werror=shadow]
   cc1: all warnings being treated as errors
   
   
   This seems to be a general issue unrelated to Xen.
   
version targeted for testing:
 libvirt  b487bb810ec95df862e7e80468c8e861ed80b0cb
baseline version:
 libvirt  225aa80246d5e4a9e3a16ebd4c482525045da3db
   
   After a quick glance I don't see a fix post-b487bb810ec9 either in
   master or on the libvirt list.
   
   Looking at the range under test it looks like one or more of John's
   changes is adding parameters called index, shadowing index(3) from
   strings.h.
  
  Yeah, we've had this problem several times before - we usually just
  do a s/index/idx/ or similar to address it.
 
 I see this is now fixed in libvirt.git#master, thanks.
 
 However, I would just comment that contrary to the commit message, I
 don't think there is anything the Xen build has done which caused this,
 I think it's down to the LIBVIRT_COMPILE_WARNINGS macro which ends up
 enabling Wshadow if it was available on the system which ran autogen and
 on the compiling system.
 
 Our builds run on Debian Wheezy, which IIRC uses gcc 4.4 which isn't
 unusual...

Yeah, there is nothing Xen specific about the problem - it is entirely
down to the build toolchain  compiler options.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [libvirt] [libvirt test] 50401: regressions - FAIL

[Daniel P. Berrange]

On Tue, Apr 14, 2015 at 10:33:45AM +0100, Ian Campbell wrote:
 On Tue, 2015-04-14 at 02:27 +, osstest service user wrote:
  flight 50401 libvirt real [real]
  http://logs.test-lab.xenproject.org/osstest/logs/50401/
  
  Regressions :-(
  
  Tests which did not succeed and are blocking,
  including tests which could not be run:
   build-armhf-libvirt   5 libvirt-build fail REGR. vs. 
  50368
 [...]
 Per
 http://logs.test-lab.xenproject.org/osstest/logs/50401/build-armhf-libvirt/5.ts-libvirt-build.log
  this is:
 
 qemu/qemu_driver.c: In function 'qemuDomainAddCgroupForThread':
 qemu/qemu_driver.c:4641:34: error: declaration of 'index' shadows a global 
 declaration [-Werror=shadow]
 qemu/qemu_driver.c: In function 'qemuDomainHotplugAddPin':
 qemu/qemu_driver.c:4674:29: error: declaration of 'index' shadows a global 
 declaration [-Werror=shadow]
 qemu/qemu_driver.c: In function 'qemuDomainHotplugPinThread':
 qemu/qemu_driver.c:4702:32: error: declaration of 'index' shadows a global 
 declaration [-Werror=shadow]
 qemu/qemu_driver.c: In function 'qemuDomainDelCgroupForThread':
 qemu/qemu_driver.c:4733:34: error: declaration of 'index' shadows a global 
 declaration [-Werror=shadow]
 cc1: all warnings being treated as errors
 
 
 This seems to be a general issue unrelated to Xen.
 
  version targeted for testing:
   libvirt  b487bb810ec95df862e7e80468c8e861ed80b0cb
  baseline version:
   libvirt  225aa80246d5e4a9e3a16ebd4c482525045da3db
 
 After a quick glance I don't see a fix post-b487bb810ec9 either in
 master or on the libvirt list.
 
 Looking at the range under test it looks like one or more of John's
 changes is adding parameters called index, shadowing index(3) from
 strings.h.

Yeah, we've had this problem several times before - we usually just
do a s/index/idx/ or similar to address it.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [libvirt] libvirt/libxl implemetation of get_online_cpu / virNodeGetCPUMap?

[Daniel P. Berrange]

On Tue, Feb 24, 2015 at 03:00:16PM +, Anthony PERARD wrote:
 On Tue, Feb 24, 2015 at 01:22:19PM +, Daniel P. Berrange wrote:
  On Tue, Feb 24, 2015 at 01:15:57PM +, Anthony PERARD wrote:
   On Tue, Feb 24, 2015 at 12:46:44PM +, Daniel P. Berrange wrote:
On Tue, Feb 24, 2015 at 12:41:01PM +, Anthony PERARD wrote:
 Hi,
 
 A recent OpenStack nova commit make use of virNodeGetCPUMap to get 
 the list
 of online cpu of a host. But this API is not implemented for the 
 libvirt
 xen driver.
 
 The commit:
   Add handling for offlined CPUs to the nova libvirt driver.
 https://review.openstack.org/gitweb?p=openstack/nova.git;a=commitdiff;h=0696a5cd5f0fdc08951a074961bb8ce0c3310086

FWIW, this should not impact Xen based on my understanding. The code
path in question should only be used when Nova is setup todo NUMA
pinning support, and that is not supported with Xen in OpenStack,
only KVM.  Did it actually cause failures for you, or are you simply
keeping track of all used APIs in Nova as a sanity check ?
   
   It prevent nova from starting. I do the setup with DevStack.
   
   The error:
   libvirtError: this function is not supported by the connection driver: 
   virNodeGetCPUMap
   
   And a part of the traceback:
 File /opt/stack/nova/nova/openstack/common/service.py, line 491, in 
   run_service
   service.start()
 File /opt/stack/nova/nova/service.py, line 181, in start
   self.manager.pre_start_hook()
 File /opt/stack/nova/nova/compute/manager.py, line 1188, in 
   pre_start_hook
   self.update_available_resource(nova.context.get_admin_context())
 File /opt/stack/nova/nova/compute/manager.py, line 6062, in 
   update_available_resource
   rt.update_available_resource(context)
 File /opt/stack/nova/nova/compute/resource_tracker.py, line 315, in 
   update_available_resource
   resources = self.driver.get_available_resource(self.nodename)
 File /opt/stack/nova/nova/virt/libvirt/driver.py, line 4896, in 
   get_available_resource
   numa_topology = self._get_host_numa_topology()
 File /opt/stack/nova/nova/virt/libvirt/driver.py, line 4749, in 
   _get_host_numa_topology
   online_cpus = self._host.get_online_cpus()
 File /opt/stack/nova/nova/virt/libvirt/host.py, line 599, in 
   get_online_cpus
   (cpus, cpu_map, online) = self.get_connection().getCPUMap()
   
   I'll look into why nova is going through NUMA code paths then.
  
  Oh damn, yes, I understand why now. Please file a bug against Nova for
  this, as we must fix it as a high pripority. It was certainly not my
  intention to break Xen when I approved this change
 
 Here is the bug report: https://bugs.launchpad.net/nova/+bug/1425115

If you are able to test a fix easily then try the patch here:

  https://review.openstack.org/#/c/159106/

If you're able to add a comment to the review indicating that you've
confirmed it fixes Xen, that'd be useful, since there's no automated
testing of Xen that reports on reviews.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [libvirt] libvirt/libxl implemetation of get_online_cpu / virNodeGetCPUMap?

[Daniel P. Berrange]

On Wed, Feb 25, 2015 at 10:24:37AM +0100, Dario Faggioli wrote:
 On Tue, 2015-02-24 at 13:10 +, Ian Campbell wrote:
  On Tue, 2015-02-24 at 12:41 +, Anthony PERARD wrote:
 
   What libxl API those provide this information, if it exist?
   
   I found libxl_get_online_cpus() but that not enough. They want a
   bitmap.
  
  I think that is all which currently exists, at least at the libxl level,
  you may need to add a new interface.
  
  It'd be worth looking into the various host numa interfaces -- perhaps
  one of them indirectly exposes what you want?
  
 Given Daniel's latest emails, I'm not sure this is useful but
 libxl_get_cpu_topology() should put LIBXL_CPUTOPOLOGY_INVALID_ENTRY in
 all the fields of the i-eth element of the array it returns, if the
 i-eth pcpu is offline (see the implementation of XEN_SYSCTL_topologyinfo
 in xen/common/sysctl.c).
 
 So, scanning that array and constructing the bitmap according to whether
 or not we find that marker on the various elements would be the way to
 go, I would say.
 
 I've actually never tested this, i.e., I've never tried offlining a pcpu
 on the host. I'll give it a go as soon as I find 5 minutes, and let know
 if it works.

FWIW, this code in openstack was only added for benefit of s390
architecture where apparently it is common to have hosts with
CPUs offlined. Presumably you have to pay IBM for each extra
CPU you turn online :)

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [libvirt] libvirt/libxl implemetation of get_online_cpu / virNodeGetCPUMap?

[Daniel P. Berrange]

On Wed, Feb 25, 2015 at 03:13:36PM +, Ian Campbell wrote:
 On Wed, 2015-02-25 at 15:03 +, Daniel P. Berrange wrote:
  FWIW, this code in openstack was only added for benefit of s390
  architecture where apparently it is common to have hosts with
  CPUs offlined. Presumably you have to pay IBM for each extra
  CPU you turn online :)
 
 Presumably :-)
 
 OOI, why does the code care which CPUs are online rather than just the
 total number (IOW why a bitmap)?

When doing strict CPU pinning,the openstack scheduler needs to have the
list of all pCPUs available in the host. It then tries to place guests
on pCPUs such that the guest does not span across NUMA nodes. To do this
it needs to know which particular pCPUs in the host are available. So we
need the full bitmap rather than just a total count.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [libvirt] [PATCH v3 1/3] Add virXMLValidateAgainstSchema helper method

[Daniel P. Berrange]

On Fri, Jan 16, 2015 at 01:58:27PM +, Ian Campbell wrote:
 Hello,
 
 On Tue, 2015-01-13 at 17:00 +, Daniel P. Berrange wrote:
  +#  define VIR_WARNINGS_NO_PRINTF \
  +_Pragma (GCC diagnostic push) \
  +_Pragma (GCC diagnostic ignored \-Wsuggest-attribute=format\)
 
 Xen automated tests are failing to build on all architectures with:
 
 util/virxml.c: In function 'catchRNGError':
 util/virxml.c:1094:9: error: unknown option after '#pragma GCC 
 diagnostic' kind [-Werror=pragmas]
 
 which I think must be down to one of these additions.
 
 (helpful of gcc not to print the unknown option in question!)
 
 test overview:
 http://www.chiark.greenend.org.uk/~xensrcts/logs/33443/
 specific failure log:
 http://www.chiark.greenend.org.uk/~xensrcts/logs/33443/build-amd64-libvirt/5.ts-libvirt-build.log
 
 We use Debian Wheezy's gcc, which is 4.6.3 AFAIK.

The configure logs show

checking whether C compiler handles -Wsuggest-attribute=const... yes
checking whether C compiler handles -Wsuggest-attribute=format... no
checking whether C compiler handles -Wsuggest-attribute=noreturn... yes
checking whether C compiler handles -Wsuggest-attribute=pure... yes

So, can someone with a Debian machine check if it helps to modify

_Pragma (GCC diagnostic ignored \-Wsuggest-attribute=format\)

To be just

_Pragma (GCC diagnostic ignored \-Wsuggest-attribute\)

THough, I guess some very old gcc might not support -Wsuggest-attribute
at all, so perhaps we need to check this fully

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [libvirt] [PATCH v3 1/3] Add virXMLValidateAgainstSchema helper method

[Daniel P. Berrange]

On Fri, Jan 16, 2015 at 02:47:42PM +, Ian Campbell wrote:
 On Fri, 2015-01-16 at 14:19 +, Daniel P. Berrange wrote:
  On Fri, Jan 16, 2015 at 01:58:27PM +, Ian Campbell wrote:
   Hello,
   
   On Tue, 2015-01-13 at 17:00 +, Daniel P. Berrange wrote:
+#  define VIR_WARNINGS_NO_PRINTF \
+_Pragma (GCC diagnostic push) \
+_Pragma (GCC diagnostic ignored \-Wsuggest-attribute=format\)
   
   Xen automated tests are failing to build on all architectures with:
   
   util/virxml.c: In function 'catchRNGError':
   util/virxml.c:1094:9: error: unknown option after '#pragma GCC 
   diagnostic' kind [-Werror=pragmas]
   
   which I think must be down to one of these additions.
   
   (helpful of gcc not to print the unknown option in question!)
   
   test overview:
   http://www.chiark.greenend.org.uk/~xensrcts/logs/33443/
   specific failure log:
   http://www.chiark.greenend.org.uk/~xensrcts/logs/33443/build-amd64-libvirt/5.ts-libvirt-build.log
   
   We use Debian Wheezy's gcc, which is 4.6.3 AFAIK.
  
  The configure logs show
  
  checking whether C compiler handles -Wsuggest-attribute=const... yes
  checking whether C compiler handles -Wsuggest-attribute=format... no
  checking whether C compiler handles -Wsuggest-attribute=noreturn... yes
  checking whether C compiler handles -Wsuggest-attribute=pure... yes
  
  So, can someone with a Debian machine check if it helps to modify
  
  _Pragma (GCC diagnostic ignored \-Wsuggest-attribute=format\)
  
  To be just
  
  _Pragma (GCC diagnostic ignored \-Wsuggest-attribute\)
 
 I'm afraid it doesn't seem to. Specifically:
 diff --git a/src/internal.h b/src/internal.h
 index 9855c49..508f8b5 100644
 --- a/src/internal.h
 +++ b/src/internal.h
 @@ -236,7 +236,7 @@
  _Pragma (GCC diagnostic ignored \-Wcast-align\)
  #  define VIR_WARNINGS_NO_PRINTF \
  _Pragma (GCC diagnostic push) \
 -_Pragma (GCC diagnostic ignored \-Wsuggest-attribute=format\)
 +_Pragma (GCC diagnostic ignored \-Wsuggest-attribute\)
  
  #  define VIR_WARNINGS_RESET \
  _Pragma (GCC diagnostic pop)
 
 Didn't help.
 
 According to
 https://gcc.gnu.org/onlinedocs/gcc-4.6.3/gcc/Warning-Options.html#Warning-Options
  the valid -Wsuggest-attributes=FOO in that version are pure const and 
 noreturn.
 
 =format seems to have arrived in 4.8, FWIW.

I just copied you on an alternative patch that would hopefully fix
it - I explicitly check if suggest-attribute=format exists in the
gcc version used.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [libvirt] [PATCH RFC] libxl: fix paths in capability string

[Daniel P. Berrange]

On Tue, Jan 06, 2015 at 10:28:13AM -0700, Eric Blake wrote:
 On 01/06/2015 09:12 AM, Wei Liu wrote:
  Currently libxl driver hardcodes some paths in its capability string,
  which might not be the correct paths.
  
  This patch introduces --with-libxl-prefix, so that user can specify the
  prefix used to build Xen tools. The default value is /usr/local which is
  the default --prefix for Xen tools.
 
 Why can't you just make '--with-libxl=/path/to/alt' work?  That is,
 --with-libxl=yes uses a default (which matches the prefix where LIBVIRT
 will be installed [1]), --with-libxl=no turns it off, and specifying a
 path says to use that path.  Then you don't need to add a new option.

Yep, that would be preferrable.

Alternatively the next best would be for xen to include a pkg-config
configuration file, with a custom variable that reports the paths
required

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel