[libvirt] [PATCHv3 3/4] conf: Automatically use NUMA memory size in case NUMA is enabled

2015-03-04 Thread Peter Krempa
Use the NUMA total instead of the configured size both in XML and for uses in the code once NUMA is enabled for a domain. One test case change is necessary as the rounding of the individual cell sizes was not matching the rounding of the total size. --- src/conf/domain_conf.c

[libvirt] [PATCHv2.5 00/10] Add support for memory hotplug

2015-03-04 Thread Peter Krempa
Rebased version after changes to the prequel series. This applies on top of http://www.redhat.com/archives/libvir-list/2015-March/msg00134.html Convenience branch to pull from: git fetch git://pipo.sk/pipo/libvirt.git memory-hotplug-2.5 Peter Krempa (10): conf: Add support for parsing and

[libvirt] [PATCHv2.5 01/10] conf: Add support for parsing and formatting max memory and slot count

2015-03-04 Thread Peter Krempa
Add a XML element that will allow to specify maximum supportable memory and the count of memory slots to use with memory hotplug. To avoid possible confusion and misuse of the new element this patch also explicitly forbids the use of the maxMemory setting in individual drivers's post parse

[libvirt] [PATCHv2.5 07/10] qemu: add support for memory devices

2015-03-04 Thread Peter Krempa
Add support to start qemu instance with 'pc-dimm' device. Thanks to the refactors we are able to reuse the existing function to determine the parameters. --- src/qemu/qemu_command.c| 130 - src/qemu/qemu_domain.c | 26

Re: [libvirt] direct device assignment

2015-03-04 Thread Laine Stump
On 03/04/2015 10:27 AM, Raymond Durand wrote: I read this in the context of direct device assignment: All of the guest's memory must kept permanently in memory. This is because the guest may program the device with any address in its address space and the hypervisor has no way of handling a

Re: [libvirt] direct device assignment

2015-03-04 Thread Michal Privoznik
On 04.03.2015 16:27, Raymond Durand wrote: I read this in the context of direct device assignment: All of the guest's memory must kept permanently in memory. This is because the guest may program the device with any address in its address space and the hypervisor has no way of handling a DMA

[libvirt] [PATCHv3 4/4] conf: Make specifying memory optional

2015-03-04 Thread Peter Krempa
Now that the size of guest's memory can be inferred from the NUMA configuration (if present) make it optional to specify memory explicitly. To make sure that memory is specified add a check that some form of memory size was specified. One side effect of this change is that it is no longer

[libvirt] [PATCHv3 1/4] conf: Replace access to def-mem.max_balloon with accessor functions

2015-03-04 Thread Peter Krempa
As there are two possible approaches to define a domain's memory size - one used with legacy, non-NUMA VMs configured in the memory element and per-node based approach on NUMA machines - the user needs to make sure that both are specified correctly in the NUMA case. To avoid this burden on the

[libvirt] [PATCHv2.5 09/10] qemu: Implement memory device hotplug

2015-03-04 Thread Peter Krempa
Add code to hot-add memory devices to running qemu instances. --- src/qemu/qemu_command.c | 4 +-- src/qemu/qemu_command.h | 15 + src/qemu/qemu_driver.c | 5 ++- src/qemu/qemu_hotplug.c | 85 + src/qemu/qemu_hotplug.h | 3 ++ 5 files

Re: [libvirt] [PATCH] parallels: Cleanup partly opened drivers on connect open failure

2015-03-04 Thread Ján Tomko
On Thu, Feb 26, 2015 at 05:29:53PM +0100, Michal Privoznik wrote: Well, the parallelsConnectOpen() joins several sub-driver openings into one big if condition. If any of sub-driver fails to open, the whole API finishes immediately. The problem is, sub-drivers may have left some memory

[libvirt] [PATCHv3 2/4] qemu: command: Add helper to align memory sizes

2015-03-04 Thread Peter Krempa
The memory sizes in qemu are aligned up to 1 MiB boundaries. There are two places where this was done once for the total size and then for individual NUMA cell sizes. Add a function that will align the sizes in one place so that it's clear where the sizes are aligned. --- src/qemu/qemu_command.c

[libvirt] [PATCHv2.5 10/10] qemu: Implement memory device hotunplug

2015-03-04 Thread Peter Krempa
Add code to hot-remove memory devices from qemu. Unfortunately QEMU doesn't support this right now, so this is just for completenes. --- src/qemu/qemu_driver.c | 4 ++- src/qemu/qemu_hotplug.c | 91 - src/qemu/qemu_hotplug.h | 3 ++ 3 files

[libvirt] [PATCHv2.5 08/10] qemu: conf: Add support for memory device cold(un)plug

2015-03-04 Thread Peter Krempa
Add a few helpers that allow to operate with memory device definitions on the domain config and use them to implement memory device coldplug in the qemu driver. --- src/conf/domain_conf.c | 100 +++ src/conf/domain_conf.h | 10 +

[libvirt] [PATCH 3/3] memtune: change the way how we store unlimited value

2015-03-04 Thread Pavel Hrdina
There was a mess in the way how we store unlimited value for memory limits and how we handled values provided by user. Internally there were two possible ways how to store unlimited value: as 0 value or as VIR_DOMAIN_MEMORY_PARAM_UNLIMITED. Because we chose to store memory limits as unsigned

[libvirt] [PATCHv2.5 06/10] qemu: migration: Forbid migration with memory modules lacking info

2015-03-04 Thread Peter Krempa
Make sure that libvirt has all vital information needed to reliably represent configuration of guest's memory devices in case of a migration. This patch forbids migration in case the required slot number and module base address are not present (failed to be loaded from qemu via monitor). ---

[libvirt] [PATCHv2.5 04/10] conf: Add interface to parse and format memory device information

2015-03-04 Thread Peter Krempa
This patch adds code that parses and formats configuration for memory devices. A simple configuration would be: memory model='dimm' target size unit='KiB'524287/size node0/node /target /memory A complete configuration of a memory device: memory model='dimm' source pagesize

[libvirt] [PATCHv2.5 05/10] qemu: memdev: Add infrastructure to load memory device information

2015-03-04 Thread Peter Krempa
When using 'dimm' memory devices with qemu, some of the information like the slot number and base address need to be reloaded from qemu after process start so that it reflects the actual state. The state then allows to use memory devices across migrations. --- src/qemu/qemu_domain.c | 49

[libvirt] direct device assignment

2015-03-04 Thread Raymond Durand
I read this in the context of direct device assignment: All of the guest's memory must kept permanently in memory. This is because the guest may program the device with any address in its address space and the hypervisor has no way of handling a DMA page fault is it still true? Thanks, Regards,

[libvirt] QEMU, libvirt, and KVM are participating in Outreachy May-August 2015

2015-03-04 Thread Stefan Hajnoczi
QEMU, libvirt, and KVM are participating in the Outreachy (formerly known as Outreach Program for Women) May-August 2015 round. Outreachy is a 12-week full-time paid internship for working on open source projects. We are able to participate thanks to sponsors who will be listed on the Outreachy

Re: [libvirt] [PATCH 2/2] Tests : Add test for 'ppc64le' architecture.

2015-03-04 Thread Ján Tomko
On Thu, Feb 26, 2015 at 10:45:54PM +0530, Prerna Saxena wrote: Tests : Add test for 'ppc64le' architecture. Signed-off-by: Prerna Saxena pre...@linux.vnet.ibm.com --- .../qemuxml2argv-pseries-cpu-le.args | 7 + .../qemuxml2argv-pseries-cpu-le.xml| 17

[libvirt] [PATCHv2.5 03/10] conf: Add device address type for dimm devices

2015-03-04 Thread Peter Krempa
Dimm devices are described by the slot and base address. Add a new address type to be able to describe such address. --- docs/schemas/domaincommon.rng | 18 +++ src/conf/domain_conf.c| 74 ++- src/conf/domain_conf.h| 9 ++ 3

[libvirt] [PATCH 1/3] virsh: fix memtune to also accept 0 as valid value

2015-03-04 Thread Pavel Hrdina
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1146539 Signed-off-by: Pavel Hrdina phrd...@redhat.com --- tools/virsh-domain.c | 62 +--- 1 file changed, 20 insertions(+), 42 deletions(-) diff --git a/tools/virsh-domain.c

[libvirt] [PATCHv2.5 02/10] qemu: Implement setup of memory hotplug parameters

2015-03-04 Thread Peter Krempa
To enable memory hotplug the maximum memory size and slot count need to be specified. As qemu supports now other units than mebibytes when specifying memory, use the new interface in this case. --- docs/formatdomain.html.in | 2 +- src/qemu/qemu_command.c

[libvirt] [PATCH 2/3] virutil: introduce helper to set memory limits

2015-03-04 Thread Pavel Hrdina
Using this macro will ensure that the value stored in domain def will never be greater than VIR_DOMAIN_MEMORY_PARAM_UNLIMITED. Signed-off-by: Pavel Hrdina phrd...@redhat.com --- src/util/virutil.h | 4 1 file changed, 4 insertions(+) diff --git a/src/util/virutil.h b/src/util/virutil.h

[libvirt] [PATCH 0/3] unify memtune value representation in libvirt

2015-03-04 Thread Pavel Hrdina
The first patch rewrites virsh memtune command to accept 0 as valid value instead of ignoring it. In the second patch I'll introduce a simple helper to crop the *_limit values to VIR_DOMAIN_MEMORY_PARAM_UNLIMITED as its used several times while reading values from cgroups. The last patch

Re: [libvirt] [PATCH v3 4/9] virsh: Add 'iothreads' command

2015-03-04 Thread Ján Tomko
On Tue, Feb 17, 2015 at 04:03:53PM -0500, John Ferlan wrote: Add the 'iothreads' command to display IOThread Info data. Allow for [--live] or [--config] options in order to display live or config data for an active domain. An active domain may return: $ virsh iothreads $dom IOThread ID

Re: [libvirt] [PATCH v3 3/9] qemu: Implement the qemu driver fetch for IOThreads

2015-03-04 Thread Ján Tomko
On Tue, Feb 17, 2015 at 04:03:52PM -0500, John Ferlan wrote: Depending on the flags passed, either attempt to return the active/live IOThread data for the domain or the config data. The active/live path will call into the Monitor in order to get the IOThread data and then correlate the

Re: [libvirt] [PATCH v3 5/9] Implement public API for virDomainSetIOThreads

2015-03-04 Thread Ján Tomko
On Tue, Feb 17, 2015 at 04:03:54PM -0500, John Ferlan wrote: https://bugzilla.redhat.com/show_bug.cgi?id=1135491 Add the libvirt API infrastructure to support setting IOThread data. For now this is the pinned CPU information, but eventually will also support hot plug add/del The API will

Re: [libvirt] [PATCH v3 1/9] Implement public API for virDomainGetIOThreadsInfo

2015-03-04 Thread Ján Tomko
On Tue, Feb 17, 2015 at 04:03:50PM -0500, John Ferlan wrote: Add virDomainGetIOThreadsInfo in order to return a list of virDomainIOThreadsInfoPtr structures which list the IOThread ID, the CPU Affinity map, and associated resources for each IOThread for the domain. For an active domain, the

Re: [libvirt] [PATCH 09/10] libxl: Stubdom emulator type

2015-03-04 Thread Marek Marczykowski-Górecki
On Thu, Feb 19, 2015 at 10:19:22PM +0100, Marek Marczykowski-Górecki wrote: On Thu, Feb 19, 2015 at 01:45:52PM -0700, Jim Fehlig wrote: Marek Marczykowski-Górecki wrote: Xen have feature of having device model in separate domain (called stub domain). Add a 'type' attribute to 'emulator'

[libvirt] [PATCHv3 0/4] Automaticaly fill memory element for NUMA enabled guests

2015-03-04 Thread Peter Krempa
While there is still a discussion open upstream on a possible change of naming here I'm posting this so that comments can be held against this version. Peter Krempa (4): conf: Replace access to def-mem.max_balloon with accessor functions qemu: command: Add helper to align memory sizes

Re: [libvirt] [PATCH 2/2] Tests : Add test for 'ppc64le' architecture.

2015-03-04 Thread Prerna Saxena
On Wednesday 04 March 2015 09:28 PM, Ján Tomko wrote: On Thu, Feb 26, 2015 at 10:45:54PM +0530, Prerna Saxena wrote: Tests : Add test for 'ppc64le' architecture. Signed-off-by: Prerna Saxena pre...@linux.vnet.ibm.com --- .../qemuxml2argv-pseries-cpu-le.args | 7 +

Re: [libvirt] [PATCH 05/10] parallels: introduce and use string constants for network types and names

2015-03-04 Thread Maxim Nestratov
Just found out that I lost one necessary chunk of code, which appeared to be in a separate commit I didn't send. Sorry for that. Should I resend the whole series or this letter is enough? diff --git a/src/parallels/parallels_network.c b/src/parallels/parallels_network.c index 45d1e4c..efa6da2

Re: [libvirt] [PATCH 09/10] libxl: Stubdom emulator type

2015-03-04 Thread Daniel P. Berrange
On Thu, Feb 19, 2015 at 10:19:22PM +0100, Marek Marczykowski-Górecki wrote: On Thu, Feb 19, 2015 at 01:45:52PM -0700, Jim Fehlig wrote: Marek Marczykowski-Górecki wrote: Xen have feature of having device model in separate domain (called stub domain). Add a 'type' attribute to 'emulator'

Re: [libvirt] [PATCH v2] qemu: snapshot: remove the redundant 'if' check

2015-03-04 Thread Shanzhi Yu
On 03/04/2015 09:12 PM, Peter Krempa wrote: On Sat, Feb 28, 2015 at 17:51:36 +0800, Shanzhi Yu wrote: When the domain's source disk type is network, if source protocol is rbd or sheepdog, the 'if().. break' will end the current case, which lead to miss check the driver type is raw or qcow2.

[libvirt] [PATCH v3] util: don't fail if no PortData is found while getting migrateData

2015-03-04 Thread zhang bo
Introduced by f6a2f97e Problem Description: After multiple times of migrating a domain, which has an ovs interface with no portData set, with non-shared disk, nbd ports got overflowed. The steps to reproduce the problem: 1 define and start a domain with its network configured as: interface

[libvirt] [PATCH] Error out on an address for isa-serial in QEMU driver.

2015-03-04 Thread Ján Tomko
We've never formatted them on the qemu command line. https://bugzilla.redhat.com/show_bug.cgi?id=1164053 --- src/qemu/qemu_command.c | 12 +++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index c112619..e7b8c66 100644

Re: [libvirt] [libvirt-designer 0/7]

2015-03-04 Thread Christophe Fergeau
On Wed, Mar 04, 2015 at 10:32:34AM +0100, Michal Privoznik wrote: On 03.03.2015 12:11, Christophe Fergeau wrote: Hey, After seeing the libvirt-designer GSoC idea, I remembered about a bunch of libvirt-designer patches I had never sent :) Here they are, a few leak fixes/build cleanups,

[libvirt] [PATCH] conf: Remove duplicate entries in metadata by namespace

2015-03-04 Thread Peter Krempa
Since the APIs support just one element per namespace and while modifying an element all duplicates would be removed, let's do this right away in the post parse callback. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1190590 --- src/conf/domain_conf.c | 42

Re: [libvirt] [libvirt-designer 0/7]

2015-03-04 Thread Michal Privoznik
On 03.03.2015 12:11, Christophe Fergeau wrote: Hey, After seeing the libvirt-designer GSoC idea, I remembered about a bunch of libvirt-designer patches I had never sent :) Here they are, a few leak fixes/build cleanups, and addition of 2 new methods for completeness. Christophe ACK

[libvirt] [libvirt-designer][PATCH] test: Add new SimpleDisk test

2015-03-04 Thread Michal Privoznik
This test is meant for adding CDROM, floppy and disk to a domain. For now, each method has _file and _device variant, which are tested for now. Signed-off-by: Michal Privoznik mpriv...@redhat.com --- Funny, when reviewing Chritophe's patches, I've found one forgotten branch too.

Re: [libvirt] [PATCH v1 29/31] bridge_driver: Drop networkDriverLock() from almost everywhere

2015-03-04 Thread Michal Privoznik
On 03.03.2015 16:26, Peter Krempa wrote: On Thu, Feb 26, 2015 at 15:17:38 +0100, Michal Privoznik wrote: Now that we have fine grained locks, there's no need to lock the whole driver. We can rely on self-locking APIs. Signed-off-by: Michal Privoznik mpriv...@redhat.com ---

Re: [libvirt] [PATCH] parallels: Cleanup partly opened drivers on connect open failure

2015-03-04 Thread Maxim Nestratov
26.02.2015 19:29, Michal Privoznik пишет: Well, the parallelsConnectOpen() joins several sub-driver openings into one big if condition. If any of sub-driver fails to open, the whole API finishes immediately. The problem is, sub-drivers may have left some memory allocated. Fortunately, we have a

Re: [libvirt] overlayfs support to lxc driver

2015-03-04 Thread Daniel P. Berrange
On Mon, Mar 02, 2015 at 07:45:18PM +0400, Vasiliy Tolstov wrote: 2015-03-02 18:22 GMT+03:00 Vasiliy Tolstov v.tols...@selfip.ru: So as i understand i need to add overlayfs like virStorageBackendFileSystem for example virStorageBackendOvlFileSystem but i don't understand how can this pool be

Re: [libvirt] [PATCH v1 24/31] bridge_driver: Use virNetworkObjEndAPI

2015-03-04 Thread Michal Privoznik
On 03.03.2015 14:40, Peter Krempa wrote: On Thu, Feb 26, 2015 at 15:17:33 +0100, Michal Privoznik wrote: So far, this is pure code replacement. But once we introduce reference counting to virNetworkObj this will be more handy as there'll be only one function to change: virNetworkObjEndAPI().

[libvirt] [libvirt-designer 2/2] Fix gvir_designer_domain_get_guest_full() leak

2015-03-04 Thread Christophe Fergeau
One GVirConfigCapabilitiesGuestArch instance was created for each loop iteration, but it was never unref'ed, causing a memory leak. --- libvirt-designer/libvirt-designer-domain.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/libvirt-designer/libvirt-designer-domain.c

[libvirt] [libvirt-designer 1/2] Fix gvir_designer_domain_get_fallback_disk_controller leak

2015-03-04 Thread Christophe Fergeau
When the list of devices is empty (osinfo_list_get_length(devices) == 0), the 'devices' object would be leaked. --- libvirt-designer/libvirt-designer-domain.c | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/libvirt-designer/libvirt-designer-domain.c

Re: [libvirt] [libvirt-designer][PATCH] test: Add new SimpleDisk test

2015-03-04 Thread Michal Privoznik
On 04.03.2015 13:34, Christophe Fergeau wrote: On Wed, Mar 04, 2015 at 10:50:57AM +0100, Michal Privoznik wrote: This test is meant for adding CDROM, floppy and disk to a domain. For now, each method has _file and _device variant, which are tested for now. Signed-off-by: Michal Privoznik

Re: [libvirt] [libvirt-designer][PATCH] test: Add new SimpleDisk test

2015-03-04 Thread Christophe Fergeau
On Wed, Mar 04, 2015 at 10:50:57AM +0100, Michal Privoznik wrote: This test is meant for adding CDROM, floppy and disk to a domain. For now, each method has _file and _device variant, which are tested for now. Signed-off-by: Michal Privoznik mpriv...@redhat.com --- Funny, when reviewing

Re: [libvirt] [libvirt-designer 1/2] Fix gvir_designer_domain_get_fallback_disk_controller leak

2015-03-04 Thread Michal Privoznik
On 04.03.2015 13:26, Christophe Fergeau wrote: When the list of devices is empty (osinfo_list_get_length(devices) == 0), the 'devices' object would be leaked. --- libvirt-designer/libvirt-designer-domain.c | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) ACK to both. Michal

[libvirt] [libvirt-designer][PATCH] Makefile: Build tests at the same time as the rest

2015-03-04 Thread Michal Privoznik
Currently, since we were using check_PROGRAMS in the Makefile template, the test program build was postponed until 'make check' was run. That's not optimal. Lets build (not run!) test program in the 'make all' phase. Signed-off-by: Michal Privoznik mpriv...@redhat.com --- From my libvirt

Re: [libvirt] [PATCH v2] qemu: snapshot: remove the redundant 'if' check

2015-03-04 Thread Peter Krempa
On Sat, Feb 28, 2015 at 17:51:36 +0800, Shanzhi Yu wrote: When the domain's source disk type is network, if source protocol is rbd or sheepdog, the 'if().. break' will end the current case, which lead to miss check the driver type is raw or qcow2. Libvirt will allow to create internal snapshot

Re: [libvirt] [libvirt-designer 1/2] Fix gvir_designer_domain_get_fallback_disk_controller leak

2015-03-04 Thread Christophe Fergeau
On Wed, Mar 04, 2015 at 01:50:51PM +0100, Michal Privoznik wrote: On 04.03.2015 13:26, Christophe Fergeau wrote: When the list of devices is empty (osinfo_list_get_length(devices) == 0), the 'devices' object would be leaked. --- libvirt-designer/libvirt-designer-domain.c | 6 -- 1

Re: [libvirt] [libvirt-designer][PATCH] Makefile: Build tests at the same time as the rest

2015-03-04 Thread Christophe Fergeau
Hey, On Wed, Mar 04, 2015 at 02:05:07PM +0100, Michal Privoznik wrote: Currently, since we were using check_PROGRAMS in the Makefile template, the test program build was postponed until 'make check' was run. That's not optimal. Lets build (not run!) test program in the 'make all' phase.

Re: [libvirt] [PATCH] virsh: tweak domif-getlink link state reporting message

2015-03-04 Thread Peter Krempa
On Wed, Feb 25, 2015 at 09:01:49 +0100, Peter Krempa wrote: On Tue, Feb 24, 2015 at 17:25:55 +0100, Erik Skultety wrote: According to docs, we only support 2 link states for an interface up/down, 'up' being the default state if link state is unspecified in domain's XML, so the message when

Re: [libvirt] [libvirt-designer][PATCH] Makefile: Build tests at the same time as the rest

2015-03-04 Thread Michal Privoznik
On 04.03.2015 14:54, Christophe Fergeau wrote: Hey, On Wed, Mar 04, 2015 at 02:05:07PM +0100, Michal Privoznik wrote: Currently, since we were using check_PROGRAMS in the Makefile template, the test program build was postponed until 'make check' was run. That's not optimal. Lets build (not

[libvirt] [PATCH v3 4/4] migration: Expose 'cancelling' status to user

2015-03-04 Thread zhanghailiang
'cancelling' status is introduced by commit 51cf4c1a, which is mainly avoid possible starting a new migration process while the previous one still exist. But we don't expose this status to user, instead by return a 'active' state. Here, we expose it to the user (such as libvirt), 'cancelling'

Re: [libvirt] [RFC] Memory hotplug naming [Was: [PATCHv2 3/7] conf: Replace access to def-mem.max_balloon with accessor functions]

2015-03-04 Thread Peter Krempa
Adding Dan Eric to cc On Thu, Feb 26, 2015 at 14:06:55 +0100, Martin Kletzander wrote: On Wed, Feb 25, 2015 at 11:22:02AM +0100, Peter Krempa wrote: As there are two possible approaches to define a domain's memory size - one used with legacy, non-NUMA VMs configured in the memory element and

Re: [libvirt] [PATCH] storage: tweak condition to properly test lseek

2015-03-04 Thread Peter Krempa
On Tue, Mar 03, 2015 at 12:10:46 -0700, Eric Blake wrote: On 03/03/2015 08:57 AM, Erik Skultety wrote: According to the POSIX standard, off_t (returned by lseek) is defined as signed integral type no shorter than int. Because our offset variable is defined as unsigned long long, the