Re: [libvirt] [PATCH] add lock priv->lock protection in remoteClientCloseFunc()

2019-12-04 Thread Lance Liu
Hi: Need any more informations about my patch? LanceLiu 于2019年12月2日周一 上午11:16写道: > sometimes, virsh console session may closed by virsh console existed > session(when virsh console client > exit) and new virsh console --force session simutaneously. So in one > thread(Job worker), it calls

Re: [libvirt] [PATCH v2 20/25] qemu: Implement backup job APIs and qemu handling

2019-12-04 Thread Eric Blake
On 12/3/19 11:17 AM, Peter Krempa wrote: This allows to start and manage the backup job. Signed-off-by: Peter Krempa --- po/POTFILES.in | 1 + src/qemu/Makefile.inc.am | 2 + src/qemu/qemu_backup.c | 941 +++ Large patch, but I'm not

Re: [libvirt] [PATCH v2 19/25] doc: Document quirk of getting block job info for a 'backup' blockjob

2019-12-04 Thread Eric Blake
On 12/3/19 11:17 AM, Peter Krempa wrote: The stats reported for a blockjob which is member of a domain pull backup refer to the utilization of the scratch file rather than the progress of the backup as the progress of the backup depends on the client. Note this quirk in the docs. Signed-off-by:

Re: [libvirt] [PATCH v2 18/25] conf: backup: Add fields for tracking stats of completed sub-jobs

2019-12-04 Thread Eric Blake
On 12/3/19 11:17 AM, Peter Krempa wrote: We need a place to store stats of completed sub-jobs so that we can later report accurate stats. Signed-off-by: Peter Krempa --- src/conf/backup_conf.h | 7 +++ 1 file changed, 7 insertions(+) diff --git a/src/conf/backup_conf.h

Re: [libvirt] [PATCH v2 17/25] tests: qemustatusxml2xml: Add test for 'pull' type backup job

2019-12-04 Thread Eric Blake
On 12/3/19 11:17 AM, Peter Krempa wrote: Signed-off-by: Peter Krempa --- .../qemustatusxml2xmldata/backup-pull-in.xml | 608 ++ .../qemustatusxml2xmldata/backup-pull-out.xml | 1 + tests/qemuxml2xmltest.c | 2 + 3 files changed, 611 insertions(+)

Re: [libvirt] [PATCH v2 16/25] qemu: blockjob: Track internal data for 'backup' blockjob

2019-12-04 Thread Eric Blake
On 12/3/19 11:17 AM, Peter Krempa wrote: A backup blockjob needs to be able to notify the parent backup job as well as track all data to be able to clean up the bitmap and blockdev used for the backup. Add the data structure, job allocation function and status XML formatter and parser.

Re: [libvirt] [PATCH v2 15/25] qemu: domain: Track backup job data in the status XML

2019-12-04 Thread Eric Blake
On 12/3/19 11:17 AM, Peter Krempa wrote: Store the data of a backup job along with the index counter for new backup jobs in the status XML. Currently we will support only one backup job and thus there's no necessity to add arrays of jobs. Signed-off-by: Peter Krempa ---

Re: [libvirt] [PATCH v2 14/25] qemu: monitor: Add support for blockdev-backup via 'transaction'

2019-12-04 Thread Eric Blake
On 12/3/19 11:17 AM, Peter Krempa wrote: Implement the transaction actions generator for blockdev-backup. Signed-off-by: Peter Krempa --- Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org --

Re: [libvirt] [PATCH v2 13/25] Add 'backup' block job type

2019-12-04 Thread Eric Blake
On 12/3/19 11:17 AM, Peter Krempa wrote: A backup job may consist of many backup sub-blockjobs. Add the new blockjob type and add all type converter strings. Signed-off-by: Peter Krempa --- Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer Red Hat, Inc.

Re: [libvirt] [PATCH v2 12/25] qemu: domain: Introduce QEMU_ASYNC_JOB_BACKUP async job type

2019-12-04 Thread Eric Blake
On 12/3/19 11:17 AM, Peter Krempa wrote: We will want to use the async job infrastructure along with all the APIs and event for the backup job so add the backup job as a new async job type. Signed-off-by: Peter Krempa --- Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer

Re: [libvirt] [PATCH v2 11/25] qemu: Add infrastructure for statistics of a backup job

2019-12-04 Thread Eric Blake
On 12/3/19 11:17 AM, Peter Krempa wrote: Introduce QEMU_DOMAIN_JOB_STATS_TYPE_BACKUP and the convertors and other plumbing to be able to report statistics for the backup job. Signed-off-by: Peter Krempa --- src/qemu/qemu_domain.c | 62 ++

Re: [libvirt] [PATCH v2 09/25] tests: genericxml2xml: Add testing of backup XML files

2019-12-04 Thread Eric Blake
On 12/3/19 11:17 AM, Peter Krempa wrote: Now that the parser and formatter are in place we can excercise it on exercise the test files. Signed-off-by: Peter Krempa --- Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226

Re: [libvirt] [python PATCH] Fix RPM deps for centos 7

2019-12-04 Thread Andrea Bolognani
On Wed, 2019-12-04 at 16:24 +, Daniel P. Berrangé wrote: > +%if 0%{?rhel} == 7 > +BuildRequires: python36-nose > +BuildRequires: python36-lxml > +%else > BuildRequires: python3-nose > BuildRequires: python3-lxml > +%endif This should work just fine in an actual RHEL 7 machine, provided that

Re: [libvirt] [jenkins-ci PATCH] lcitool: Add 'install=' to the extra_arg

2019-12-04 Thread Andrea Bolognani
On Wed, 2019-12-04 at 17:19 +0100, Fabiano Fidêncio wrote: > Add "install=" unconditionally to the extra_arg parameter passed to > virt-install. It's needed as the virt-install present on CentOS CI host > does not use the correct kernel URL argument when installing OpenSUSE. > > This does not

[libvirt] [PATCH] qemuProcessStop: Remove image metadata only when allowed

2019-12-04 Thread Michal Privoznik
In v5.9.0-370-g8fa0374c5b I've tried to fix a bug by removing some stale XATTRs in qemuProcessStop(). However, I forgot to do nothing when the VIR_QEMU_PROCESS_STOP_NO_RELABEL flag was specified. 臘 Signed-off-by: Michal Privoznik --- src/qemu/qemu_process.c | 12 +++- 1 file changed, 7

[libvirt] [python PATCH] Fix RPM deps for centos 7

2019-12-04 Thread Daniel P . Berrangé
Signed-off-by: Daniel P. Berrangé --- libvirt-python.spec.in | 5 + 1 file changed, 5 insertions(+) diff --git a/libvirt-python.spec.in b/libvirt-python.spec.in index 1e92f3c..33d2775 100644 --- a/libvirt-python.spec.in +++ b/libvirt-python.spec.in @@ -21,8 +21,13 @@ Url: http://libvirt.org

[libvirt] [jenkins-ci PATCH] lcitool: Add 'install=' to the extra_arg

2019-12-04 Thread Fabiano Fidêncio
Add "install=" unconditionally to the extra_arg parameter passed to virt-install. It's needed as the virt-install present on CentOS CI host does not use the correct kernel URL argument when installing OpenSUSE. This does not cause issues on other installations as the other distros will just

Re: [libvirt] [PATCH] build: only support python3 binary

2019-12-04 Thread Daniel P . Berrangé
On Wed, Dec 04, 2019 at 09:59:48AM -0600, Eric Blake wrote: > On 12/4/19 4:14 AM, Daniel P. Berrangé wrote: > > python2 will be end of life by the time of the next > > libvirt release. All our supported build targets, including > > CentOS7, have a python3 build available. > > > > Signed-off-by:

Re: [libvirt] [PATCH] build: only support python3 binary

2019-12-04 Thread Eric Blake
On 12/4/19 4:14 AM, Daniel P. Berrangé wrote: python2 will be end of life by the time of the next libvirt release. All our supported build targets, including CentOS7, have a python3 build available. Signed-off-by: Daniel P. Berrangé --- +++ b/configure.ac @@ -701,9 +701,9 @@ if test

Re: [libvirt] [jenkins-ci PATCH] Drop support for python 2

2019-12-04 Thread Andrea Bolognani
On Wed, 2019-12-04 at 13:26 +, Daniel P. Berrangé wrote: [...] > +++ b/guests/host_vars/libvirt-centos-7/main.yml > @@ -16,5 +16,3 @@ package_format: 'rpm' > package_manager: 'yum' > os_name: 'CentOS' > os_version: '7' > - > -ansible_python_interpreter: /usr/bin/python2 This will not work:

Re: [libvirt] [jenkins-ci PATCH] lcitool: Decouple Python interpreters for Ansible and builds

2019-12-04 Thread Daniel P . Berrangé
On Wed, Dec 04, 2019 at 03:23:25PM +0100, Andrea Bolognani wrote: > We need to keep using Python 2 for Ansible, because Python 3 is > only supported by the dnf module and on CentOS 7 we have to use > the yum module instead; at the same time, all libvirt projects > have now dropped Python 2 support

[libvirt] [PATCH 13/30] conf: drop virCapsPtr param from post parse data alloc callback

2019-12-04 Thread Daniel P . Berrangé
The only user of this callback did not require the virCapsPtr parameter. Signed-off-by: Daniel P. Berrangé --- src/conf/domain_conf.c | 4 ++-- src/conf/domain_conf.h | 1 - src/qemu/qemu_domain.c | 1 - 3 files changed, 2 insertions(+), 4 deletions(-) diff --git a/src/conf/domain_conf.c

[libvirt] [PATCH 30/30] qemu: stop creating capabilities at driver startup

2019-12-04 Thread Daniel P . Berrangé
Now that nearly all internal APIs use the QEMU capabilities or other QEMU driver data directly, there's no compelling benefit to create virCapsPtr at driver startup. Skipping this means we don't probe capabilities for all 30 system emulator targets at startup, only those emulators which are

[libvirt] [PATCH 05/30] conf: pass in default architecture via domain XML options

2019-12-04 Thread Daniel P . Berrangé
When parsing the guest XML we must fill in the default guest arch if it is not already present because later parts of the parsing process need this information. If no arch is specified we lookup the first guest in the capabilities data matching the os type and virt type. In most cases this will

[libvirt] [PATCH 26/30] qemu: use NUMA capabilities object directly

2019-12-04 Thread Daniel P . Berrangé
Avoid grabbing the whole virCapsPtr object when we only need the NUMA information. Signed-off-by: Daniel P. Berrangé --- src/qemu/qemu_capabilities.c | 7 --- src/qemu/qemu_conf.c | 17 +++ src/qemu/qemu_conf.h | 6 ++ src/qemu/qemu_domain.c | 10

[libvirt] [jenkins-ci PATCH] lcitool: Decouple Python interpreters for Ansible and builds

2019-12-04 Thread Andrea Bolognani
We need to keep using Python 2 for Ansible, because Python 3 is only supported by the dnf module and on CentOS 7 we have to use the yum module instead; at the same time, all libvirt projects have now dropped Python 2 support so we need to make sure we use Python 3 for building them. Decouple the

[libvirt] [PATCH 04/30] conf: sanitize virDomainObjFormat & virDomainDefFormat* APIs

2019-12-04 Thread Daniel P . Berrangé
Moving their instance parameter to be the first one, and give consistent ordering of other parameters across all functions. Ensure that the xml options are passed into both functions in prep for future work. Signed-off-by: Daniel P. Berrangé --- src/bhyve/bhyve_driver.c | 4 +--

[libvirt] [PATCH 17/30] conf: drop virCapsPtr param from domain formatting APIs

2019-12-04 Thread Daniel P . Berrangé
This parameter is now unused and can be removed entirely. Signed-off-by: Daniel P. Berrangé --- src/bhyve/bhyve_driver.c | 16 ++-- src/conf/checkpoint_conf.c | 4 ++-- src/conf/domain_conf.c | 26 +++--- src/conf/domain_conf.h

[libvirt] [PATCH 24/30] qemu: drop virCapsPtr param & vars from many APIs

2019-12-04 Thread Daniel P . Berrangé
Now that the domain XML APIs don't use virCapsPtr we can stop passing it around many QEMU driver methods. Signed-off-by: Daniel P. Berrangé --- src/qemu/qemu_checkpoint.c | 24 +++ src/qemu/qemu_checkpoint.h | 1 - src/qemu/qemu_domain.c | 3 +-- src/qemu/qemu_domain.h

[libvirt] [PATCH 25/30] conf: move NUMA capabilities into self contained object

2019-12-04 Thread Daniel P . Berrangé
The NUMA cells are stored directly in the virCapsHostPtr struct. This moves them into their own struct allowing them to be stored independantly of the rest of the host capabilities. The change is used as an excuse to switch the representation to use a GPtrArray too. Signed-off-by: Daniel P.

[libvirt] [PATCH 29/30] qemu: directly create virResctrlInfo ignoring capabilities

2019-12-04 Thread Daniel P . Berrangé
We always refresh the capabilities object when using virResctrlInfo during process startup. This is undesirable overhead, because we can just directly create a virResctrlInfo instead. Signed-off-by: Daniel P. Berrangé --- src/qemu/qemu_process.c | 24

[libvirt] [PATCH 28/30] qemu: use host CPU object directly

2019-12-04 Thread Daniel P . Berrangé
Avoid grabbing the whole virCapsPtr object when we only need the host CPU information. Signed-off-by: Daniel P. Berrangé --- src/qemu/qemu_capabilities.c | 3 -- src/qemu/qemu_conf.c | 17 + src/qemu/qemu_conf.h | 6 src/qemu/qemu_driver.c | 6 ++--

[libvirt] [PATCH 23/30] conf: drop virCapsPtr param from snapshot & checkpoint APIs

2019-12-04 Thread Daniel P . Berrangé
Signed-off-by: Daniel P. Berrangé --- src/conf/checkpoint_conf.c | 15 --- src/conf/checkpoint_conf.h | 2 -- src/conf/snapshot_conf.c| 14 -- src/conf/snapshot_conf.h| 3 --- src/esx/esx_driver.c

[libvirt] [PATCH 18/30] conf: drop virCapsPtr param from domain post parse & validate APIs

2019-12-04 Thread Daniel P . Berrangé
Signed-off-by: Daniel P. Berrangé --- src/conf/domain_conf.c | 10 -- src/conf/domain_conf.h | 2 -- src/libxl/xen_xl.c | 2 +- src/libxl/xen_xm.c | 2 +- src/lxc/lxc_native.c| 4 ++-- src/qemu/qemu_driver.c | 12 ++-- src/qemu/qemu_process.c | 4 ++--

[libvirt] [PATCH 27/30] conf: make virCPUDef into a ref counted struct

2019-12-04 Thread Daniel P . Berrangé
Annoyingly there was no existing constructor, and identifying all the places which do a VIR_ALLOC(cpu) is a bit error prone. Hopefully this has found & converted them all. Signed-off-by: Daniel P. Berrangé --- src/conf/cpu_conf.c| 29 ++---

[libvirt] [PATCH 11/30] conf: drop virCapsPtr param from assign addresses callback

2019-12-04 Thread Daniel P . Berrangé
The virCapsPtr param is not used by any of the virt drivers providing this callback. Signed-off-by: Daniel P. Berrangé --- src/bhyve/bhyve_domain.c | 1 - src/conf/domain_conf.c | 2 +- src/conf/domain_conf.h | 1 - src/qemu/qemu_domain.c | 1 - 4 files changed, 1 insertion(+), 4

[libvirt] [PATCH 22/30] conf: drop virCapsPtr param from domain obj list APIs

2019-12-04 Thread Daniel P . Berrangé
Signed-off-by: Daniel P. Berrangé --- src/bhyve/bhyve_driver.c| 2 -- src/conf/virdomainobjlist.c | 5 - src/conf/virdomainobjlist.h | 1 - src/libxl/libxl_driver.c| 3 --- src/lxc/lxc_driver.c| 14 -- src/qemu/qemu_driver.c | 8 +--- 6 files

[libvirt] [PATCH 19/30] conf: drop virCapsPtr param from domain parse APIs

2019-12-04 Thread Daniel P . Berrangé
Signed-off-by: Daniel P. Berrangé --- src/bhyve/bhyve_driver.c | 17 +++- src/conf/checkpoint_conf.c | 4 +-- src/conf/domain_conf.c | 29 +++- src/conf/domain_conf.h | 6 - src/conf/snapshot_conf.c | 6 ++---

[libvirt] [PATCH 12/30] conf: drop virCapsPtr param from basic post parse callback

2019-12-04 Thread Daniel P . Berrangé
The QEMU impl of the callback can directly use the QEMU capabilities cache to resolve the emulator binary name, allowing virCapsPtr to be dropped. Signed-off-by: Daniel P. Berrangé --- src/conf/domain_conf.c | 2 +- src/conf/domain_conf.h | 1 - src/qemu/qemu_capabilities.c | 28

[libvirt] [PATCH 14/30] conf: drop virCapsPtr param from post parse callback

2019-12-04 Thread Daniel P . Berrangé
No impl of this callback requires the virCapsPtr anymore. Signed-off-by: Daniel P. Berrangé --- src/bhyve/bhyve_domain.c | 1 - src/conf/domain_conf.c | 2 +- src/conf/domain_conf.h | 1 - src/libxl/libxl_domain.c | 1 - src/lxc/lxc_domain.c | 1 - src/openvz/openvz_conf.c |

[libvirt] [PATCH 08/30] conf: pass netprefix in the domain parser config struct

2019-12-04 Thread Daniel P . Berrangé
Instead of using the virCapsPtr information, pass the driver specific netprefix in the domain parser struct. This eliminates one more use of virCapsPtr from the XML parsing/formatting code. Signed-off-by: Daniel P. Berrangé --- src/conf/domain_conf.c | 24 +++-

[libvirt] [PATCH 15/30] conf: drop virCapsPtr param from device post parse callback

2019-12-04 Thread Daniel P . Berrangé
None of the impls of this callback require the virCapsPtr param. Signed-off-by: Daniel P. Berrangé --- src/bhyve/bhyve_domain.c | 1 - src/conf/domain_conf.c | 20 +++- src/conf/domain_conf.h | 1 - src/libxl/libxl_domain.c | 1 - src/lxc/lxc_domain.c | 1 -

[libvirt] [PATCH 21/30] conf: drop virCapsPtr param from APIs for saving domains

2019-12-04 Thread Daniel P . Berrangé
Signed-off-by: Daniel P. Berrangé --- src/bhyve/bhyve_driver.c| 9 +-- src/bhyve/bhyve_process.c | 16 +- src/conf/domain_conf.c | 7 +-- src/conf/domain_conf.h | 3 - src/libxl/libxl_domain.c| 2 +- src/libxl/libxl_driver.c| 34 +--

[libvirt] [PATCH 16/30] conf: drop virCapsPtr param from domain validate callback

2019-12-04 Thread Daniel P . Berrangé
None of the impls of this callback require the virCapsPtr param. Signed-off-by: Daniel P. Berrangé --- src/conf/domain_conf.c | 4 ++-- src/conf/domain_conf.h | 1 - src/qemu/qemu_domain.c | 1 - src/vz/vz_driver.c | 1 - 4 files changed, 2 insertions(+), 5 deletions(-) diff --git

[libvirt] [PATCH 09/30] conf: move seclabel validation into post-parse phase

2019-12-04 Thread Daniel P . Berrangé
Currently the disk and chardev seclabels are validated immediately at the time their data is parsed. This forces the parser to fill in the top level secmodel at time of parsing which is an undesirable thing. This validation conceptually should be done in the post-parse phase instead.

[libvirt] [PATCH 20/30] conf: drop virCapsPtr param from domain APIs for copying config

2019-12-04 Thread Daniel P . Berrangé
Signed-off-by: Daniel P. Berrangé --- src/conf/domain_conf.c | 17 ++--- src/conf/domain_conf.h | 9 ++--- src/libxl/libxl_domain.c| 6 +++--- src/libxl/libxl_driver.c| 15 ++- src/libxl/libxl_migration.c | 3 +-- src/lxc/lxc_driver.c| 22

[libvirt] [PATCH 03/30] conf: sanitize virDomainSaveStatus & virDomainSaveConfig APIs

2019-12-04 Thread Daniel P . Berrangé
Our normal practice is for the object type to be the name prefix, and the object instance be the first parameter passed in. Rename these to virDomainObjSave and virDomainDefSave moving their primary parameter to be the first one. Ensure that the xml options are passed into both functions in prep

[libvirt] [PATCH 02/30] qemu: add machines & arch to fake test capabilities cache

2019-12-04 Thread Daniel P . Berrangé
Currently the virQEMUCapsPtr objects are just empty. Future patches are going to expect them to contain real data. Start off by populating the machine types and arch information. Signed-off-by: Daniel P. Berrangé --- src/qemu/qemu_capabilities.c | 70 +++-- src/qemu/qemu_capspriv.h | 10 +

[libvirt] [PATCH 10/30] conf: pass default sec model in parser config

2019-12-04 Thread Daniel P . Berrangé
Instead of using the virCapsPtr to get the default security model, pass this in via the parser config. Signed-off-by: Daniel P. Berrangé --- src/conf/domain_conf.c | 14 +++--- src/conf/domain_conf.h | 1 + src/qemu/qemu_conf.c | 4 +++- src/qemu/qemu_conf.h | 3 ++-

[libvirt] [PATCH 07/30] conf: don't use passed in caps in post parse method

2019-12-04 Thread Daniel P . Berrangé
To enable the virCapsPtr parameter to the post parse method to be eliminated, the drivers must fetch the virCapsPtr from their own driver via the opaque parameter, or use an alternative approach to validate the parsed data. Signed-off-by: Daniel P. Berrangé --- src/bhyve/bhyve_domain.c |

[libvirt] [PATCH 06/30] conf: move virt type / os type / arch validation to post-parse

2019-12-04 Thread Daniel P . Berrangé
The XML parser currently calls virCapabilitiesDomainDataLookup during parsing to find the domain capabilities matching the triple (virt type, os type, arch) This is, however, bogus with the QEMU driver as it assumes that there is an emulator known to the default driver capabilities that

[libvirt] [PATCH 01/30] qemu: cache host arch separately from virCapsPtr

2019-12-04 Thread Daniel P . Berrangé
As part of a goal to eliminate the need to use virCapsPtr for anything other than the virConnectGetCapabilies() API impl, cache the host arch against the QEMU driver struct and use that field directly. In the tests we move virArchFromHost() globally in testutils.c so that every test runs with a

[libvirt] [PATCH 00/30] conf: eliminate use of virCapsPtr for parsing/formatting XML

2019-12-04 Thread Daniel P . Berrangé
The virCapsPtr object is a serious source of performance problems for the QEMU driver, because populating it means probing every QEMU system emulator binary installed on the host. This is is crazy because parsing any single XML document only ever needs information about a single QEMU binary.

Re: [libvirt] [jenkins-ci PATCH] Drop support for python 2

2019-12-04 Thread Fabiano Fidêncio
Daniel, [snip] > diff --git a/guests/host_vars/libvirt-centos-7/main.yml > b/guests/host_vars/libvirt-centos-7/main.yml > index 94e29af..d730aa2 100644 > --- a/guests/host_vars/libvirt-centos-7/main.yml > +++ b/guests/host_vars/libvirt-centos-7/main.yml > @@ -16,5 +16,3 @@ package_format: 'rpm'

Re: [libvirt] [PATCH] tests: Make check-file-access.pl accept files through argv

2019-12-04 Thread Daniel P . Berrangé
On Wed, Dec 04, 2019 at 03:01:31PM +0100, Michal Privoznik wrote: > The script needs two files to open: > > $(builddir)/test_file_access.txt, and > $(srcdir)/file_access_whitelist.txt. > > However, the script is opening the files from the $CWD which > won't work for a VPATH build. Make the

Re: [libvirt] [PATCH v5 17/23] tests: rewrite file access checker in Python

2019-12-04 Thread Michal Privoznik
On 12/4/19 2:19 PM, Daniel P. Berrangé wrote: On Mon, Nov 18, 2019 at 02:11:33PM -0500, Cole Robinson wrote: On 11/11/19 9:38 AM, Daniel P. Berrangé wrote: As part of an goal to eliminate Perl from libvirt build tools, rewrite the check-file-access.pl tool in Python. This was a straight

[libvirt] [PATCH] tests: Make check-file-access.pl accept files through argv

2019-12-04 Thread Michal Privoznik
The script needs two files to open: $(builddir)/test_file_access.txt, and $(srcdir)/file_access_whitelist.txt. However, the script is opening the files from the $CWD which won't work for a VPATH build. Make the script accept paths to the files through @ARGV and tune the Makefile.am to pass

[libvirt] [jenkins-ci PATCH] Drop support for python 2

2019-12-04 Thread Daniel P . Berrangé
CentOS7 now has support for python 3 out of the box Signed-off-by: Daniel P. Berrangé --- guests/host_vars/libvirt-centos-7/main.yml | 2 -- guests/playbooks/update/tasks/bootstrap.yml | 6 -- guests/vars/mappings.yml| 24 -

Re: [libvirt] [PATCH v5 17/23] tests: rewrite file access checker in Python

2019-12-04 Thread Daniel P . Berrangé
On Mon, Nov 18, 2019 at 02:11:33PM -0500, Cole Robinson wrote: > On 11/11/19 9:38 AM, Daniel P. Berrangé wrote: > > As part of an goal to eliminate Perl from libvirt build tools, > > rewrite the check-file-access.pl tool in Python. > > > > This was a straight conversion, manually going

Re: [libvirt] [PATCH v3 0/3] Update valid range of scsi unit

2019-12-04 Thread Michal Privoznik
On 12/4/19 10:35 AM, Han Han wrote: Fix spelling typo. Separate virtio serial port and ccid slot from driveUnit. v2: https://www.redhat.com/archives/libvir-list/2019-December/msg4.html Han Han (3): conf: fix logic error for scsi units rng: Separate virtio serial port and ccid slot

Re: [libvirt] [PATCH v2 00/15] docs: some refactoring and new docs about RPM deployment

2019-12-04 Thread Michal Privoznik
On 11/22/19 3:46 PM, Daniel P. Berrangé wrote: This refactors existing docs related to the remote driver/daemon and URIs. It then also adds a kbase page about RPM package options. This introduces the use of RST for docs as a replacement for HTML. The intent is that all new docs should use RST

Re: [libvirt] [PATCH v2 25/25] qemu: Add support for VIR_DOMAIN_CAPS_FEATURE_BACKUP

2019-12-04 Thread Daniel P . Berrangé
On Tue, Dec 03, 2019 at 06:17:47PM +0100, Peter Krempa wrote: > Signed-off-by: Peter Krempa > --- > src/qemu/qemu_capabilities.c | 1 + > tests/domaincapsdata/qemu_1.5.3-q35.x86_64.xml| 1 + > tests/domaincapsdata/qemu_1.5.3-tcg.x86_64.xml| 1 + >

Re: [libvirt] [PATCH v2 24/25] conf: domaincaps: Add 'backup' feature flag

2019-12-04 Thread Daniel P . Berrangé
On Tue, Dec 03, 2019 at 06:17:46PM +0100, Peter Krempa wrote: > This flag will allow figuring out whether the hypervisor supports the > incremental backup and checkpoint features. > > Signed-off-by: Peter Krempa > --- > docs/formatdomaincaps.html.in | 8 > docs/schemas/domaincaps.rng

Re: [libvirt] [PATCH v2 22/25] qemu: driver: Allow cancellation of the backup job

2019-12-04 Thread Daniel P . Berrangé
On Tue, Dec 03, 2019 at 06:17:44PM +0100, Peter Krempa wrote: > Use the helper which cancels all blockjobs to perform the backup job > cancellation in qemuDomainAbortJob. > > Signed-off-by: Peter Krempa > --- > src/qemu/qemu_driver.c | 15 ++- > 1 file changed, 10 insertions(+), 5

Re: [libvirt] [PATCH] tests: stop static linking to libvirt code in tests

2019-12-04 Thread Michal Privoznik
On 12/3/19 4:25 PM, Daniel P. Berrangé wrote: If we static link to libvirt_util.la then we can't override functions in this file by simply implementing them in the test code. Any tests should dynamic link to the main libvirt.la and ensure symbols are exported. Signed-off-by: Daniel P. Berrangé

Re: [libvirt] [PATCH v2 19/25] doc: Document quirk of getting block job info for a 'backup' blockjob

2019-12-04 Thread Daniel P . Berrangé
On Tue, Dec 03, 2019 at 06:17:41PM +0100, Peter Krempa wrote: > The stats reported for a blockjob which is member of a domain pull > backup refer to the utilization of the scratch file rather than the > progress of the backup as the progress of the backup depends on the > client. Note this quirk

Re: [libvirt] [PATCH v2 18/25] conf: backup: Add fields for tracking stats of completed sub-jobs

2019-12-04 Thread Daniel P . Berrangé
On Tue, Dec 03, 2019 at 06:17:40PM +0100, Peter Krempa wrote: > We need a place to store stats of completed sub-jobs so that we can > later report accurate stats. Its kind of gross using the public struct for internal only data but I guess we can fix that any time we want to later. > >

Re: [libvirt] [PATCH v2 17/25] tests: qemustatusxml2xml: Add test for 'pull' type backup job

2019-12-04 Thread Daniel P . Berrangé
On Tue, Dec 03, 2019 at 06:17:39PM +0100, Peter Krempa wrote: > Signed-off-by: Peter Krempa > --- > .../qemustatusxml2xmldata/backup-pull-in.xml | 608 ++ > .../qemustatusxml2xmldata/backup-pull-out.xml | 1 + > tests/qemuxml2xmltest.c | 2 + > 3 files

Re: [libvirt] [PATCH v2 16/25] qemu: blockjob: Track internal data for 'backup' blockjob

2019-12-04 Thread Daniel P . Berrangé
On Tue, Dec 03, 2019 at 06:17:38PM +0100, Peter Krempa wrote: > A backup blockjob needs to be able to notify the parent backup job as > well as track all data to be able to clean up the bitmap and blockdev > used for the backup. > > Add the data structure, job allocation function and status XML

Re: [libvirt] [PATCH v2 15/25] qemu: domain: Track backup job data in the status XML

2019-12-04 Thread Daniel P . Berrangé
On Tue, Dec 03, 2019 at 06:17:37PM +0100, Peter Krempa wrote: > Store the data of a backup job along with the index counter for new > backup jobs in the status XML. Currently we will support only one > backup job and thus there's no necessity to add arrays of jobs. > > Signed-off-by: Peter Krempa

Re: [libvirt] [PATCH] scripts: fix list indexing when printing mis-ordered symbols

2019-12-04 Thread Michal Privoznik
On 12/3/19 6:15 PM, Daniel P. Berrangé wrote: The python array slice syntax expects the first and last indexes, not the first length and element count. Signed-off-by: Daniel P. Berrangé --- scripts/check-symsorting.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) Reviewed-by:

Re: [libvirt] [PATCH v2 14/25] qemu: monitor: Add support for blockdev-backup via 'transaction'

2019-12-04 Thread Daniel P . Berrangé
On Tue, Dec 03, 2019 at 06:17:36PM +0100, Peter Krempa wrote: > Implement the transaction actions generator for blockdev-backup. > > Signed-off-by: Peter Krempa > --- > src/qemu/qemu_monitor.c | 13 + > src/qemu/qemu_monitor.h | 15 +++ >

Re: [libvirt] [PATCH v2 13/25] Add 'backup' block job type

2019-12-04 Thread Daniel P . Berrangé
On Tue, Dec 03, 2019 at 06:17:35PM +0100, Peter Krempa wrote: > A backup job may consist of many backup sub-blockjobs. Add the new > blockjob type and add all type converter strings. > > Signed-off-by: Peter Krempa > --- > examples/c/misc/event-test.c | 3 +++ >

Re: [libvirt] [PATCH v2 12/25] qemu: domain: Introduce QEMU_ASYNC_JOB_BACKUP async job type

2019-12-04 Thread Daniel P . Berrangé
On Tue, Dec 03, 2019 at 06:17:34PM +0100, Peter Krempa wrote: > We will want to use the async job infrastructure along with all the APIs > and event for the backup job so add the backup job as a new async job > type. > > Signed-off-by: Peter Krempa > --- > src/qemu/qemu_domain.c| 3 +++ >

Re: [libvirt] [PATCH v2 11/25] qemu: Add infrastructure for statistics of a backup job

2019-12-04 Thread Daniel P . Berrangé
On Tue, Dec 03, 2019 at 06:17:33PM +0100, Peter Krempa wrote: > Introduce QEMU_DOMAIN_JOB_STATS_TYPE_BACKUP and the convertors and other > plumbing to be able to report statistics for the backup job. > > Signed-off-by: Peter Krempa > --- > src/qemu/qemu_domain.c | 62

Re: [libvirt] [python PATCH] Drop support for python 2

2019-12-04 Thread Michal Privoznik
On 12/4/19 11:41 AM, Daniel P. Berrangé wrote: Signed-off-by: Daniel P. Berrangé --- examples/consolecallback.py | 2 +- examples/dhcpleases.py| 2 +- examples/dominfo.py | 2 +- examples/domipaddrs.py| 2

Re: [libvirt] [PATCH v2 10/25] backup: Implement virsh support for backup

2019-12-04 Thread Daniel P . Berrangé
On Tue, Dec 03, 2019 at 06:17:32PM +0100, Peter Krempa wrote: > From: Eric Blake > > Introduce virsh commands for performing backup jobs. > > Signed-off-by: Eric Blake > Signed-off-by: Peter Krempa > --- > po/POTFILES.in | 1 + > tools/Makefile.am| 1 + > tools/virsh-backup.c |

Re: [libvirt] [PATCH] build: only support python3 binary

2019-12-04 Thread Michal Privoznik
On 12/4/19 11:14 AM, Daniel P. Berrangé wrote: python2 will be end of life by the time of the next libvirt release. All our supported build targets, including CentOS7, have a python3 build available. Signed-off-by: Daniel P. Berrangé --- configure.ac | 4 ++--

Re: [libvirt] [PATCH v2 09/25] tests: genericxml2xml: Add testing of backup XML files

2019-12-04 Thread Daniel P . Berrangé
On Tue, Dec 03, 2019 at 06:17:31PM +0100, Peter Krempa wrote: > Now that the parser and formatter are in place we can excercise it on > the test files. > > Signed-off-by: Peter Krempa > --- > tests/Makefile.am | 1 + > .../backup-pull-seclabel.xml |

Re: [libvirt] [PATCH v2 08/25] backup: Parse and output backup XML

2019-12-04 Thread Daniel P . Berrangé
On Tue, Dec 03, 2019 at 06:17:30PM +0100, Peter Krempa wrote: > From: Eric Blake > > Accept XML describing a generic block job, and output it again as > needed. This may still need a few tweaks to match the documented XML > and RNG schema. > > Signed-off-by: Eric Blake > --- > po/POTFILES.in

Re: [libvirt] [PATCH v2 07/25] backup: Implement backup APIs for remote driver

2019-12-04 Thread Daniel P . Berrangé
On Tue, Dec 03, 2019 at 06:17:29PM +0100, Peter Krempa wrote: > From: Eric Blake > > This one is fairly straightforward - the generator already does what > we need. > > Signed-off-by: Eric Blake > Signed-off-by: Peter Krempa > --- > src/remote/remote_driver.c | 2 ++ >

Re: [libvirt] [PATCH v2 06/25] API: Add domain job operation for backups

2019-12-04 Thread Daniel P . Berrangé
On Tue, Dec 03, 2019 at 06:17:28PM +0100, Peter Krempa wrote: > Introduce VIR_DOMAIN_JOB_OPERATION_BACKUP into virDomainJobOperation. > > Signed-off-by: Peter Krempa > --- > include/libvirt/libvirt-domain.h | 1 + > tools/virsh-domain.c | 4 +++- > 2 files changed, 4 insertions(+),

Re: [libvirt] [PATCH v2 05/25] backup: Introduce virDomainBackup APIs

2019-12-04 Thread Daniel P . Berrangé
On Tue, Dec 03, 2019 at 06:17:27PM +0100, Peter Krempa wrote: > From: Eric Blake > > Introduce a few new public APIs related to incremental backups. This > builds on the previous notion of a checkpoint (without an existing > checkpoint, the new API is a full backup, differing from >

Re: [libvirt] [PATCH v2 00/15] docs: some refactoring and new docs about RPM deployment

2019-12-04 Thread Daniel P . Berrangé
ping On Fri, Nov 22, 2019 at 02:46:47PM +, Daniel P. Berrangé wrote: > This refactors existing docs related to the remote driver/daemon and > URIs. It then also adds a kbase page about RPM package options. > > This introduces the use of RST for docs as a replacement for HTML. > The intent is

Re: [libvirt] [PATCH v2 04/25] backup: Document new XML for backups

2019-12-04 Thread Daniel P . Berrangé
On Tue, Dec 03, 2019 at 06:17:26PM +0100, Peter Krempa wrote: > From: Eric Blake > > Prepare for new backup APIs by describing the XML that will represent > a backup. The XML resembles snapshots and checkpoints in being able > to select actions for a set of disks, but has other differences. It

Re: [libvirt] [PATCH v2 01/25] qemu: domain: Export qemuDomainGetImageIds

2019-12-04 Thread Daniel P . Berrangé
On Tue, Dec 03, 2019 at 06:17:23PM +0100, Peter Krempa wrote: > Signed-off-by: Peter Krempa > --- > src/qemu/qemu_domain.c | 2 +- > src/qemu/qemu_domain.h | 7 +++ > 2 files changed, 8 insertions(+), 1 deletion(-) Reviewed-by: Daniel P. Berrangé Regards, Daniel -- |:

[libvirt] [python PATCH] Drop support for python 2

2019-12-04 Thread Daniel P . Berrangé
Signed-off-by: Daniel P. Berrangé --- examples/consolecallback.py | 2 +- examples/dhcpleases.py| 2 +- examples/dominfo.py | 2 +- examples/domipaddrs.py| 2 +- examples/domrestore.py| 2 +-

[libvirt] [PATCH] build: only support python3 binary

2019-12-04 Thread Daniel P . Berrangé
python2 will be end of life by the time of the next libvirt release. All our supported build targets, including CentOS7, have a python3 build available. Signed-off-by: Daniel P. Berrangé --- configure.ac | 4 ++-- docs/apibuild.py | 4 +---

[libvirt] [PATCH] qemu: keep capabilities when running QEMU as root

2019-12-04 Thread Daniel P . Berrangé
When QEMU uid/gid is set to non-root this is pointless as if we just used a regular setuid/setgid call, the process will have all its capabilities cleared anyway by the kernel. When QEMU uid/gid is set to root, this is almost (always?) never what people actually want. People make QEMU run as root

[libvirt] [PATCH 5/9] libxl: Don't use dom->conn to lookup virNetwork

2019-12-04 Thread Michal Privoznik
When using the monolithic daemon, then dom->conn has all driver tables filled in properly and thus it's safe to call an API other than virDomain*(). However, when using split daemons then dom->conn has only hypervisor driver table set (dom->conn->driver) and the rest is NULL. Therefore, if we want

[libvirt] [PATCH 4/9] qemuGetDHCPInterfaces: Switch to GLib

2019-12-04 Thread Michal Privoznik
If we use glib alloc functions, we can drop the 'cleanup' label and @rv variable and also simplify the code a bit. Signed-off-by: Michal Privoznik --- src/qemu/qemu_driver.c | 40 +++- 1 file changed, 11 insertions(+), 29 deletions(-) diff --git

[libvirt] [PATCH 0/9] Cleanup virConnectPtr usage

2019-12-04 Thread Michal Privoznik
I've noticed this problem when reviewing a patch on the list [1]. Long story short, dom->conn is not guaranteed to have all driver pointers set (consider split daemons). I haven't identified any other violation than what I'm fixing here. But something might have slipped through my git grep. 1:

[libvirt] [PATCH 8/9] lxc: Cleanup virConnectPtr usage

2019-12-04 Thread Michal Privoznik
There are some functions which pass virConnectPtr around for one reason and one reason only: to obtain virLXCDriverPtr in the end. Might replace the argument and pass a pointer to the driver right from the start. Signed-off-by: Michal Privoznik --- src/lxc/lxc_driver.c | 12 +---

[libvirt] [PATCH 9/9] get_nonnull_domain: Drop useless comment

2019-12-04 Thread Michal Privoznik
The intent of get_nonnull_domain() is not to validate virDomain as sent by the client but just to construct the virDomain structure. The validation is then done in each API when looking up the domain in our internal hash tables. Signed-off-by: Michal Privoznik ---

[libvirt] [PATCH 1/9] qemu_driver: Push qemuDomainInterfaceAddresses() a few lines down

2019-12-04 Thread Michal Privoznik
If we place qemuDomainInterfaceAddresses() a few lines below the two functions its using then we can drop forward declarations of those functions. Signed-off-by: Michal Privoznik --- src/qemu/qemu_driver.c | 125 - 1 file changed, 60 insertions(+), 65

[libvirt] [PATCH 3/9] qemuGetDHCPInterfaces: Move some variables inside the loop

2019-12-04 Thread Michal Privoznik
Some variables are not used outside of the for() loop. Move their declaration to clean up the code a bit. Signed-off-by: Michal Privoznik --- src/qemu/qemu_driver.c | 15 --- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/src/qemu/qemu_driver.c

[libvirt] [PATCH 6/9] libxlGetDHCPInterfaces: Move some variables inside the loop

2019-12-04 Thread Michal Privoznik
Some variables are not used outside of the for() loop. Move their declaration to clean up the code a bit. Signed-off-by: Michal Privoznik --- src/libxl/libxl_driver.c | 16 +--- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/src/libxl/libxl_driver.c

[libvirt] [PATCH 2/9] qemu: Don't use dom->conn to lookup virNetwork

2019-12-04 Thread Michal Privoznik
When using the monolithic daemon, then dom->conn has all driver tables filled in properly and thus it's safe to call an API other than virDomain*(). However, when using split daemons then dom->conn has only hypervisor driver table set (dom->conn->driver) and the rest is NULL. Therefore, if we want

[libvirt] [PATCH 7/9] libxlGetDHCPInterfaces: Switch to GLib

2019-12-04 Thread Michal Privoznik
If we use glib alloc functions, we can drop the 'cleanup' label and @rv variable and also simplify the code a bit. Signed-off-by: Michal Privoznik --- src/libxl/libxl_driver.c | 41 +++- 1 file changed, 11 insertions(+), 30 deletions(-) diff --git

[libvirt] [PATCH v3 1/3] conf: fix logic error for scsi units

2019-12-04 Thread Han Han
Introduced in c8007fdc5d2, it should use 'greater than max' instead of 'equal or greater than max' for the condition of checking invalid scsi unit. Signed-off-by: Han Han --- src/conf/domain_conf.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/conf/domain_conf.c

  1   2   >