[libvirt PATCH 05/29] cpu-gather: Move cpuid call to new script

2020-12-15 Thread Tim Wiederhake
Turn the comment on how to aquire cpuid into a runtime error message. Use "http" instead of "https" in the URL, as the latter is broken. Signed-off-by: Tim Wiederhake --- tests/cputestdata/cpu-gather.py | 25 + tests/cputestdata/cpu-gather.sh | 7 --- 2 files

[libvirt PATCH 02/29] cpu-gather: Create python wrapper for shell script

2020-12-15 Thread Tim Wiederhake
This changes the invocation from ./cpu-gather.sh | ./cpu-parse.sh to ./cpu-gather.py | ./cpu-parse.sh Signed-off-by: Tim Wiederhake --- tests/cputestdata/cpu-gather.py | 13 + tests/cputestdata/cpu-gather.sh | 5 + 2 files changed, 18 insertions(+) create mode 100755

[libvirt PATCH 11/29] cpu-gather: Move simple model extraction to new script

2020-12-15 Thread Tim Wiederhake
Signed-off-by: Tim Wiederhake --- tests/cputestdata/cpu-gather.py | 60 + tests/cputestdata/cpu-gather.sh | 13 --- 2 files changed, 60 insertions(+), 13 deletions(-) diff --git a/tests/cputestdata/cpu-gather.py b/tests/cputestdata/cpu-gather.py index

[libvirt PATCH 19/29] cpu-parse: Move file name generation to new script

2020-12-15 Thread Tim Wiederhake
Signed-off-by: Tim Wiederhake --- tests/cputestdata/cpu-gather.py | 18 ++ tests/cputestdata/cpu-parse.sh | 13 - 2 files changed, 18 insertions(+), 13 deletions(-) diff --git a/tests/cputestdata/cpu-gather.py b/tests/cputestdata/cpu-gather.py index

[libvirt PATCH 21/29] cpu-parse: Move json output to new script

2020-12-15 Thread Tim Wiederhake
Signed-off-by: Tim Wiederhake --- tests/cputestdata/cpu-gather.py | 30 ++ tests/cputestdata/cpu-parse.sh| 16 tests/cputestdata/cpu-reformat.py | 9 - 3 files changed, 30 insertions(+), 25 deletions(-) delete mode 100755

[libvirt PATCH 06/29] cpu-gather: Allow overwriting cpuid binary location

2020-12-15 Thread Tim Wiederhake
This is useful if cpuid was compiled from source in a non-standard location. Signed-off-by: Tim Wiederhake --- tests/cputestdata/cpu-gather.py | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/tests/cputestdata/cpu-gather.py b/tests/cputestdata/cpu-gather.py index

[libvirt PATCH 15/29] cpu-gather: Separate data input and output

2020-12-15 Thread Tim Wiederhake
This is a preparatory step to replace the output format with something more readable. Signed-off-by: Tim Wiederhake --- tests/cputestdata/cpu-gather.py | 50 + 1 file changed, 32 insertions(+), 18 deletions(-) diff --git a/tests/cputestdata/cpu-gather.py

[libvirt PATCH 0/2] Schema fixes for virsh [hypervisor-]cpu-compare

2020-12-15 Thread Tim Wiederhake
See individual commit messages for more details. Tim Wiederhake (2): schemas: Deduplicate cpuTopology in cputypes.rng schema: Allow counter element in host cpu definition docs/schemas/cputypes.rng | 15 +-- 1 file changed, 9 insertions(+), 6 deletions(-) -- 2.26.2

[libvirt PATCH 2/2] schema: Allow counter element in host cpu definition

2020-12-15 Thread Tim Wiederhake
If the capabilities include a counter element, e.g. the XML could not be validated: $ virsh capabilities > cap.xml $ virsh [hypervisor-]cpu-compare cap.xml --validate error: Failed to compare hypervisor CPU with cap.txt error: XML document failed to validate against schema: Unable to

[libvirt PATCH 1/2] schemas: Deduplicate cpuTopology in cputypes.rng

2020-12-15 Thread Tim Wiederhake
The duplicate had the "dies" attribute missing, causing $ virsh capabilities > cap.xml $ virsh [hypervisor-]cpu-compare cap.xml --validate to fail with error: Failed to compare hypervisor CPU with cap.xml error: XML document failed to validate against schema: Unable to validate doc

[libvirt PATCH 20/29] cpu-parse: Move xml output to new script

2020-12-15 Thread Tim Wiederhake
Signed-off-by: Tim Wiederhake --- tests/cputestdata/cpu-gather.py | 40 + tests/cputestdata/cpu-parse.sh | 18 --- 2 files changed, 40 insertions(+), 18 deletions(-) diff --git a/tests/cputestdata/cpu-gather.py b/tests/cputestdata/cpu-gather.py index

[libvirt PATCH 08/29] cpu-gather: Move qemu detection to new script

2020-12-15 Thread Tim Wiederhake
Signed-off-by: Tim Wiederhake --- tests/cputestdata/cpu-gather.py | 17 + tests/cputestdata/cpu-gather.sh | 8 2 files changed, 17 insertions(+), 8 deletions(-) diff --git a/tests/cputestdata/cpu-gather.py b/tests/cputestdata/cpu-gather.py index db2348b460..7f9479f78d

[libvirt PATCH 14/29] cpu-gather: Delete old script

2020-12-15 Thread Tim Wiederhake
Signed-off-by: Tim Wiederhake --- tests/cputestdata/cpu-gather.py | 5 - tests/cputestdata/cpu-gather.sh | 6 -- 2 files changed, 11 deletions(-) delete mode 100755 tests/cputestdata/cpu-gather.sh diff --git a/tests/cputestdata/cpu-gather.py b/tests/cputestdata/cpu-gather.py index

[libvirt PATCH 16/29] cpu-parse: Wrap with python script

2020-12-15 Thread Tim Wiederhake
This changes the invocation from ./cpu-gather.py | ./cpu-parse.sh to ./cpu-gather.py [--gather] | ./cpu-gather.py --parse Signed-off-by: Tim Wiederhake --- tests/cputestdata/cpu-gather.py | 29 +++-- tests/cputestdata/cpu-parse.sh | 6 -- 2 files changed, 31

[libvirt PATCH 25/29] cpu-gather: Ignore shutdown messages from qemu

2020-12-15 Thread Tim Wiederhake
Signed-off-by: Tim Wiederhake --- tests/cputestdata/cpu-gather.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/cputestdata/cpu-gather.py b/tests/cputestdata/cpu-gather.py index 10dbeb87cf..49c72df320 100755 --- a/tests/cputestdata/cpu-gather.py +++

[libvirt PATCH 26/29] cpu-gather: Parse cpuid leaves early

2020-12-15 Thread Tim Wiederhake
Signed-off-by: Tim Wiederhake --- tests/cputestdata/cpu-gather.py | 48 ++--- 1 file changed, 26 insertions(+), 22 deletions(-) diff --git a/tests/cputestdata/cpu-gather.py b/tests/cputestdata/cpu-gather.py index 49c72df320..fe660c486e 100755 ---

[libvirt PATCH 22/29] cpu-parse: Move call to cpu-cpuid.py to new script

2020-12-15 Thread Tim Wiederhake
Signed-off-by: Tim Wiederhake --- tests/cputestdata/cpu-gather.py | 10 ++ tests/cputestdata/cpu-parse.sh | 10 -- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/tests/cputestdata/cpu-gather.py b/tests/cputestdata/cpu-gather.py index 2c3f39ab35..373f179c8d

[libvirt PATCH 12/29] cpu-gather: Move full model extraction to new script

2020-12-15 Thread Tim Wiederhake
Signed-off-by: Tim Wiederhake --- tests/cputestdata/cpu-gather.py | 16 +++- tests/cputestdata/cpu-gather.sh | 20 2 files changed, 15 insertions(+), 21 deletions(-) diff --git a/tests/cputestdata/cpu-gather.py b/tests/cputestdata/cpu-gather.py index

[libvirt PATCH 27/29] cpu-gather: Allow gathering and parsing data in one step.

2020-12-15 Thread Tim Wiederhake
Make ./cpu-gather.py --gather --parse an alias of ./cpu-gather.py [--gather] | ./cpu-gather.py --parse Signed-off-by: Tim Wiederhake --- tests/cputestdata/cpu-gather.py | 26 ++ 1 file changed, 14 insertions(+), 12 deletions(-) diff --git

[libvirt PATCH 23/29] cpu-parse: Delete old script

2020-12-15 Thread Tim Wiederhake
Signed-off-by: Tim Wiederhake --- tests/cputestdata/cpu-gather.py | 31 --- tests/cputestdata/cpu-parse.sh | 8 2 files changed, 39 deletions(-) delete mode 100755 tests/cputestdata/cpu-parse.sh diff --git a/tests/cputestdata/cpu-gather.py

[libvirt PATCH 13/29] cpu-gather: Merge model gathering logic

2020-12-15 Thread Tim Wiederhake
Signed-off-by: Tim Wiederhake --- tests/cputestdata/cpu-gather.py | 13 - 1 file changed, 4 insertions(+), 9 deletions(-) diff --git a/tests/cputestdata/cpu-gather.py b/tests/cputestdata/cpu-gather.py index ea1d42f1ec..c639c433d2 100755 --- a/tests/cputestdata/cpu-gather.py +++

[libvirt PATCH 01/29] cpu-cpuid: Shorten overly long line

2020-12-15 Thread Tim Wiederhake
flake8 was complaining. Signed-off-by: Tim Wiederhake --- tests/cputestdata/cpu-cpuid.py | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/tests/cputestdata/cpu-cpuid.py b/tests/cputestdata/cpu-cpuid.py index 37d00e3c97..dac43debb6 100755 ---

[libvirt PATCH 00/29] Refactor scripts in tests/cputestdata

2020-12-15 Thread Tim Wiederhake
This series refactors the various scripts found in tests/cputestdata and adds support for CORE_CAPABILITY MSR, as found on e.g. SnowRidge. Acquiring test data on a new system is a two step process. "cpu-gather.sh" gathers information on the target machine and has as few dependencies as possible.

[libvirt PATCH 29/29] cpu-gather: Add IA32_CORE_CAPABILITY_MSR

2020-12-15 Thread Tim Wiederhake
Signed-off-by: Tim Wiederhake --- tests/cputestdata/cpu-gather.py | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/cputestdata/cpu-gather.py b/tests/cputestdata/cpu-gather.py index 9c9eec6d93..a01c615504 100755 --- a/tests/cputestdata/cpu-gather.py +++ b/tests/cputestdata/cpu-gather.py

[libvirt PATCH 18/29] cpu-parse: Move model name detection to new script

2020-12-15 Thread Tim Wiederhake
Signed-off-by: Tim Wiederhake --- tests/cputestdata/cpu-gather.py | 1 + tests/cputestdata/cpu-parse.sh | 2 -- 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/tests/cputestdata/cpu-gather.py b/tests/cputestdata/cpu-gather.py index 7c51a47353..0b1019456c 100755 ---

[libvirt PATCH 28/29] cpu-gather: Prepare gather_msr for reading multiple msr

2020-12-15 Thread Tim Wiederhake
Signed-off-by: Tim Wiederhake --- tests/cputestdata/cpu-gather.py | 23 ++- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git a/tests/cputestdata/cpu-gather.py b/tests/cputestdata/cpu-gather.py index 46997c8a48..9c9eec6d93 100755 ---

[libvirt PATCH 24/29] cpu-gather: Ignore empty responses from qemu

2020-12-15 Thread Tim Wiederhake
Signed-off-by: Tim Wiederhake --- tests/cputestdata/cpu-gather.py | 8 ++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/tests/cputestdata/cpu-gather.py b/tests/cputestdata/cpu-gather.py index b6acae9eb7..10dbeb87cf 100755 --- a/tests/cputestdata/cpu-gather.py +++

[libvirt PATCH 10/29] cpu-gather: Move static model extraction to new script

2020-12-15 Thread Tim Wiederhake
Signed-off-by: Tim Wiederhake --- tests/cputestdata/cpu-gather.py | 13 ++--- tests/cputestdata/cpu-gather.sh | 5 - 2 files changed, 10 insertions(+), 8 deletions(-) diff --git a/tests/cputestdata/cpu-gather.py b/tests/cputestdata/cpu-gather.py index e4a82fc949..0c8d8e93d0 100755

[libvirt PATCH 03/29] cpu-gather: Move model_name to new script

2020-12-15 Thread Tim Wiederhake
Signed-off-by: Tim Wiederhake --- tests/cputestdata/cpu-gather.py | 13 + tests/cputestdata/cpu-gather.sh | 2 -- 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/tests/cputestdata/cpu-gather.py b/tests/cputestdata/cpu-gather.py index f7030eb48b..1b02df6ec7 100755 ---

[libvirt PATCH 04/29] cpu-gather: Allow overwriting model name

2020-12-15 Thread Tim Wiederhake
Some hardware, e.g. exotic platforms or pre-production hardware, may report wrong or random data for the cpu model name. As the name of the created files is derived from that name, this may lead to issues. Signed-off-by: Tim Wiederhake --- tests/cputestdata/cpu-gather.py | 19

[libvirt PATCH 17/29] cpu-gather: Transport data as json

2020-12-15 Thread Tim Wiederhake
More reliable, easier to parse, easier to edit. Signed-off-by: Tim Wiederhake --- tests/cputestdata/cpu-gather.py | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/tests/cputestdata/cpu-gather.py b/tests/cputestdata/cpu-gather.py index 83f175d342..7c51a47353 100755 ---

Re: [RFC PATCH 3/6] docs: added rng schema and formatdomain for NFS

2020-12-15 Thread Laine Stump
On 12/14/20 5:57 PM, Ryan Gahagan wrote: On Thu, Dec 10, 2020 at 10:37 PM Han Han > wrote: On Fri, Dec 11, 2020 at 4:00 AM Ryan Gahagan mailto:rgaha...@cs.utexas.edu>> wrote: diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst index

[libvirt PATCH 09/29] cpu-gather: Move static model expansion to new script

2020-12-15 Thread Tim Wiederhake
Signed-off-by: Tim Wiederhake --- tests/cputestdata/cpu-gather.py | 45 + tests/cputestdata/cpu-gather.sh | 7 - 2 files changed, 45 insertions(+), 7 deletions(-) diff --git a/tests/cputestdata/cpu-gather.py b/tests/cputestdata/cpu-gather.py index

[libvirt PATCH 07/29] cpu-gather: Move msr decoding to new script

2020-12-15 Thread Tim Wiederhake
Fixes the leaking file descriptors. Does not silently ignore errors (e.g. permission denied on /dev/cpu/0/msr if run as non-root) and always attempt to read from /dev/kvm if /dev/cpu/0/msr failed. 'gather_msr()' returns a dictionary of values, as a later patch will add more registers to be

Re: [RFC PATCH 1/6] compatibilities: added in flags for NFS support

2020-12-15 Thread Laine Stump
(answering for Peter since he is on "vacation", so it may be awhile before he gets around to this) On 12/14/20 5:54 PM, Ryan Gahagan wrote: On Fri, Dec 11, 2020 at 3:35 AM Peter Krempa > wrote: In subject/summary. We don't have anything which we'd prefix with

Re: [PATCHv3 0/5] netdev: Extract GenerateName/ReserveName as common functions

2020-12-15 Thread Laine Stump
On 12/13/20 8:50 PM, Shi Lei wrote: V2 here: https://www.redhat.com/archives/libvir-list/2020-December/msg00563.html Since V2: * Fix libxl driver for missing changing virNetDevMacVLanReserveName V1 here: https://www.redhat.com/archives/libvir-list/2020-December/msg00308.html Since V1:

[libvirt][PATCH v1 2/3] schema: add 'restrictive' config option for mode in numatune

2020-12-15 Thread Luyao Zhong
support 'restrictive' mode in memory element and memnode element in numatune: ... ... --- docs/schemas/domaincommon.rng | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng index

[PATCH 3/5] qemu: remove redundant code that adds "template" netdev name

2020-12-15 Thread Laine Stump
The lower level function virNetDevGenerateName() now understands that a blank ifname should be replaced with a generated name based on a template that it knows about itself - there is no need for the higher level functions to stuff a template name ("vnet%d") into ifname. Signed-off-by: Laine

[libvirt][PATCH v1 0/3] introduce 'restrictive' mode in numatune

2020-12-15 Thread Luyao Zhong
Before this patch set, numatune only has three memory modes: static, interleave and prefered. These memory policies are ultimately set by mbind() system call. Memory policy could be 'hard coded' into the kernel, but none of above policies fit our requirment under this case. mbind() support

[libvirt][PATCH v1 1/3] docs: add docs for 'restrictive' option for mode in numatune

2020-12-15 Thread Luyao Zhong
When user would like use cgroups to restrict the allowed memory nodes, and require not setting any specific memory policy, then 'restrictive' mode is useful. --- docs/formatdomain.rst | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/docs/formatdomain.rst

[libvirt][PATCH v1 3/3] qemu: add parser and formatter for 'restrictive' mode in numatune

2020-12-15 Thread Luyao Zhong
--- include/libvirt/libvirt-domain.h | 1 + src/conf/numa_conf.c | 9 + src/qemu/qemu_command.c | 6 ++- src/qemu/qemu_process.c | 27 + src/util/virnuma.c| 3 ++

Re: [PATCHv3 5/5] netdev: Prevent functions that call virNetDevTapCreate from adding 'vnet%d' into ifname

2020-12-15 Thread Laine Stump
(NB: As I looked further into the details of this, I decided it would be simpler for me to just post a few patches to take the place of this one patch. I've left in the reasoning for a couple of those patches which I had typed before I made the decision to just make a new series, but this

[PATCH 1/5] util: fix tap device name auto-generation for FreeBSD

2020-12-15 Thread Laine Stump
The Linux implementation of virNetDevCreate() no longer requires a template ifname (e.g. "vnet%d") when it is called, but just generates a new name if ifname is empty. The FreeBSD implementation requires that the caller actually fill in a template ifname, and will fail if ifname is empty. Since we

[PATCH 2/5] bhyve: remove redundant code that adds "template" netdev name

2020-12-15 Thread Laine Stump
The FreeBSD version of virNetDevTapCreate() now calls virNetDevGenerateName(), and virNetDevGenerateName() understands that a blank ifname should be replaced with a generated name based on a device-type-specific template - so there is no longer any need for the higher level functions to stuff a

[PATCH 0/5] Followup to virNetDevGenerateName() patches

2020-12-15 Thread Laine Stump
A few issues came up during review of this series that were better fixed in cleanups rather than requiring yet another round of review. (A couple of things I noticed after the other patches were already pushed). Laine Stump (5): util: fix tap device name auto-generation for FreeBSD bhyve:

[PATCH 4/5] util: simplify virNetDevMacVLanCreateWithVPortProfile()

2020-12-15 Thread Laine Stump
Since commit 282d135ddbb the parser for has cleared out any interface name from the input XML that used the macvtap/macvlan name as a prefix. Along with that, the switch to use the new virNetDevGenerateName() function for auto-generating macvtap/macvlan device names (commit 9b5d741a9), has

[PATCH 5/5] util: minor comment/formatting changes to virNetDevTapCreate()

2020-12-15 Thread Laine Stump
The comment about auto-generating names was obsoleted by recent changes, and there was an unnecessary set of braces around a single line conditional body. Signed-off-by: Laine Stump --- src/util/virnetdevtap.c | 11 +-- 1 file changed, 5 insertions(+), 6 deletions(-) diff --git

[PATCHv3 0/3] Use netlink to create veth device pair when netlink is supported

2020-12-15 Thread Shi Lei
V2 here: https://www.redhat.com/archives/libvir-list/2020-November/msg01239.html Since V2: * Remove the argument 'status' of virNetDevVethCreateInternal * fix a memory leak in virLXCProcessSetupInterfaceTap V1 here: https://www.redhat.com/archives/libvir-list/2020-November/msg00898.html

[PATCHv3 2/3] util:veth: Create veth device pair by netlink

2020-12-15 Thread Shi Lei
When netlink is supported, use netlink to create veth device pair rather than 'ip link' command. Signed-off-by: Shi Lei --- src/util/virnetdevveth.c | 126 ++- 1 file changed, 72 insertions(+), 54 deletions(-) diff --git a/src/util/virnetdevveth.c

[PATCHv3 3/3] lxc: fix a memory leak

2020-12-15 Thread Shi Lei
In virLXCProcessSetupInterfaceTap, containerVeth needs to be freed on failure. Signed-off-by: Shi Lei --- src/lxc/lxc_process.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/lxc/lxc_process.c b/src/lxc/lxc_process.c index 85d0287a..0f7c9295 100644 ---

[PATCHv3 1/3] util:netlink: Enable virNetlinkNewLink to support veth

2020-12-15 Thread Shi Lei
Signed-off-by: Shi Lei --- src/util/virnetlink.c | 14 ++ src/util/virnetlink.h | 1 + 2 files changed, 15 insertions(+) diff --git a/src/util/virnetlink.c b/src/util/virnetlink.c index fdd3a6a4..8625b896 100644 --- a/src/util/virnetlink.c +++ b/src/util/virnetlink.c @@ -24,6 +24,7

Re: [libvirt PATCH] ci: containers: Refresh the Dockerfiles

2020-12-15 Thread Daniel P . Berrangé
On Mon, Dec 14, 2020 at 08:30:56PM +0100, Erik Skultety wrote: > Contains changes utilizing "nosync" and "eatmydata" for speedup as well > as fixes for CentOS-8 repoid regression. > ci-commit: b098ec6631a85880f818f2dd25c437d509e53680 > > Signed-off-by: Erik Skultety > --- > Pipeline on my

Re: [PATCH] lxc: don't try to reserve macvtap name for LXC domains

2020-12-15 Thread Daniel P . Berrangé
On Mon, Dec 14, 2020 at 04:30:48PM -0500, Laine Stump wrote: > Commit 729a06c41 added code to the LXC driver (patterned after similar > code in the QEMU driver) that called > virNetDevMacVlanReserveName(net->ifname) for all type='direct' > interfaces during a libvirtd restart, to prevent other

Re: Libvirt-lxc: iptables not working in containers

2020-12-15 Thread Michal Privoznik
On 12/14/20 12:05 AM, John Hurnett wrote: Hi, I can't get iptables to work in libvirt-lxc containers. "iptables -L" command shows empty chains. However I tested the same scenario with pure lxc and iptables works as it should. Has anyone experienced that? It seems like a bug, but maybe there is

Libvirt-lxc: iptables not working in containers

2020-12-15 Thread John Hurnett
Hi, I can't get iptables to work in libvirt-lxc containers. "iptables -L" command shows empty chains. However I tested the same scenario with pure lxc and iptables works as it should. Has anyone experienced that? It seems like a bug, but maybe there is some libvirt xml parameter I am missing? BR