[libvirt] [PATCH v10 06/10] backup: Implement virsh support for backup

2019-08-21 Thread Eric Blake
Introduce a few more new virsh commands for performing backup jobs. At this time, I did not opt for a convenience command 'backup-begin-as' that cobbles together appropriate XML from the user's command line arguments, but that may be a viable future extension. Similarly, since backup is a

[libvirt] [PATCH v10 10/10] backup: Implement qemu incremental pull backup

2019-08-21 Thread Eric Blake
Complete wiring up incremental backup, by adding in support for creating a checkpoint at the same time as a backup (make the transaction have a few more steps) as well as exposing the dirty bitmap for a prior backup over NBD (requires creating a temporary bitmap, merging all appropriate bitmaps

[libvirt] [PATCH v10 03/10] backup: Introduce virDomainBackup APIs

2019-08-21 Thread Eric Blake
Introduce a few new public APIs related to incremental backups. This builds on the previous notion of a checkpoint (without an existing checkpoint, the new API is a full backup, differing from virDomainBlockCopy in the point of time chosen and in operation on multiple disks at once); and also

[libvirt] [PATCH v10 07/10] backup: qemu: Implement framework for backup job APIs

2019-08-21 Thread Eric Blake
A later patch will be added to actually kick off the right QMP commands, but at least this framework allows validation of backup XML, including the fact that a backup job can survive a libvirtd restart. Atomically creating a checkpoint alongside the backup is also left for a later patch.

[libvirt] [PATCH v10 08/10] backup: Wire up qemu full pull backup commands over QMP

2019-08-21 Thread Eric Blake
Time to actually issue the QMP transactions that start and stop backup commands (for now, just pull mode, not push). Starting a job has to kick off several pre-req steps, then a transaction, and additionally spawn an NBD server for pull mode; ending a job as well as failing partway through

[libvirt] [PATCH v10 05/10] backup: Parse and output backup XML

2019-08-21 Thread Eric Blake
Accept XML describing a generic block job, and output it again as needed. This may still need a few tweaks to match the documented XML and RNG schema. Signed-off-by: Eric Blake --- src/conf/backup_conf.h | 94 +++ src/conf/virconftypes.h | 3 + src/conf/Makefile.inc.am | 2 +

[libvirt] [PATCH v10 00/10] incremental backup

2019-08-21 Thread Eric Blake
This is not the final version of incremental backup - before we can accept this series, it needs a lot of polish to pick up cleanups made possible by Peter's blockdev work, and we need to settle on our Job API addition (so that the return value of BackupBegin and the parameter to BackupGetXMLDesc

[libvirt] [PATCH v10 04/10] backup: Implement backup APIs for remote driver

2019-08-21 Thread Eric Blake
This one is fairly straightforward - the generator already does what we need. Signed-off-by: Eric Blake Reviewed-by: Daniel P. Berrangé --- src/remote/remote_driver.c | 3 ++ src/remote/remote_protocol.x | 54 +++- src/remote_protocol-structs | 28

[libvirt] [PATCH v10 09/10] backup: qemu: Wire up qemu full push backup commands over QMP

2019-08-21 Thread Eric Blake
Update the code to support push backups; for now, the destination file still has to be local, although the XML could be extended into supporting remote destinations (where we will have to use the full power of blockdev-add). This also touches up the event handling to inform the user when the job

[libvirt] [PATCH v10 02/10] backup: Document new XML for backups

2019-08-21 Thread Eric Blake
Prepare for new backup APIs by describing the XML that will represent a backup. The XML resembles snapshots and checkpoints in being able to select actions for a set of disks, but has other differences. It can support both push model (the hypervisor does the backup directly into the destination

[libvirt] [PATCH v10 01/10] backup: qemu: Implement VIR_DOMAIN_CHECKPOINT_XML_SIZE flag

2019-08-21 Thread Eric Blake
Once a checkpoint has been created, it is desirable to estimate the size of the disk delta that is represented between the checkpoint and the current operation. To do this, we have to scrape information out of QMP query-block on a request from the user. Signed-off-by: Eric Blake ---

[libvirt] [PATCH v2 8/9] qemu: Implement virDomainGetGuestInfo()

2019-08-21 Thread Jonathon Jongsma
Iimplements the new guest information API by querying requested information via the guest agent. Signed-off-by: Jonathon Jongsma --- src/qemu/qemu_driver.c | 110 + 1 file changed, 110 insertions(+) diff --git a/src/qemu/qemu_driver.c

[libvirt] [PATCH v2 4/9] qemu: add helper function for querying OS info

2019-08-21 Thread Jonathon Jongsma
This function queries the guest operating system information and adds the returned information to an array of typed parameters with field names intended to be returned in virDomainGetGuestInfo(). Signed-off-by: Jonathon Jongsma --- src/qemu/qemu_agent.c | 53 ++

[libvirt] [PATCH v2 7/9] qemu: add helper for getting full FSInfo

2019-08-21 Thread Jonathon Jongsma
This function adds the complete filesystem information returned by the qemu agent to an array of typed parameters with field names intended to to be returned by virDomainGetGuestInfo() --- src/qemu/qemu_agent.c | 89 ++ src/qemu/qemu_agent.h | 5 + tests/qemuagenttest.c | 210

[libvirt] [PATCH v2 5/9] qemu: add helper for querying timezone info

2019-08-21 Thread Jonathon Jongsma
This function queries timezone information within the guest and adds the information to an array of typed parameters with field names intended to be returned to virDomainGetGuestInfo() Signed-off-by: Jonathon Jongsma --- src/qemu/qemu_agent.c | 46 ++

[libvirt] [PATCH v2 3/9] qemu: add helper for getting guest users

2019-08-21 Thread Jonathon Jongsma
This function fetches the list of logged-in users from the qemu agent and adds them to a list of typed parameters so that they can be used internally in libvirt. Also add some basic tests for the function. Signed-off-by: Jonathon Jongsma --- src/qemu/qemu_agent.c | 91 +++

[libvirt] [PATCH v2 6/9] qemu: add support for new fields in FSInfo

2019-08-21 Thread Jonathon Jongsma
Since version 3.0, qemu has returned disk usage statistics in guest-get-fsinfo. And since 3.1, it has returned information about the disk serial number and device node of disks that are targeted by the filesystem. Unfortunately, the public API virDomainGetFSInfo() returns the filesystem info

[libvirt] [PATCH v2 9/9] virsh: add 'guestinfo' command

2019-08-21 Thread Jonathon Jongsma
The 'guestinfo' command uses the new virDomainGetGuestInfo() API to query information about the specified domain and print it out for the user. The output is modeled roughly on the 'domstats' command. Signed-off-by: Jonathon Jongsma --- tools/virsh-domain.c | 85

[libvirt] [PATCH v2 2/9] remote: implement virDomainGetGuestInfo

2019-08-21 Thread Jonathon Jongsma
Add daemon and client code to serialize/deserialize virDomainGetGuestInfo(). Signed-off-by: Jonathon Jongsma --- src/remote/remote_daemon_dispatch.c | 41 ++ src/remote/remote_driver.c | 53 + src/remote/remote_protocol.x| 21

[libvirt] [PATCH v2 0/9] Add virDomainGetGuestInfo()

2019-08-21 Thread Jonathon Jongsma
This series adds several bits of guest information provided by a new API function virDomainGetGuestInfo(). There is an implementation for qemu using the guest agent. In particular, it adds information about logged-in users, guest OS, timezone, hostname, and filesystem info. This is the second

[libvirt] [PATCH v2 1/9] lib: add virDomainGetGuestInfo()

2019-08-21 Thread Jonathon Jongsma
This API is intended to aggregate several guest agent information queries and is ispired by stats API virDomainListGetStats(). It is anticipated that this information will be provided by a guest agent running within the domain. Signed-off-by: Jonathon Jongsma ---

Re: [libvirt] QEMU bitmap backup usability FAQ

2019-08-21 Thread John Snow
On 8/21/19 10:21 AM, Vladimir Sementsov-Ogievskiy wrote: > [CC Nikolay] > > 21.08.2019 1:25, John Snow wrote: >> Hi, downstream here at Red Hat I've been fielding some questions about >> the usability and feature readiness of Bitmaps (and related features) in >> QEMU. >> >> Here are some

Re: [libvirt] Fwd: libvirtd failing on MacOS in setgroups

2019-08-21 Thread Roman Bolshakov
On Wed, Aug 21, 2019 at 05:55:51PM +0100, Daniel P. Berrangé wrote: > On Wed, Aug 21, 2019 at 12:47:03PM -0400, Marcus Furlong wrote: > > On Wed, 21 Aug 2019 at 08:23, Daniel P. Berrangé > > wrote: > > > > > > On Tue, Aug 20, 2019 at 11:11:07AM -0400, Marcus Furlong wrote: > > > > Resend to

Re: [libvirt] [PATCH v3] tools: console: Relax stream EOF handling

2019-08-21 Thread Daniel Henrique Barboza
On 8/21/19 10:33 AM, Roman Bolshakov wrote: Regular VM shutdown triggers the error for existing session of virsh console and it returns with non-zero exit code: error: internal error: console stream EOF The message and status code are misleading because there's no real error.

Re: [libvirt] [PATCH 00/11] Fix 10 tests on macOS

2019-08-21 Thread Daniel P . Berrangé
On Wed, Aug 21, 2019 at 06:18:11PM +0100, Daniel P. Berrangé wrote: > On Wed, Aug 21, 2019 at 07:13:12PM +0300, Roman Bolshakov wrote: > > Hi! > > > > This patch series attempts to reduce the number of failing tests on macOS. > > > > The fixes involve some funk with macOS dynamic and static

Re: [libvirt] [PATCH 00/11] Fix 10 tests on macOS

2019-08-21 Thread Daniel P . Berrangé
On Wed, Aug 21, 2019 at 07:13:12PM +0300, Roman Bolshakov wrote: > Hi! > > This patch series attempts to reduce the number of failing tests on macOS. > > The fixes involve some funk with macOS dynamic and static linkers, dyld and > ld64, respectively. > > As result, instead of 15 failing tests

Re: [libvirt] [PATCH 11/11] tests: Make references to global symbols indirect in test drivers

2019-08-21 Thread Daniel P . Berrangé
On Wed, Aug 21, 2019 at 07:13:23PM +0300, Roman Bolshakov wrote: > A library has to be built with -flat_namespace to get all references to > global symbols indirected. That can also be achieved with two-level > namespace interposition but we're not using explicit symbol > interposition since it's

Re: [libvirt] [PATCH 10/11] tests: Avoid gnulib replacements in mocks

2019-08-21 Thread Daniel P . Berrangé
On Wed, Aug 21, 2019 at 07:13:22PM +0300, Roman Bolshakov wrote: > gnulib headers change stat, lstat and open to replacement functions, > even for function definitions. This effectively disables standard > library overrides in virfilewrapper and virmockstathelpers since they > are never reached. >

Re: [libvirt] [PATCH 09/11] tests: Use flat namespace on macOS

2019-08-21 Thread Daniel P . Berrangé
On Wed, Aug 21, 2019 at 07:13:21PM +0300, Roman Bolshakov wrote: > Test executables and mocks have assumption that any symbol can be > replaced with LD_PRELOAD. That's not a case for macOS unless flat > namespace is used, because every external symbol reference records the > library to be looked

Re: [libvirt] [PATCH 08/11] tests: Lookup extended stat/lstat in mocks

2019-08-21 Thread Daniel P . Berrangé
On Wed, Aug 21, 2019 at 07:13:20PM +0300, Roman Bolshakov wrote: > macOS syscall interface (/usr/lib/system/libsystem_kernel.dylib) has > three kinds of stat but only one of them can be used to fill > "struct stat": stat$INODE64. > > virmockstathelpers looks up regular stat instead of

Re: [libvirt] [PATCH 07/11] build: Use flat namespace for libvirt on macOS

2019-08-21 Thread Daniel P . Berrangé
On Wed, Aug 21, 2019 at 07:13:19PM +0300, Roman Bolshakov wrote: > >From ld(1): > > By default all references resolved to a dynamic library record the > library to which they were resolved. At runtime, dyld uses that > information to directly resolve symbols. The alternative is to use the >

Re: [libvirt] [PATCH 06/11] tests: Drop /private CWD prefix in commandhelper

2019-08-21 Thread Daniel P . Berrangé
On Wed, Aug 21, 2019 at 07:13:18PM +0300, Roman Bolshakov wrote: > /tmp is a symbolic link to /private/tmp on macOS. That causes failures > in commandtest, because getcwd returns /private/tmp and the expected > output doesn't match to "CWD: /tmp". > > Rathern than making a copy of commanddata

Re: [libvirt] [PATCH 05/11] tests: Remove -module flag for mocks

2019-08-21 Thread Daniel P . Berrangé
On Wed, Aug 21, 2019 at 07:13:17PM +0300, Roman Bolshakov wrote: > macOS has two kinds of loadable libraries: MH_BUNDLE, and MH_DYLIB. > bundle is used for plugins that are loaded with dlopen/dlsym/dlclose. > And there's no way to preload a bundle into an application. dynamic > linker (dyld) will

Re: [libvirt] [PATCH 04/11] tests: Add lib- prefix to all mocks

2019-08-21 Thread Daniel P . Berrangé
On Wed, Aug 21, 2019 at 07:13:16PM +0300, Roman Bolshakov wrote: > In preparation libtool "-module" flag removal, add lib prefix to all > mock shared objects. > > While at it, introduce VIR_TEST_MOCK macros that makes path out of mock > name to be used with VIR_TEST_PRELOAD or

Re: [libvirt] [PATCH 03/11] tests: Preload mocks with DYLD_INSERT_LIBRARIES on macOS

2019-08-21 Thread Daniel P . Berrangé
On Wed, Aug 21, 2019 at 07:13:15PM +0300, Roman Bolshakov wrote: > LD_PRELOAD has no effect on macOS. Instead, dyld(1) provides a way for > symbol hooking via DYLD_INSERT_LIBRARIES. The variable should contain > colon-separated paths to the dylibs to be inserted. > > Signed-off-by: Roman

Re: [libvirt] [PATCH 02/11] tests: Avoid IPv4-translated IPv6 address in sockettest

2019-08-21 Thread Daniel P . Berrangé
On Wed, Aug 21, 2019 at 07:13:14PM +0300, Roman Bolshakov wrote: > getnameinfo on macOS formats certain IPv6 addresses as IPv4-translated > addresses. The following pattern has been observed: > :: is formated as ::0.0.255.255 > ::fffe is formated as ::0.0.255.254 > :::0 is formated

Re: [libvirt] [PATCH 01/11] tests: Don't test octal localhost IP in sockettest on macOS

2019-08-21 Thread Daniel P . Berrangé
On Wed, Aug 21, 2019 at 07:13:13PM +0300, Roman Bolshakov wrote: > getaddrinfo on macOS doesn't interpret octal IPv4 addresses. Only > inet_aton can be used for that. Therefore, from macOS standpoint > "0177.0.0.01" is not the same as "127.0.0.1". > > The issue was also discovered by python and

Re: [libvirt] Fwd: libvirtd failing on MacOS in setgroups

2019-08-21 Thread Daniel P . Berrangé
On Wed, Aug 21, 2019 at 12:47:03PM -0400, Marcus Furlong wrote: > On Wed, 21 Aug 2019 at 08:23, Daniel P. Berrangé wrote: > > > > On Tue, Aug 20, 2019 at 11:11:07AM -0400, Marcus Furlong wrote: > > > Resend to libvir-list in case that is more appropriate: > > > > > > > > > Hi, > > > > > > I get

Re: [libvirt] Fwd: libvirtd failing on MacOS in setgroups

2019-08-21 Thread Marcus Furlong
On Wed, 21 Aug 2019 at 08:23, Daniel P. Berrangé wrote: > > On Tue, Aug 20, 2019 at 11:11:07AM -0400, Marcus Furlong wrote: > > Resend to libvir-list in case that is more appropriate: > > > > > > Hi, > > > > I get the following error when running libvirtd on MacOS as root: > > > > 2019-07-11

Re: [libvirt] [PATCH 00/10] ci: Several fixes and improvements

2019-08-21 Thread Ján Tomko
On Fri, Aug 16, 2019 at 11:49:44AM +0200, Andrea Bolognani wrote: See the individual commits for details, but the gist of it is that after this series it's possible for users to hook into the build process and customize it according to their needs; on top of that, the whole thing is made more

[libvirt] [PATCH 06/11] tests: Drop /private CWD prefix in commandhelper

2019-08-21 Thread Roman Bolshakov
/tmp is a symbolic link to /private/tmp on macOS. That causes failures in commandtest, because getcwd returns /private/tmp and the expected output doesn't match to "CWD: /tmp". Rathern than making a copy of commanddata solely for macOS, the /private prefix is stripped. Signed-off-by: Roman

[libvirt] [PATCH 11/11] tests: Make references to global symbols indirect in test drivers

2019-08-21 Thread Roman Bolshakov
A library has to be built with -flat_namespace to get all references to global symbols indirected. That can also be achieved with two-level namespace interposition but we're not using explicit symbol interposition since it's more verbose and requires massive changes to the mocks. This provides a

[libvirt] [PATCH 07/11] build: Use flat namespace for libvirt on macOS

2019-08-21 Thread Roman Bolshakov
>From ld(1): By default all references resolved to a dynamic library record the library to which they were resolved. At runtime, dyld uses that information to directly resolve symbols. The alternative is to use the -flat_namespace option. With flat namespace, the library is not

[libvirt] [PATCH 08/11] tests: Lookup extended stat/lstat in mocks

2019-08-21 Thread Roman Bolshakov
macOS syscall interface (/usr/lib/system/libsystem_kernel.dylib) has three kinds of stat but only one of them can be used to fill "struct stat": stat$INODE64. virmockstathelpers looks up regular stat instead of stat$INODE64. That causes a failure in qemufirmwaretest because "struct stat" is laid

[libvirt] [PATCH 10/11] tests: Avoid gnulib replacements in mocks

2019-08-21 Thread Roman Bolshakov
gnulib headers change stat, lstat and open to replacement functions, even for function definitions. This effectively disables standard library overrides in virfilewrapper and virmockstathelpers since they are never reached. Rename the functions and provide a declartion that uses correct assembler

[libvirt] [PATCH 00/11] Fix 10 tests on macOS

2019-08-21 Thread Roman Bolshakov
Hi! This patch series attempts to reduce the number of failing tests on macOS. The fixes involve some funk with macOS dynamic and static linkers, dyld and ld64, respectively. As result, instead of 15 failing tests we get only 5. The tests have been fixed: qemublocktest qemumonitorjsontest

[libvirt] [PATCH 04/11] tests: Add lib- prefix to all mocks

2019-08-21 Thread Roman Bolshakov
In preparation libtool "-module" flag removal, add lib prefix to all mock shared objects. While at it, introduce VIR_TEST_MOCK macros that makes path out of mock name to be used with VIR_TEST_PRELOAD or VIR_TEST_MAIN_PRELOAD. That, hopefully, improves readability, reduces line length and allows

[libvirt] [PATCH 05/11] tests: Remove -module flag for mocks

2019-08-21 Thread Roman Bolshakov
macOS has two kinds of loadable libraries: MH_BUNDLE, and MH_DYLIB. bundle is used for plugins that are loaded with dlopen/dlsym/dlclose. And there's no way to preload a bundle into an application. dynamic linker (dyld) will reject it when finds it in DYLD_INSERT_LIBRARIES. Unfortunately, a

[libvirt] [PATCH 09/11] tests: Use flat namespace on macOS

2019-08-21 Thread Roman Bolshakov
Test executables and mocks have assumption that any symbol can be replaced with LD_PRELOAD. That's not a case for macOS unless flat namespace is used, because every external symbol reference records the library to be looked up. And the symbols cannot be replaced unless dyld interposing is used.

[libvirt] [PATCH 03/11] tests: Preload mocks with DYLD_INSERT_LIBRARIES on macOS

2019-08-21 Thread Roman Bolshakov
LD_PRELOAD has no effect on macOS. Instead, dyld(1) provides a way for symbol hooking via DYLD_INSERT_LIBRARIES. The variable should contain colon-separated paths to the dylibs to be inserted. Signed-off-by: Roman Bolshakov --- tests/testutils.h | 10 -- 1 file changed, 8 insertions(+),

[libvirt] [PATCH 01/11] tests: Don't test octal localhost IP in sockettest on macOS

2019-08-21 Thread Roman Bolshakov
getaddrinfo on macOS doesn't interpret octal IPv4 addresses. Only inet_aton can be used for that. Therefore, from macOS standpoint "0177.0.0.01" is not the same as "127.0.0.1". The issue was also discovered by python and dotnet core: https://bugs.python.org/issue27612

[libvirt] [PATCH 02/11] tests: Avoid IPv4-translated IPv6 address in sockettest

2019-08-21 Thread Roman Bolshakov
getnameinfo on macOS formats certain IPv6 addresses as IPv4-translated addresses. The following pattern has been observed: :: is formated as ::0.0.255.255 ::fffe is formated as ::0.0.255.254 :::0 is formated as ::255.255.0.0 ::fffe:0 is formated as ::255.254.0.0 :::0:0 is

Re: [libvirt] [PATCH 2/3] qemu: add hypervisor feature cpu-pm support for kvm

2019-08-21 Thread Ján Tomko
On Mon, Aug 19, 2019 at 10:32:25AM +0200, Menno Lageman wrote: From: Wim ten Have QEMU introduced a CPU power management feature with commit 6f131f13e68d ("kvm: support -overcommit cpu-pm=on|off"). This patch series adds support for controlling QEMU's "-overcommit" command line switch:

Re: [libvirt] [PATCH 1/3] qemu: introduce qemuBuildOvercommitCommandLine()

2019-08-21 Thread Ján Tomko
On Mon, Aug 19, 2019 at 10:32:24AM +0200, Menno Lageman wrote: In preparation for adding support for '-overcommit cpu-pm=[on|off]', add qemuBuildOvercommitCommandLine() to generate the '-overcommit' commandline. Move the existing '-overcommit mem-lock=[on|off]' generation code from

Re: [libvirt] [PATCH v2 1/9] vbox: Add various vir*Flags API

2019-08-21 Thread Eric Blake
On 7/10/19 6:50 AM, Eric Blake wrote: > On 7/10/19 2:02 AM, Peter Krempa wrote: >> On Tue, Jul 09, 2019 at 12:46:30 -0500, Eric Blake wrote: >>> Even though we don't accept any flags, it is unfriendly to callers >>> that use the modern API to have to fall back to the flag-free API. >>> >>>

Re: [libvirt] [PATCH 05/10] qemu: Split out preparing of single snapshot from qemuDomainSnapshotDiskDataCollect

2019-08-21 Thread Ján Tomko
On Fri, Aug 16, 2019 at 03:54:39PM +0200, Peter Krempa wrote: Move the internals into qemuDomainSnapshotDiskDataCollectOne to make it obvious what's happening after moving more code here. Signed-off-by: Peter Krempa --- src/qemu/qemu_driver.c | 104 +++-- 1

Re: [libvirt] [PATCH 04/10] qemu: snapshot: Restrict file existance check only for local storage

2019-08-21 Thread Ján Tomko
s/existance/existence/ in the commit summary On Fri, Aug 16, 2019 at 03:54:38PM +0200, Peter Krempa wrote: Soon we'll allow more protocols and storage types with snapshots where we in some cases can't check whether the storage already exists. Restrict the sanity checks whether the destination

Re: [libvirt] [PATCH 03/10] qemu: Remove cleanup label in qemuDomainSnapshotPrepareDiskExternal

2019-08-21 Thread Ján Tomko
On Fri, Aug 16, 2019 at 03:54:37PM +0200, Peter Krempa wrote: Refactor the code to avoid having a cleanup label. This will simplify the change necessary when restricting this check in an upcoming patch. Signed-off-by: Peter Krempa --- src/qemu/qemu_driver.c | 28 +++- 1

Re: [libvirt] [PATCH 01/10] qemu: snapshot: Don't modify persistent XML if disk source is different

2019-08-21 Thread Ján Tomko
On Fri, Aug 16, 2019 at 03:54:35PM +0200, Peter Krempa wrote: While the VM is running the persistent source of a disk might differ e.g. as the 'newDef' was redefined. Our snapshot code would blindly rewrite the source of such disk if it shared the 'target'. Fix this by checking whether the

Re: [libvirt] [PATCH 02/10] qemu: driver: Remove dead code from qemuDomainSnapshotUpdateDiskSources

2019-08-21 Thread Ján Tomko
On Fri, Aug 16, 2019 at 03:54:36PM +0200, Peter Krempa wrote: dd->src is always allocated in this function as it contains the new source for the snapshot which is meant to replace the disk source. The label handling code executed if that source was not present thus is dead code. Remove it.

Re: [libvirt] [PATCH 5/5] security_util: Remove stale XATTRs

2019-08-21 Thread Jiri Denemark
On Wed, Aug 14, 2019 at 16:33:23 +0200, Michal Privoznik wrote: > It may happen that we leave some XATTRs behind. For instance, on > a sudden power loss, the host just shuts down without calling > restore on domain paths. This creates a problem, because when the > host starts up again, the XATTRs

Re: [libvirt] [PATCH 4/5] util: Introduce virhostuptime

2019-08-21 Thread Pino Toscano
On Wednesday, 14 August 2019 16:33:22 CEST Michal Privoznik wrote: > +int > +virHostGetBootTime(unsigned long long *when ATTRIBUTE_UNUSED) > +{ > +#if defined(__linux__) || defined(__FreeBSD__) > +struct utmpx id = {.ut_type = BOOT_TIME}; > +struct utmpx *res = NULL; > + > +if (!(res =

Re: [libvirt] [PATCH 14/14] util: storagefile: Flag backing store strings with authentication

2019-08-21 Thread Ján Tomko
On Fri, Aug 16, 2019 at 12:39:35PM +0200, Peter Krempa wrote: Using inline authentication for storage volumes will not work properly as libvirt requires use of the secret driver for the auth data and thus would not be able to represent the passwords stored in the backing store string. Make sure

Re: [libvirt] [PATCH 13/14] util: storagefile: Don't traverse storage sources unusable by VM

2019-08-21 Thread Ján Tomko
On Fri, Aug 16, 2019 at 12:39:34PM +0200, Peter Krempa wrote: virStorageFileGetMetadataRecurse would include in the backing chain files which would not really be usable by libvirt directly e.g. when include files in the backing chain such file would be promoted to the top layer by a active

Re: [libvirt] [PATCH 12/14] util: storagefile: Clarify docs for '@report_broken' of virStorageFileGetMetadata

2019-08-21 Thread Ján Tomko
On Fri, Aug 16, 2019 at 12:39:33PM +0200, Peter Krempa wrote: virStorageFileGetMetadata does not report error if we can't interrogate the file somehow. Clarify this in the description of the @report_broken flag as it implies we should report an error in that case. The problem is that we don't

Re: [libvirt] [PATCH 10/14] tests: virstorage: Allow testing return value of virStorageSourceNewFromBackingAbsolute

2019-08-21 Thread Ján Tomko
On Fri, Aug 16, 2019 at 12:39:31PM +0200, Peter Krempa wrote: Modiy testBackingParse to allow testing other return values of the Modify backing store string parser. Signed-off-by: Peter Krempa --- tests/virstoragetest.c | 24 ++-- 1 file changed, 18 insertions(+), 6

Re: [libvirt] [PATCH 11/14] util: storagefile: Add handling of unusable storage sources

2019-08-21 Thread Ján Tomko
On Fri, Aug 16, 2019 at 12:39:32PM +0200, Peter Krempa wrote: Introduce new semantics to virStorageSourceNewFromBacking and some of the helpers used by it which propagate the return value from the callers. The new return value introduced by this patch allows to notify the calller that the

Re: [libvirt] [PATCH 4/5] util: Introduce virhostuptime

2019-08-21 Thread Jiri Denemark
On Wed, Aug 14, 2019 at 16:33:22 +0200, Michal Privoznik wrote: > This module contains function to get host boot time. > > Signed-off-by: Michal Privoznik > --- > src/libvirt_private.syms | 4 +++ > src/util/Makefile.inc.am | 2 ++ > src/util/virhostuptime.c | 61

Re: [libvirt] [PATCH 09/14] util: storagefile: Modify arguments of virStorageSourceNewFromBackingAbsolue

2019-08-21 Thread Ján Tomko
On Fri, Aug 16, 2019 at 12:39:30PM +0200, Peter Krempa wrote: Return the parsed storage source via an pointer in arguments and return an integer from the function. Describe the semantics with a comment for the function and adjust callers to the new semantics. Signed-off-by: Peter Krempa ---

Re: [libvirt] [PATCH 08/14] util: storagefile: Preserve return value in virStorageSourceParseBackingJSONUriStr

2019-08-21 Thread Ján Tomko
On Fri, Aug 16, 2019 at 12:39:29PM +0200, Peter Krempa wrote: virStorageSourceParseBackingURI will report special return values in some cases. Preserve it in virStorageSourceParseBackingJSONUriStr. Signed-off-by: Peter Krempa --- src/util/virstoragefile.c | 6 -- 1 file changed, 4

Re: [libvirt] [PATCH 07/14] util: storage: Modify return value of virStorageSourceNewFromBacking

2019-08-21 Thread Ján Tomko
On Fri, Aug 16, 2019 at 12:39:28PM +0200, Peter Krempa wrote: Return the storage source definition via a pointer in the arguments and document the returned values. This will simplify the possibility to ignore certain backing store types which are not representable by libvirt. Signed-off-by:

Re: [libvirt] [PATCH 3/5] security_util: Document virSecurityMoveRememberedLabel

2019-08-21 Thread Jiri Denemark
On Wed, Aug 14, 2019 at 16:33:21 +0200, Michal Privoznik wrote: > Signed-off-by: Michal Privoznik > --- > src/security/security_util.c | 13 + > 1 file changed, 13 insertions(+) Reviewed-by: Jiri Denemark -- libvir-list mailing list libvir-list@redhat.com

Re: [libvirt] [PATCH 2/5] security_util: Use more VIR_AUTOFREE()

2019-08-21 Thread Jiri Denemark
On Wed, Aug 14, 2019 at 16:33:20 +0200, Michal Privoznik wrote: > Signed-off-by: Michal Privoznik > --- > src/security/security_util.c | 78 +++- > 1 file changed, 32 insertions(+), 46 deletions(-) Reviewed-by: Jiri Denemark -- libvir-list mailing list

Re: [libvirt] [PATCH 1/5] virUUIDFormat: s/VIR_UUID_RAW_LEN/VIR_UUID_BUFLEN/ in comment

2019-08-21 Thread Jiri Denemark
On Wed, Aug 14, 2019 at 16:33:19 +0200, Michal Privoznik wrote: > The function takes raw UUID and formats it into string > representation. However, the comment mistakenly states that the > expected size of raw UUID buffer is VIR_UUID_RAW_LEN bytes. We > don't have such constant since v0.3.2~24. It

Re: [libvirt] [PATCH 06/14] tests: storage: Refactor cleanup in testBackingParse

2019-08-21 Thread Ján Tomko
On Fri, Aug 16, 2019 at 12:39:27PM +0200, Peter Krempa wrote: Automatically clean the temporary buffer and get rid of the cleanup label. Signed-off-by: Peter Krempa --- tests/virstoragetest.c | 22 -- 1 file changed, 8 insertions(+), 14 deletions(-) Reviewed-by: Ján Tomko

Re: [libvirt] [PATCH 05/14] tests: viruri: Add test for password in URI userinfo

2019-08-21 Thread Ján Tomko
On Fri, Aug 16, 2019 at 12:39:26PM +0200, Peter Krempa wrote: While it's a bad idea to use userinfo to pass credentials via an URI add s/an URI/a URI/ a test that we at least do the correct thing. Signed-off-by: Peter Krempa --- tests/viruritest.c | 1 + 1 file changed, 1 insertion(+)

Re: [libvirt] [PATCH 04/14] util: storagefile: Simplify cleanup in virStorageSourceParseBackingJSON

2019-08-21 Thread Ján Tomko
On Fri, Aug 16, 2019 at 12:39:25PM +0200, Peter Krempa wrote: Automatically free the 'root' temporary variable to get rid fo some s/fo/of/ complexity. Signed-off-by: Peter Krempa --- src/util/virstoragefile.c | 8 ++-- 1 file changed, 2 insertions(+), 6 deletions(-) Reviewed-by: Ján

Re: [libvirt] [PATCH 03/14] util: storagefile: Simplify cleanup handling in virStorageSourceParseBackingURI

2019-08-21 Thread Ján Tomko
On Fri, Aug 16, 2019 at 12:39:24PM +0200, Peter Krempa wrote: Automatically clean the 'uri' variable and get rid of the 'cleanup' label. Signed-off-by: Peter Krempa --- src/util/virstoragefile.c | 31 +-- 1 file changed, 13 insertions(+), 18 deletions(-) diff --git

Re: [libvirt] [PATCH 02/14] util: storagefile: Remove cleanup label from virStorageSourceParseBackingJSONiSCSI

2019-08-21 Thread Ján Tomko
On Fri, Aug 16, 2019 at 12:39:23PM +0200, Peter Krempa wrote: There is no cleanup code. Do not try and bend the cleanup code. Signed-off-by: Peter Krempa --- src/util/virstoragefile.c | 20 1 file changed, 8 insertions(+), 12 deletions(-) Reviewed-by: Ján Tomko

Re: [libvirt] [PATCH 01/14] util: storage: Simplify cleanup path handling in virStorageSourceParseBackingJSONInternal

2019-08-21 Thread Ján Tomko
On Fri, Aug 16, 2019 at 12:39:22PM +0200, Peter Krempa wrote: Automatically free the intermediate JSON data to get rid of the cleanup section. Signed-off-by: Peter Krempa --- src/util/virstoragefile.c | 20 +++- 1 file changed, 7 insertions(+), 13 deletions(-) Reviewed-by:

Re: [libvirt] QEMU bitmap backup usability FAQ

2019-08-21 Thread Vladimir Sementsov-Ogievskiy
[CC Nikolay] 21.08.2019 1:25, John Snow wrote: > Hi, downstream here at Red Hat I've been fielding some questions about > the usability and feature readiness of Bitmaps (and related features) in > QEMU. > > Here are some questions I answered internally that I am copying to the > list for two

[libvirt] [PATCH v3] tools: console: Relax stream EOF handling

2019-08-21 Thread Roman Bolshakov
Regular VM shutdown triggers the error for existing session of virsh console and it returns with non-zero exit code: error: internal error: console stream EOF The message and status code are misleading because there's no real error. virStreamRecv returns 0 correctly when EOF is reached.

Re: [libvirt] [PATCH] virt-aa-helper: Drop unnecessary AppArmor rule

2019-08-21 Thread Martin Kletzander
On Wed, Aug 21, 2019 at 09:45:01AM +0200, Andrea Bolognani wrote: Apparently /proc/self is automatically converted to /proc/@{pid} before checking rules, which makes spelling it out explicitly redundant. Because it is usually a symlink. Reviewed-by: Martin Kletzander Suggested-by: Jamie

Re: [libvirt] [PATCH 00/12] Drop KVM assignment

2019-08-21 Thread Daniel Henrique Barboza
On 8/21/19 9:53 AM, Ján Tomko wrote: On Tue, Aug 20, 2019 at 02:53:32PM -0300, Daniel Henrique Barboza wrote: On 8/20/19 11:30 AM, Michal Privoznik wrote: The KVM style of PCI assignment is not used, and it hasn't been for a while. Any attempt to start a domain with it would result in

Re: [libvirt] [PATCH v3 1/2] qemu: formatting XML from domain def choosing the root name

2019-08-21 Thread Jiri Denemark
On Wed, Aug 14, 2019 at 11:47:21 -0300, Maxiwell S. Garcia wrote: > The function virDomainDefFormatInternal() has the predefined root name > "domain" to format the XML. But to save both active and inactive domain > in the snapshot XML, the new root name "inactiveDomain" was created. > So, this

Re: [libvirt] [PATCH 00/12] Drop KVM assignment

2019-08-21 Thread Ján Tomko
On Tue, Aug 20, 2019 at 02:53:32PM -0300, Daniel Henrique Barboza wrote: On 8/20/19 11:30 AM, Michal Privoznik wrote: The KVM style of PCI assignment is not used, and it hasn't been for a while. Any attempt to start a domain with it would result in error as kernel dropped its support in

Re: [libvirt] [PATCH 12/12] news: Document KVM assignment removal

2019-08-21 Thread Ján Tomko
On Tue, Aug 20, 2019 at 04:30:32PM +0200, Michal Privoznik wrote: Signed-off-by: Michal Privoznik --- docs/news.xml | 13 + 1 file changed, 13 insertions(+) diff --git a/docs/news.xml b/docs/news.xml index d63fca3b48..6137ebd943 100644 --- a/docs/news.xml +++ b/docs/news.xml @@

Re: [libvirt] [PATCH 10/12] virpcimock: Don't create new_id or remove_id files

2019-08-21 Thread Ján Tomko
On Tue, Aug 20, 2019 at 04:30:30PM +0200, Michal Privoznik wrote: Now that PCI attach/detach happens solely via driver_override these two files are no longer needed. Signed-off-by: Michal Privoznik --- tests/virpcimock.c | 112 - 1 file changed, 112

Re: [libvirt] [PATCH 11/12] virpcimock: Drop @driverActions enum

2019-08-21 Thread Ján Tomko
On Tue, Aug 20, 2019 at 04:30:31PM +0200, Michal Privoznik wrote: This enum was introduced to model how RHEL-7 kernel behaves - for some reason going with the old way (via new_id + bind) fails but using driver_override succeeds. Well, we don't need to care about that anymore since we don't

Re: [libvirt] [PATCH 08/12] virpci: Drop newid style of PCI device detach

2019-08-21 Thread Ján Tomko
On Tue, Aug 20, 2019 at 04:30:28PM +0200, Michal Privoznik wrote: As stated in 84f9358b18346 all kernels that we are interested in have 'drivers_override'. Drop the other, older style of overriding PCI device driver - newid. Signed-off-by: Michal Privoznik --- src/util/virpci.c | 284

Re: [libvirt] [PATCH 09/12] virpcimock: Don't create "pci-stub" driver

2019-08-21 Thread Ján Tomko
On Tue, Aug 20, 2019 at 04:30:29PM +0200, Michal Privoznik wrote: Now that nothing supports "pci-stub" driver (aka KVM style of PCI device assignment) there is no need for virpcimock to create it. Signed-off-by: Michal Privoznik --- tests/virpcimock.c | 1 - 1 file changed, 1 deletion(-)

Re: [libvirt] [PATCH 07/12] virpci: Remove unused virPCIDeviceWaitForCleanup

2019-08-21 Thread Ján Tomko
On Tue, Aug 20, 2019 at 04:30:27PM +0200, Michal Privoznik wrote: This function is no longer used after previous commit. Signed-off-by: Michal Privoznik --- src/libvirt_private.syms | 1 - src/util/virpci.c| 108 --- src/util/virpci.h| 1 -

Re: [libvirt] [PATCH 06/12] virpci: Drop 'pci-stub' driver

2019-08-21 Thread Ján Tomko
On Tue, Aug 20, 2019 at 04:30:26PM +0200, Michal Privoznik wrote: Now that no one uses KVM style of PCI assignment we can safely remove 'pci-stub' backend. Signed-off-by: Michal Privoznik --- src/util/virpci.c | 11 --- src/util/virpci.h | 1 - 2 files changed, 12 deletions(-)

Re: [libvirt] [PATCH 05/12] virhostdev: Disable legacy kvm assignment

2019-08-21 Thread Ján Tomko
On Tue, Aug 20, 2019 at 04:30:25PM +0200, Michal Privoznik wrote: The KVM assignment is going to be removed shortly. Don't let the hostdev module configure it. Signed-off-by: Michal Privoznik --- src/util/virhostdev.c | 12 1 file changed, 8 insertions(+), 4 deletions(-)

Re: [libvirt] Fwd: libvirtd failing on MacOS in setgroups

2019-08-21 Thread Daniel P . Berrangé
On Tue, Aug 20, 2019 at 11:11:07AM -0400, Marcus Furlong wrote: > Resend to libvir-list in case that is more appropriate: > > > Hi, > > I get the following error when running libvirtd on MacOS as root: > > 2019-07-11 00:12:33.673+: 123145573953536: error : > qemuProcessQMPLaunch:8501 :

Re: [libvirt] [PATCH 04/12] qemu: Drop unused qemuOpenPCIConfig()

2019-08-21 Thread Ján Tomko
On Tue, Aug 20, 2019 at 04:30:24PM +0200, Michal Privoznik wrote: After previous commits, the function is not used anymore. Remove it. Signed-off-by: Michal Privoznik --- src/qemu/qemu_command.c | 22 -- src/qemu/qemu_command.h | 2 -- 2 files changed, 24 deletions(-)

Re: [libvirt] [PATCH 03/12] virhostdev: Unify virDomainHostdevDef to virPCIDevice translation

2019-08-21 Thread Ján Tomko
On Tue, Aug 20, 2019 at 04:30:23PM +0200, Michal Privoznik wrote: There are two places where we need to create virPCIDevice from given virDomainHostdevDef. In both places the code is duplicated. Move them into a single function and call it from those two places. Signed-off-by: Michal Privoznik

Re: [libvirt] [PATCH 02/12] tests: Remove 'kvm' PCI backend from domaincapstest

2019-08-21 Thread Ján Tomko
On Tue, Aug 20, 2019 at 04:30:22PM +0200, Michal Privoznik wrote: The KVM assignment was removed in qemu driver in previous commit. Remove it from domaincapstest too which is hard coding it. Signed-off-by: Michal Privoznik --- tests/domaincapsschemadata/qemu_1.7.0.x86_64.xml| 1 -

Re: [libvirt] [PATCH 01/12] qemu: Drop KVM assignment

2019-08-21 Thread Ján Tomko
On Tue, Aug 20, 2019 at 04:30:21PM +0200, Michal Privoznik wrote: KVM style of PCI devices assignment was dropped in kernel in favor of vfio pci (see kernel commit v4.12-rc1~68^2~65). Since vfio is around for quite some time now and is far superior discourage people in using KVM style. Ideally,

[libvirt] Fwd: libvirtd failing on MacOS in setgroups

2019-08-21 Thread Marcus Furlong
Resend to libvir-list in case that is more appropriate: Hi, I get the following error when running libvirtd on MacOS as root: 2019-07-11 00:12:33.673+: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-x86_64 for

  1   2   >