Re: [libvirt] [PATCH] secret: Add check/validation for correct usage when LookupByUUID

2018-12-12 Thread John Ferlan
On 12/12/18 9:29 AM, Michal Privoznik wrote: > On 12/4/18 9:32 PM, John Ferlan wrote: >> If virSecretGetSecretString is using by secretLookupByUUID, >> then it's possible the found sec->usageType doesn't match the >> desired @secretUsageType. If this occurs for the encrypted >> volume creation

Re: [libvirt] [PATCH v2 2/4] qemu: caps: add QEMU_CAPS_QCOW2_L2_CACHE_SIZE

2018-12-12 Thread Nikolay Shirokovskiy
On 12.12.2018 17:28, John Ferlan wrote: > > > On 12/12/18 5:59 AM, Nikolay Shirokovskiy wrote: >> >> >> On 11.12.2018 19:41, John Ferlan wrote: >>> >>> >>> On 12/11/18 6:04 AM, Nikolay Shirokovskiy wrote: On 10.12.2018 19:58, John Ferlan wrote: > > > On 11/8/18 8:02

[libvirt] [PATCH RFC 06/51] qemu: blockjob: Remove header dependency on qemu_domain.h

2018-12-12 Thread Peter Krempa
The blockjob module uses 'qemuDomainAsyncJob' in it's public headers. As I plan adding a new structure containing job data which will need to be included in "qemu_domain.h" it's necessary to break the circular dependency. Convert 'qemuDomainAsyncJob' type to 'int' as it's an enum. Signed-off-by:

[libvirt] [PATCH RFC 02/51] qemu: blockjob: Emit VIR_DOMAIN_EVENT_ID_BLOCK_JOB only for local disks

2018-12-12 Thread Peter Krempa
The event reports the disk path to identify the disk which makes sense only for local disks. Additionally network backed disks like NBD don't need to have a path so the callback would return NULL. Report VIR_DOMAIN_EVENT_ID_BLOCK_JOB only for non-empty local disks. Signed-off-by: Peter Krempa

[libvirt] [PATCH RFC 23/51] qemu: blockjob: Convert qemuBlockJobSyncBeginDisk to work with the job

2018-12-12 Thread Peter Krempa
Modify qemuBlockJobSyncBeginDisk to operate on qemuBlockt sJobDataPtr and rename it accordingly. Signed-off-by: Peter Krempa --- src/qemu/qemu_blockjob.c | 11 +++ src/qemu/qemu_blockjob.h | 2 +- src/qemu/qemu_driver.c| 10 +- src/qemu/qemu_migration.c | 11 +++

[libvirt] [PATCH RFC 19/51] qemu: blockjob: Drop unnecessary calls to qemuBlockJobSyncEndDisk

2018-12-12 Thread Peter Krempa
If the job wasn't started, we don't need to end the synchronous job. Add a note and drop the unnecessary calls. Signed-off-by: Peter Krempa --- src/qemu/qemu_blockjob.c | 4 +++- src/qemu/qemu_migration.c | 8 ++-- 2 files changed, 5 insertions(+), 7 deletions(-) diff --git

[libvirt] [PATCH RFC 20/51] qemu: blockjob: Add reference to disk into struct qemuBlockJobData

2018-12-12 Thread Peter Krempa
Block jobs can also happen on objects which are not a disk at a given point (e.g. the frontend was not hotplugged yet) and thus will be eventually kept separately. Add a reference back to the disk for blockjobs which do correspond to a disk. Signed-off-by: Peter Krempa ---

[libvirt] [PATCH RFC 22/51] qemu: blockjob: Pass in job to qemuBlockJobEventProcessLegacy

2018-12-12 Thread Peter Krempa
Don't split out individual fields, just pass in the job. Signed-off-by: Peter Krempa --- src/qemu/qemu_blockjob.c | 26 +++--- 1 file changed, 11 insertions(+), 15 deletions(-) diff --git a/src/qemu/qemu_blockjob.c b/src/qemu/qemu_blockjob.c index 531de035af..c6b70c5388

[libvirt] [PATCH RFC 15/51] qemu: blockjob: Turn struct qemuBlockJobData into a virObject

2018-12-12 Thread Peter Krempa
Reference counting will simplify semantics of the lifecycle of the object. Signed-off-by: Peter Krempa --- src/qemu/qemu_blockjob.c | 33 - src/qemu/qemu_blockjob.h | 5 +++-- src/qemu/qemu_domain.c | 4 ++-- 3 files changed, 33 insertions(+), 9 deletions(-)

[libvirt] [PATCH RFC 27/51] qemu: blockjob: Consume new block job state in the processing function

2018-12-12 Thread Peter Krempa
The processing function modifies the job state so it should make sure that the variable holding the new state is cleared properly and not the caller. The caller should only deal with the job state and not the transition that happened. Signed-off-by: Peter Krempa --- src/qemu/qemu_blockjob.c |

[libvirt] [PATCH RFC 18/51] qemu: blockjob: Add functions for block job state control

2018-12-12 Thread Peter Krempa
Rather than direclty modifying fields in the qemuBlockJobDataPtr structure add a bunch of fields which allow to do the transitions. This will help later when adding more complexity to the job handing. APIs introduced in this patch are: qemuBlockJobDiskNew - prepare for starting a new blockjob

[libvirt] [PATCH RFC 01/51] qemu: blockjob: Extract emitting of libvirt events

2018-12-12 Thread Peter Krempa
Put the emitting of VIR_DOMAIN_EVENT_ID_BLOCK_JOB and VIR_DOMAIN_EVENT_ID_BLOCK_JOB_2 into a separate function. Signed-off-by: Peter Krempa --- src/qemu/qemu_blockjob.c | 37 ++--- 1 file changed, 26 insertions(+), 11 deletions(-) diff --git

[libvirt] [PATCH RFC 00/51] refactor block job handling and add support for new event/commands (blockdev-add saga)

2018-12-12 Thread Peter Krempa
This series refactors how libvirt stores block job data and handles block job events in general. This is done in preparation for -blockdev support where we'll need to add support for the 'blockdev-create' job which is not associated to any frontend 'disk' in some cases. This series introduces a

[libvirt] [PATCH RFC 03/51] qemu: processBlockJobEvent: Use qemuBlockJobUpdate to process block job events

2018-12-12 Thread Peter Krempa
Replace use of qemuBlockJobEventProcess with the general helper. A small tweak is required to pass in the 'type' and 'status' of the job via the appropriate private data variables. Signed-off-by: Peter Krempa --- src/qemu/qemu_driver.c | 14 -- 1 file changed, 12 insertions(+), 2

[libvirt] [PATCH RFC 04/51] qemu: blockjob: Unexport qemuBlockJobEventProcess

2018-12-12 Thread Peter Krempa
The function is now only called locally. Some code movement was necessary to avoid forward declarations. Signed-off-by: Peter Krempa --- src/qemu/qemu_blockjob.c | 80 src/qemu/qemu_blockjob.h | 6 --- 2 files changed, 40 insertions(+), 46 deletions(-)

[libvirt] [PATCH RFC 05/51] qemu: blockjob: Rename public APIs

2018-12-12 Thread Peter Krempa
All the public APIs of the qemu_blockjob module operate on a 'disk'. Since I'll be adding APIs which operate on a job later let's rename the existing ones. Signed-off-by: Peter Krempa --- src/qemu/qemu_blockjob.c | 28 ++-- src/qemu/qemu_blockjob.h | 16

Re: [libvirt] [PATCH 1/2] qemu: don't log error for missing optional sources on stats

2018-12-12 Thread Nikolay Shirokovskiy
On 12.12.2018 16:28, John Ferlan wrote: > > > On 12/12/18 3:04 AM, Nikolay Shirokovskiy wrote: >> >> >> On 11.12.2018 17:33, John Ferlan wrote: >>> >>> >>> On 12/11/18 2:34 AM, Nikolay Shirokovskiy wrote: On 11.12.2018 01:05, John Ferlan wrote: > > $SUBJ: > >

Re: [libvirt] [PATCH 1/2] qemu: don't log error for missing optional sources on stats

2018-12-12 Thread John Ferlan
On 12/12/18 9:32 AM, Nikolay Shirokovskiy wrote: > > > On 12.12.2018 16:28, John Ferlan wrote: >> >> >> On 12/12/18 3:04 AM, Nikolay Shirokovskiy wrote: >>> >>> >>> On 11.12.2018 17:33, John Ferlan wrote: On 12/11/18 2:34 AM, Nikolay Shirokovskiy wrote: > > > On

Re: [libvirt] [PATCH 4/4] conf: domain: gfx: Iterate over graphics devices when doing validation

2018-12-12 Thread Erik Skultety
On Tue, Dec 11, 2018 at 06:54:31PM -0500, John Ferlan wrote: > > > On 12/7/18 9:47 AM, Erik Skultety wrote: > > The validation code for graphics has been in place for a while, but > > because it is only executed from the device iterator, that validation > > code was never truly run. The

[libvirt] [PATCH RFC 14/51] qemu: migration: Simplify cancellation of migration blockjobs

2018-12-12 Thread Peter Krempa
When cancelling job after a reconnect we can now use the disk block job state rather than having to re-detect it in the migration code. Signed-off-by: Peter Krempa --- src/qemu/qemu_migration.c | 63 +++ 1 file changed, 18 insertions(+), 45 deletions(-) diff

[libvirt] [PATCH RFC 35/51] qemu: blockjob: Add string convertors for blockjob type and state enums

2018-12-12 Thread Peter Krempa
Later on we'll format these values into the status XML so the from/to string functions will come handy. The implementation also notes that these will be used in the status XML to avoid somebody changing the values. Signed-off-by: Peter Krempa --- src/qemu/qemu_blockjob.c | 7 +++

[libvirt] [PATCH RFC 12/51] qemu: process: refresh block jobs on reconnect

2018-12-12 Thread Peter Krempa
Block job state was widely untracked by libvirt across restarts which was allowed by a stateless block job finishing handler which discarded disk state and redetected it. This is undesirable since we'll need to track more information for individual blockjobs due to -blockdev integration

[libvirt] [PATCH RFC 38/51] qemu: monitor: Add new fields for 'block-stream' command

2018-12-12 Thread Peter Krempa
Allow using the node name to specify the base of the 'stream' operation, allow specifying explicit job name and add support for delayed dismiss of the job so that we can reap the state even if libvirtd was not running when qemu emitted the job completion event. Signed-off-by: Peter Krempa ---

[libvirt] [PATCH RFC 36/51] qemu: domain: Store blockjob data in the status XML

2018-12-12 Thread Peter Krempa
We need to store the block job state in the status XML so that we can properly recover any data when reconnecting after startup and also in the end to be able to do any transition of the backing chain that happened while libvirt was not connected to the monitor. First step is to note the name,

[libvirt] [PATCH RFC 37/51] tests: qemustatusxml2xml: Add test case for block job tracking

2018-12-12 Thread Peter Krempa
Signed-off-by: Peter Krempa --- .../blockjob-blockdev-in.xml | 364 ++ .../blockjob-blockdev-out.xml | 1 + tests/qemuxml2xmltest.c | 2 + 3 files changed, 367 insertions(+) create mode 100644

[libvirt] [PATCH RFC 11/51] qemu: migration: Properly note that non-shared-storage migration uses a blockjob

2018-12-12 Thread Peter Krempa
Internally we do a 'block-copy' to accomodate non-shared storage migration but the code did not fill in that the block job was active on the disk when starting the copy job. Since we handle block jobs finishes regardless of having it registered it's not a problem but soon will become one.

[libvirt] [PATCH RFC 39/51] qemu: monitor: Add new fields for 'block-commit' command

2018-12-12 Thread Peter Krempa
Allow using the node name to specify the base and top of the 'commit' operation, allow specifying explicit job name and add support for delayed dismiss of the job so that we can reap the state even if libvirtd was not running when qemu emitted the job completion event. Signed-off-by: Peter Krempa

[libvirt] [PATCH RFC 28/51] qemu: blockjob: Pass job into qemuBlockJobUpdateDisk and rename it

2018-12-12 Thread Peter Krempa
Instead of passing in the disk information, pass in the job and name the function accordingly. Few callers needed to be modified to have the job pointer handy. Signed-off-by: Peter Krempa --- src/qemu/qemu_blockjob.c | 20 src/qemu/qemu_blockjob.h | 6 +++---

[libvirt] [PATCH RFC 16/51] qemu: blockjob: Clarify that job 'status' field contains new state

2018-12-12 Thread Peter Krempa
The field is used to note the state the job has transitioned to while handling the blockjob state change event. Rename the field so that it's obvious that this is the new state and not the general state of the blockjob. Signed-off-by: Peter Krempa --- src/qemu/qemu_blockjob.c | 12 ++--

[libvirt] [PATCH RFC 10/51] qemu: blockjob: Split out handling of comlpleted jobs

2018-12-12 Thread Peter Krempa
qemuBlockJobEventProcessLegacy was getting too big. Remove handling of completed jobs in a separate function. Signed-off-by: Peter Krempa --- src/qemu/qemu_blockjob.c | 118 +-- 1 file changed, 65 insertions(+), 53 deletions(-) diff --git

[libvirt] [PATCH RFC 09/51] qemu: blockjob: Rename qemuBlockJobEventProcess to qemuBlockJobEventProcessLegacy

2018-12-12 Thread Peter Krempa
This will handle blockjob finalizing for the old approach so rename it accordingly. Signed-off-by: Peter Krempa --- src/qemu/qemu_blockjob.c | 18 +- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/src/qemu/qemu_blockjob.c b/src/qemu/qemu_blockjob.c index

[libvirt] [PATCH RFC 25/51] qemu: migration: Extract reporting of disk migration error

2018-12-12 Thread Peter Krempa
The same message is reported in 3 distinct places. Move it out into a single function. Signed-off-by: Peter Krempa --- src/qemu/qemu_migration.c | 43 --- 1 file changed, 18 insertions(+), 25 deletions(-) diff --git a/src/qemu/qemu_migration.c

[libvirt] [PATCH RFC 29/51] qemu: Allocate diskPriv->blockjob only when there's a blockjob

2018-12-12 Thread Peter Krempa
Rather than storing the presence of the blockjob in a flag we can bind together the lifecycle of the job with the lifecycle of the object which is tracking the data for it. Signed-off-by: Peter Krempa --- src/qemu/qemu_blockjob.c | 76 +---

[libvirt] [PATCH RFC 13/51] qemu: driver: Remove block job status reprobing from qemuDomainBlockPivot

2018-12-12 Thread Peter Krempa
Now that we reprobe the status of blockjobs when reconnecting in addition to handling job status events, the status reprobing can be removed as we always track the correct status internally. Signed-off-by: Peter Krempa --- src/qemu/qemu_driver.c | 20 +--- 1 file changed, 1

[libvirt] [PATCH RFC 32/51] qemu: blockjob: Add job name into the data

2018-12-12 Thread Peter Krempa
Currently the job name corresponds to the disk the job belongs to. For jobs which will not correspond to disks we'll need to track the name separately. Signed-off-by: Peter Krempa --- src/qemu/qemu_blockjob.c | 20 src/qemu/qemu_blockjob.h | 7 +--

[libvirt] [PATCH RFC 33/51] qemu: domain: Add global table of blockjobs

2018-12-12 Thread Peter Krempa
Block jobs currently belong to disks only so we can look up the block job data for them in the corresponding disks. This won't be the case when using blockdev as certain jobs don't even correspond to a disk and most of them can run on a part of the backing chain. Add a global table of blockjobs

[libvirt] [PATCH RFC 34/51] qemu: blockjob: Register new and running blockjobs in the global table

2018-12-12 Thread Peter Krempa
Add the job structure to the table when instantiating a new job and remove it when it terminates/fails. Signed-off-by: Peter Krempa --- src/qemu/qemu_blockjob.c | 31 --- src/qemu/qemu_blockjob.h | 6 -- src/qemu/qemu_driver.c| 16

[libvirt] [PATCH RFC 08/51] qemu: process: Consolidate error paths in qemuProcessHandleBlockJob

2018-12-12 Thread Peter Krempa
'cleanup' label was accessed only from a jump to 'error'. Consolidate everyting into 'cleanup'. Signed-off-by: Peter Krempa --- src/qemu/qemu_process.c | 14 +++--- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index

[libvirt] [PATCH RFC 07/51] qemu: Consolidate disk blockjob variables into a structure

2018-12-12 Thread Peter Krempa
Struct qemuDomainDiskPrivate was holding multiple variables connected to a disk block job. Consolidate them into a new struct qemuBlockJobData. This will also allow simpler extensions to the block job mechanisms. Signed-off-by: Peter Krempa --- src/qemu/qemu_blockjob.c | 36

[libvirt] [PATCH RFC 42/51] qemu: monitor: Add support for 'job-cancel' command

2018-12-12 Thread Peter Krempa
This belongs to the new job management API which can manage also non-block based jobs. Signed-off-by: Peter Krempa --- src/qemu/qemu_monitor.c | 12 src/qemu/qemu_monitor.h | 4 src/qemu/qemu_monitor_json.c | 27 +++

[libvirt] [PATCH RFC 43/51] qemu: monitor: Add support for 'job-complete' command

2018-12-12 Thread Peter Krempa
This belongs to the new job management API which can manage also non-block based jobs. Signed-off-by: Peter Krempa --- src/qemu/qemu_monitor.c | 12 src/qemu/qemu_monitor.h | 4 src/qemu/qemu_monitor_json.c | 27 +++

[libvirt] [PATCH RFC 40/51] qemu: monitor: Add new fields for 'blockdev-mirror' command

2018-12-12 Thread Peter Krempa
Allow using the delayed dismiss of the job so that we can reap the state even if libvirtd was not running when qemu emitted the job completion event. Signed-off-by: Peter Krempa --- src/qemu/qemu_migration.c| 2 +- src/qemu/qemu_monitor.c | 9 + src/qemu/qemu_monitor.h |

[libvirt] [PATCH RFC 41/51] qemu: monitor: Add support for 'job-dismiss' command

2018-12-12 Thread Peter Krempa
This belongs to the new job management API which can manage also non-block based jobs. The dismiss command is meant to remove a concluded job after we were able to get the final status. Signed-off-by: Peter Krempa --- src/qemu/qemu_monitor.c | 12 src/qemu/qemu_monitor.h

[libvirt] [PATCH RFC 45/51] qemu: blockjob: Add 'concluded' state for a block job

2018-12-12 Thread Peter Krempa
This new state is entered when qemu finished the job but libvirt does not know whether it was successful or not. Signed-off-by: Peter Krempa --- src/qemu/qemu_blockjob.c | 2 +- src/qemu/qemu_blockjob.h | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-) diff --git

[libvirt] [PATCH RFC 44/51] qemu: monitor: Add infrastructure for 'query-jobs'

2018-12-12 Thread Peter Krempa
Signed-off-by: Peter Krempa --- src/qemu/qemu_monitor.c | 23 + src/qemu/qemu_monitor.h | 49 ++ src/qemu/qemu_monitor_json.c | 86 ++ src/qemu/qemu_monitor_json.h | 6 ++

[libvirt] [PATCH RFC 17/51] qemu: migration: Separate startup of disk mirror from migration logic

2018-12-12 Thread Peter Krempa
Extract the disk mirroring startup code from the loop into a separate function to allow cleaner cleanup paths. Signed-off-by: Peter Krempa --- src/qemu/qemu_migration.c | 85 +-- 1 file changed, 55 insertions(+), 30 deletions(-) diff --git

[libvirt] [PATCH RFC 21/51] qemu: blockjob: Record job type when starting the job

2018-12-12 Thread Peter Krempa
We can properly track the job type when starting the job so that we don't have to infer it later. This patch also adds an enum of block job types specific to qemu (qemuBlockjobType) which mirrors the public block job types (virDomainBlockJobType) but allows for other types to be added later which

[libvirt] [PATCH RFC 26/51] qemu: blockjob: Remove error propagation from qemuBlockJobUpdateDisk

2018-12-12 Thread Peter Krempa
The job error can be safely accessed in the job structure, so we don't need to propagate it through qemuBlockJobUpdateDisk. Drop the propagation and refactor any caller that pased non-NULL error. Signed-off-by: Peter Krempa --- src/qemu/qemu_blockjob.c | 12 ++--

[libvirt] [PATCH RFC 24/51] qemu: blockjob: Track current state of blockjob

2018-12-12 Thread Peter Krempa
Add a field tracking the current state of job so that it can be queried later. Until now the job state e.g. that the job is _READY for finalizing was tracked only for mirror jobs. Add tracking of state for all jobs. Similarly to 'qemuBlockJobType' this maps the existing states of the blockjob

[libvirt] [PATCH RFC 30/51] qemu: migration: Don't call qemuBlockJobSyncEndDisk when block job has terminated

2018-12-12 Thread Peter Krempa
Now that the data is per-job, we don't really need to bother with finishing the synchronous job handling if the job is already terminated. Signed-off-by: Peter Krempa --- src/qemu/qemu_migration.c | 1 - 1 file changed, 1 deletion(-) diff --git a/src/qemu/qemu_migration.c

[libvirt] [PATCH RFC 31/51] qemu: blockjob: Convert qemuBlockJobSyncEndDisk to take job instead of disk

2018-12-12 Thread Peter Krempa
And rename it in accordance with the change. Signed-off-by: Peter Krempa --- src/qemu/qemu_blockjob.c | 20 ++-- src/qemu/qemu_blockjob.h | 6 +++--- src/qemu/qemu_driver.c| 4 ++-- src/qemu/qemu_migration.c | 2 +- 4 files changed, 16 insertions(+), 16 deletions(-)

Re: [libvirt] [PATCH 0/8] Add tests for Storage Pool startup command line generation

2018-12-12 Thread John Ferlan
On 12/12/18 9:04 AM, Michal Privoznik wrote: > On 12/4/18 5:47 PM, John Ferlan wrote: >> Similar to qemuxml2argv and storagevolxml2argv, add storagepoolxml2argvtest >> in order to check the command line creation for the pool 'Start' commands. >> >> Only applicable for pool types with a

[libvirt] [PATCH RFC 49/51] qemu: Add handler for job state change event

2018-12-12 Thread Peter Krempa
Add support for handling the event either synchronously or asynchronously using the event thread. Signed-off-by: Peter Krempa --- src/qemu/qemu_domain.c | 3 ++ src/qemu/qemu_domain.h | 1 + src/qemu/qemu_driver.c | 23 ++ src/qemu/qemu_process.c | 66

[libvirt] [PATCH RFC 47/51] qemu: process: Don't trigger BLOCK_JOB* events with -blockdev

2018-12-12 Thread Peter Krempa
With blockdev we'll need to use the JOB_STATUS_CHANGE so gate the old events by the blockdev capability. Signed-off-by: Peter Krempa --- src/qemu/qemu_process.c | 7 +++ 1 file changed, 7 insertions(+) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index

[libvirt] [PATCH RFC 46/51] qemu: monitor: Implement support for 'JOB_STATUS_CHANGE' event

2018-12-12 Thread Peter Krempa
This new event is a superset of the BLOCK_JOB* events and also covers jobs which don't bind to a VM disk. In this patch the monitor part is implemented. Signed-off-by: Peter Krempa --- src/qemu/qemu_monitor.c | 13 + src/qemu/qemu_monitor.h | 9 +

[libvirt] [PATCH RFC 48/51] qemu: blockjob: Add helper to convert monitor job status to internal state

2018-12-12 Thread Peter Krempa
Signed-off-by: Peter Krempa --- src/qemu/qemu_blockjob.c | 41 src/qemu/qemu_blockjob.h | 3 +++ 2 files changed, 44 insertions(+) diff --git a/src/qemu/qemu_blockjob.c b/src/qemu/qemu_blockjob.c index ee545fc8de..020e7b3994 100644 ---

[libvirt] [PATCH RFC 51/51] qemu: process: Refresh -blockdev based blockjobs on reconnect to qemu

2018-12-12 Thread Peter Krempa
Refresh the state of the jobs and process any events that might have happened while libvirt was not running. The job state processing requires some care to figure out if a job needs to be bumped.t Signed-off-by: Peter Krempa --- src/qemu/qemu_blockjob.c | 57

[libvirt] [PATCH RFC 50/51] qemu: blockjob: Add modern block job event handler

2018-12-12 Thread Peter Krempa
Add the infrastructure to handle block job events in the -blockdev era. Some complexity is required as qemu does not bother to notify whether the job was concluded successfully or failed. Thus it's necessary to re-query the monitor. To minimize the possibility of stuck jobs save the state into

Re: [libvirt] [PATCH v2 2/4] qemu: caps: add QEMU_CAPS_QCOW2_L2_CACHE_SIZE

2018-12-12 Thread John Ferlan
On 12/12/18 5:59 AM, Nikolay Shirokovskiy wrote: > > > On 11.12.2018 19:41, John Ferlan wrote: >> >> >> On 12/11/18 6:04 AM, Nikolay Shirokovskiy wrote: >>> >>> >>> On 10.12.2018 19:58, John Ferlan wrote: On 11/8/18 8:02 AM, Nikolay Shirokovskiy wrote: > For qemu capable of

Re: [libvirt] [PATCH] secret: Add check/validation for correct usage when LookupByUUID

2018-12-12 Thread Michal Privoznik
On 12/4/18 9:32 PM, John Ferlan wrote: > If virSecretGetSecretString is using by secretLookupByUUID, > then it's possible the found sec->usageType doesn't match the > desired @secretUsageType. If this occurs for the encrypted > volume creation processing and a subsequent pool refresh is >

Re: [libvirt] [PATCH v2 0/4] util: Fixing libvirt errors on cavium/thunder-nicvf

2018-12-12 Thread dann frazier
On Sat, Nov 17, 2018 at 1:17 PM Radoslaw Biernacki wrote: > > ThunderX is Cavium SoC. This platform contain SRIOV NIC. > Unlike other commonly known network devices it does not have VF functionality > duplicated in its PF. PF is purely management device (on HW level). > > This creates several

Re: [libvirt] [PATCHv2 01/16] qemu: Add KVM CPUs into cache only if KVM is present

2018-12-12 Thread Roman Bolshakov
On Fri, Nov 23, 2018 at 09:46:36PM +0300, Roman Bolshakov wrote: > On Fri, Nov 23, 2018 at 06:16:46PM +0100, Jiri Denemark wrote: > > On Fri, Nov 23, 2018 at 18:55:00 +0300, Roman Bolshakov wrote: > > > On Fri, Nov 23, 2018 at 04:30:13PM +0100, Jiri Denemark wrote: > > > > On Fri, Nov 23, 2018 at

Re: [libvirt] [PATCH 4/8] tests: Introduce tests for storage pool xml to argv checks

2018-12-12 Thread John Ferlan
On 12/12/18 9:04 AM, Michal Privoznik wrote: > On 12/4/18 5:47 PM, John Ferlan wrote: >> Similar to qemuxml2argv and storagevolxml2argv, let's create some >> tests to ensure that the XML generates a consistent command line. >> >> Using the same list of pools as storagepoolxml2xmltest, start

Re: [libvirt] [PATCH v2 0/2] qemu: Don't use -mem-prealloc among with .prealloc=yes

2018-12-12 Thread John Ferlan
On 12/12/18 7:58 AM, Michal Privoznik wrote: > v2 of: > > https://www.redhat.com/archives/libvir-list/2018-November/msg00159.html > > diff to v1: > - Patch 01/02 is completely new > - Patch 02/02 has reworded commit message > > Michal Prívozník (2): > qemuBuildMemoryBackendProps: Pass @priv

[libvirt] [PATCH] qemu-options: Remove deprecated "-virtioconsole" option

2018-12-12 Thread Thomas Huth
It's been deprecated since QEMU 3.0, and nobody complained so far, so it is time to remove this option now. Signed-off-by: Thomas Huth --- docs/qdev-device-use.txt | 4 include/hw/boards.h | 1 - qemu-deprecated.texi | 5 qemu-options.hx | 10 vl.c

[libvirt] [PATCH 1/2] vircgroup: introduce virCgroupKillRecursiveCB

2018-12-12 Thread Pavel Hrdina
The rewrite to support cgroup v2 missed this function. In cgroup v2 we have different files to track tasks. We would fail to remove cgroup on non-systemd OSes if there is any extra process assigned to guest cgroup because we would not kill any process form the guest cgroup. Signed-off-by: Pavel

[libvirt] [PATCH 0/2] random cgroup v2 fixes

2018-12-12 Thread Pavel Hrdina
While working on cgroup v2 devices and doing some extended testing I figured out that there are two issues with the current code. Pavel Hrdina (2): vircgroup: introduce virCgroupKillRecursiveCB vircgroupv2: fix virCgroupV2ValidateMachineGroup src/util/vircgroup.c| 69

[libvirt] [PATCH 2/2] vircgroupv2: fix virCgroupV2ValidateMachineGroup

2018-12-12 Thread Pavel Hrdina
When libvirt is reconnecting to running domain that uses cgroup v2 the QEMU process reports cgroup for the emulator directory because the main thread is in that cgroup. We need to remove the "/emulator" part in order to match with the root cgroup directory name for that domain. Signed-off-by:

Re: [libvirt] [PATCH v2 2/4] qemu: caps: add QEMU_CAPS_QCOW2_L2_CACHE_SIZE

2018-12-12 Thread Nikolay Shirokovskiy
On 11.12.2018 19:41, John Ferlan wrote: > > > On 12/11/18 6:04 AM, Nikolay Shirokovskiy wrote: >> >> >> On 10.12.2018 19:58, John Ferlan wrote: >>> >>> >>> On 11/8/18 8:02 AM, Nikolay Shirokovskiy wrote: For qemu capable of setting l2-cache-size for qcow2 images to INT64_MAX and

Re: [libvirt] [PATCH 3/4] tests: Introduce negative versions of DO_TEST_CAPS_LATEST

2018-12-12 Thread Erik Skultety
On Tue, Dec 11, 2018 at 06:51:56PM -0500, John Ferlan wrote: > > > On 12/7/18 9:47 AM, Erik Skultety wrote: > > As commit d8266ebe161 demonstrated, it's so easy to forget to add a > > single capability which in turn can easily fool the test suite so that > > tests expecting a failure can fail with

Re: [libvirt] [PATCH 1/4] qemu: domain: gfx: Fix shadowing the ptr argument to graphics validation

2018-12-12 Thread Erik Skultety
On Tue, Dec 11, 2018 at 06:48:26PM -0500, John Ferlan wrote: > > > On 12/7/18 9:47 AM, Erik Skultety wrote: > > Since the code was never run, this stupid mistake could have only been > > spotted by an accident. > > > > Signed-off-by: Erik Skultety > > --- > > src/qemu/qemu_domain.c | 4 +--- > >

Re: [libvirt] [PATCH 1/2] qemu: don't log error for missing optional sources on stats

2018-12-12 Thread Nikolay Shirokovskiy
On 11.12.2018 17:33, John Ferlan wrote: > > > On 12/11/18 2:34 AM, Nikolay Shirokovskiy wrote: >> >> >> On 11.12.2018 01:05, John Ferlan wrote: >>> >>> $SUBJ: >>> >>> 'storage sources' >>> >>> On 11/12/18 7:58 AM, Nikolay Shirokovskiy wrote: Every time we call all domain stats for

Re: [libvirt] [PATCH 2/4] conf: domain: Introduce virDomainDeviceInfoIterate flags

2018-12-12 Thread Erik Skultety
On Tue, Dec 11, 2018 at 06:50:19PM -0500, John Ferlan wrote: > > > On 12/7/18 9:47 AM, Erik Skultety wrote: > > One of the usages of the device iterator is to run config validation. > > That's a problem for graphics devices, because they don't have any @info > > data (graphics shouldn't have been

Re: [libvirt] [PATCH v2 1/4] xml: add disk driver metadata_cache_size option

2018-12-12 Thread Nikolay Shirokovskiy
On 11.12.2018 19:34, John Ferlan wrote: > > > On 12/11/18 5:22 AM, Nikolay Shirokovskiy wrote: >> >> >> On 10.12.2018 19:56, John Ferlan wrote: >>> >>> On 11/8/18 8:02 AM, Nikolay Shirokovskiy wrote: Signed-off-by: Nikolay Shirokovskiy --- docs/formatdomain.html.in

Re: [libvirt] [PATCH 4/4] conf: domain: gfx: Iterate over graphics devices when doing validation

2018-12-12 Thread Erik Skultety
On Tue, Dec 11, 2018 at 06:54:31PM -0500, John Ferlan wrote: > > > On 12/7/18 9:47 AM, Erik Skultety wrote: > > The validation code for graphics has been in place for a while, but > > because it is only executed from the device iterator, that validation > > code was never truly run. The

Re: [libvirt] [PATCH 2/4] conf: domain: Introduce virDomainDeviceInfoIterate flags

2018-12-12 Thread John Ferlan
On 12/12/18 3:29 AM, Erik Skultety wrote: > On Tue, Dec 11, 2018 at 06:50:19PM -0500, John Ferlan wrote: >> >> >> On 12/7/18 9:47 AM, Erik Skultety wrote: >>> One of the usages of the device iterator is to run config validation. >>> That's a problem for graphics devices, because they don't have

Re: [libvirt] [PATCH 8/8] storage: Add tests for logical backend startup

2018-12-12 Thread Michal Privoznik
On 12/4/18 5:47 PM, John Ferlan wrote: > Add the logical storage pool startup validation (xml2argv) tests. > > Signed-off-by: John Ferlan > --- > src/storage/storage_backend_logical.c | 6 +- > src/storage/storage_util.c | 11 +++ >

Re: [libvirt] [PATCH 4/8] tests: Introduce tests for storage pool xml to argv checks

2018-12-12 Thread Michal Privoznik
On 12/4/18 5:47 PM, John Ferlan wrote: > Similar to qemuxml2argv and storagevolxml2argv, let's create some > tests to ensure that the XML generates a consistent command line. > > Using the same list of pools as storagepoolxml2xmltest, start with > the file system tests (fs, netfs, netfs-cifs,

Re: [libvirt] [PATCH 0/8] Add tests for Storage Pool startup command line generation

2018-12-12 Thread Michal Privoznik
On 12/4/18 5:47 PM, John Ferlan wrote: > Similar to qemuxml2argv and storagevolxml2argv, add storagepoolxml2argvtest > in order to check the command line creation for the pool 'Start' commands. > > Only applicable for pool types with a "@startPool" function - that is disk, > fs, iscsi, logical,

[libvirt] [PATCH v3 03/18] security_dac: Restore label on failed chown() attempt

2018-12-12 Thread Michal Privoznik
It's important to keep XATTRs untouched (well, in the same state they were in when entering the function). Otherwise our refcounting would be messed up. Signed-off-by: Michal Privoznik Reviewed-by: Daniel P. Berrangé --- src/security/security_dac.c | 20 +++- 1 file changed, 19

[libvirt] [PATCH v3 00/18] Implement original label remembering

2018-12-12 Thread Michal Privoznik
v3 of: https://www.redhat.com/archives/libvir-list/2018-November/msg01070.html diff to v2: - dropped 01/18 from v2 - Introduced a test - Couple of minor adjustments as suggested in review of v2 Michal Prívozník (18): util: Introduce xattr getter/setter/remover security: Include

[libvirt] [PATCH v3 02/18] security: Include security_util

2018-12-12 Thread Michal Privoznik
This file implements wrappers over XATTR getter/setter. It ensures the proper XATTR namespace is used. Signed-off-by: Michal Privoznik --- src/security/Makefile.inc.am | 2 + src/security/security_util.c | 256 +++ src/security/security_util.h | 32 + 3

[libvirt] [RFC v3 4/4] nvdimm: update news.xml

2018-12-12 Thread Luyao Zhong
add more configure options for NVDIMM Signed-off-by: Luyao Zhong --- docs/news.xml | 9 + 1 file changed, 9 insertions(+) diff --git a/docs/news.xml b/docs/news.xml index 5bdbd34..0112c91 100644 --- a/docs/news.xml +++ b/docs/news.xml @@ -38,6 +38,15 @@ would normally

[libvirt] [RFC v3 2/4] nvdimm: add nvdimm-related qemucapabilities check

2018-12-12 Thread Luyao Zhong
-object memory-backend-file[,align=][,pmem=] Signed-off-by: Luyao Zhong --- src/qemu/qemu_capabilities.c | 8 +++- src/qemu/qemu_capabilities.h | 4 tests/qemucapabilitiesdata/caps_2.12.0.aarch64.xml | 1 +

[libvirt] [RFC v3 1/4] nvdimm: introduce more config elements into xml for NVDIMM memory

2018-12-12 Thread Luyao Zhong
1.alignsize The 'alignsize' option allows users to specify the proper alignment. 2.pmem The 'pmem' option allows users to specify whether the backend storage of memory-backend-file is a real persistent memory. 3.unarmed The 'unarmed' option allows users to mark vNVDIMM read-only. These options

[libvirt] [RFC v3 0/4] update NVDIMM support

2018-12-12 Thread Luyao Zhong
Hi libvirt experts, This is the RFC v3 for updating NVDIMM support in libvirt. There are some gaps between qemu and libvirt, libvirt has not supported several config options about NVDIMM memory while qemu is ready now, including 'align', 'pmem', 'unarmed'. I reworded and recoded my patches

[libvirt] [PATCH v2 0/2] qemu: Don't use -mem-prealloc among with .prealloc=yes

2018-12-12 Thread Michal Privoznik
v2 of: https://www.redhat.com/archives/libvir-list/2018-November/msg00159.html diff to v1: - Patch 01/02 is completely new - Patch 02/02 has reworded commit message Michal Prívozník (2): qemuBuildMemoryBackendProps: Pass @priv instead of its individual members qemu: Don't use

Re: [libvirt] [PATCH 0/2] Add mem cold/hot plug supported check to virDomainDefCompatibleDevice

2018-12-12 Thread Michal Privoznik
On 12/7/18 6:27 PM, John Ferlan wrote: > https://bugzilla.redhat.com/show_bug.cgi?id=1624336 > > Details are in patch2, but essentially the issue is the check > for whether cold/hot plug of memory is supported occurs during > qemuDomainDefValidateMemoryHotplug; however, that is called > after

Re: [libvirt] [PATCH v2 1/4] xml: add disk driver metadata_cache_size option

2018-12-12 Thread John Ferlan
On 12/12/18 3:39 AM, Nikolay Shirokovskiy wrote: > > > On 11.12.2018 19:34, John Ferlan wrote: >> >> >> On 12/11/18 5:22 AM, Nikolay Shirokovskiy wrote: >>> >>> >>> On 10.12.2018 19:56, John Ferlan wrote: On 11/8/18 8:02 AM, Nikolay Shirokovskiy wrote: > Signed-off-by: Nikolay

[libvirt] [PATCH v3 06/18] virSecurityDACRestoreAllLabel: Restore more labels

2018-12-12 Thread Michal Privoznik
We are setting label on kernel, initrd, dtb and slic_table files. But we never restored it. Signed-off-by: Michal Privoznik Reviewed-by: Daniel P. Berrangé --- src/security/security_dac.c | 16 1 file changed, 16 insertions(+) diff --git a/src/security/security_dac.c

[libvirt] [PATCH v3 14/18] virSecuritySELinuxRestoreAllLabel: Reorder device relabeling

2018-12-12 Thread Michal Privoznik
It helps whe trying to match calls with virSecuritySELinuxSetAllLabel if the order in which devices are set/restored is the same in both functions. Signed-off-by: Michal Privoznik Reviewed-by: Daniel P. Berrangé --- src/security/security_selinux.c | 14 +++--- 1 file changed, 7

[libvirt] [PATCH v3 05/18] virSecurityDACRestoreAllLabel: Reorder device relabeling

2018-12-12 Thread Michal Privoznik
It helps whe trying to match calls with virSecurityDACSetAllLabel if the order in which devices are set/restored is the same in both functions. Signed-off-by: Michal Privoznik Reviewed-by: Daniel P. Berrangé --- src/security/security_dac.c | 36 ++-- 1 file

[libvirt] [PATCH v3 10/18] security_selinux: Track if transaction is restore

2018-12-12 Thread Michal Privoznik
It is going to be important to know if the current transaction we are running is a restore operation or set label operation so that we know whether to call virSecurityGetRememberedLabel() or virSecuritySetRememberedLabel(). That is, whether we are in a restore and therefore have to fetch the

[libvirt] [PATCH v3 01/18] util: Introduce xattr getter/setter/remover

2018-12-12 Thread Michal Privoznik
Signed-off-by: Michal Privoznik --- src/libvirt_private.syms | 3 + src/util/virfile.c | 121 +++ src/util/virfile.h | 11 3 files changed, 135 insertions(+) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index

[libvirt] [PATCH v3 16/18] tests: Introduce qemusecuritytest

2018-12-12 Thread Michal Privoznik
This test checks if security label remembering works correctly. It uses qemuSecurity* APIs to do that. And some mocking (even though it's not real mocking as we are used to from other tests like virpcitest). So far, only DAC driver is tested. Signed-off-by: Michal Privoznik --- cfg.mk

[libvirt] [PATCH v3 04/18] virSecurityDACTransactionRun: Implement rollback

2018-12-12 Thread Michal Privoznik
When iterating over list of paths/disk sources to relabel it may happen that the process fails at some point. In that case, for the sake of keeping seclabel refcount (stored in XATTRs) in sync with reality we have to perform rollback. However, if that fails too the only thing we can do is warn

[libvirt] [PATCH v3 15/18] virSecuritySELinuxRestoreAllLabel: Restore more labels

2018-12-12 Thread Michal Privoznik
We are setting label on kernel, initrd, dtb and slic_table files. But we never restored it. Signed-off-by: Michal Privoznik Reviewed-by: Daniel P. Berrangé --- src/security/security_selinux.c | 16 1 file changed, 16 insertions(+) diff --git a/src/security/security_selinux.c

[libvirt] [PATCH v3 13/18] virSecuritySELinuxTransactionRun: Implement rollback

2018-12-12 Thread Michal Privoznik
When iterating over list of paths/disk sources to relabel it may happen that the process fails at some point. In that case, for the sake of keeping seclabel refcount (stored in XATTRs) in sync with reality we have to perform rollback. However, if that fails too the only thing we can do is warn

[libvirt] [PATCH v3 07/18] security_dac: Allow callers to enable/disable label remembering/recall

2018-12-12 Thread Michal Privoznik
Because the implementation that will be used for label remembering/recall is not atomic we have to give callers a chance to enable or disable it. That is, enable it if and only if metadata locking is enabled. Otherwise the feature MUST be turned off. Signed-off-by: Michal Privoznik Reviewed-by:

[libvirt] [PATCH v3 17/18] tools: Provide a script to recover fubar'ed XATTRs setup

2018-12-12 Thread Michal Privoznik
Our code is not bug free. The refcounting I introduced will almost certainly not work in some use cases. Provide a script that will remove all the XATTRs set by libvirt so that it can start cleanly. Signed-off-by: Michal Privoznik --- tools/Makefile.am | 1 +

  1   2   >