Re: [libvirt] [PATCH 2/5] qemu: simplify graphics port releasing

2018-07-18 Thread Nikolay Shirokovskiy



On 17.07.2018 23:15, John Ferlan wrote:
> 
> 
> On 07/04/2018 07:03 AM, Nikolay Shirokovskiy wrote:
>> Originally port allocator have 2 functions to release ports: one
>> for for manually reserved ports and one for autoallocated ports. Thus
>> a bit complicated code of port releasing. Now we have only one releasing
>> function.
> 
> But there's a reason for the difference between manually allocated and
> automatically allocated. This patch I believe will blur those lines.
> 
> Also "technically" it's not 2 functions it's different conditions during
> qemuProcessStop which must be handled.

I mean look at src/util/virportallocator.c before 5dbda5e972. We use
virPortAllocatorRelease to release auto generated ports and 
virPortAllocatorSetUsed($value, false) to release manually specified ports.

> 
>>
>> Let's use *reserved flag whenever we manually/automatically allocate
>> port so that we can use it later for releasing.
>>
>> Actually we set *reserved flag on autoallocation already on reconnection,
>> which lead to uncleared flag on stop. So this step looks natural.
>>
>> qemuProcessGraphicsReservePorts is called on reconnect. As a result
>> portReserved is set not only for manual ports but autoports too. Now
>> due to the way port releasing is written in qemuProcessStop portReserved
>> stays set for autoports after domain stop. Now imagine one redefine
>>
> 
> ^^^ This whole commit message is really difficult to read and you seem
> to have lost your thought in the last sentence of the last paragraph.

Hmm, somehow I just lost a part of text in message.

I thought we can have issues with port releasing in current code because we use 
portReserved
a bit innacurately. For example we set it for manual ports on domain start 
so we know we need to release port on domain stop. But on reconnect
we set portReserved for all ports and do not clear this flag for auto
ports on domain stop. Fortunately for autoports this flags is never checked and 
on
domain config update if we change autoport to manual port we reset this
flag as it is not part of user defined configuration.

So using portReserved is inaccurate but we don't have any issues yet.
One can just clear portReserved unconditionally on domain stop to make
it more accurate. But I think we can instead make releasing ports on
domain stop more starightforward. Instead of make distinction auto/manual
let's use portReserved for both auto and manual ports. We need this
flag for manual ports why not use it for autoports too?

In short, let's set portReserved for any port reserving via
virPortAllocatorSetUsed or virPortAllocatorAcquire so we know
we need to release port later with virPortAllocatorRelease.

Nikolay


> 
>> Signed-off-by: Nikolay Shirokovskiy 
>> ---
>>  src/conf/domain_conf.h  |  1 +
>>  src/qemu/qemu_process.c | 48 
>> ++--
>>  2 files changed, 23 insertions(+), 26 deletions(-)
>>
> NACK without a better explanation as to why this is important.
> 
>> diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h
>> index 41d2748..1da6b8f 100644
>> --- a/src/conf/domain_conf.h
>> +++ b/src/conf/domain_conf.h
>> @@ -1610,6 +1610,7 @@ struct _virDomainGraphicsDef {
>>  int port;
>>  bool portReserved;
>>  int websocket;
>> +bool websocketReserved;
>>  bool websocketGenerated;
>>  bool autoport;
>>  char *keymap;
>> diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c
>> index da5656d..de2e84b 100644
>> --- a/src/qemu/qemu_process.c
>> +++ b/src/qemu/qemu_process.c
>> @@ -3628,12 +3628,14 @@ qemuProcessVNCAllocatePorts(virQEMUDriverPtr driver,
>>  if (virPortAllocatorAcquire(driver->remotePorts, &port) < 0)
>>  return -1;
>>  graphics->data.vnc.port = port;
>> +graphics->data.vnc.portReserved = true;
> 
> Why?  "autoport" says acquire some/any port not a specific port.  So on
> restart we can use anything and it's not a specifically reserved port.
> So on reconnect IIUC we don't need to "block" usage of specific ports.
> Keeping track of what port number got autoallocated is a different issue
> I would think and if that's what you're after, then that would be a
> different set of patches, wouldn't it?
> 
>>  }
>>  
>>  if (graphics->data.vnc.websocket == -1) {
> 
> Recall from:
> 
> struct _virDomainGraphicsDef {
> /* Port value discipline:
>  * Value -1 is legacy syntax indicating that it should be
> auto-allocated.
>  * Value 0 means port wasn't specified in XML at all.
>  * Positive value is actual port number given in XML.
>  */
> 
>>  if (virPortAllocatorAcquire(driver->webSocketPorts, &port) < 0)
>>  return -1;
>>  graphics->data.vnc.websocket = port;
>> +graphics->data.vnc.websocketReserved = true;
>>  graphics->data.vnc.websocketGenerated = true;
> 
> So again, why? Not sure I agree - the value was Generate

Re: [libvirt] [PATCH 3/5] qemu: vnc: mark websocket as used on reconnect

2018-07-18 Thread Nikolay Shirokovskiy



On 17.07.2018 23:16, John Ferlan wrote:
> 
> 
> On 07/04/2018 07:03 AM, Nikolay Shirokovskiy wrote:
>> Signed-off-by: Nikolay Shirokovskiy 
>> ---
>>  src/qemu/qemu_process.c | 3 ++-
>>  1 file changed, 2 insertions(+), 1 deletion(-)
>>
> 
> So your contention is that commit id 84067696 missed a condition?
> 
> I agree, but would like you to reference that commit id in your commit
> message before pushing - especially since AIUI websocket > 0 means it's
> a port we've specifically wanted reserved.
> 

Honestly I don't undestand why I wrote this patch. On reconnection whether
websocket is auto generated or not it will be marked as used in 
qemuProcessGraphicsReservePorts
because it is > 0.

Nikolay

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list


Re: [libvirt] [PATCHv2 11/11] qemu_driver: BaselineHypervisorCPU supports S390 using QEMU/QMP

2018-07-18 Thread David Hildenbrand
On 18.07.2018 00:39, Collin Walling wrote:
> On 07/17/2018 05:01 PM, David Hildenbrand wrote:
>> On 13.07.2018 18:00, Jiri Denemark wrote:
>>> On Mon, Jul 09, 2018 at 22:56:55 -0500, Chris Venteicher wrote:
 Transient S390 configurations require using QEMU to compute CPU Model
 Baseline and to do CPU Feature Expansion.

 Start and use a single QEMU instance to do both the baseline and
 expansion transactions required by BaselineHypervisorCPU.

 CPU Feature Expansion uses true / false to indicate if property is/isn't
 included in model. Baseline only returns property list where all
 enumerated properties are included.
>>>
>>> So are you saying on s390 there's no chance there would be a CPU model
>>> with some feature which is included in the CPU model disabled for some
>>> reason? Sounds too good to be true :-) (This is the question I referred
>>> to in one of my replies to the other patches.)
>>
>> Giving some background information: When we expand/baseline CPU models,
>> we always expand them to the "-base" variants of our CPU models, which
>> contain some set of features we expect to be around in all sane
>> configurations ("minimal feature set").
>>
>> It is very unlikely that we ever have something like
>> "z14-base,featx=off", but it could happen
>>  - When using an emulator (TCG)
>>  - When running nested and the guest hypervisor is started with such a
>>strange CPU model
>>  - When something in the HW is very wrong or eventually removed in the
>>future (unlikely but possible)
>>
>> On some very weird inputs to a baseline request, such a strange model
>> can also be the result. But it is very unusual.
>>
>> I assume something like "baseline z14-base,featx=off with z14-base" will
>> result in "z14-base,featx=off", too.
>>
>>
> 
> That's correct. A CPU model with a feature disabled that is baseline with a 
> CPU 
> model with that same feature enabled will omit that feature in the QMP 
> response.
> 
> e.g. if z14-base has features x, y, z then
> 
> "baseline z14-base,featx=off with z14-base" will result in 
> "z14-base,featy=on,featz=on"

Usually we try to not chose a model with stripped off base features ("we
try to produce a model that looks sane"), but instead fallback to some
very ancient CPU model. E.g.

{ "execute": "query-cpu-model-baseline", "arguments" : { "modela": {
"name": "z14-base", "props": {"msa" : false}}, "modelb": { "name": "z14"}} }

-> {"return": {"model": {"name": "z800-base", "props": {"etf2": true,
"ldisp": true

We might want to change that behavior in the future however (or maybe it
already is like this for some corner cases) - assume some base feature
gets dropped by HW in a new CPU generation. We don't always want to
fallback to a z900 or so when baselining. So one should assume that we
can have disabled features here.

Especially as there is a BUG in QEMU I'll have to fix:

{ "execute": "query-cpu-model-baseline", "arguments" : { "modela": {
"name": "z14-base", "props": {"esan3" : false}}, "modelb": { "name":
"z14"}} }

-> Segmentation fault

This would have to produce a model with esan3 disabled (very very
unlikely to ever happen in real life :) )

The result should be something like {"model": {"name": "z900-base",
"props": {"esan3": false}}} or an error that they cannot be baselined.



> 
> Since baseline will just report a base cpu and its minimal feature set... I 
> wonder if it
> makes sense for libvirt to just report the features resulting from the 
> baseline command 
> instead of later calling expansion?

Yes it does and the docs say:

"Baseline two CPU models, creating a compatible third model. The created
model will always be a static, migration-safe CPU model (see "static"
CPU model expansion for details)"

> 
>>>
>>> Most of the code you added in this patch is indented by three spaces
>>> while we use four spaces in libvirt.
>>>
 ---
  src/qemu/qemu_driver.c | 74 +-
  1 file changed, 65 insertions(+), 9 deletions(-)

 diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c
 index 9a35e04a85..6c6107f077 100644
 --- a/src/qemu/qemu_driver.c
 +++ b/src/qemu/qemu_driver.c
 @@ -13400,10 +13400,13 @@ qemuConnectBaselineHypervisorCPU(virConnectPtr 
 conn,
  virArch arch;
  virDomainVirtType virttype;
  virDomainCapsCPUModelsPtr cpuModels;
 -bool migratable;
 +bool migratable_only;
>>>
>>> Why? The bool says the user specified
>>> VIR_CONNECT_BASELINE_CPU_MIGRATABLE which means they want a migratable
>>> CPU back. What does the "_only" part mean? This API does not return
>>> several CPUs, it only returns a single one and it's either migratable or
>>> not.
>>>
  virCPUDefPtr cpu = NULL;
  char *cpustr = NULL;
  char **features = NULL;
 +virQEMUCapsInitQMPCommandPtr cmd = NULL;
 +bool forceTCG = false;
 +qemuMonitorCPUModelInfoPtr modelInfo = NULL;

Re: [libvirt] [PATCH 1/4] tests: qemu: Remove disk from graphics-vnc-tls

2018-07-18 Thread Peter Krempa
On Tue, Jul 17, 2018 at 17:15:53 +0200, Ján Tomko wrote:
> The disk command line is tested elsewhere.
> 
> Signed-off-by: Ján Tomko 
> ---
>  tests/qemuxml2argvdata/graphics-vnc-tls.args  | 2 --
>  tests/qemuxml2argvdata/graphics-vnc-tls.xml   | 6 --
>  tests/qemuxml2xmloutdata/graphics-vnc-tls.xml | 6 --
>  3 files changed, 14 deletions(-)

ACK


signature.asc
Description: PGP signature
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH 3/4] tests: qemu: test more versions for graphics-vnc-tls

2018-07-18 Thread Peter Krempa
On Tue, Jul 17, 2018 at 17:15:55 +0200, Ján Tomko wrote:
> Add a test with QEMU 2.4.0 capabilites, as well as the latest caps.
> 
> The code paths for formatting TLS options will be altered and
> 2.4.0 is the newest version where QEMU_CAPS_OBJECT_TLS_CREDS_X509
> is not supported.
> 
> Signed-off-by: Ján Tomko 
> ---
>  .../qemuxml2argvdata/graphics-vnc-tls.x86_64-2.4.0.args | 12 +++-
>  .../graphics-vnc-tls.x86_64-latest.args | 17 
> +++--
>  tests/qemuxml2argvtest.c|  2 ++
>  3 files changed, 20 insertions(+), 11 deletions(-)

ACK


signature.asc
Description: PGP signature
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH 4/4] qemu: vnc: switch to tls-creds-x509

2018-07-18 Thread Daniel P . Berrangé
On Tue, Jul 17, 2018 at 05:15:56PM +0200, Ján Tomko wrote:
> The tls, x509 and x509verify options were deprecated in QEMU v2.5.0:
> 
> commit 3e305e4a4752f70c0b5c3cf5b43ec957881714f7
> Author: Daniel P. Berrange 
> 
> ui: convert VNC server to use QCryptoTLSSession
> 
> Use the tls-creds-x509 object when available.
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=1598167
> 
> Signed-off-by: Ján Tomko 
> ---
>  src/qemu/qemu_command.c| 26 
> +-
>  .../graphics-vnc-tls.x86_64-latest.args|  4 +++-
>  2 files changed, 23 insertions(+), 7 deletions(-)
> 
> diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c
> index 44ae8dcef7..9326abbe63 100644
> --- a/src/qemu/qemu_command.c
> +++ b/src/qemu/qemu_command.c
> @@ -7917,13 +7917,27 @@ 
> qemuBuildGraphicsVNCCommandLine(virQEMUDriverConfigPtr cfg,
>  virBufferAddLit(&opt, ",password");
>  
>  if (cfg->vncTLS) {
> -virBufferAddLit(&opt, ",tls");
> -if (cfg->vncTLSx509verify) {
> -virBufferAddLit(&opt, ",x509verify=");
> -virQEMUBuildBufferEscapeComma(&opt, cfg->vncTLSx509certdir);
> +if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_OBJECT_TLS_CREDS_X509)) {
> +const char *alias = "vnc-tls-creds0";
> +if (qemuBuildTLSx509CommandLine(cmd,
> +cfg->vncTLSx509certdir,
> +true,
> +cfg->vncTLSx509verify,
> +NULL,
> +alias,
> +qemuCaps) < 0)
> +goto error;
> +
> +virBufferAsprintf(&opt, ",tls-creds=%s", alias);
>  } else {
> -virBufferAddLit(&opt, ",x509=");
> -virQEMUBuildBufferEscapeComma(&opt, cfg->vncTLSx509certdir);
> +virBufferAddLit(&opt, ",tls");
> +if (cfg->vncTLSx509verify) {
> +virBufferAddLit(&opt, ",x509verify=");
> +virQEMUBuildBufferEscapeComma(&opt, cfg->vncTLSx509certdir);
> +} else {
> +virBufferAddLit(&opt, ",x509=");
> +virQEMUBuildBufferEscapeComma(&opt, cfg->vncTLSx509certdir);
> +}
>  }
>  }

So this is better than what we have today, but it is still not comparable
with what John did for the other TLS features. Specifically it is missing
support for encrypted keys, so we're still broken if the user has editted
qemu.conf to set a "default_tls_x509_secret_uuid". We should also have
a new "vnc_tls_x509_secret_uuid" to match what's done for chardev and
migration.

We need to use more of the helpers that John created for dealing with
TLS creds to handle this correctly.


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH 4/4] qemu: vnc: switch to tls-creds-x509

2018-07-18 Thread Ján Tomko

On Wed, Jul 18, 2018 at 08:32:44AM +0100, Daniel P. Berrangé wrote:

On Tue, Jul 17, 2018 at 05:15:56PM +0200, Ján Tomko wrote:

The tls, x509 and x509verify options were deprecated in QEMU v2.5.0:

commit 3e305e4a4752f70c0b5c3cf5b43ec957881714f7
Author: Daniel P. Berrange 

ui: convert VNC server to use QCryptoTLSSession

Use the tls-creds-x509 object when available.

https://bugzilla.redhat.com/show_bug.cgi?id=1598167

Signed-off-by: Ján Tomko 
---
 src/qemu/qemu_command.c| 26 +-
 .../graphics-vnc-tls.x86_64-latest.args|  4 +++-
 2 files changed, 23 insertions(+), 7 deletions(-)

diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c
index 44ae8dcef7..9326abbe63 100644
--- a/src/qemu/qemu_command.c
+++ b/src/qemu/qemu_command.c
@@ -7917,13 +7917,27 @@ qemuBuildGraphicsVNCCommandLine(virQEMUDriverConfigPtr 
cfg,
 virBufferAddLit(&opt, ",password");

 if (cfg->vncTLS) {
-virBufferAddLit(&opt, ",tls");
-if (cfg->vncTLSx509verify) {
-virBufferAddLit(&opt, ",x509verify=");
-virQEMUBuildBufferEscapeComma(&opt, cfg->vncTLSx509certdir);
+if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_OBJECT_TLS_CREDS_X509)) {
+const char *alias = "vnc-tls-creds0";
+if (qemuBuildTLSx509CommandLine(cmd,
+cfg->vncTLSx509certdir,
+true,
+cfg->vncTLSx509verify,
+NULL,
+alias,
+qemuCaps) < 0)
+goto error;
+
+virBufferAsprintf(&opt, ",tls-creds=%s", alias);
 } else {
-virBufferAddLit(&opt, ",x509=");
-virQEMUBuildBufferEscapeComma(&opt, cfg->vncTLSx509certdir);
+virBufferAddLit(&opt, ",tls");
+if (cfg->vncTLSx509verify) {
+virBufferAddLit(&opt, ",x509verify=");
+virQEMUBuildBufferEscapeComma(&opt, cfg->vncTLSx509certdir);
+} else {
+virBufferAddLit(&opt, ",x509=");
+virQEMUBuildBufferEscapeComma(&opt, cfg->vncTLSx509certdir);
+}
 }
 }


So this is better than what we have today, but it is still not comparable
with what John did for the other TLS features. Specifically it is missing
support for encrypted keys, so we're still broken if the user has editted
qemu.conf to set a "default_tls_x509_secret_uuid". We should also have
a new "vnc_tls_x509_secret_uuid" to match what's done for chardev and
migration.



I'm aware of that, as I said in the bugzilla comment:
https://bugzilla.redhat.com/show_bug.cgi?id=1598167#c1

Do you consider the lack of this feature a blocker for this patch aiming
to prevent a regression when QEMU stops accepting the old syntax?

Jano


We need to use more of the helpers that John created for dealing with
TLS creds to handle this correctly.


Regards,
Daniel
--
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|


signature.asc
Description: Digital signature
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH 4/4] qemu: vnc: switch to tls-creds-x509

2018-07-18 Thread Daniel P . Berrangé
On Wed, Jul 18, 2018 at 09:55:47AM +0200, Ján Tomko wrote:
> On Wed, Jul 18, 2018 at 08:32:44AM +0100, Daniel P. Berrangé wrote:
> > On Tue, Jul 17, 2018 at 05:15:56PM +0200, Ján Tomko wrote:
> > > The tls, x509 and x509verify options were deprecated in QEMU v2.5.0:
> > > 
> > > commit 3e305e4a4752f70c0b5c3cf5b43ec957881714f7
> > > Author: Daniel P. Berrange 
> > > 
> > > ui: convert VNC server to use QCryptoTLSSession
> > > 
> > > Use the tls-creds-x509 object when available.
> > > 
> > > https://bugzilla.redhat.com/show_bug.cgi?id=1598167
> > > 
> > > Signed-off-by: Ján Tomko 
> > > ---
> > >  src/qemu/qemu_command.c| 26 
> > > +-
> > >  .../graphics-vnc-tls.x86_64-latest.args|  4 +++-
> > >  2 files changed, 23 insertions(+), 7 deletions(-)
> > > 
> > > diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c
> > > index 44ae8dcef7..9326abbe63 100644
> > > --- a/src/qemu/qemu_command.c
> > > +++ b/src/qemu/qemu_command.c
> > > @@ -7917,13 +7917,27 @@ 
> > > qemuBuildGraphicsVNCCommandLine(virQEMUDriverConfigPtr cfg,
> > >  virBufferAddLit(&opt, ",password");
> > > 
> > >  if (cfg->vncTLS) {
> > > -virBufferAddLit(&opt, ",tls");
> > > -if (cfg->vncTLSx509verify) {
> > > -virBufferAddLit(&opt, ",x509verify=");
> > > -virQEMUBuildBufferEscapeComma(&opt, cfg->vncTLSx509certdir);
> > > +if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_OBJECT_TLS_CREDS_X509)) {
> > > +const char *alias = "vnc-tls-creds0";
> > > +if (qemuBuildTLSx509CommandLine(cmd,
> > > +cfg->vncTLSx509certdir,
> > > +true,
> > > +cfg->vncTLSx509verify,
> > > +NULL,
> > > +alias,
> > > +qemuCaps) < 0)
> > > +goto error;
> > > +
> > > +virBufferAsprintf(&opt, ",tls-creds=%s", alias);
> > >  } else {
> > > -virBufferAddLit(&opt, ",x509=");
> > > -virQEMUBuildBufferEscapeComma(&opt, cfg->vncTLSx509certdir);
> > > +virBufferAddLit(&opt, ",tls");
> > > +if (cfg->vncTLSx509verify) {
> > > +virBufferAddLit(&opt, ",x509verify=");
> > > +virQEMUBuildBufferEscapeComma(&opt, 
> > > cfg->vncTLSx509certdir);
> > > +} else {
> > > +virBufferAddLit(&opt, ",x509=");
> > > +virQEMUBuildBufferEscapeComma(&opt, 
> > > cfg->vncTLSx509certdir);
> > > +}
> > >  }
> > >  }
> > 
> > So this is better than what we have today, but it is still not comparable
> > with what John did for the other TLS features. Specifically it is missing
> > support for encrypted keys, so we're still broken if the user has editted
> > qemu.conf to set a "default_tls_x509_secret_uuid". We should also have
> > a new "vnc_tls_x509_secret_uuid" to match what's done for chardev and
> > migration.
> > 
> 
> I'm aware of that, as I said in the bugzilla comment:
> https://bugzilla.redhat.com/show_bug.cgi?id=1598167#c1
> 
> Do you consider the lack of this feature a blocker for this patch aiming
> to prevent a regression when QEMU stops accepting the old syntax?

Even today if you use an encrypted key for default_tls_x509_dir the VNC
server config will be broken, so it isn't a regression, and thus not a
blocker.

I should have made it clearer in the BZ though, that the intention was to
get parity for TLS config across all servers, so that we both avoid the
deprecated feature & fix the existing bug wrt encrypted TLS keys. Perhaps
better file a separate BZ for the latter now.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [jenkins-ci PATCH v3 00/12] lcitool: Rewrite in Python (and add Dockefile generator)

2018-07-18 Thread Katerina Koukiou
On Tue, Jul 17, 2018 at 06:35:16PM +0200, Andrea Bolognani wrote:
> pylint is reasonably happy with the script now:
> 
>   $ pylint lcitool
>   No config file found, using default configuration
>   * Module lcitool
>   C:  1, 0: Missing module docstring (missing-docstring)
>   C: 37, 0: Missing class docstring (missing-docstring)
>   C: 47, 4: Missing method docstring (missing-docstring)
>   W:108,15: Catching too general exception Exception (broad-except)
>   R:427, 4: Too many branches (14/12) (too-many-branches)
>   R:289, 0: Too few public methods (1/2) (too-few-public-methods)
> 
>   ---
>   Your code has been rated at 9.21/10 (previous run: 9.21/10, +0.00)
> 
> The remaining issues are not considered blockers and will be
> addressed, if at all, later down the line.
> 
> Changes from [v2]:
> 
>   * address review comments;
>   * improve pycodestyle and pylint compliance;
>   * replace FSF address with an URL.
> 
> Changes from [v1]:
> 
>   * add Dockerfile generator;
>   * rename the 'list' action to 'hosts' to better fit along with
> the additional 'projects' action;
>   * always list items in alphabetical order;
>   * move some generic functions to an Util class.
> 
> [v2] https://www.redhat.com/archives/libvir-list/2018-July/msg00795.html
> [v1] https://www.redhat.com/archives/libvir-list/2018-July/msg00717.html
> 
> Andrea Bolognani (12):
>   lcitool: Drop shell implementation
>   lcitool: Stub out Python implementation
>   lcitool: Add tool configuration handling
>   lcitool: Add inventory handling
>   lcitool: Implement the 'hosts' action
>   lcitool: Implement the 'install' action
>   lcitool: Implement the 'update' action
>   guests: Update documentation
>   guests: Add Docker-related information to the inventory
>   lcitool: Add projects information handling
>   lcitool: Implement the 'projects' action
>   lcitool: Implement the 'dockerfile' action
> 
>  guests/README.markdown|   8 +-
>  guests/host_vars/libvirt-centos-7/docker.yml  |   2 +
>  guests/host_vars/libvirt-debian-8/docker.yml  |   2 +
>  guests/host_vars/libvirt-debian-9/docker.yml  |   2 +
>  .../host_vars/libvirt-debian-sid/docker.yml   |   2 +
>  guests/host_vars/libvirt-fedora-27/docker.yml |   2 +
>  guests/host_vars/libvirt-fedora-28/docker.yml |   2 +
>  .../libvirt-fedora-rawhide/docker.yml |   2 +
>  guests/host_vars/libvirt-ubuntu-16/docker.yml |   2 +
>  guests/host_vars/libvirt-ubuntu-18/docker.yml |   2 +
>  guests/lcitool| 730 --
>  11 files changed, 529 insertions(+), 227 deletions(-)
>  create mode 100644 guests/host_vars/libvirt-centos-7/docker.yml
>  create mode 100644 guests/host_vars/libvirt-debian-8/docker.yml
>  create mode 100644 guests/host_vars/libvirt-debian-9/docker.yml
>  create mode 100644 guests/host_vars/libvirt-debian-sid/docker.yml
>  create mode 100644 guests/host_vars/libvirt-fedora-27/docker.yml
>  create mode 100644 guests/host_vars/libvirt-fedora-28/docker.yml
>  create mode 100644 guests/host_vars/libvirt-fedora-rawhide/docker.yml
>  create mode 100644 guests/host_vars/libvirt-ubuntu-16/docker.yml
>  create mode 100644 guests/host_vars/libvirt-ubuntu-18/docker.yml
> 
> -- 
> 2.17.1
> 
> --
> libvir-list mailing list
> libvir-list@redhat.com
> https://www.redhat.com/mailman/listinfo/libvir-list

For all patches in this series.

Reviewed-by: Katerina Koukiou 


signature.asc
Description: PGP signature
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH 2/2] docs: formatdomain: clarify period cputune subelement

2018-07-18 Thread Ján Tomko

On Tue, Jul 17, 2018 at 05:47:32PM +0200, Katerina Koukiou wrote:

Although the name of the element is not self-explanatory,
it's affecting only the vcpu threads.

Signed-off-by: Katerina Koukiou 
---
docs/formatdomain.html.in | 10 +-
1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in
index 7fa448acdc..61930e4907 100644
--- a/docs/formatdomain.html.in
+++ b/docs/formatdomain.html.in
@@ -831,11 +831,11 @@
  
The optional quota element specifies the maximum allowed
bandwidth(unit: microseconds). A domain with quota as any


There is a missing space before the parenthesis here, and it's copied
multiple times for all the similar attributes. It's a good indicator
that we should only write the repetititive text once, and collect the
differences in a neat table, e.g.:

+---++-+-+
+ quota element | period element | affects | since   |
+---++-+-+
+ quota | period | vCPUs   | 0.9.4, LXC since 0.9.10 |
+ global_guota  | global_period  | all | ... |
+---++-+-+

(But, of course, even this change is better than nothing)

Jano


-negative value indicates that the domain has infinite bandwidth, which
-means that it is not bandwidth controlled. The value should be in range
-[1000, 18446744073709551] or less than 0. A quota with value 0 means no
-value. You can use this feature to ensure that all vcpus run at the 
same
-speed.


signature.asc
Description: Digital signature
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH 1/2] docs: formatdomain: add info about global_period and global_quota for cputune

2018-07-18 Thread Erik Skultety
On Tue, Jul 17, 2018 at 05:47:31PM +0200, Katerina Koukiou wrote:
> Patch series
> https://www.redhat.com/archives/libvir-list/2016-January/msg00482.html

The commit hashes would be better IMO, i.e. commits 4d92d5 and 55ecda.

> introduced the parameters but didn't update the docs.

>
> Signed-off-by: Katerina Koukiou 
> ---
>  docs/formatdomain.html.in | 21 +
>  1 file changed, 21 insertions(+)
>
> diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in
> index a3afe137bf..7fa448acdc 100644
> --- a/docs/formatdomain.html.in
> +++ b/docs/formatdomain.html.in
> @@ -747,6 +747,8 @@
>  2048
>  100
>  -1
> +100
> +-1
>  100
>  -1
>  100
> @@ -837,6 +839,25 @@
>  Only QEMU driver support since 0.9.4, LXC since
>  0.9.10
>
> +  global_period
> +  
> +The optional global_period element specifies the
> +enforcement period for a quota (unit: microseconds) for whole domain,
> +when using posix scheduler. The value should be in range

s/ whole/the whole/

How about "... element specifies the enforcement CFS scheduler interval for the
whole domain in contrast with  which enforces the interval per vcpu."

> +[1000, 100]. A global_period with value 0 means no value.

s/\(global_period\)/\1/

> +Only QEMU driver support since 1.3.3

^This bit can stay as is.

> +  
> +  global_quota
> +  
> +The optional global_quota element specifies the maximum
> +allowed bandwidth (unit: microseconds) to be used within a period for

s/to be used//
s/ whole/the whole/

> +whole domain. A domain with global_quota as any negative
> +value indicates that the domain has infinite bandwidth, which means 
> that
> +it is not bandwidth controlled. The value should be in range
> +[1000, 18446744073709551] or less than 0. A global_quota with value 0
> +means no value.

s/\(global_quota\)/\1/

> +Only QEMU driver support since 1.3.3
> +  
>
>emulator_period
>
> --

Reviewed-by: Erik Skultety 

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list


[libvirt] [PATCH 7/9] conf: Refactor virDomainCachetuneDefParse

2018-07-18 Thread bing . niu
From: Bing Niu 

Refactoring virDomainCachetuneDefParse, extracting vcpus extracting,
overlapping detecting and new resctrl allocation creating logic.
Those two logic is common among different resource allocation
technologies.

Signed-off-by: Bing Niu 
---
 src/conf/domain_conf.c | 195 +
 1 file changed, 131 insertions(+), 64 deletions(-)

diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
index 24fefd1..695981c 100644
--- a/src/conf/domain_conf.c
+++ b/src/conf/domain_conf.c
@@ -18884,6 +18884,115 @@ virDomainDefParseBootOptions(virDomainDefPtr def,
 
 
 static int
+virDomainRestuneParseVcpus(virDomainDefPtr def,
+   xmlNodePtr node,
+   virBitmapPtr *vcpus)
+{
+char *vcpus_str = NULL;
+int ret = -1;
+
+vcpus_str = virXMLPropString(node, "vcpus");
+if (!vcpus_str) {
+virReportError(VIR_ERR_XML_ERROR, "%s",
+   _("Missing cachetune attribute 'vcpus'"));
+goto cleanup;
+}
+if (virBitmapParse(vcpus_str, vcpus, VIR_DOMAIN_CPUMASK_LEN) < 0) {
+virReportError(VIR_ERR_XML_ERROR,
+   _("Invalid cachetune attribute 'vcpus' value '%s'"),
+   vcpus_str);
+goto cleanup;
+}
+
+/* We need to limit the bitmap to number of vCPUs.  If there's nothing 
left,
+ * then we can just clean up and return 0 immediately */
+virBitmapShrink(*vcpus, def->maxvcpus);
+
+ret = 0;
+ cleanup:
+VIR_FREE(vcpus_str);
+return ret;
+}
+
+
+static int
+virDomainFindResctrlAlloc(virDomainDefPtr def,
+  virBitmapPtr vcpus,
+  virResctrlAllocPtr *alloc)
+{
+ssize_t i = 0;
+
+for (i = 0; i < def->nrestunes; i++) {
+/* vcpus group has been created, directly use the existing one.
+ * Just updating memory allocation information of that group
+ */
+if (virBitmapEqual(def->restunes[i]->vcpus, vcpus)) {
+*alloc = def->restunes[i]->alloc;
+break;
+}
+if (virBitmapOverlaps(def->restunes[i]->vcpus, vcpus)) {
+virReportError(VIR_ERR_XML_ERROR, "%s",
+   _("Overlapping vcpus in restunes"));
+return -1;
+}
+}
+return 0;
+}
+
+
+static int
+virDomainUpdateRestune(virDomainDefPtr def,
+   xmlNodePtr node,
+   virResctrlAllocPtr alloc,
+   virBitmapPtr vcpus,
+   unsigned int flags)
+{
+char *vcpus_str = NULL;
+char *alloc_id = NULL;
+virDomainRestuneDefPtr tmp_restune = NULL;
+int ret = -1;
+
+if (VIR_ALLOC(tmp_restune) < 0)
+goto cleanup;
+
+/* We need to format it back because we need to be consistent in the naming
+ * even when users specify some "sub-optimal" string there. */
+vcpus_str = virBitmapFormat(vcpus);
+if (!vcpus_str)
+goto cleanup;
+
+if (!(flags & VIR_DOMAIN_DEF_PARSE_INACTIVE))
+alloc_id = virXMLPropString(node, "id");
+
+if (!alloc_id) {
+/* The number of allocations is limited and the directory structure is 
flat,
+ * not hierarchical, so we need to have all same allocations in one
+ * directory, so it's nice to have it named appropriately.  For now 
it's
+ * 'vcpus_...' but it's designed in order for it to be changeable in 
the
+ * future (it's part of the status XML). */
+if (virAsprintf(&alloc_id, "vcpus_%s", vcpus_str) < 0)
+goto cleanup;
+}
+
+if (virResctrlAllocSetID(alloc, alloc_id) < 0)
+goto cleanup;
+
+tmp_restune->vcpus = vcpus;
+tmp_restune->alloc = alloc;
+
+if (VIR_APPEND_ELEMENT(def->restunes, def->nrestunes, tmp_restune) < 0)
+goto cleanup;
+
+ret = 0;
+ cleanup:
+virDomainRestuneDefFree(tmp_restune);
+VIR_FREE(alloc_id);
+VIR_FREE(vcpus_str);
+return ret;
+}
+
+
+static int
 virDomainCachetuneDefParseCache(xmlXPathContextPtr ctxt,
 xmlNodePtr node,
 virResctrlAllocPtr alloc)
@@ -18966,39 +19075,16 @@ virDomainCachetuneDefParse(virDomainDefPtr def,
 xmlNodePtr oldnode = ctxt->node;
 xmlNodePtr *nodes = NULL;
 virBitmapPtr vcpus = NULL;
-virResctrlAllocPtr alloc = virResctrlAllocNew();
-virDomainRestuneDefPtr tmp_restune = NULL;
-char *tmp = NULL;
-char *vcpus_str = NULL;
-char *alloc_id = NULL;
+virResctrlAllocPtr alloc = NULL;
 ssize_t i = 0;
 int n;
 int ret = -1;
+bool new_alloc = false;
 
 ctxt->node = node;
 
-if (!alloc)
-goto cleanup;
-
-if (VIR_ALLOC(tmp_restune) < 0)
-goto cleanup;
-
-vcpus_str = virXMLPropString(node, "vcpus");
-if (!vcpus_str) {
-virReportError(VIR_ERR_XML_ERROR, "%s",
-   _("Missing cachetune attribute 'vcpus'"));
-goto 

[libvirt] [PATCH 1/9] util: Rename some functions of virresctrl

2018-07-18 Thread bing . niu
From: Bing Niu 

Some functions in virresctrl are for CAT only, while some of other
functions are for resource allocation, not just CAT. So change
their names to reflect the reality.

Signed-off-by: Bing Niu 
---
 src/conf/domain_conf.c   |  8 
 src/libvirt_private.syms |  4 ++--
 src/util/virresctrl.c| 30 +++---
 src/util/virresctrl.h| 26 +-
 4 files changed, 34 insertions(+), 34 deletions(-)

diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
index 7396616..f259b4c 100644
--- a/src/conf/domain_conf.c
+++ b/src/conf/domain_conf.c
@@ -18946,7 +18946,7 @@ virDomainCachetuneDefParseCache(xmlXPathContextPtr ctxt,
   ULLONG_MAX, true) < 0)
 goto cleanup;
 
-if (virResctrlAllocSetSize(alloc, level, type, cache, size) < 0)
+if (virResctrlAllocSetCacheSize(alloc, level, type, cache, size) < 0)
 goto cleanup;
 
 ret = 0;
@@ -26921,9 +26921,9 @@ virDomainCachetuneDefFormat(virBufferPtr buf,
 int ret = -1;
 
 virBufferSetChildIndent(&childrenBuf, buf);
-virResctrlAllocForeachSize(cachetune->alloc,
-   virDomainCachetuneDefFormatHelper,
-   &childrenBuf);
+virResctrlAllocForeachCache(cachetune->alloc,
+virDomainCachetuneDefFormatHelper,
+&childrenBuf);
 
 
 if (virBufferCheckError(&childrenBuf) < 0)
diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms
index 1caecb9..47e1b18 100644
--- a/src/libvirt_private.syms
+++ b/src/libvirt_private.syms
@@ -2646,14 +2646,14 @@ virCacheTypeToString;
 virResctrlAllocAddPID;
 virResctrlAllocCreate;
 virResctrlAllocDeterminePath;
-virResctrlAllocForeachSize;
+virResctrlAllocForeachCache;
 virResctrlAllocFormat;
 virResctrlAllocGetID;
 virResctrlAllocGetUnused;
 virResctrlAllocNew;
 virResctrlAllocRemove;
+virResctrlAllocSetCacheSize;
 virResctrlAllocSetID;
-virResctrlAllocSetSize;
 virResctrlInfoGetCache;
 virResctrlInfoNew;
 
diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c
index e492a63..6d69c8d 100644
--- a/src/util/virresctrl.c
+++ b/src/util/virresctrl.c
@@ -754,11 +754,11 @@ virResctrlAllocCheckCollision(virResctrlAllocPtr alloc,
 
 
 int
-virResctrlAllocSetSize(virResctrlAllocPtr alloc,
-   unsigned int level,
-   virCacheType type,
-   unsigned int cache,
-   unsigned long long size)
+virResctrlAllocSetCacheSize(virResctrlAllocPtr alloc,
+unsigned int level,
+virCacheType type,
+unsigned int cache,
+unsigned long long size)
 {
 if (virResctrlAllocCheckCollision(alloc, level, type, cache)) {
 virReportError(VIR_ERR_XML_ERROR,
@@ -773,9 +773,9 @@ virResctrlAllocSetSize(virResctrlAllocPtr alloc,
 
 
 int
-virResctrlAllocForeachSize(virResctrlAllocPtr alloc,
-   virResctrlAllocForeachSizeCallback cb,
-   void *opaque)
+virResctrlAllocForeachCache(virResctrlAllocPtr alloc,
+virResctrlAllocForeachCacheCallback cb,
+void *opaque)
 {
 int ret = 0;
 unsigned int level = 0;
@@ -939,9 +939,9 @@ virResctrlAllocParseProcessCache(virResctrlInfoPtr resctrl,
 
 
 static int
-virResctrlAllocParseProcessLine(virResctrlInfoPtr resctrl,
-virResctrlAllocPtr alloc,
-char *line)
+virResctrlAllocParseCacheLine(virResctrlInfoPtr resctrl,
+  virResctrlAllocPtr alloc,
+  char *line)
 {
 char **caches = NULL;
 char *tmp = NULL;
@@ -1009,7 +1009,7 @@ virResctrlAllocParse(virResctrlInfoPtr resctrl,
 
 lines = virStringSplitCount(schemata, "\n", 0, &nlines);
 for (i = 0; i < nlines; i++) {
-if (virResctrlAllocParseProcessLine(resctrl, alloc, lines[i]) < 0)
+if (virResctrlAllocParseCacheLine(resctrl, alloc, lines[i]) < 0)
 goto cleanup;
 }
 
@@ -1401,8 +1401,8 @@ virResctrlAllocCopyMasks(virResctrlAllocPtr dst,
  * transforming `sizes` into `masks`.
  */
 static int
-virResctrlAllocMasksAssign(virResctrlInfoPtr resctrl,
-   virResctrlAllocPtr alloc)
+virResctrlAllocAssign(virResctrlInfoPtr resctrl,
+  virResctrlAllocPtr alloc)
 {
 int ret = -1;
 unsigned int level = 0;
@@ -1526,7 +1526,7 @@ virResctrlAllocCreate(virResctrlInfoPtr resctrl,
 if (lockfd < 0)
 goto cleanup;
 
-if (virResctrlAllocMasksAssign(resctrl, alloc) < 0)
+if (virResctrlAllocAssign(resctrl, alloc) < 0)
 goto cleanup;
 
 alloc_str = virResctrlAllocFormat(alloc);
diff --git a/src/util/virresctrl.h b/src/util/virresctrl.h
index 9052a2b..d657c06 100644
--- a/src/util/virresctrl.h
+++ b/src/util

[libvirt] [PATCH 9/9] conf: Add memory bandwidth allocation capability of host

2018-07-18 Thread bing . niu
From: Bing Niu 

Add new XML section to report host's memory bandwidth allocation
capability. The format as below example:

 
 .
   
 
   
 
   


granularity    granularity of memory bandwidth, unit percentage.
min    minimum memory bandwidth allowed, unit percentage.
maxAllocs  maximum memory bandwidth allocation group supported.

Signed-off-by: Bing Niu 
---
 docs/schemas/capability.rng|  33 +++
 src/conf/capabilities.c| 108 +
 src/conf/capabilities.h|  11 +++
 src/util/virresctrl.c  |  20 
 src/util/virresctrl.h  |  15 +++
 .../linux-resctrl/resctrl/info/MB/bandwidth_gran   |   1 +
 .../linux-resctrl/resctrl/info/MB/min_bandwidth|   1 +
 .../linux-resctrl/resctrl/info/MB/num_closids  |   1 +
 tests/vircaps2xmldata/vircaps-x86_64-resctrl.xml   |   8 ++
 tests/virresctrldata/resctrl.schemata  |   1 +
 10 files changed, 199 insertions(+)
 create mode 100644 
tests/vircaps2xmldata/linux-resctrl/resctrl/info/MB/bandwidth_gran
 create mode 100644 
tests/vircaps2xmldata/linux-resctrl/resctrl/info/MB/min_bandwidth
 create mode 100644 
tests/vircaps2xmldata/linux-resctrl/resctrl/info/MB/num_closids

diff --git a/docs/schemas/capability.rng b/docs/schemas/capability.rng
index 52164d5..d61515c 100644
--- a/docs/schemas/capability.rng
+++ b/docs/schemas/capability.rng
@@ -51,6 +51,9 @@
   
 
   
+  
+
+  
   
 
   
@@ -326,6 +329,36 @@
 
   
 
+  
+
+  
+
+  
+
+  
+  
+
+  
+  
+
+  
+
+  
+
+  
+
+
+
+  
+
+  
+
+  
+
+  
+
+  
+
   
 
   
diff --git a/src/conf/capabilities.c b/src/conf/capabilities.c
index 7a810ef..3f52296 100644
--- a/src/conf/capabilities.c
+++ b/src/conf/capabilities.c
@@ -198,6 +198,16 @@ virCapabilitiesFreeNUMAInfo(virCapsPtr caps)
 }
 
 static void
+virCapsHostMemBWNodeFree(virCapsHostMemBWNodePtr ptr)
+{
+if (!ptr)
+return;
+
+virBitmapFree(ptr->cpus);
+VIR_FREE(ptr);
+}
+
+static void
 virCapabilitiesClearSecModel(virCapsHostSecModelPtr secmodel)
 {
 size_t i;
@@ -239,6 +249,11 @@ virCapsDispose(void *object)
 virCapsHostCacheBankFree(caps->host.caches[i]);
 VIR_FREE(caps->host.caches);
 
+for (i = 0; i < caps->host.nnodes; i++)
+virCapsHostMemBWNodeFree(caps->host.nodes[i]);
+VIR_FREE(caps->host.nodes);
+
+
 VIR_FREE(caps->host.netprefix);
 VIR_FREE(caps->host.pagesSize);
 virCPUDefFree(caps->host.cpu);
@@ -957,6 +972,58 @@ virCapabilitiesFormatCaches(virBufferPtr buf,
 return 0;
 }
 
+static int
+virCapabilitiesFormatMemoryBandwidth(virBufferPtr buf,
+ size_t nnodes,
+ virCapsHostMemBWNodePtr *nodes)
+{
+size_t i = 0;
+virBuffer controlBuf = VIR_BUFFER_INITIALIZER;
+
+if (!nnodes)
+return 0;
+
+virBufferAddLit(buf, "\n");
+virBufferAdjustIndent(buf, 2);
+
+for (i = 0; i < nnodes; i++) {
+virCapsHostMemBWNodePtr node = nodes[i];
+virResctrlInfoMemBWPerNodePtr control = &node->control;
+char *cpus_str = virBitmapFormat(node->cpus);
+
+if (!cpus_str)
+return -1;
+
+virBufferAsprintf(buf,
+  "id, cpus_str);
+VIR_FREE(cpus_str);
+
+virBufferSetChildIndent(&controlBuf, buf);
+virBufferAsprintf(&controlBuf,
+  "\n",
+  control->granularity, control->min,
+  control->max_allocation);
+
+if (virBufferCheckError(&controlBuf) < 0)
+return -1;
+
+if (virBufferUse(&controlBuf)) {
+virBufferAddLit(buf, ">\n");
+virBufferAddBuffer(buf, &controlBuf);
+virBufferAddLit(buf, "\n");
+} else {
+virBufferAddLit(buf, "/>\n");
+}
+}
+
+virBufferAdjustIndent(buf, -2);
+virBufferAddLit(buf, "\n");
+
+return 0;
+}
+
 /**
  * virCapabilitiesFormatXML:
  * @caps: capabilities to format
@@ -1060,6 +1127,10 @@ virCapabilitiesFormatXML(virCapsPtr caps)
 caps->host.caches) < 0)
 goto error;
 
+if (virCapabilitiesFormatMemoryBandwidth(&buf, caps->host.nnodes,
+ caps->host.nodes) < 0)
+goto error;
+
 for (i = 0; i < caps->host.nsecModels; i++) {
 virBufferAddLit(&buf, "\n");
 virBufferAdjustIndent(&buf, 2);
@@ -1602,6 +1673,40 @@ virCapabilitiesInitResctrl(virCapsPtr caps)
 }
 
 
+static int
+virCapabilitiesInitR

[libvirt] [PATCH 0/9] Introduce RDT memory bandwidth allocation support

2018-07-18 Thread bing . niu
From: Bing Niu 

This series is to introduce RDT memory bandwidth allocation support by extending
current virresctrl implementation.

The Memory Bandwidth Allocation (MBA) feature provides indirect and approximate
control over memory bandwidth available per-core. This feature provides a 
method to
control applications which may be over-utilizing bandwidth relative to their 
priority
in environments such as the data-center. The details can be found in Intel's 
SDM 17.19.7.
Kernel supports MBA through resctrl file system same as CAT. Each resctrl group 
have a
MB parameter to control how much memory bandwidth it can utilize in unit of 
percentage.

In this series, MBA is enabled by enhancing existing virresctrl implementation. 
The
policy employed for MBA is similar with CAT: The sum of each MBA group's 
bandwidth
dose not exceed 100%.

The enhancement of virresctrl include two main parts:

Part 1:  Add two new structures virResctrlInfoMemMB and virResctrlAllocMemBW 
for collecting
 host system MBA capability and domain memory bandwidth allocation. 
Those two
 structures are the extension of existing virResctrlInfo and 
virResctrlAlloc. With
 them, virresctrl framework can support MBA and CAT concurrently. Each 
virResctrlAlloc
 represent a resource allocation including CAT, or MBA, or CAT&MBA. The 
policy of MBA is
 that: total memory bandwidth of each resctrl group, created by 
virresctrl, does not
 exceed to 100%.

Part 2:  On XML part, add new elements to host capabilities query and domain 
allocation to support
 memory bandwidth allocation.
 
-
 For host capabilities XML, new XML format like below example,
   
.
 
   
 
   
 
   

   granularity --- memory bandwidth granularity
   min --- minimum memory bandwidth allowed
   maxAllocs   --- maximum concurrent memory bandwidth allocation 
allowed.

 
-
 For domain XML, new format as below example
   
 ..
 
   ..
   1024
   
 
   
 
 ..
   

  id --- node where memory bandwidth allocation will happen
  bandwidth  --- bandwidth allocated in percentage

 
--

With this extension of the virresctrl, the overall working follow of CAT and 
MBA is described by below
picture. XML parser will aggregate MBA and CAT configuration and represents it 
in one virresctrl object.
The methods of virresctrl class will manipulate resctrl interface to allocate 
corresponding resources.


 
+-+
  |
XML   |   +
   parser +---+
  |
  |
 +--+
  |
  |
internal object+--v--+  
virResctrlAlloc|   backing object|
   +--+--+
  |
  |
 +--+
  |
   +--v---+
   |  |
   | schemata |
 /sys/fs/resctrl   | tasks|
   |   .  |
   |   .  |
   |  |
   +--+
-


previous RFC versions and discussion can be found at
v2: https://www.redhat.com/archives/libvir-list/2018-June/msg01268.html
v1: https://www.redhat.com/archives/libvir-list/2018-May/msg02101.html

Changelog:
   RFC_v2->this: John's comment: 1. use name MemBW to replace MB for a more 
clear description.
 2. split rename patch and put refactor 
function part separately.
 3. split virResctrlInfoMemMB and 
virResctrlAllocMemBW to different
patches.
 4. add docs/schemas/*.rng for XML related 
patches.
 5. some cleanup for coding conventions.
   RFC_ v1->RFC_v2:
Jano's comment: 1. put renaming parts into separated patches.
2. set the initial return value as -1.
3. using full name in structure definition.

[libvirt] [PATCH 4/9] util: Add MBA capability information query to resctrl

2018-07-18 Thread bing . niu
From: Bing Niu 

Introducing virResctrlInfoMemBW for the information memory bandwidth
allocation information. Those information is used for memory
bandwidth allocating.

Signed-off-by: Bing Niu 
---
 src/util/virresctrl.c | 104 ++
 1 file changed, 104 insertions(+)

diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c
index 1515de2..06e2702 100644
--- a/src/util/virresctrl.c
+++ b/src/util/virresctrl.c
@@ -80,6 +80,9 @@ typedef virResctrlInfoPerType *virResctrlInfoPerTypePtr;
 typedef struct _virResctrlInfoPerLevel virResctrlInfoPerLevel;
 typedef virResctrlInfoPerLevel *virResctrlInfoPerLevelPtr;
 
+typedef struct _virResctrlInfoMemBW virResctrlInfoMemBW;
+typedef virResctrlInfoMemBW *virResctrlInfoMemBWPtr;
+
 typedef struct _virResctrlAllocPerType virResctrlAllocPerType;
 typedef virResctrlAllocPerType *virResctrlAllocPerTypePtr;
 
@@ -116,11 +119,31 @@ struct _virResctrlInfoPerLevel {
 virResctrlInfoPerTypePtr *types;
 };
 
+/* Information about memory bandwidth allocation */
+struct _virResctrlInfoMemBW {
+/* minimum memory bandwidth allowed */
+unsigned int min_bandwidth;
+/* bandwidth granularity */
+unsigned int bandwidth_granularity;
+/* Maximum number of simultaneous allocations */
+unsigned int max_allocation;
+/* level number of last level cache */
+unsigned int last_level_cache;
+/* max id of last level cache, this is used to track
+ * how many last level cache available in host system,
+ * the number of memory bandwidth allocation controller
+ * is identical with last level cache.
+ */
+unsigned int max_id;
+};
+
 struct _virResctrlInfo {
 virObject parent;
 
 virResctrlInfoPerLevelPtr *levels;
 size_t nlevels;
+
+virResctrlInfoMemBWPtr membw_info;
 };
 
 
@@ -146,6 +169,7 @@ virResctrlInfoDispose(void *obj)
 VIR_FREE(level);
 }
 
+VIR_FREE(resctrl->membw_info);
 VIR_FREE(resctrl->levels);
 }
 
@@ -442,6 +466,65 @@ virResctrlGetCacheInfo(virResctrlInfoPtr resctrl, DIR 
*dirp)
 
 
 static int
+virResctrlGetMemoryBandwidthInfo(virResctrlInfoPtr resctrl)
+{
+int ret = -1;
+int rv = -1;
+virResctrlInfoMemBWPtr i_membw = NULL;
+
+/* query memory bandwidth allocation info */
+if (VIR_ALLOC(i_membw) < 0)
+goto cleanup;
+rv = virFileReadValueUint(&i_membw->bandwidth_granularity,
+  SYSFS_RESCTRL_PATH "/info/MB/bandwidth_gran");
+if (rv == -2) {
+/* The file doesn't exist, so it's unusable for us,
+ *  properly mba unsupported */
+VIR_INFO("The path '" SYSFS_RESCTRL_PATH "/info/MB/bandwidth_gran'"
+ "does not exist");
+ret = 0;
+goto cleanup;
+} else if (rv < 0) {
+/* Other failures are fatal, so just quit */
+goto cleanup;
+}
+
+rv = virFileReadValueUint(&i_membw->min_bandwidth,
+  SYSFS_RESCTRL_PATH "/info/MB/min_bandwidth");
+if (rv == -2) {
+/* If the previous file exists, so should this one.  Hence -2 is
+ * fatal in this case (errors out in next condition) - the
+ * kernel interface might've changed too much or something else is
+ * wrong. */
+
+virReportError(VIR_ERR_INTERNAL_ERROR, "%s",
+   _("Cannot get min bandwidth from resctrl memory info"));
+}
+if (rv < 0)
+goto cleanup;
+
+rv = virFileReadValueUint(&i_membw->max_allocation,
+  SYSFS_RESCTRL_PATH "/info/MB/num_closids");
+if (rv == -2) {
+/* If the previous file exists, so should this one.  Hence -2 is
+ * fatal in this case as well (errors out in next condition) - the
+ * kernel interface might've changed too much or something else is
+ * wrong. */
+virReportError(VIR_ERR_INTERNAL_ERROR, "%s",
+   _("Cannot get max allocation from resctrl memory 
info"));
+}
+if (rv < 0)
+goto cleanup;
+
+VIR_STEAL_PTR(resctrl->membw_info, i_membw);
+ret = 0;
+ cleanup:
+VIR_FREE(i_membw);
+return ret;
+}
+
+
+static int
 virResctrlGetInfo(virResctrlInfoPtr resctrl)
 {
 DIR *dirp = NULL;
@@ -451,6 +534,10 @@ virResctrlGetInfo(virResctrlInfoPtr resctrl)
 if (ret <= 0)
 goto cleanup;
 
+ret = virResctrlGetMemoryBandwidthInfo(resctrl);
+if (ret < 0)
+goto cleanup;
+
 ret = virResctrlGetCacheInfo(resctrl, dirp);
 if (ret < 0)
 goto cleanup;
@@ -492,6 +579,9 @@ virResctrlInfoIsEmpty(virResctrlInfoPtr resctrl)
 if (!resctrl)
 return true;
 
+if (resctrl->membw_info)
+return false;
+
 for (i = 0; i < resctrl->nlevels; i++) {
 virResctrlInfoPerLevelPtr i_level = resctrl->levels[i];
 
@@ -517,12 +607,26 @@ virResctrlInfoGetCache(virResctrlInfoPtr resctrl,
 {
 virResctrlInfoPerLevelPtr i_level = NULL;
 virResctrlInfoPerTypePtr i_type = NULL;
+v

[libvirt] [PATCH 8/9] conf: Introduce cputune/memorytune to support memory bandwidth allocation

2018-07-18 Thread bing . niu
From: Bing Niu 

Introduce a new section memorytune to support memory bandwidth allocation.
This is consistent with existing cachetune. As the example:
below:
  
..

  

  

vpus  --- vpus subjected to this memory bandwidth.
id--- on which node memory bandwidth to be set.
bandwidth --- the memory bandwidth percent to set.

Signed-off-by: Bing Niu 
---
 docs/formatdomain.html.in  |  35 
 docs/schemas/domaincommon.rng  |  17 ++
 src/conf/domain_conf.c | 199 +
 .../memorytune-colliding-allocs.xml|  30 
 .../memorytune-colliding-cachetune.xml |  32 
 tests/genericxml2xmlindata/memorytune.xml  |  33 
 tests/genericxml2xmltest.c |   5 +
 7 files changed, 351 insertions(+)
 create mode 100644 tests/genericxml2xmlindata/memorytune-colliding-allocs.xml
 create mode 100644 
tests/genericxml2xmlindata/memorytune-colliding-cachetune.xml
 create mode 100644 tests/genericxml2xmlindata/memorytune.xml

diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in
index a3afe13..4e38446 100644
--- a/docs/formatdomain.html.in
+++ b/docs/formatdomain.html.in
@@ -757,6 +757,10 @@
   
   
 
+
+  
+
+
   
   ...
 
@@ -950,7 +954,38 @@
 
   
 
+  
 
+  memorytune
+  
+Optional memorytune element can control allocations for
+memory bandwidth using the resctrl on the host. Whether or not is this
+supported can be gathered from capabilities where some limitations like
+minimum bandwidth and required granularity are reported as well. The
+required attribute vcpus specifies to which vCPUs this
+allocation applies. A vCPU can only be member of one
+memorytune element allocations. vcpus 
specified
+by memorytune can be identical to those specified by
+cachetune. However they are not allowed to overlap each 
other.
+Supported subelements are:
+
+  node
+  
+This element controls the allocation of CPU memory bandwidth and 
has the
+following attributes:
+
+  id
+  
+Host node id from which to allocate memory bandwidth.
+  
+  bandwidth
+  
+The memory bandwidth to allocate from this node. The value by 
default
+is in percentage.
+  
+
+  
+
   
 
 
diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng
index f24a563..b4cd96b 100644
--- a/docs/schemas/domaincommon.rng
+++ b/docs/schemas/domaincommon.rng
@@ -983,6 +983,23 @@
 
   
 
+
+  
+
+  
+
+
+  
+
+  
+
+
+  
+
+  
+
+  
+
   
 
   
diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
index 695981c..ea9276f 100644
--- a/src/conf/domain_conf.c
+++ b/src/conf/domain_conf.c
@@ -19139,6 +19139,128 @@ virDomainCachetuneDefParse(virDomainDefPtr def,
 }
 
 
+static int
+virDomainMemorytuneDefParseMemory(xmlXPathContextPtr ctxt,
+  xmlNodePtr node,
+  virResctrlAllocPtr alloc)
+{
+xmlNodePtr oldnode = ctxt->node;
+unsigned int id;
+unsigned int bandwidth;
+char *tmp = NULL;
+int ret = -1;
+
+ctxt->node = node;
+
+tmp = virXMLPropString(node, "id");
+if (!tmp) {
+virReportError(VIR_ERR_XML_ERROR, "%s",
+   _("Missing memorytune attribute 'id'"));
+goto cleanup;
+}
+if (virStrToLong_uip(tmp, NULL, 10, &id) < 0) {
+virReportError(VIR_ERR_XML_ERROR,
+   _("Invalid memorytune attribute 'id' value '%s'"),
+   tmp);
+goto cleanup;
+}
+VIR_FREE(tmp);
+
+tmp = virXMLPropString(node, "bandwidth");
+if (!tmp) {
+virReportError(VIR_ERR_XML_ERROR, "%s",
+   _("Missing memorytune attribute 'bandwidth'"));
+goto cleanup;
+}
+if (virStrToLong_uip(tmp, NULL, 10, &bandwidth) < 0) {
+virReportError(VIR_ERR_XML_ERROR,
+   _("Invalid memorytune attribute 'bandwidth' value 
'%s'"),
+   tmp);
+goto cleanup;
+}
+VIR_FREE(tmp);
+if (virResctrlSetMemoryBandwidth(alloc, id, bandwidth) < 0)
+goto cleanup;
+
+ret = 0;
+ cleanup:
+ctxt->node = oldnode;
+ 

[libvirt] [PATCH 3/9] util: Refactor virResctrlAllocFormat of virresctrl

2018-07-18 Thread bing . niu
From: Bing Niu 

Refactor virResctrlAllocFormat so that it is easy to support other
resource allocation technologies.

Signed-off-by: Bing Niu 
---
 src/util/virresctrl.c | 43 ---
 1 file changed, 28 insertions(+), 15 deletions(-)

diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c
index 98e7296..1515de2 100644
--- a/src/util/virresctrl.c
+++ b/src/util/virresctrl.c
@@ -848,17 +848,13 @@ virResctrlAllocGetID(virResctrlAllocPtr alloc)
 }
 
 
-char *
-virResctrlAllocFormat(virResctrlAllocPtr alloc)
+static int
+virResctrlAllocFormatCache(virResctrlAllocPtr alloc, virBufferPtr buf)
 {
-virBuffer buf = VIR_BUFFER_INITIALIZER;
 unsigned int level = 0;
 unsigned int type = 0;
 unsigned int cache = 0;
 
-if (!alloc)
-return NULL;
-
 for (level = 0; level < alloc->nlevels; level++) {
 virResctrlAllocPerLevelPtr a_level = alloc->levels[level];
 
@@ -871,7 +867,7 @@ virResctrlAllocFormat(virResctrlAllocPtr alloc)
 if (!a_type)
 continue;
 
-virBufferAsprintf(&buf, "L%u%s:", level, 
virResctrlTypeToString(type));
+virBufferAsprintf(buf, "L%u%s:", level, 
virResctrlTypeToString(type));
 
 for (cache = 0; cache < a_type->nmasks; cache++) {
 virBitmapPtr mask = a_type->masks[cache];
@@ -881,21 +877,38 @@ virResctrlAllocFormat(virResctrlAllocPtr alloc)
 continue;
 
 mask_str = virBitmapToString(mask, false, true);
-if (!mask_str) {
-virBufferFreeAndReset(&buf);
-return NULL;
-}
+if (!mask_str)
+return -1;
 
-virBufferAsprintf(&buf, "%u=%s;", cache, mask_str);
+virBufferAsprintf(buf, "%u=%s;", cache, mask_str);
 VIR_FREE(mask_str);
 }
 
-virBufferTrim(&buf, ";", 1);
-virBufferAddChar(&buf, '\n');
+virBufferTrim(buf, ";", 1);
+virBufferAddChar(buf, '\n');
 }
 }
 
-virBufferCheckError(&buf);
+if (virBufferCheckError(buf) < 0)
+return -1;
+else
+return 0;
+}
+
+
+char *
+virResctrlAllocFormat(virResctrlAllocPtr alloc)
+{
+virBuffer buf = VIR_BUFFER_INITIALIZER;
+
+if (!alloc)
+return NULL;
+
+if (virResctrlAllocFormatCache(alloc, &buf) < 0) {
+virBufferFreeAndReset(&buf);
+return NULL;
+}
+
 return virBufferContentAndReset(&buf);
 }
 
-- 
2.7.4

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list


[libvirt] [PATCH 2/9] util: Refactor virResctrlGetInfo in virresctrl

2018-07-18 Thread bing . niu
From: Bing Niu 

Separate resctrl common information parts from CAT specific parts,
so that common information parts can be reused among different
resource allocation technologies.

Signed-off-by: Bing Niu 
---
 src/util/virresctrl.c | 31 ++-
 1 file changed, 22 insertions(+), 9 deletions(-)

diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c
index 6d69c8d..98e7296 100644
--- a/src/util/virresctrl.c
+++ b/src/util/virresctrl.c
@@ -318,9 +318,8 @@ virResctrlUnlock(int fd)
 
 /* virResctrlInfo-related definitions */
 static int
-virResctrlGetInfo(virResctrlInfoPtr resctrl)
+virResctrlGetCacheInfo(virResctrlInfoPtr resctrl, DIR *dirp)
 {
-DIR *dirp = NULL;
 char *endptr = NULL;
 char *tmp_str = NULL;
 int ret = -1;
@@ -332,12 +331,6 @@ virResctrlGetInfo(virResctrlInfoPtr resctrl)
 virResctrlInfoPerLevelPtr i_level = NULL;
 virResctrlInfoPerTypePtr i_type = NULL;
 
-rv = virDirOpenIfExists(&dirp, SYSFS_RESCTRL_PATH "/info");
-if (rv <= 0) {
-ret = rv;
-goto cleanup;
-}
-
 while ((rv = virDirRead(dirp, &ent, SYSFS_RESCTRL_PATH "/info")) > 0) {
 VIR_DEBUG("Parsing info type '%s'", ent->d_name);
 if (ent->d_name[0] != 'L')
@@ -443,12 +436,32 @@ virResctrlGetInfo(virResctrlInfoPtr resctrl)
 
 ret = 0;
  cleanup:
-VIR_DIR_CLOSE(dirp);
 VIR_FREE(i_type);
 return ret;
 }
 
 
+static int
+virResctrlGetInfo(virResctrlInfoPtr resctrl)
+{
+DIR *dirp = NULL;
+int ret = -1;
+
+ret = virDirOpenIfExists(&dirp, SYSFS_RESCTRL_PATH "/info");
+if (ret <= 0)
+goto cleanup;
+
+ret = virResctrlGetCacheInfo(resctrl, dirp);
+if (ret < 0)
+goto cleanup;
+
+ret = 0;
+ cleanup:
+VIR_DIR_CLOSE(dirp);
+return ret;
+}
+
+
 virResctrlInfoPtr
 virResctrlInfoNew(void)
 {
-- 
2.7.4

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list


[libvirt] [PATCH 5/9] util: Add MBA allocation to virresctrl

2018-07-18 Thread bing . niu
From: Bing Niu 

Add memory bandwidth allocation support to virresctrl class.
Introducing virResctrlAllocMemBW which is used for allocating memory
bandwidth. Following virResctrlAllocPerType, it also employs a
nested sparse array to indicate whether allocation is available for
particular last level cache.

Signed-off-by: Bing Niu 
---
 src/util/virresctrl.c | 346 --
 src/util/virresctrl.h |  13 ++
 2 files changed, 346 insertions(+), 13 deletions(-)

diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c
index 06e2702..bec2afd 100644
--- a/src/util/virresctrl.c
+++ b/src/util/virresctrl.c
@@ -36,9 +36,9 @@ VIR_LOG_INIT("util.virresctrl")
 
 
 /* Resctrl is short for Resource Control.  It might be implemented for various
- * resources, but at the time of this writing this is only supported for cache
- * allocation technology (aka CAT).  Hence the reson for leaving 'Cache' out of
- * all the structure and function names for now (can be added later if needed.
+ * resources. Currently this supports cache allocation technology (aka CAT) and
+ * memory bandwidth allocation (aka MBA). More resources technologies may be
+ * added in feature.
  */
 
 
@@ -89,6 +89,8 @@ typedef virResctrlAllocPerType *virResctrlAllocPerTypePtr;
 typedef struct _virResctrlAllocPerLevel virResctrlAllocPerLevel;
 typedef virResctrlAllocPerLevel *virResctrlAllocPerLevelPtr;
 
+typedef struct _virResctrlAllocMemBW virResctrlAllocMemBW;
+typedef virResctrlAllocMemBW *virResctrlAllocMemBWPtr;
 
 /* Class definitions and initializations */
 static virClassPtr virResctrlInfoClass;
@@ -181,7 +183,10 @@ virResctrlInfoDispose(void *obj)
  * consequently a directory under /sys/fs/resctrl).  Since it can have multiple
  * parts of multiple caches allocated it is represented as bunch of nested
  * sparse arrays (by sparse I mean array of pointers so that each might be NULL
- * in case there is no allocation for that particular one (level, cache, ...)).
+ * in case there is no allocation for that particular cache allocation (level,
+ * cache, ...) or memory allocation for particular node).
+ *
+ * =Cache allocation technology (CAT)=
  *
  * Since one allocation can be made for caches on different levels, the first
  * nested sparse array is of types virResctrlAllocPerLevel.  For example if you
@@ -206,6 +211,16 @@ virResctrlInfoDispose(void *obj)
  * all of them.  While doing that we store the bitmask in a sparse array of
  * virBitmaps named `masks` indexed the same way as `sizes`.  The upper bounds
  * of the sparse arrays are stored in nmasks or nsizes, respectively.
+ *
+ * =Memory Bandwidth allocation technology (MBA)=
+ *
+ * The memory bandwidth allocation support in virResctrlAlloc works in the same
+ * fashion as CAT. However, memory bandwidth controller doesn't have a 
hierarchy
+ * organization as cache, each node have one memory bandwidth controller to
+ * memory bandwidth distribution. The number of memory bandwidth controller is
+ * identical with number of last level cache. So MBA also employs a sparse 
array
+ * to represent whether a memory bandwidth allocation happens on corresponding 
node.
+ * The available memory controller number is collected in 'virResctrlInfo'.
  */
 struct _virResctrlAllocPerType {
 /* There could be bool saying whether this is set or not, but since 
everything
@@ -226,12 +241,24 @@ struct _virResctrlAllocPerLevel {
  * VIR_CACHE_TYPE_LAST number of items */
 };
 
+/*
+ * virResctrlAllocMemBW represents one memory bandwidth allocation. Since it 
can have
+ * several last level caches in a NUMA system, it is also represented as a 
nested
+ * sparse arrays as virRestrlAllocPerLevel.
+ */
+struct _virResctrlAllocMemBW {
+unsigned int **bandwidths;
+size_t nbandwidths;
+};
+
 struct _virResctrlAlloc {
 virObject parent;
 
 virResctrlAllocPerLevelPtr *levels;
 size_t nlevels;
 
+virResctrlAllocMemBWPtr mem_bw;
+
 /* The identifier (any unique string for now) */
 char *id;
 /* libvirt-generated path in /sys/fs/resctrl for this particular
@@ -275,6 +302,13 @@ virResctrlAllocDispose(void *obj)
 VIR_FREE(level);
 }
 
+if (alloc->mem_bw) {
+virResctrlAllocMemBWPtr mem_bw = alloc->mem_bw;
+for (i = 0; i < mem_bw->nbandwidths; i++)
+VIR_FREE(mem_bw->bandwidths[i]);
+}
+
+VIR_FREE(alloc->mem_bw);
 VIR_FREE(alloc->id);
 VIR_FREE(alloc->path);
 VIR_FREE(alloc->levels);
@@ -697,6 +731,9 @@ virResctrlAllocIsEmpty(virResctrlAllocPtr alloc)
 if (!alloc)
 return true;
 
+if (alloc->mem_bw)
+return false;
+
 for (i = 0; i < alloc->nlevels; i++) {
 virResctrlAllocPerLevelPtr a_level = alloc->levels[i];
 
@@ -890,6 +927,27 @@ virResctrlAllocSetCacheSize(virResctrlAllocPtr alloc,
 
 
 int
+virResctrlAllocForeachMemory(virResctrlAllocPtr alloc,
+ virResctrlAllocForeachMemoryCallback cb,
+

[libvirt] [PATCH 6/9] conf: Rename cachetune to restune

2018-07-18 Thread bing . niu
From: Bing Niu 

Resctrl not only supports cache tuning, but also memory bandwidth
tuning. Renaming cachetune to restune(resource tuning) to reflect
that. With restune, all allocation for different resources (cache,
memory bandwidth) are aggregated and represented by a
virResctrlAllocPtr inside virDomainRestuneDef.

Signed-off-by: Bing Niu 
---
 src/conf/domain_conf.c  | 44 ++--
 src/conf/domain_conf.h  | 10 +-
 src/qemu/qemu_domain.c  |  2 +-
 src/qemu/qemu_process.c | 18 +-
 4 files changed, 37 insertions(+), 37 deletions(-)

diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
index f259b4c..24fefd1 100644
--- a/src/conf/domain_conf.c
+++ b/src/conf/domain_conf.c
@@ -2963,14 +2963,14 @@ virDomainLoaderDefFree(virDomainLoaderDefPtr loader)
 
 
 static void
-virDomainCachetuneDefFree(virDomainCachetuneDefPtr cachetune)
+virDomainRestuneDefFree(virDomainRestuneDefPtr restune)
 {
-if (!cachetune)
+if (!restune)
 return;
 
-virObjectUnref(cachetune->alloc);
-virBitmapFree(cachetune->vcpus);
-VIR_FREE(cachetune);
+virObjectUnref(restune->alloc);
+virBitmapFree(restune->vcpus);
+VIR_FREE(restune);
 }
 
 
@@ -3160,9 +3160,9 @@ void virDomainDefFree(virDomainDefPtr def)
 virDomainShmemDefFree(def->shmems[i]);
 VIR_FREE(def->shmems);
 
-for (i = 0; i < def->ncachetunes; i++)
-virDomainCachetuneDefFree(def->cachetunes[i]);
-VIR_FREE(def->cachetunes);
+for (i = 0; i < def->nrestunes; i++)
+virDomainRestuneDefFree(def->restunes[i]);
+VIR_FREE(def->restunes);
 
 VIR_FREE(def->keywrap);
 
@@ -18967,7 +18967,7 @@ virDomainCachetuneDefParse(virDomainDefPtr def,
 xmlNodePtr *nodes = NULL;
 virBitmapPtr vcpus = NULL;
 virResctrlAllocPtr alloc = virResctrlAllocNew();
-virDomainCachetuneDefPtr tmp_cachetune = NULL;
+virDomainRestuneDefPtr tmp_restune = NULL;
 char *tmp = NULL;
 char *vcpus_str = NULL;
 char *alloc_id = NULL;
@@ -18980,7 +18980,7 @@ virDomainCachetuneDefParse(virDomainDefPtr def,
 if (!alloc)
 goto cleanup;
 
-if (VIR_ALLOC(tmp_cachetune) < 0)
+if (VIR_ALLOC(tmp_restune) < 0)
 goto cleanup;
 
 vcpus_str = virXMLPropString(node, "vcpus");
@@ -19021,8 +19021,8 @@ virDomainCachetuneDefParse(virDomainDefPtr def,
 goto cleanup;
 }
 
-for (i = 0; i < def->ncachetunes; i++) {
-if (virBitmapOverlaps(def->cachetunes[i]->vcpus, vcpus)) {
+for (i = 0; i < def->nrestunes; i++) {
+if (virBitmapOverlaps(def->restunes[i]->vcpus, vcpus)) {
 virReportError(VIR_ERR_XML_ERROR, "%s",
_("Overlapping vcpus in cachetunes"));
 goto cleanup;
@@ -19052,16 +19052,16 @@ virDomainCachetuneDefParse(virDomainDefPtr def,
 if (virResctrlAllocSetID(alloc, alloc_id) < 0)
 goto cleanup;
 
-VIR_STEAL_PTR(tmp_cachetune->vcpus, vcpus);
-VIR_STEAL_PTR(tmp_cachetune->alloc, alloc);
+VIR_STEAL_PTR(tmp_restune->vcpus, vcpus);
+VIR_STEAL_PTR(tmp_restune->alloc, alloc);
 
-if (VIR_APPEND_ELEMENT(def->cachetunes, def->ncachetunes, tmp_cachetune) < 
0)
+if (VIR_APPEND_ELEMENT(def->restunes, def->nrestunes, tmp_restune) < 0)
 goto cleanup;
 
 ret = 0;
  cleanup:
 ctxt->node = oldnode;
-virDomainCachetuneDefFree(tmp_cachetune);
+virDomainRestuneDefFree(tmp_restune);
 virObjectUnref(alloc);
 virBitmapFree(vcpus);
 VIR_FREE(alloc_id);
@@ -26913,7 +26913,7 @@ virDomainCachetuneDefFormatHelper(unsigned int level,
 
 static int
 virDomainCachetuneDefFormat(virBufferPtr buf,
-virDomainCachetuneDefPtr cachetune,
+virDomainRestuneDefPtr restune,
 unsigned int flags)
 {
 virBuffer childrenBuf = VIR_BUFFER_INITIALIZER;
@@ -26921,7 +26921,7 @@ virDomainCachetuneDefFormat(virBufferPtr buf,
 int ret = -1;
 
 virBufferSetChildIndent(&childrenBuf, buf);
-virResctrlAllocForeachCache(cachetune->alloc,
+virResctrlAllocForeachCache(restune->alloc,
 virDomainCachetuneDefFormatHelper,
 &childrenBuf);
 
@@ -26934,14 +26934,14 @@ virDomainCachetuneDefFormat(virBufferPtr buf,
 goto cleanup;
 }
 
-vcpus = virBitmapFormat(cachetune->vcpus);
+vcpus = virBitmapFormat(restune->vcpus);
 if (!vcpus)
 goto cleanup;
 
 virBufferAsprintf(buf, "alloc);
+const char *alloc_id = virResctrlAllocGetID(restune->alloc);
 if (!alloc_id)
 goto cleanup;
 
@@ -27062,8 +27062,8 @@ virDomainCputuneDefFormat(virBufferPtr buf,
  def->iothreadids[i]->iothread_id);
 }
 
-for (i = 0; i < def->ncachetunes; i++)
-virDomainCachetuneDefFormat(&childrenBuf, def->cachetunes[i], flags);
+for (i = 0; i < def->nrestunes; i++)
+virDomainCachetuneDefFormat

[libvirt] [PATCHv2] examples: Add clean-traffic-gateway into nwfilters

2018-07-18 Thread Ales Musil
The filter purpose is to simulate isolated private VLAN.

The behavior can be achieved by limiting network traffic
to traffic between VM and gateway. Because there is no
concept of the PVLAN in the linux bridge.

The filter also contains parts from clean-traffic
to prevent VM from spoofing its IP and MAC address.

To use this filter the user just needs to set
the GATEWAY_MAC variable to gateway MAC address.

Signed-off-by: Ales Musil 
---
 examples/xml/nwfilter/clean-traffic-gateway.xml | 34 +
 1 file changed, 34 insertions(+)
 create mode 100644 examples/xml/nwfilter/clean-traffic-gateway.xml

diff --git a/examples/xml/nwfilter/clean-traffic-gateway.xml 
b/examples/xml/nwfilter/clean-traffic-gateway.xml
new file mode 100644
index 000..b8c2040
--- /dev/null
+++ b/examples/xml/nwfilter/clean-traffic-gateway.xml
@@ -0,0 +1,34 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
-- 
1.8.3.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list


Re: [libvirt] [PATCH 2/2] docs: formatdomain: clarify period cputune subelement

2018-07-18 Thread Erik Skultety
On Tue, Jul 17, 2018 at 06:01:50PM +0200, Fabiano Fidêncio wrote:
> On Tue, Jul 17, 2018 at 5:47 PM, Katerina Koukiou  wrote:
> > Although the name of the element is not self-explanatory,
> > it's affecting only the vcpu threads.
> >
> > Signed-off-by: Katerina Koukiou 
> > ---
> >  docs/formatdomain.html.in | 10 +-
> >  1 file changed, 5 insertions(+), 5 deletions(-)
> >
> > diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in
> > index 7fa448acdc..61930e4907 100644
> > --- a/docs/formatdomain.html.in
> > +++ b/docs/formatdomain.html.in
> > @@ -831,11 +831,11 @@
> >
> >  The optional quota element specifies the maximum 
> > allowed
> >  bandwidth(unit: microseconds). A domain with quota as 
> > any
> > -negative value indicates that the domain has infinite bandwidth, 
> > which
> > -means that it is not bandwidth controlled. The value should be in 
> > range
> > -[1000, 18446744073709551] or less than 0. A quota with value 0 
> > means no
> > -value. You can use this feature to ensure that all vcpus run at 
> > the same
> > -speed.
> > +negative value indicates that the domain has infinite bandwidth for
> > +VCPU threads, which means that it is not bandwidth controlled. The 
> > value
>
> Just a nitpick here.
> I'd go for vCPU instead of VCPU.
>
> I know both terms are used, it's just that vCPU seems to be used more
> often than VCPU:
> [ffidenci@pessoa libvirt]$ grep --color "vCPU" docs/formatdomain.html.in | wc 
> -l
> 10
> [ffidenci@pessoa libvirt]$ grep --color "VCPU" docs/formatdomain.html.in | wc 
> -l
> 5
>
> I'd follow up with an additional patch just to standardize the name
> for the most used one.

even "vcpu" would be fine, but I agree with Fabiano.

If you adjust that, you have my
Reviewed-by: Erik Skultety 

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH 2/2] docs: formatdomain: clarify period cputune subelement

2018-07-18 Thread Katerina Koukiou
On Wed, Jul 18, 2018 at 10:05:30AM +0200, Ján Tomko wrote:
> On Tue, Jul 17, 2018 at 05:47:32PM +0200, Katerina Koukiou wrote:
> > Although the name of the element is not self-explanatory,
> > it's affecting only the vcpu threads.
> > 
> > Signed-off-by: Katerina Koukiou 
> > ---
> > docs/formatdomain.html.in | 10 +-
> > 1 file changed, 5 insertions(+), 5 deletions(-)
> > 
> > diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in
> > index 7fa448acdc..61930e4907 100644
> > --- a/docs/formatdomain.html.in
> > +++ b/docs/formatdomain.html.in
> > @@ -831,11 +831,11 @@
> >   
> > The optional quota element specifies the maximum 
> > allowed
> > bandwidth(unit: microseconds). A domain with quota as 
> > any
> 
> There is a missing space before the parenthesis here, and it's copied
> multiple times for all the similar attributes. It's a good indicator
> that we should only write the repetititive text once, and collect the
> differences in a neat table, e.g.:
> 
> +---++-+-+
> + quota element | period element | affects | since   |
> +---++-+-+
> + quota | period | vCPUs   | 0.9.4, LXC since 0.9.10 |
> + global_guota  | global_period  | all | ... |
> +---++-+-+
> 
> (But, of course, even this change is better than nothing)

Yes, this approach looks indeed cleaner. I 'll merge this patch with
Fabianos/Erics comment and I 'll keep moving to table format as a followup.

> 
> Jano
> 
> > -negative value indicates that the domain has infinite bandwidth, 
> > which
> > -means that it is not bandwidth controlled. The value should be in 
> > range
> > -[1000, 18446744073709551] or less than 0. A quota with value 0 
> > means no
> > -value. You can use this feature to ensure that all vcpus run at 
> > the same
> > -speed.




signature.asc
Description: PGP signature
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [jenkins-ci PATCH v2 10/12] lcitool: Add projects information handling

2018-07-18 Thread Andrea Bolognani
On Tue, 2018-07-17 at 19:55 +0200, Katerina Koukiou wrote:
> I meant only having a variable for the playbook location.
> 
> playbook_path = os.getenv('PLAYBOOK_PATH', './'))
> 
> And then use os.path.join for all the others relative paths to the
> playbook path.
> So that you can actually run the lcitool script from wherever you want.

That wouldn't be quite enough: you'd also have to make sure you
call ansible / ansible-playbook like

  ANSIBLE_CONFIG="$PLAYBOOK_PATH/ansible.cfg" \
  ansible --playbook-dir "$PLAYBOOK_PATH" ...

otherwise it won't work. And at that point, the script might as
well just figure out the base directory itself based on its own
location :)

Should be easy enough. I'll look into it.

-- 
Andrea Bolognani / Red Hat / Virtualization

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list


[libvirt] CPU Support

2018-07-18 Thread Hetz Ben Hamo
Hi,

I've been looking at the CPU list and although I see lots of CPU's, I
cannot find 2 CPU families:

* AMD Ryzen
* AMD Threadripper

Although EPYC has been added recently.

Are there any missing details which preventing adding those CPU's to the
list?

Thanks
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [dbus PATCH] tests: Explicitly spawn a session libvirt-dbus instance

2018-07-18 Thread Andrea Bolognani
Tests are performed using the session D-Bus instance, so we
should launch libvirt-dbus in session mode as well.

This was working fine when running the tests as a regular
user, because in that case libvirt-dbus would default to
session mode, but fail when running them as root because
libvirt-dbus would run in system mode and consequently not
show up on the session bus.

Of course building and runnning the test suite as root is
a pretty bad idea in general, but a lot of distributions
run at least part of their package build steps with pretend
root privileges (eg. fakeroot), so we have to make sure it
works in that scenario too.

Signed-off-by: Andrea Bolognani 
---
 tests/libvirttest.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/libvirttest.py b/tests/libvirttest.py
index 3741abd..4653055 100644
--- a/tests/libvirttest.py
+++ b/tests/libvirttest.py
@@ -33,7 +33,7 @@ class BaseTestClass():
 """Start libvirt-dbus for each test function
 """
 os.environ['LIBVIRT_DEBUG'] = '3'
-self.libvirt_dbus = subprocess.Popen([exe])
+self.libvirt_dbus = subprocess.Popen([exe, '--session'])
 self.bus = dbus.SessionBus()
 
 for i in range(10):
-- 
2.17.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list


Re: [libvirt] CPU Support

2018-07-18 Thread Daniel P . Berrangé
On Wed, Jul 18, 2018 at 12:41:48PM +0300, Hetz Ben Hamo wrote:
> Hi,
> 
> I've been looking at the CPU list and although I see lots of CPU's, I
> cannot find 2 CPU families:
> 
> * AMD Ryzen
> * AMD Threadripper
> 
> Although EPYC has been added recently.
> 
> Are there any missing details which preventing adding those CPU's to the
> list?

Libvirt adds CPU models based on what QEMU supports. So from libvirt side the
answer is simply that QEMU doesn't expose any models for Ryzen/Threadripper,
but I'm not clear why it doesn't...

For a while I thought Ryzen/Threadripper would have same feature set as
EPYC, but I've seen bugs recently suggesting that is not in fact the
case. So it does look like having those models exposed by QEMU might
be useful.

Copy'ing QEMU devel & the CPU model maintainers for opinions.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list


[libvirt] [PATCH] docs: formatdomain: unify naming for CPUs/vCPUs

2018-07-18 Thread Katerina Koukiou
CPU is an acronym and should be written in uppercase
when part of plain text and not refering to an element.

Signed-off-by: Katerina Koukiou 
---

As asked in the review here
https://www.redhat.com/archives/libvir-list/2018-July/msg01093.html

 docs/formatdomain.html.in | 84 +++
 1 file changed, 42 insertions(+), 42 deletions(-)

diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in
index b00971a945..d08ede9ab5 100644
--- a/docs/formatdomain.html.in
+++ b/docs/formatdomain.html.in
@@ -631,45 +631,45 @@
   
   vcpus
   
-The vcpus element allows to control state of individual vcpus.
+The vcpus element allows to control state of individual vCPUs.
 
 The id attribute specifies the vCPU id as used by libvirt
-in other places such as vcpu pinning, scheduler information and NUMA
-assignment. Note that the vcpu ID as seen in the guest may differ from
-libvirt ID in certain cases. Valid IDs are from 0 to the maximum vcpu
+in other places such as vCPU pinning, scheduler information and NUMA
+assignment. Note that the vCPU ID as seen in the guest may differ from
+libvirt ID in certain cases. Valid IDs are from 0 to the maximum vCPU
 count as set by the vcpu element minus 1.
 
 The enabled attribute allows to control the state of the
-vcpu. Valid values are yes and no.
+vCPU. Valid values are yes and no.
 
-hotpluggable controls whether given vcpu can be hotplugged
-and hotunplugged in cases when the cpu is enabled at boot. Note that
-all disabled vcpus must be hotpluggable. Valid values are
+hotpluggable controls whether given vCPU can be hotplugged
+and hotunplugged in cases when the CPU is enabled at boot. Note that
+all disabled vCPUs must be hotpluggable. Valid values are
 yes and no.
 
-order allows to specify the order to add the online vcpus.
-For hypervisors/platforms that require to insert multiple vcpus at once
-the order may be duplicated across all vcpus that need to be
-enabled at once. Specifying order is not necessary, vcpus are then
+order allows to specify the order to add the online vCPUs.
+For hypervisors/platforms that require to insert multiple vCPUs at once
+the order may be duplicated across all vCPUs that need to be
+enabled at once. Specifying order is not necessary, vCPUs are then
 added in an arbitrary order. If order info is used, it must be used for
-all online vcpus. Hypervisors may clear or update ordering information
+all online vCPUs. Hypervisors may clear or update ordering information
 during certain operations to assure valid configuration.
 
-Note that hypervisors may create hotpluggable vcpus differently from
-boot vcpus thus special initialization may be necessary.
+Note that hypervisors may create hotpluggable vCPUs differently from
+boot vCPUs thus special initialization may be necessary.
 
-Hypervisors may require that vcpus enabled on boot which are not
+Hypervisors may require that vCPUs enabled on boot which are not
 hotpluggable are clustered at the beginning starting with ID 0. It may
-be also required that vcpu 0 is always present and non-hotpluggable.
+be also required that vCPU 0 is always present and non-hotpluggable.
 
-Note that providing state for individual cpus may be necessary to 
enable
+Note that providing state for individual CPUs may be necessary to 
enable
 support of addressable vCPU hotplug and this feature may not be
 supported by all hypervisors.
 
-For QEMU the following conditions are required. Vcpu 0 needs to be
-enabled and non-hotpluggable. On PPC64 along with it vcpus that are in
-the same core need to be enabled as well. All non-hotpluggable cpus
-present at boot need to be grouped after vcpu 0.
+For QEMU the following conditions are required. vCPU 0 needs to be
+enabled and non-hotpluggable. On PPC64 along with it vCPUs that are in
+the same core need to be enabled as well. All non-hotpluggable CPUs
+present at boot need to be grouped after vCPU 0.
 Since 2.2.0 (QEMU only)
   
 
@@ -774,11 +774,11 @@
   vcpupin
   
 The optional vcpupin element specifies which of host's
-physical CPUs the domain VCPU will be pinned to. If this is omitted,
+physical CPUs the domain vCPU will be pinned to. If this is omitted,
 and attribute cpuset of element vcpu is
 not specified, the vCPU is pinned to all the physical CPUs by default.
 It contains two required attributes, the attribute vcpu
-specifies vcpu id, and the attribute cpuset is same as
+specifies vCPU id, and the attribute cpuset is same

Re: [libvirt] CPU Support

2018-07-18 Thread Pavel Hrdina
On Wed, Jul 18, 2018 at 10:50:34AM +0100, Daniel P. Berrangé wrote:
> On Wed, Jul 18, 2018 at 12:41:48PM +0300, Hetz Ben Hamo wrote:
> > Hi,
> > 
> > I've been looking at the CPU list and although I see lots of CPU's, I
> > cannot find 2 CPU families:
> > 
> > * AMD Ryzen
> > * AMD Threadripper
> > 
> > Although EPYC has been added recently.
> > 
> > Are there any missing details which preventing adding those CPU's to the
> > list?
> 
> Libvirt adds CPU models based on what QEMU supports. So from libvirt side the
> answer is simply that QEMU doesn't expose any models for Ryzen/Threadripper,
> but I'm not clear why it doesn't...
> 
> For a while I thought Ryzen/Threadripper would have same feature set as
> EPYC, but I've seen bugs recently suggesting that is not in fact the
> case. So it does look like having those models exposed by QEMU might
> be useful.
> 
> Copy'ing QEMU devel & the CPU model maintainers for opinions.

I think that QEMU should figure out some pattern for naming CPU models
because it's one big mess.  EPYC and Ryzen are bad names for QEMU as
Core/Xeon would be for Intel CPUs.  It's the name of a model families
and it will probably remain the same but with different
microarchitecture.

Better name would be similarly like for the latest Inter CPUs,
Skylake-Client and Skylake-Server.  Currently AMD has already two
microarchitectures, Zen and Zen+ and there is third one Zen 2 planned.

Zen has AMD Ryzen, AMD Ryzen Threadripper and AMD Epyc.
Zen+ has AMD Ryzen, AMD Ryzen Threadripper

And I bet that Zen 2 will follow the same model families.

We probably cannot rename EPYC now, but before we introduce Ryzen and
Threadripper let's thing about it and come up with better names, for
example Zen-Client/Zen-Server Zen+-Client or something like that.

Pavel


signature.asc
Description: PGP signature
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [dbus PATCH] tests: Explicitly spawn a session libvirt-dbus instance

2018-07-18 Thread Katerina Koukiou
On Wed, Jul 18, 2018 at 11:50:33AM +0200, Andrea Bolognani wrote:
> Tests are performed using the session D-Bus instance, so we
> should launch libvirt-dbus in session mode as well.
> 
> This was working fine when running the tests as a regular
> user, because in that case libvirt-dbus would default to
> session mode, but fail when running them as root because
> libvirt-dbus would run in system mode and consequently not
> show up on the session bus.
> 
> Of course building and runnning the test suite as root is
> a pretty bad idea in general, but a lot of distributions
> run at least part of their package build steps with pretend
> root privileges (eg. fakeroot), so we have to make sure it
> works in that scenario too.
> 
> Signed-off-by: Andrea Bolognani 
> ---
>  tests/libvirttest.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 

Reviewed-by: Katerina Koukiou 


signature.asc
Description: PGP signature
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATCH] spec: remove libcgroup and cgconfig

2018-07-18 Thread Pavel Hrdina
RHEL-6/CentOS-6 is no longer supported, let's remove dependency on
libcgroup and code that enables/starts cgconfig service.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1602407

Signed-off-by: Pavel Hrdina 
---
 libvirt.spec.in | 19 ---
 1 file changed, 19 deletions(-)

diff --git a/libvirt.spec.in b/libvirt.spec.in
index 6f0d399064..6f360c5579 100644
--- a/libvirt.spec.in
+++ b/libvirt.spec.in
@@ -80,7 +80,6 @@
 
 # A few optional bits off by default, we enable later
 %define with_fuse  0%{!?_without_fuse:0}
-%define with_cgconfig  0%{!?_without_cgconfig:0}
 %define with_sanlock   0%{!?_without_sanlock:0}
 %define with_systemd   0%{!?_without_systemd:0}
 %define with_numad 0%{!?_without_numad:0}
@@ -216,11 +215,6 @@
 %endif
 %endif
 
-# Pull in cgroups config system
-%if %{with_qemu} || %{with_lxc}
-%define with_cgconfig 0%{!?_without_cgconfig:1}
-%endif
-
 # Force QEMU to run as non-root
 %define qemu_user  qemu
 %define qemu_group  qemu
@@ -496,9 +490,6 @@ Requires: polkit >= 0.112
 %else
 Requires: polkit >= 0.93
 %endif
-%if %{with_cgconfig}
-Requires: libcgroup
-%endif
 %ifarch %{ix86} x86_64 ia64
 # For virConnectGetSysinfo
 Requires: dmidecode
@@ -1490,16 +1481,6 @@ if [ $1 -eq 1 ] ; then
 fi
 %endif
 %else
-%if %{with_cgconfig}
-# Starting with Fedora 16/RHEL-7, systemd automounts all cgroups,
-# and cgconfig is no longer a necessary service.
-%if 0%{?rhel} && 0%{?rhel} < 7
-if [ "$1" -eq "1" ]; then
-/sbin/chkconfig cgconfig on
-fi
-%endif
-%endif
-
 /sbin/chkconfig --add libvirtd
 /sbin/chkconfig --add virtlogd
 /sbin/chkconfig --add virtlockd
-- 
2.17.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list


Re: [libvirt] [PATCH] docs: formatdomain: unify naming for CPUs/vCPUs

2018-07-18 Thread Erik Skultety
On Wed, Jul 18, 2018 at 12:09:12PM +0200, Katerina Koukiou wrote:
> CPU is an acronym and should be written in uppercase
> when part of plain text and not refering to an element.
>
> Signed-off-by: Katerina Koukiou 
> ---
>
> As asked in the review here
> https://www.redhat.com/archives/libvir-list/2018-July/msg01093.html

This should be squashed in:
diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in
index d08ede9ab5..679690d060 100644
--- a/docs/formatdomain.html.in
+++ b/docs/formatdomain.html.in
@@ -768,7 +768,7 @@
   cputune
   
  The optional cputune element provides details
- regarding the cpu tunable parameters for the domain.
+ regarding the CPU tunable parameters for the domain.
  Since 0.9.0
   
   vcpupin

Reviewed-by: Erik Skultety 

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list


Re: [libvirt] [PATCH] spec: remove libcgroup and cgconfig

2018-07-18 Thread Daniel P . Berrangé
On Wed, Jul 18, 2018 at 12:40:16PM +0200, Pavel Hrdina wrote:
> RHEL-6/CentOS-6 is no longer supported, let's remove dependency on
> libcgroup and code that enables/starts cgconfig service.
> 
> Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1602407
> 
> Signed-off-by: Pavel Hrdina 
> ---
>  libvirt.spec.in | 19 ---
>  1 file changed, 19 deletions(-)

Reviewed-by: Daniel P. Berrangé 


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH 1/3] add functions: load 8021q module, create/destroy vlan-dev

2018-07-18 Thread Shi Lei



> Sent: Wednesday, July 18, 2018 at 12:12 AM
> From: "John Ferlan" 
> To: "Shi Lei" , libvir-list@redhat.com
> Subject: Re: [libvirt] [PATCH 1/3] add functions: load 8021q module, 
> create/destroy vlan-dev
>
> 
> 
> On 07/05/2018 11:36 PM, Shi Lei wrote:
> > Signed-off-by: Shi Lei 
> > ---
> >  configure.ac |   5 +-
> >  src/libvirt_private.syms |   4 +
> >  src/util/virnetdev.c | 195 
> > +++
> >  src/util/virnetdev.h |  14 
> >  4 files changed, 216 insertions(+), 2 deletions(-)
> > 
> 
> Not exactly in my wheelhouse of knowledge and I suspect when Laine gets
> back from vacation he'd want to look as well, but ... let's see what
> feedback I can provide. I'll probably make a mess, but hopefully its
> understandable.
> 
> First off kudos for actually providing patches first time out that
> compile and pass make check syntax-check!

Thanks for your reviewing, John!
I have thought that my patches have been ignored ...

> 
> > diff --git a/configure.ac b/configure.ac
> > index 59d2d09..141d5b2 100644
> > --- a/configure.ac
> > +++ b/configure.ac
> > @@ -769,8 +769,9 @@ then
> >  fi
> >  AM_CONDITIONAL([WITH_NODE_DEVICES], [test "$with_nodedev" = "yes"])
> >  
> > -dnl GET_VLAN_VID_CMD is required for virNetDevGetVLanID
> 
> Losing this comment for a more generic one probably isn't good.

OK

> 
> > -AC_CHECK_DECLS([GET_VLAN_VID_CMD], [], [], [[#include ]])
> > +dnl GET_VLAN_VID_CMD, ADD_VLAN_CMD, DEL_VLAN_CMD is required
> 
> Rather than combining into one and losing the virNetDevGetVLanID why not
> add separate comments "ADD_VLAN_CMD is required for
> virNetDevCreateVLanDev" and "DEL_VLAN_CMD is required for
> virNetDevDestroyVLanDev".
> 

OK. I'll fix it.

> > +AC_CHECK_DECLS([GET_VLAN_VID_CMD, ADD_VLAN_CMD, DEL_VLAN_CMD],
> > +   [], [], [[#include ]])
> >  
> >  # Check for Linux vs. BSD ifreq members
> >  AC_CHECK_MEMBERS([struct ifreq.ifr_newname,
> > diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms
> > index 3e30490..a61ba02 100644
> > --- a/src/libvirt_private.syms
> > +++ b/src/libvirt_private.syms
> > @@ -2266,7 +2266,9 @@ virModuleLoad;
> >  
> >  # util/virnetdev.h
> >  virNetDevAddMulti;
> > +virNetDevCreateVLanDev;
> >  virNetDevDelMulti;
> > +virNetDevDestroyVLanDev;
> >  virNetDevExists;
> >  virNetDevFeatureTypeFromString;
> >  virNetDevFeatureTypeToString;
> > @@ -2287,10 +2289,12 @@ virNetDevGetRxFilter;
> >  virNetDevGetVirtualFunctionIndex;
> >  virNetDevGetVirtualFunctionInfo;
> >  virNetDevGetVirtualFunctions;
> > +virNetDevGetVLanDevName;
> >  virNetDevGetVLanID;
> >  virNetDevIfStateTypeFromString;
> >  virNetDevIfStateTypeToString;
> >  virNetDevIsVirtualFunction;
> > +virNetDevLoad8021Q;
> >  virNetDevPFGetVF;
> >  virNetDevReadNetConfig;
> >  virNetDevRunEthernetScript;
> > diff --git a/src/util/virnetdev.c b/src/util/virnetdev.c
> > index c20022f..50a81d2 100644
> > --- a/src/util/virnetdev.c
> > +++ b/src/util/virnetdev.c
> > @@ -44,6 +44,7 @@
> >  # include 
> >  # include 
> >  # define VIR_NETDEV_FAMILY AF_UNIX
> > +# include "virkmod.h"
> >  #elif defined(HAVE_STRUCT_IFREQ) && defined(AF_LOCAL)
> >  # define VIR_NETDEV_FAMILY AF_LOCAL
> >  #else
> > @@ -1048,6 +1049,200 @@ int virNetDevGetVLanID(const char *ifname 
> > ATTRIBUTE_UNUSED,
> >  #endif /* ! SIOCGIFVLAN */
> >  
> >  
> > +#if defined(HAVE_STRUCT_IFREQ)
> > +
> > +# define MODULE_8021Q "8021q"
> > +# define PROC_NET_VLAN_CONFIG "/proc/net/vlan/config"
> > +
> > +static int
> > +validate8021Q(void)
> 
> I think this could be replaced in the callers with a:
> 
> virFileExists(PROC_NET_VLAN_CONFIG)
> 
> since all you seem to care is that the file got created. The open here
> is not doing much other than proving that this process could open the
> file. There's nothing be read AFAICT.

Yes. It's better to use virFileExists here. I didn't know it before.

> 
> > +{
> > +int fd;
> > +if ((fd = open(PROC_NET_VLAN_CONFIG, O_RDONLY)) < 0)
> > +return -1;
> > +VIR_FORCE_CLOSE(fd);
> > +return 0;
> > +}
> > +
> 
> Adding new functions should keep 2 empty lines between each.

OK.

> 
> > +static int
> > +getVlanDevName(const char *ifname, unsigned int vlanid,
> > +   char vdname[], size_t vdname_size)
> 
> One line per argument please.

OK.

> 
> Why is it 'char vdname[]' and not 'char **'?

OK. I'll change this.
Generally I prefer to use stack variable if I know the limit of its size.

> 
> > +{
> 
> I think a :
> 
> if (!ifname) {
> virReportError(VIR_ERR_INTERNAL_ERROR, "%s",
>_("Interface name not provided"));
> return -1;
> }
> 
> should be added.

I'll add it.

> 
> > +if (!vdname || vdname_size < IFNAMSIZ)
> > +return -1;
> 
> So if this fails eventually a caller is going to want to know why.
> Adding a virReportError here about invalid arguments would be OK, but
> perhaps unnecessary.  The @vdname is an output that if not

Re: [libvirt] [PATCH 2/3] support new forward mode of vlan for virtual network

2018-07-18 Thread Shi Lei



> Sent: Wednesday, July 18, 2018 at 12:38 AM
> From: "John Ferlan" 
> To: "Shi Lei" , libvir-list@redhat.com
> Subject: Re: [libvirt] [PATCH 2/3] support new forward mode of vlan for 
> virtual network
>
> 
> 
> On 07/05/2018 11:36 PM, Shi Lei wrote:
> > Signed-off-by: Shi Lei 
> > ---
> >  src/conf/network_conf.c | 12 ---
> >  src/conf/network_conf.h |  1 +
> >  src/network/bridge_driver.c | 80 
> > +
> >  3 files changed, 82 insertions(+), 11 deletions(-)
> > 
> 
> This would be missing RNG changes in docs/schemas/network.rng,
> documentation changes in docs/formatnetwork.html.in in order to describe
> things, and networkxml2* tests.  Usually if you look at the last person
> to add a new element to something it'll give you a good hint. In this
> case, see commit id 25e8112d7 when "open" was added. I use gitk to chase
> changes, but others use straight git - whatever works for you.

OK. Thank you! I will add these in patch v2.

> 
> > diff --git a/src/conf/network_conf.c b/src/conf/network_conf.c
> > index 630a87f..6e1de6c 100644
> > --- a/src/conf/network_conf.c
> > +++ b/src/conf/network_conf.c
> > @@ -50,7 +50,7 @@ VIR_ENUM_IMPL(virNetworkForward,
> >VIR_NETWORK_FORWARD_LAST,
> >"none", "nat", "route", "open",
> >"bridge", "private", "vepa", "passthrough",
> > -  "hostdev")
> > +  "hostdev", "vlan")
> >  
> >  VIR_ENUM_IMPL(virNetworkBridgeMACTableManager,
> >VIR_NETWORK_BRIDGE_MAC_TABLE_MANAGER_LAST,
> > @@ -1876,6 +1876,7 @@ virNetworkDefParseXML(xmlXPathContextPtr ctxt)
> >   */
> >  switch (def->forward.type) {
> >  case VIR_NETWORK_FORWARD_NONE:
> > +case VIR_NETWORK_FORWARD_VLAN:
> >  break;
> >  
> >  case VIR_NETWORK_FORWARD_ROUTE:
> > @@ -1963,9 +1964,10 @@ virNetworkDefParseXML(xmlXPathContextPtr ctxt)
> >  (def->forward.type != VIR_NETWORK_FORWARD_NONE &&
> >   def->forward.type != VIR_NETWORK_FORWARD_NAT &&
> >   def->forward.type != VIR_NETWORK_FORWARD_ROUTE &&
> > - def->forward.type != VIR_NETWORK_FORWARD_OPEN)) {
> > + def->forward.type != VIR_NETWORK_FORWARD_OPEN &&
> > + def->forward.type != VIR_NETWORK_FORWARD_VLAN)) {
> >  virReportError(VIR_ERR_XML_ERROR,
> > -   _("mtu size only allowed in open, route, nat, "
> > +   _("mtu size only allowed in open, route, nat, vlan "
> >   "and isolated mode, not in %s (network '%s')"),
> > virNetworkForwardTypeToString(def->forward.type),
> > def->name);
> > @@ -2474,6 +2476,7 @@ virNetworkDefFormatBuf(virBufferPtr buf,
> >  def->forward.type == VIR_NETWORK_FORWARD_NAT ||
> >  def->forward.type == VIR_NETWORK_FORWARD_ROUTE ||
> >  def->forward.type == VIR_NETWORK_FORWARD_OPEN ||
> > +def->forward.type == VIR_NETWORK_FORWARD_VLAN ||
> >  def->bridge || def->macTableManager) {
> >  
> >  virBufferAddLit(buf, " > @@ -2481,7 +2484,8 @@ virNetworkDefFormatBuf(virBufferPtr buf,
> >  if (def->forward.type == VIR_NETWORK_FORWARD_NONE ||
> >  def->forward.type == VIR_NETWORK_FORWARD_NAT ||
> >  def->forward.type == VIR_NETWORK_FORWARD_ROUTE ||
> > -def->forward.type == VIR_NETWORK_FORWARD_OPEN) {
> > +def->forward.type == VIR_NETWORK_FORWARD_OPEN ||
> > +def->forward.type == VIR_NETWORK_FORWARD_VLAN) {
> >  virBufferAsprintf(buf, " stp='%s' delay='%ld'",
> >def->stp ? "on" : "off", def->delay);
> >  }
> > diff --git a/src/conf/network_conf.h b/src/conf/network_conf.h
> > index 54c8ed1..47bb83e 100644
> > --- a/src/conf/network_conf.h
> > +++ b/src/conf/network_conf.h
> > @@ -53,6 +53,7 @@ typedef enum {
> >  VIR_NETWORK_FORWARD_VEPA,
> >  VIR_NETWORK_FORWARD_PASSTHROUGH,
> >  VIR_NETWORK_FORWARD_HOSTDEV,
> > +VIR_NETWORK_FORWARD_VLAN,
> >  
> >  VIR_NETWORK_FORWARD_LAST,
> >  } virNetworkForwardType;
> > diff --git a/src/network/bridge_driver.c b/src/network/bridge_driver.c
> > index da3c32e..314f78c 100644
> > --- a/src/network/bridge_driver.c
> > +++ b/src/network/bridge_driver.c
> > @@ -451,6 +451,7 @@ networkUpdateState(virNetworkObjPtr obj,
> >  case VIR_NETWORK_FORWARD_NAT:
> >  case VIR_NETWORK_FORWARD_ROUTE:
> >  case VIR_NETWORK_FORWARD_OPEN:
> > +case VIR_NETWORK_FORWARD_VLAN:
> >  /* If bridge doesn't exist, then mark it inactive */
> >  if (!(def->bridge && virNetDevExists(def->bridge) == 1))
> >  virNetworkObjSetActive(obj, false);
> > @@ -2092,7 +2093,8 @@ networkRefreshDaemonsHelper(virNetworkObjPtr obj,
> >  ((def->forward.type == VIR_NETWORK_FORWARD_NONE) ||
> >   (def->forward.type == VIR_NETWORK_FORWARD_NAT) ||
> >   (def->forward.type == VIR_NETW

[libvirt] [PATCH 11/10] qemu: hotplug: Add/remove managed PR objects on media change

2018-07-18 Thread Peter Krempa
When changing cdrom media we did not handle the managed PR objects thus
we'd either have a stale PR object left behind or the media change would
fail.

Signed-off-by: Peter Krempa 
---

Opps this one was also left behind in the blockdev-add series but
belongs to this posting.

 src/qemu/qemu_hotplug.c | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c
index f416db2c60..a1e523013d 100644
--- a/src/qemu/qemu_hotplug.c
+++ b/src/qemu/qemu_hotplug.c
@@ -595,6 +595,9 @@ qemuDomainChangeEjectableMedia(virQEMUDriverPtr driver,
 if (qemuHotplugPrepareDiskAccess(driver, vm, disk, newsrc, false) < 0)
 goto cleanup;

+if (qemuHotplugAttachManagedPR(driver, vm, newsrc, QEMU_ASYNC_JOB_NONE) < 
0)
+goto cleanup;
+
 rc = qemuDomainChangeMediaLegacy(driver, vm, disk, newsrc, force);

 virDomainAuditDisk(vm, disk->src, newsrc, "update", rc >= 0);
@@ -610,6 +613,9 @@ qemuDomainChangeEjectableMedia(virQEMUDriverPtr driver,

 virStorageSourceFree(disk->src);
 VIR_STEAL_PTR(disk->src, newsrc);
+
+ignore_value(qemuHotplugRemoveManagedPR(driver, vm, QEMU_ASYNC_JOB_NONE));
+
 ret = 0;

  cleanup:
-- 
2.16.2

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list


Re: [libvirt] [PATCH 4/4] qemu: vnc: switch to tls-creds-x509

2018-07-18 Thread Peter Krempa
On Tue, Jul 17, 2018 at 17:15:56 +0200, Ján Tomko wrote:
> The tls, x509 and x509verify options were deprecated in QEMU v2.5.0:
> 
> commit 3e305e4a4752f70c0b5c3cf5b43ec957881714f7
> Author: Daniel P. Berrange 
> 
> ui: convert VNC server to use QCryptoTLSSession
> 
> Use the tls-creds-x509 object when available.
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=1598167
> 
> Signed-off-by: Ján Tomko 
> ---
>  src/qemu/qemu_command.c| 26 
> +-
>  .../graphics-vnc-tls.x86_64-latest.args|  4 +++-
>  2 files changed, 23 insertions(+), 7 deletions(-)

ACK, as we've established that the we'll need to deal with the encrypted
key part later.


signature.asc
Description: PGP signature
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH 3/3] fix other functions to add VIR_NETWORK_FORWARD_VLAN

2018-07-18 Thread Shi Lei



> Sent: Wednesday, July 18, 2018 at 12:38 AM
> From: "John Ferlan" 
> To: "Shi Lei" , libvir-list@redhat.com
> Subject: Re: [libvirt] [PATCH 3/3] fix other functions to add 
> VIR_NETWORK_FORWARD_VLAN
>
> 
> 
> On 07/05/2018 11:36 PM, Shi Lei wrote:
> > Signed-off-by: Shi Lei 
> > ---
> >  src/conf/domain_conf.c   | 1 +
> >  src/conf/virnetworkobj.c | 1 +
> >  src/qemu/qemu_process.c  | 1 +
> >  3 files changed, 3 insertions(+)
> > 
> 
> This should have been merged into the previous patch; however, this
> doing point out to me that perhaps rather than "if" type conditions,
> maybe it'd be better if there was a typed switch() with case:'s that
> would then be noted during compilation when a new VIR_NETWORK_FORWARD_*
> type was added That could be a patch added *prior to* patch 2.
> 
> John

OK and maybe I can try to make the patch (from 'if' to 'switch').

Thanks again!

Shi Lei

> 
> > diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
> > index f4e59f6..bd8b050 100644
> > --- a/src/conf/domain_conf.c
> > +++ b/src/conf/domain_conf.c
> > @@ -29912,6 +29912,7 @@ virDomainNetResolveActualType(virDomainNetDefPtr 
> > iface)
> >  if ((def->forward.type == VIR_NETWORK_FORWARD_NONE) ||
> >  (def->forward.type == VIR_NETWORK_FORWARD_NAT) ||
> >  (def->forward.type == VIR_NETWORK_FORWARD_ROUTE) ||
> > +(def->forward.type == VIR_NETWORK_FORWARD_VLAN) ||
> >  (def->forward.type == VIR_NETWORK_FORWARD_OPEN)) {
> >  /* for these forward types, the actual net type really *is*
> >   * NETWORK; we just keep the info from the portgroup in
> > diff --git a/src/conf/virnetworkobj.c b/src/conf/virnetworkobj.c
> > index e00c8a7..3869021 100644
> > --- a/src/conf/virnetworkobj.c
> > +++ b/src/conf/virnetworkobj.c
> > @@ -1012,6 +1012,7 @@ virNetworkLoadConfig(virNetworkObjListPtr nets,
> >  if (def->forward.type == VIR_NETWORK_FORWARD_NONE ||
> >  def->forward.type == VIR_NETWORK_FORWARD_NAT ||
> >  def->forward.type == VIR_NETWORK_FORWARD_ROUTE ||
> > +def->forward.type == VIR_NETWORK_FORWARD_VLAN ||
> >  def->forward.type == VIR_NETWORK_FORWARD_OPEN) {
> >  
> >  if (!def->mac_specified) {
> > diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c
> > index 40d35cb..0e3e1af 100644
> > --- a/src/qemu/qemu_process.c
> > +++ b/src/qemu/qemu_process.c
> > @@ -4434,6 +4434,7 @@ qemuProcessGetNetworkAddress(const char *netname,
> >  case VIR_NETWORK_FORWARD_NAT:
> >  case VIR_NETWORK_FORWARD_ROUTE:
> >  case VIR_NETWORK_FORWARD_OPEN:
> > +case VIR_NETWORK_FORWARD_VLAN:
> >  ipdef = virNetworkDefGetIPByIndex(netdef, AF_UNSPEC, 0);
> >  if (!ipdef) {
> >  virReportError(VIR_ERR_INTERNAL_ERROR,
> > 
> 

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list


Re: [libvirt] [PATCH 1/3] add functions: load 8021q module, create/destroy vlan-dev

2018-07-18 Thread John Ferlan
[...]

>>> +
>>> +static int
>>> +controlVlanDev(unsigned int cmd,
>>> +   const char *ifname, unsigned int vlanid)
>>
>> One line per argument.
> 
> OK
> 
>>
>>> +{
>>
>> int ret = -1;
>>
> 
> Why need ret?
> 

I was going to suggest changing the code to use a cleanup: label like
many other functions, but then figured it wasn't necessary, but forgot
to go back and remove this line.

John

>>> +int fd;
>>> +struct vlan_ioctl_args if_request;
>>> +memset(&if_request, 0, sizeof(struct vlan_ioctl_args));
>>> +if_request.cmd = cmd;
>>> +

[...]

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list


Re: [libvirt] [PATCHv2] examples: Add clean-traffic-gateway into nwfilters

2018-07-18 Thread Martin Kletzander

On Wed, Jul 18, 2018 at 10:33:03AM +0200, Ales Musil wrote:

The filter purpose is to simulate isolated private VLAN.

The behavior can be achieved by limiting network traffic
to traffic between VM and gateway. Because there is no
concept of the PVLAN in the linux bridge.

The filter also contains parts from clean-traffic
to prevent VM from spoofing its IP and MAC address.

To use this filter the user just needs to set
the GATEWAY_MAC variable to gateway MAC address.

Signed-off-by: Ales Musil 
---
examples/xml/nwfilter/clean-traffic-gateway.xml | 34 +
1 file changed, 34 insertions(+)
create mode 100644 examples/xml/nwfilter/clean-traffic-gateway.xml



Looks OK.

Reviewed-by: Martin Kletzander 

And pushed.

signature.asc
Description: Digital signature
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [RFC PATCHv2 00/10] x86 RDT Cache Monitoring Technology (CMT)

2018-07-18 Thread Martin Kletzander

On Wed, Jul 18, 2018 at 02:29:32AM +, Wang, Huaqiang wrote:




-Original Message-
From: Martin Kletzander [mailto:mklet...@redhat.com]
Sent: Tuesday, July 17, 2018 5:11 PM
To: Wang, Huaqiang 
Cc: libvir-list@redhat.com; Feng, Shaohe ; Niu, Bing
; Ding, Jian-feng ; Zang, Rui

Subject: Re: [libvirt] [RFC PATCHv2 00/10] x86 RDT Cache Monitoring
Technology (CMT)

On Tue, Jul 17, 2018 at 07:19:41AM +, Wang, Huaqiang wrote:
>Hi Martin,
>
>Thanks for your comments. Please see my reply inline.
>
>> -Original Message-
>> From: Martin Kletzander [mailto:mklet...@redhat.com]
>> Sent: Tuesday, July 17, 2018 2:27 PM
>> To: Wang, Huaqiang 
>> Cc: libvir-list@redhat.com; Feng, Shaohe ;
>> Niu, Bing ; Ding, Jian-feng
>> ; Zang, Rui 
>> Subject: Re: [libvirt] [RFC PATCHv2 00/10] x86 RDT Cache Monitoring
>> Technology (CMT)
>>
>> On Mon, Jul 09, 2018 at 03:00:48PM +0800, Wang Huaqiang wrote:
>> >
>> >This is the V2 of RFC and the POC source code for introducing x86
>> >RDT CMT feature, thanks Martin Kletzander for his review and
>> >constructive suggestion for V1.
>> >
>> >This series is trying to provide the similar functions of the perf
>> >event based CMT, MBMT and MBML features in reporting cache
>> >occupancy, total memory bandwidth utilization and local memory
>> >bandwidth utilization information in livirt. Firstly we focus on cmt.
>> >
>> >x86 RDT Cache Monitoring Technology (CMT) provides a medthod to
>> >track the cache occupancy information per CPU thread. We are
>> >leveraging the implementation of kernel resctrl filesystem and
>> >create our patches on top of that.
>> >
>> >Describing the functionality from a high level:
>> >
>> >1. Extend the output of 'domstats' and report CMT inforamtion.
>> >
>> >Comparing with perf event based CMT implementation in libvirt, this
>> >series extends the output of command 'domstat' and reports cache
>> >occupancy information like these:
>> >
>> >[root@dl-c200 libvirt]# virsh domstats vm3 --cpu-resource
>> >Domain: 'vm3'
>> >  cpu.cacheoccupancy.vcpus_2.value=4415488
>> >  cpu.cacheoccupancy.vcpus_2.vcpus=2
>> >  cpu.cacheoccupancy.vcpus_1.value=7839744
>> >  cpu.cacheoccupancy.vcpus_1.vcpus=1
>> >  cpu.cacheoccupancy.vcpus_0,3.value=53796864
>> >  cpu.cacheoccupancy.vcpus_0,3.vcpus=0,3
>> >
>> >The vcpus have been arragned into three monitoring groups, these
>> >three groups cover vcpu 1, vcpu 2 and vcpus 0,3 respectively. Take
>> >an example, the 'cpu.cacheoccupancy.vcpus_0,3.value' reports the
>> >cache occupancy information for vcpu 0 and vcpu 3, the
>> 'cpu.cacheoccupancy.vcpus_0,3.vcpus'
>> >represents the vcpu group information.
>> >
>> >To address Martin's suggestion "beware as 1-4 is something else than
>> >1,4 so you need to differentiate that.", the content of 'vcpus'
>> >(cpu.cacheoccupancy..vcpus=xxx) has been specially
>> >processed, if vcpus is a continous range, e.g. 0-2, then the output
>> >of cpu.cacheoccupancy.vcpus_0-2.vcpus will be like
>> >'cpu.cacheoccupancy.vcpus_0-2.vcpus=0,1,2'
>> >instead of
>> >'cpu.cacheoccupancy.vcpus_0-2.vcpus=0-2'.
>> >Please note that 'vcpus_0-2' is a name of this monitoring group,
>> >could be specified any other word from the XML configuration file or
>> >lively changed with the command introduced in following part.
>> >
>>
>> One small nit according to the naming (but it shouldn't block any
>> reviewers from reviewing, just keep this in mind for next version for
>> example) is that this is still inconsistent.
>
>OK.  I'll try to use words such as 'cache', 'cpu resource' and avoid
>using 'RDT', 'CMT'.
>

Oh, you misunderstood, I meant the naming in the domstats output =)

>The way domstats are structured when there is something like an
>> array could shed some light into this.  What you suggested is really
>> kind of hard to parse (although looks better).  What would you say to
something like this:
>>
>>   cpu.cacheoccupancy.count = 3
>>   cpu.cacheoccupancy.0.value=4415488
>>   cpu.cacheoccupancy.0.vcpus=2
>>   cpu.cacheoccupancy.0.name=vcpus_2
>>   cpu.cacheoccupancy.1.value=7839744
>>   cpu.cacheoccupancy.1.vcpus=1
>>   cpu.cacheoccupancy.1.name=vcpus_1
>>   cpu.cacheoccupancy.2.value=53796864
>>   cpu.cacheoccupancy.2.vcpus=0,3
>>   cpu.cacheoccupancy.2.name=0,3
>>
>
>Your arrangement looks more reasonable, thanks for your advice.
>However, as I mentioned in another email that I sent to libvirt-list
>hours ago, the kernel resctrl interface provides cache occupancy
>information for each cache block for every resource group.
>Maybe we need to expose the cache occupancy for each cache block.
>If you agree, we need to refine the 'domstats' output message, how
>about this:
>
>  cpu.cacheoccupancy.count=3
>  cpu.cacheoccupancy.0.name=vcpus_2
>  cpu.cacheoccupancy.0.vcpus=2
>  cpu.cacheoccupancy.0.block.count=2
>  cpu.cacheoccupancy.0.block.0.bytes=5488
>  cpu.cacheoccupancy.0.block.1. bytes =441
>  cpu.cacheoccupancy.1.name=vcpus_1
>  cpu.cacheoccupancy.1.vcpus=1
>  cpu.cacheoccupancy.1.block.count

Re: [libvirt] [PATCH] test: Implement virConnectListAllInterfaces

2018-07-18 Thread John Ferlan



On 07/10/2018 05:09 PM, Cole Robinson wrote:
> This adds some generic virinterfaceobj code, roughly matching what
> is used by other stateful drivers like network, storage, etc.
> 
> Signed-off-by: Cole Robinson 
> ---
>  src/conf/virinterfaceobj.c | 105 +
>  src/conf/virinterfaceobj.h |   7 +++
>  src/libvirt_private.syms   |   1 +
>  src/test/test_driver.c |  15 ++
>  4 files changed, 128 insertions(+)
> 
> diff --git a/src/conf/virinterfaceobj.c b/src/conf/virinterfaceobj.c
> index a1d7346eb2..87ce188117 100644
> --- a/src/conf/virinterfaceobj.c
> +++ b/src/conf/virinterfaceobj.c
> @@ -241,6 +241,111 @@ virInterfaceObjListFindByName(virInterfaceObjListPtr 
> interfaces,
>  return obj;
>  }
>  

2 blank lines

> +#define MATCH(FLAG) (flags & (FLAG))
> +static bool
> +virInterfaceMatch(virInterfaceObjPtr obj,

virInterfaceObjMatch

> +  unsigned int flags)
> +{
> +/* filter by active state */
> +if (MATCH(VIR_CONNECT_LIST_INTERFACES_FILTERS_ACTIVE) &&
> +!((MATCH(VIR_CONNECT_LIST_INTERFACES_ACTIVE) &&
> +   virInterfaceObjIsActive(obj)) ||
> +  (MATCH(VIR_CONNECT_LIST_INTERFACES_INACTIVE) &&
> +   !virInterfaceObjIsActive(obj
> +return false;
> +
> +return true;
> +}
> +#undef MATCH
> +
> +
> +struct virInterfaceObjListData {
> +virConnectPtr conn;
> +virInterfacePtr *ifaces;
> +virInterfaceObjListFilter filter;
> +unsigned int flags;
> +int nifaces;
> +bool error;
> +};
> +
> +static int
> +virInterfaceObjListPopulate(void *payload,

This follows the NetworkObj code and I'll just point that the naming is
still not common on what's done between:

virDomainObjListCollectIterator
virNetworkObjListPopulate
virNodeDeviceObjListExportCallback
virNWFilterBindingObjListCollectIterator
virSecretObjListExportCallback
virStoragePoolObjVolumeListExportCb
virStoragePoolObjListExportCb

Perhaps a common name could be a bite sized task. I never quite got to
where I wanted to get to with the common object code mainly because of
varying opinions and not having the energy to continue 0-).

With the virInterfaceObjMatch adjustment,

Reviewed-by: John Ferlan 

John

[...]

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list


Re: [libvirt] [RFC PATCHv2 00/10] x86 RDT Cache Monitoring Technology (CMT)

2018-07-18 Thread Wang, Huaqiang



> -Original Message-
> From: Martin Kletzander [mailto:mklet...@redhat.com]
> Sent: Wednesday, July 18, 2018 8:07 PM
> To: Wang, Huaqiang 
> Cc: libvir-list@redhat.com; Feng, Shaohe ; Niu, Bing
> ; Ding, Jian-feng ; Zang, Rui
> 
> Subject: Re: [libvirt] [RFC PATCHv2 00/10] x86 RDT Cache Monitoring
> Technology (CMT)
> 
> On Wed, Jul 18, 2018 at 02:29:32AM +, Wang, Huaqiang wrote:
> >
> >
> >> -Original Message-
> >> From: Martin Kletzander [mailto:mklet...@redhat.com]
> >> Sent: Tuesday, July 17, 2018 5:11 PM
> >> To: Wang, Huaqiang 
> >> Cc: libvir-list@redhat.com; Feng, Shaohe ;
> >> Niu, Bing ; Ding, Jian-feng
> >> ; Zang, Rui 
> >> Subject: Re: [libvirt] [RFC PATCHv2 00/10] x86 RDT Cache Monitoring
> >> Technology (CMT)
> >>
> >> On Tue, Jul 17, 2018 at 07:19:41AM +, Wang, Huaqiang wrote:
> >> >Hi Martin,
> >> >
> >> >Thanks for your comments. Please see my reply inline.
> >> >
> >> >> -Original Message-
> >> >> From: Martin Kletzander [mailto:mklet...@redhat.com]
> >> >> Sent: Tuesday, July 17, 2018 2:27 PM
> >> >> To: Wang, Huaqiang 
> >> >> Cc: libvir-list@redhat.com; Feng, Shaohe ;
> >> >> Niu, Bing ; Ding, Jian-feng
> >> >> ; Zang, Rui 
> >> >> Subject: Re: [libvirt] [RFC PATCHv2 00/10] x86 RDT Cache
> >> >> Monitoring Technology (CMT)
> >> >>
> >> >> On Mon, Jul 09, 2018 at 03:00:48PM +0800, Wang Huaqiang wrote:
> >> >> >
> >> >> >This is the V2 of RFC and the POC source code for introducing x86
> >> >> >RDT CMT feature, thanks Martin Kletzander for his review and
> >> >> >constructive suggestion for V1.
> >> >> >
> >> >> >This series is trying to provide the similar functions of the
> >> >> >perf event based CMT, MBMT and MBML features in reporting cache
> >> >> >occupancy, total memory bandwidth utilization and local memory
> >> >> >bandwidth utilization information in livirt. Firstly we focus on cmt.
> >> >> >
> >> >> >x86 RDT Cache Monitoring Technology (CMT) provides a medthod to
> >> >> >track the cache occupancy information per CPU thread. We are
> >> >> >leveraging the implementation of kernel resctrl filesystem and
> >> >> >create our patches on top of that.
> >> >> >
> >> >> >Describing the functionality from a high level:
> >> >> >
> >> >> >1. Extend the output of 'domstats' and report CMT inforamtion.
> >> >> >
> >> >> >Comparing with perf event based CMT implementation in libvirt,
> >> >> >this series extends the output of command 'domstat' and reports
> >> >> >cache occupancy information like these:
> >> >> >
> >> >> >[root@dl-c200 libvirt]# virsh domstats vm3 --cpu-resource
> >> >> >Domain: 'vm3'
> >> >> >  cpu.cacheoccupancy.vcpus_2.value=4415488
> >> >> >  cpu.cacheoccupancy.vcpus_2.vcpus=2
> >> >> >  cpu.cacheoccupancy.vcpus_1.value=7839744
> >> >> >  cpu.cacheoccupancy.vcpus_1.vcpus=1
> >> >> >  cpu.cacheoccupancy.vcpus_0,3.value=53796864
> >> >> >  cpu.cacheoccupancy.vcpus_0,3.vcpus=0,3
> >> >> >
> >> >> >The vcpus have been arragned into three monitoring groups, these
> >> >> >three groups cover vcpu 1, vcpu 2 and vcpus 0,3 respectively.
> >> >> >Take an example, the 'cpu.cacheoccupancy.vcpus_0,3.value' reports
> >> >> >the cache occupancy information for vcpu 0 and vcpu 3, the
> >> >> 'cpu.cacheoccupancy.vcpus_0,3.vcpus'
> >> >> >represents the vcpu group information.
> >> >> >
> >> >> >To address Martin's suggestion "beware as 1-4 is something else
> >> >> >than
> >> >> >1,4 so you need to differentiate that.", the content of 'vcpus'
> >> >> >(cpu.cacheoccupancy..vcpus=xxx) has been specially
> >> >> >processed, if vcpus is a continous range, e.g. 0-2, then the
> >> >> >output of cpu.cacheoccupancy.vcpus_0-2.vcpus will be like
> >> >> >'cpu.cacheoccupancy.vcpus_0-2.vcpus=0,1,2'
> >> >> >instead of
> >> >> >'cpu.cacheoccupancy.vcpus_0-2.vcpus=0-2'.
> >> >> >Please note that 'vcpus_0-2' is a name of this monitoring group,
> >> >> >could be specified any other word from the XML configuration file
> >> >> >or lively changed with the command introduced in following part.
> >> >> >
> >> >>
> >> >> One small nit according to the naming (but it shouldn't block any
> >> >> reviewers from reviewing, just keep this in mind for next version
> >> >> for
> >> >> example) is that this is still inconsistent.
> >> >
> >> >OK.  I'll try to use words such as 'cache', 'cpu resource' and avoid
> >> >using 'RDT', 'CMT'.
> >> >
> >>
> >> Oh, you misunderstood, I meant the naming in the domstats output =)
> >>
> >> >The way domstats are structured when there is something like an
> >> >> array could shed some light into this.  What you suggested is
> >> >> really kind of hard to parse (although looks better).  What would
> >> >> you say to
> >> something like this:
> >> >>
> >> >>   cpu.cacheoccupancy.count = 3
> >> >>   cpu.cacheoccupancy.0.value=4415488
> >> >>   cpu.cacheoccupancy.0.vcpus=2
> >> >>   cpu.cacheoccupancy.0.name=vcpus_2
> >> >>   cpu.cacheoccupancy.1.value=7839744
> >> >>   cpu.cacheoccupancy.1.vcpus=1
> >> >>   cpu.cacheoccupancy.1.name=vcpus_1
>

[libvirt] [PATCH v4 01/10] docs: formatdomain: unify naming for CPUs/vCPUs

2018-07-18 Thread Erik Skultety
From: Katerina Koukiou 

CPU is an acronym and should be written in uppercase
when part of plain text and not refering to an element.

Signed-off-by: Katerina Koukiou 
---
 docs/formatdomain.html.in | 84 +++
 1 file changed, 42 insertions(+), 42 deletions(-)

diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in
index b00971a945..d08ede9ab5 100644
--- a/docs/formatdomain.html.in
+++ b/docs/formatdomain.html.in
@@ -631,45 +631,45 @@
   
   vcpus
   
-The vcpus element allows to control state of individual vcpus.
+The vcpus element allows to control state of individual vCPUs.
 
 The id attribute specifies the vCPU id as used by libvirt
-in other places such as vcpu pinning, scheduler information and NUMA
-assignment. Note that the vcpu ID as seen in the guest may differ from
-libvirt ID in certain cases. Valid IDs are from 0 to the maximum vcpu
+in other places such as vCPU pinning, scheduler information and NUMA
+assignment. Note that the vCPU ID as seen in the guest may differ from
+libvirt ID in certain cases. Valid IDs are from 0 to the maximum vCPU
 count as set by the vcpu element minus 1.
 
 The enabled attribute allows to control the state of the
-vcpu. Valid values are yes and no.
+vCPU. Valid values are yes and no.
 
-hotpluggable controls whether given vcpu can be hotplugged
-and hotunplugged in cases when the cpu is enabled at boot. Note that
-all disabled vcpus must be hotpluggable. Valid values are
+hotpluggable controls whether given vCPU can be hotplugged
+and hotunplugged in cases when the CPU is enabled at boot. Note that
+all disabled vCPUs must be hotpluggable. Valid values are
 yes and no.
 
-order allows to specify the order to add the online vcpus.
-For hypervisors/platforms that require to insert multiple vcpus at once
-the order may be duplicated across all vcpus that need to be
-enabled at once. Specifying order is not necessary, vcpus are then
+order allows to specify the order to add the online vCPUs.
+For hypervisors/platforms that require to insert multiple vCPUs at once
+the order may be duplicated across all vCPUs that need to be
+enabled at once. Specifying order is not necessary, vCPUs are then
 added in an arbitrary order. If order info is used, it must be used for
-all online vcpus. Hypervisors may clear or update ordering information
+all online vCPUs. Hypervisors may clear or update ordering information
 during certain operations to assure valid configuration.
 
-Note that hypervisors may create hotpluggable vcpus differently from
-boot vcpus thus special initialization may be necessary.
+Note that hypervisors may create hotpluggable vCPUs differently from
+boot vCPUs thus special initialization may be necessary.
 
-Hypervisors may require that vcpus enabled on boot which are not
+Hypervisors may require that vCPUs enabled on boot which are not
 hotpluggable are clustered at the beginning starting with ID 0. It may
-be also required that vcpu 0 is always present and non-hotpluggable.
+be also required that vCPU 0 is always present and non-hotpluggable.
 
-Note that providing state for individual cpus may be necessary to 
enable
+Note that providing state for individual CPUs may be necessary to 
enable
 support of addressable vCPU hotplug and this feature may not be
 supported by all hypervisors.
 
-For QEMU the following conditions are required. Vcpu 0 needs to be
-enabled and non-hotpluggable. On PPC64 along with it vcpus that are in
-the same core need to be enabled as well. All non-hotpluggable cpus
-present at boot need to be grouped after vcpu 0.
+For QEMU the following conditions are required. vCPU 0 needs to be
+enabled and non-hotpluggable. On PPC64 along with it vCPUs that are in
+the same core need to be enabled as well. All non-hotpluggable CPUs
+present at boot need to be grouped after vCPU 0.
 Since 2.2.0 (QEMU only)
   
 
@@ -774,11 +774,11 @@
   vcpupin
   
 The optional vcpupin element specifies which of host's
-physical CPUs the domain VCPU will be pinned to. If this is omitted,
+physical CPUs the domain vCPU will be pinned to. If this is omitted,
 and attribute cpuset of element vcpu is
 not specified, the vCPU is pinned to all the physical CPUs by default.
 It contains two required attributes, the attribute vcpu
-specifies vcpu id, and the attribute cpuset is same as
+specifies vCPU id, and the attribute cpuset is same as
 attribute cpuset of element vcpu.
 (NB: Only

[libvirt] [PATCH v4 00/10] Enable vfio-pci 'property' for mediated device

2018-07-18 Thread Erik Skultety
RFC here:
https://www.redhat.com/archives/libvir-list/2018-May/msg02218.html

Since v1 [1]:
- remodeled egl-headless to be a standalone graphics element instead of a
result of some automagic with graphic's  element
- pushed the first 5 simple fixes that were RB'd
- fixed some nits raised during review

Since v2 [2]
- converted the 'display' and 'egl-headless' PostParse callbacks in patches 2
and 6 to Validate callbacks as they should have been since the beginning (see
the RFC)
- some minor nits
- added a standalone patch substituting some 'error' labels with 'cleanup' as
requested during review

Since v3 [3]:
- all of the patches got a RB, but there were some comments that made me send
another version
- the only major change was to move the default 'display' choice from
qemu_command.c back to a PostParse callback, so now we format display='off' by
default to every domain that utilized an mdev

[1] https://www.redhat.com/archives/libvir-list/2018-June/msg01740.html
[2] https://www.redhat.com/archives/libvir-list/2018-July/msg00485.html
[3] https://www.redhat.com/archives/libvir-list/2018-July/msg00698.html


Erik Skultety (9):
  qemu: qemuBuildHostdevCommandLine: Use a helper variable mdevsrc
  qemu: caps: Introduce a capability for egl-headless
  qemu: Introduce a new graphics display type 'headless'
  qemu: caps: Add vfio-pci.display capability
  conf: Introduce virDomainGraphicsDefHasOpenGL helper
  conf: Replace 'error' with 'cleanup' in
virDomainHostdevDefParseXMLSubsys
  conf: Introduce new  attribute 'display'
  qemu: command: Enable formatting vfio-pci.display option onto cmdline
  docs: Update news about the VNC console enablement for mdevs

Katerina Koukiou (1):
  docs: formatdomain: unify naming for CPUs/vCPUs

 docs/formatdomain.html.in  | 137 --
 docs/news.xml  |  10 ++
 docs/schemas/domaincommon.rng  |   8 ++
 src/conf/domain_conf.c | 106 +++---
 src/conf/domain_conf.h |   5 +
 src/libvirt_private.syms   |   1 +
 src/libxl/libxl_conf.c |   1 +
 src/qemu/qemu_capabilities.c   |  10 ++
 src/qemu/qemu_capabilities.h   |   4 +
 src/qemu/qemu_command.c|  23 ++-
 src/qemu/qemu_domain.c | 154 -
 src/qemu/qemu_driver.c |   2 +
 src/qemu/qemu_hotplug.c|   1 +
 src/qemu/qemu_process.c|   4 +
 src/vmx/vmx.c  |   1 +
 tests/domaincapsschemadata/full.xml|   1 +
 tests/qemucapabilitiesdata/caps_2.10.0.aarch64.xml |   1 +
 tests/qemucapabilitiesdata/caps_2.10.0.ppc64.xml   |   1 +
 tests/qemucapabilitiesdata/caps_2.10.0.s390x.xml   |   1 +
 tests/qemucapabilitiesdata/caps_2.10.0.x86_64.xml  |   1 +
 tests/qemucapabilitiesdata/caps_2.11.0.s390x.xml   |   1 +
 tests/qemucapabilitiesdata/caps_2.11.0.x86_64.xml  |   1 +
 tests/qemucapabilitiesdata/caps_2.12.0.aarch64.xml |   2 +
 tests/qemucapabilitiesdata/caps_2.12.0.ppc64.xml   |   2 +
 tests/qemucapabilitiesdata/caps_2.12.0.s390x.xml   |   2 +
 tests/qemucapabilitiesdata/caps_2.12.0.x86_64.xml  |   2 +
 tests/qemucapabilitiesdata/caps_3.0.0.ppc64.xml|   2 +
 tests/qemucapabilitiesdata/caps_3.0.0.x86_64.xml   |   2 +
 tests/qemuxml2argvdata/graphics-egl-headless.args  |  26 
 tests/qemuxml2argvdata/graphics-egl-headless.xml   |  31 +
 .../qemuxml2argvdata/graphics-sdl-egl-headless.xml |  35 +
 .../graphics-spice-egl-headless.args   |  31 +
 .../graphics-spice-egl-headless.xml|  36 +
 .../graphics-spice-invalid-egl-headless.xml|  37 +
 .../graphics-vnc-egl-headless.args |  28 
 .../qemuxml2argvdata/graphics-vnc-egl-headless.xml |  37 +
 .../hostdev-mdev-display-missing-graphics.xml  |  35 +
 ...v-display-spice-egl-headless.x86_64-latest.args |  37 +
 .../hostdev-mdev-display-spice-egl-headless.xml|  40 ++
 ...ev-mdev-display-spice-opengl.x86_64-latest.args |  36 +
 .../hostdev-mdev-display-spice-opengl.xml  |  41 ++
 ...dev-display-vnc-egl-headless.x86_64-latest.args |  37 +
 .../hostdev-mdev-display-vnc-egl-headless.xml  |  40 ++
 .../hostdev-mdev-display-vnc.x86_64-latest.args|  36 +
 .../qemuxml2argvdata/hostdev-mdev-display-vnc.xml  |  39 ++
 tests/qemuxml2argvdata/hostdev-mdev-display.xml|  39 ++
 tests/qemuxml2argvtest.c   |  24 
 .../graphics-spice-egl-headless.xml|  44 ++
 .../graphics-vnc-egl-headless.xml  |  42 ++
 tests/qemuxml2xmloutdata/hostdev-mdev-display.xml  |  47 +++
 tests/qemuxml2xmltest.c|   3 +
 51 files changed, 1214 insertions(

[libvirt] [PATCH v4 05/10] qemu: caps: Add vfio-pci.display capability

2018-07-18 Thread Erik Skultety
QEMU 2.12 introduced a new vfio-pci device option 'display=on/off/auto'.
This patch introduces the necessary capability.

Signed-off-by: Erik Skultety 
Reviewed-by: John Ferlan 
Reviewed-by: Ján Tomko 
---
 src/qemu/qemu_capabilities.c   | 4 
 src/qemu/qemu_capabilities.h   | 3 +++
 tests/qemucapabilitiesdata/caps_2.12.0.aarch64.xml | 1 +
 tests/qemucapabilitiesdata/caps_2.12.0.ppc64.xml   | 1 +
 tests/qemucapabilitiesdata/caps_2.12.0.s390x.xml   | 1 +
 tests/qemucapabilitiesdata/caps_2.12.0.x86_64.xml  | 1 +
 tests/qemucapabilitiesdata/caps_3.0.0.ppc64.xml| 1 +
 tests/qemucapabilitiesdata/caps_3.0.0.x86_64.xml   | 1 +
 8 files changed, 13 insertions(+)

diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c
index df358f8967..0fb800589a 100644
--- a/src/qemu/qemu_capabilities.c
+++ b/src/qemu/qemu_capabilities.c
@@ -504,6 +504,9 @@ VIR_ENUM_IMPL(virQEMUCaps, QEMU_CAPS_LAST,
   "machine.pseries.cap-htm",
   "usb-storage.werror",
   "egl-headless",
+
+  /* 315 */
+  "vfio-pci.display",
 );
 
 
@@ -1197,6 +1200,7 @@ static struct virQEMUCapsStringFlags 
virQEMUCapsDevicePropsPCIAssign[] = {
 
 static struct virQEMUCapsStringFlags virQEMUCapsDevicePropsVfioPCI[] = {
 { "bootindex", QEMU_CAPS_VFIO_PCI_BOOTINDEX },
+{ "display", QEMU_CAPS_VFIO_PCI_DISPLAY },
 };
 
 static struct virQEMUCapsStringFlags virQEMUCapsDevicePropsSCSIDisk[] = {
diff --git a/src/qemu/qemu_capabilities.h b/src/qemu/qemu_capabilities.h
index 55221e7e57..9e8ad5f5c3 100644
--- a/src/qemu/qemu_capabilities.h
+++ b/src/qemu/qemu_capabilities.h
@@ -489,6 +489,9 @@ typedef enum { /* virQEMUCapsFlags grouping marker for 
syntax-check */
 QEMU_CAPS_USB_STORAGE_WERROR, /* -device usb-storage,werror=..,rerror=.. */
 QEMU_CAPS_EGL_HEADLESS, /* -display egl-headless */
 
+/* 315 */
+QEMU_CAPS_VFIO_PCI_DISPLAY, /* -device vfio-pci.display */
+
 QEMU_CAPS_LAST /* this must always be the last item */
 } virQEMUCapsFlags;
 
diff --git a/tests/qemucapabilitiesdata/caps_2.12.0.aarch64.xml 
b/tests/qemucapabilitiesdata/caps_2.12.0.aarch64.xml
index 80e7afec04..0cc6327573 100644
--- a/tests/qemucapabilitiesdata/caps_2.12.0.aarch64.xml
+++ b/tests/qemucapabilitiesdata/caps_2.12.0.aarch64.xml
@@ -170,6 +170,7 @@
   
   
   
+  
   2011090
   0
   347550
diff --git a/tests/qemucapabilitiesdata/caps_2.12.0.ppc64.xml 
b/tests/qemucapabilitiesdata/caps_2.12.0.ppc64.xml
index c4b09c0003..a88da6193e 100644
--- a/tests/qemucapabilitiesdata/caps_2.12.0.ppc64.xml
+++ b/tests/qemucapabilitiesdata/caps_2.12.0.ppc64.xml
@@ -168,6 +168,7 @@
   
   
   
+  
   2011090
   0
   428334
diff --git a/tests/qemucapabilitiesdata/caps_2.12.0.s390x.xml 
b/tests/qemucapabilitiesdata/caps_2.12.0.s390x.xml
index 1ff2fe45e1..7121da27a0 100644
--- a/tests/qemucapabilitiesdata/caps_2.12.0.s390x.xml
+++ b/tests/qemucapabilitiesdata/caps_2.12.0.s390x.xml
@@ -134,6 +134,7 @@
   
   
   
+  
   2012000
   0
   375999
diff --git a/tests/qemucapabilitiesdata/caps_2.12.0.x86_64.xml 
b/tests/qemucapabilitiesdata/caps_2.12.0.x86_64.xml
index 37d17786cf..78889facce 100644
--- a/tests/qemucapabilitiesdata/caps_2.12.0.x86_64.xml
+++ b/tests/qemucapabilitiesdata/caps_2.12.0.x86_64.xml
@@ -212,6 +212,7 @@
   
   
   
+  
   2011090
   0
   416196
diff --git a/tests/qemucapabilitiesdata/caps_3.0.0.ppc64.xml 
b/tests/qemucapabilitiesdata/caps_3.0.0.ppc64.xml
index 57bf5dba11..01bb968938 100644
--- a/tests/qemucapabilitiesdata/caps_3.0.0.ppc64.xml
+++ b/tests/qemucapabilitiesdata/caps_3.0.0.ppc64.xml
@@ -168,6 +168,7 @@
   
   
   
+  
   2012050
   0
   446771
diff --git a/tests/qemucapabilitiesdata/caps_3.0.0.x86_64.xml 
b/tests/qemucapabilitiesdata/caps_3.0.0.x86_64.xml
index 431910a9e3..4bc7cfeebc 100644
--- a/tests/qemucapabilitiesdata/caps_3.0.0.x86_64.xml
+++ b/tests/qemucapabilitiesdata/caps_3.0.0.x86_64.xml
@@ -215,6 +215,7 @@
   
   
   
+  
   2012090
   0
   438109
-- 
2.14.4

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATCH v4 07/10] conf: Replace 'error' with 'cleanup' in virDomainHostdevDefParseXMLSubsys

2018-07-18 Thread Erik Skultety
The exit path is the same for both success and failure, so the label
should be called cleanup.

Signed-off-by: Erik Skultety 
Reviewed-by: Ján Tomko 
---
 src/conf/domain_conf.c | 38 +++---
 1 file changed, 19 insertions(+), 19 deletions(-)

diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
index 187a8b36a0..8deff5442f 100644
--- a/src/conf/domain_conf.c
+++ b/src/conf/domain_conf.c
@@ -7680,18 +7680,18 @@ virDomainHostdevDefParseXMLSubsys(xmlNodePtr node,
 virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
_("unknown host device source address type '%s'"),
type);
-goto error;
+goto cleanup;
 }
 } else {
 virReportError(VIR_ERR_XML_ERROR,
"%s", _("missing source address type"));
-goto error;
+goto cleanup;
 }
 
 if (!(sourcenode = virXPathNode("./source", ctxt))) {
 virReportError(VIR_ERR_XML_ERROR, "%s",
_("Missing  element in hostdev device"));
-goto error;
+goto cleanup;
 }
 
 if (def->source.subsys.type != VIR_DOMAIN_HOSTDEV_SUBSYS_TYPE_USB &&
@@ -7699,20 +7699,20 @@ virDomainHostdevDefParseXMLSubsys(xmlNodePtr node,
 virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
_("Setting startupPolicy is only allowed for USB"
  " devices"));
-goto error;
+goto cleanup;
 }
 
 if (sgio) {
 if (def->source.subsys.type != VIR_DOMAIN_HOSTDEV_SUBSYS_TYPE_SCSI) {
 virReportError(VIR_ERR_XML_ERROR, "%s",
_("sgio is only supported for scsi host device"));
-goto error;
+goto cleanup;
 }
 
 if ((scsisrc->sgio = virDomainDeviceSGIOTypeFromString(sgio)) <= 0) {
 virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
_("unknown sgio mode '%s'"), sgio);
-goto error;
+goto cleanup;
 }
 }
 
@@ -7720,14 +7720,14 @@ virDomainHostdevDefParseXMLSubsys(xmlNodePtr node,
 if (def->source.subsys.type != VIR_DOMAIN_HOSTDEV_SUBSYS_TYPE_SCSI) {
 virReportError(VIR_ERR_XML_ERROR, "%s",
_("rawio is only supported for scsi host device"));
-goto error;
+goto cleanup;
 }
 
 if ((scsisrc->rawio = virTristateBoolTypeFromString(rawio)) <= 0) {
 virReportError(VIR_ERR_XML_ERROR,
_("unknown hostdev rawio setting '%s'"),
rawio);
-goto error;
+goto cleanup;
 }
 }
 
@@ -7736,28 +7736,28 @@ virDomainHostdevDefParseXMLSubsys(xmlNodePtr node,
 virReportError(VIR_ERR_XML_ERROR, "%s",
_("'model' attribute in  is only supported 
"
  "when type='mdev'"));
-goto error;
+goto cleanup;
 }
 } else {
 if (!model) {
 virReportError(VIR_ERR_XML_ERROR, "%s",
_("Missing 'model' attribute in mediated device's "
  " element"));
-goto error;
+goto cleanup;
 }
 
 if ((mdevsrc->model = virMediatedDeviceModelTypeFromString(model)) < 
0) {
 virReportError(VIR_ERR_XML_ERROR,
_("unknown hostdev model '%s'"),
model);
-goto error;
+goto cleanup;
 }
 }
 
 switch (def->source.subsys.type) {
 case VIR_DOMAIN_HOSTDEV_SUBSYS_TYPE_PCI:
 if (virDomainHostdevSubsysPCIDefParseXML(sourcenode, def, flags) < 0)
-goto error;
+goto cleanup;
 
 backend = VIR_DOMAIN_HOSTDEV_PCI_BACKEND_DEFAULT;
 if ((backendStr = virXPathString("string(./driver/@name)", ctxt)) &&
@@ -7766,7 +7766,7 @@ virDomainHostdevDefParseXMLSubsys(xmlNodePtr node,
 virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
_("Unknown PCI device  "
  "has been specified"), backendStr);
-goto error;
+goto cleanup;
 }
 pcisrc->backend = backend;
 
@@ -7774,32 +7774,32 @@ virDomainHostdevDefParseXMLSubsys(xmlNodePtr node,
 
 case VIR_DOMAIN_HOSTDEV_SUBSYS_TYPE_USB:
 if (virDomainHostdevSubsysUSBDefParseXML(sourcenode, def) < 0)
-goto error;
+goto cleanup;
 break;
 
 case VIR_DOMAIN_HOSTDEV_SUBSYS_TYPE_SCSI:
 if (virDomainHostdevSubsysSCSIDefParseXML(sourcenode, scsisrc, ctxt) < 
0)
-goto error;
+goto cleanup;
 break;
 
 case VIR_DOMAIN_HOSTDEV_SUBSYS_TYPE_SCSI_HOST:
 if (virDomainHostdevSubsysSCSIVHostDefParseXML(sourcenode, def) < 0)
-goto error;
+goto cleanup;
 break;

[libvirt] [PATCH v4 06/10] conf: Introduce virDomainGraphicsDefHasOpenGL helper

2018-07-18 Thread Erik Skultety
A simple helper which will loop through all the graphics elements and
checks whether at least one of them enables OpenGL support, either by
containing  or being of type 'egl-headless'.

Signed-off-by: Erik Skultety 
Acked-by: Michal Privoznik 
Reviewed-by: Ján Tomko 
---
 src/conf/domain_conf.c   | 43 +++
 src/conf/domain_conf.h   |  3 +++
 src/libvirt_private.syms |  1 +
 3 files changed, 47 insertions(+)

diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
index 89c6556026..187a8b36a0 100644
--- a/src/conf/domain_conf.c
+++ b/src/conf/domain_conf.c
@@ -30274,3 +30274,46 @@ virDomainDefHasManagedPR(const virDomainDef *def)
 
 return false;
 }
+
+
+/**
+ * virDomainGraphicsDefHasOpenGL:
+ * @def: domain definition
+ *
+ * Returns true if a domain config contains at least one  element
+ * with OpenGL support enabled, false otherwise.
+ */
+bool
+virDomainGraphicsDefHasOpenGL(const virDomainDef *def)
+{
+size_t i;
+
+for (i = 0; i < def->ngraphics; i++) {
+virDomainGraphicsDefPtr graphics = def->graphics[i];
+
+/* we only care about OpenGL support for a given type here */
+switch (graphics->type) {
+case VIR_DOMAIN_GRAPHICS_TYPE_VNC:
+case VIR_DOMAIN_GRAPHICS_TYPE_RDP:
+case VIR_DOMAIN_GRAPHICS_TYPE_DESKTOP:
+continue;
+case VIR_DOMAIN_GRAPHICS_TYPE_SDL:
+if (graphics->data.sdl.gl == VIR_TRISTATE_BOOL_YES)
+return true;
+
+continue;
+case VIR_DOMAIN_GRAPHICS_TYPE_SPICE:
+if (graphics->data.spice.gl == VIR_TRISTATE_BOOL_YES)
+return true;
+
+continue;
+case VIR_DOMAIN_GRAPHICS_TYPE_EGL_HEADLESS:
+return true;
+
+case VIR_DOMAIN_GRAPHICS_TYPE_LAST:
+break;
+}
+}
+
+return false;
+}
diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h
index 26f75b15d0..3deda1d978 100644
--- a/src/conf/domain_conf.h
+++ b/src/conf/domain_conf.h
@@ -3642,4 +3642,7 @@ virDomainDiskGetDetectZeroesMode(virDomainDiskDiscard 
discard,
 bool
 virDomainDefHasManagedPR(const virDomainDef *def);
 
+bool
+virDomainGraphicsDefHasOpenGL(const virDomainDef *def);
+
 #endif /* __DOMAIN_CONF_H */
diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms
index 1caecb96b6..10496ff7dc 100644
--- a/src/libvirt_private.syms
+++ b/src/libvirt_private.syms
@@ -359,6 +359,7 @@ virDomainGetFilesystemForTarget;
 virDomainGraphicsAuthConnectedTypeFromString;
 virDomainGraphicsAuthConnectedTypeToString;
 virDomainGraphicsDefFree;
+virDomainGraphicsDefHasOpenGL;
 virDomainGraphicsGetListen;
 virDomainGraphicsListenAppendAddress;
 virDomainGraphicsListenAppendSocket;
-- 
2.14.4

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATCH v4 04/10] qemu: Introduce a new graphics display type 'headless'

2018-07-18 Thread Erik Skultety
Since 2.10 QEMU supports a new display type egl-headless which uses the
drm nodes for OpenGL rendering copying back the rendered bits back to
QEMU into a dma-buf which can be accessed by standard "display" apps
like VNC or SPICE. Although this display type can be used on its own,
for any practical use case it makes sense to pair it with either VNC or
SPICE display. The clear benefit of this display is that VNC gains
OpenGL support, which it natively doesn't have, and SPICE gains remote
OpenGL support (native OpenGL support only works locally through a UNIX
socket, i.e. listen type=socket/none)

Signed-off-by: Erik Skultety 
---
 docs/formatdomain.html.in  | 33 +++-
 docs/schemas/domaincommon.rng  |  3 ++
 src/conf/domain_conf.c |  6 ++-
 src/conf/domain_conf.h |  1 +
 src/libxl/libxl_conf.c |  1 +
 src/qemu/qemu_command.c| 14 -
 src/qemu/qemu_domain.c | 60 +-
 src/qemu/qemu_driver.c |  2 +
 src/qemu/qemu_hotplug.c|  1 +
 src/qemu/qemu_process.c|  4 ++
 src/vmx/vmx.c  |  1 +
 tests/domaincapsschemadata/full.xml|  1 +
 tests/qemuxml2argvdata/graphics-egl-headless.args  | 26 ++
 tests/qemuxml2argvdata/graphics-egl-headless.xml   | 31 +++
 .../qemuxml2argvdata/graphics-sdl-egl-headless.xml | 35 +
 .../graphics-spice-egl-headless.args   | 31 +++
 .../graphics-spice-egl-headless.xml| 36 +
 .../graphics-spice-invalid-egl-headless.xml| 37 +
 .../graphics-vnc-egl-headless.args | 28 ++
 .../qemuxml2argvdata/graphics-vnc-egl-headless.xml | 37 +
 tests/qemuxml2argvtest.c   | 17 ++
 .../graphics-spice-egl-headless.xml| 44 
 .../graphics-vnc-egl-headless.xml  | 42 +++
 tests/qemuxml2xmltest.c|  2 +
 24 files changed, 488 insertions(+), 5 deletions(-)
 create mode 100644 tests/qemuxml2argvdata/graphics-egl-headless.args
 create mode 100644 tests/qemuxml2argvdata/graphics-egl-headless.xml
 create mode 100644 tests/qemuxml2argvdata/graphics-sdl-egl-headless.xml
 create mode 100644 tests/qemuxml2argvdata/graphics-spice-egl-headless.args
 create mode 100644 tests/qemuxml2argvdata/graphics-spice-egl-headless.xml
 create mode 100644 
tests/qemuxml2argvdata/graphics-spice-invalid-egl-headless.xml
 create mode 100644 tests/qemuxml2argvdata/graphics-vnc-egl-headless.args
 create mode 100644 tests/qemuxml2argvdata/graphics-vnc-egl-headless.xml
 create mode 100644 tests/qemuxml2xmloutdata/graphics-spice-egl-headless.xml
 create mode 100644 tests/qemuxml2xmloutdata/graphics-vnc-egl-headless.xml

diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in
index d08ede9ab5..091888c879 100644
--- a/docs/formatdomain.html.in
+++ b/docs/formatdomain.html.in
@@ -6321,7 +6321,8 @@ qemu-kvm -net nic,model=? /dev/null
 
   The graphics element has a mandatory type
   attribute which takes the value sdl, vnc,
-  spice, rdp or desktop:
+  spice, rdp, desktop or
+  egl-headless:
 
 
   sdl
@@ -6380,6 +6381,11 @@ qemu-kvm -net nic,model=? /dev/null
   auto-allocation and autoport having no effect due to
   security reasons) Since 1.0.6.
 
+
+  Although VNC doesn't support OpenGL natively, it can be paired
+  with graphics type egl-headless (see below) which
+  will instruct QEMU to open and use drm nodes for OpenGL 
rendering.
+
   
   spice Since 0.8.6
   
@@ -6485,6 +6491,12 @@ qemu-kvm -net nic,model=? /dev/null
   You can enable or disable OpenGL support explicitly with
   the gl element, by setting the enable
   property. (QEMU only, since 1.3.3).
+  Note that this only works locally, since this requires usage of
+  UNIX sockets, i.e. using listen types 'socket' or
+  'none'. For accelerated OpenGL with remote support, consider
+  pairing this element with type egl-headless
+  (see below). However, this will deliver weaker performance
+  compared to native Spice OpenGL support.
 
 
   By default, QEMU will pick the first available GPU DRM render 
node.
@@ -6520,6 +6532,25 @@ qemu-kvm -net nic,model=? /dev/null
   fullscreen.
 
   
+  egl-headlessSince 
4.6.0
+  
+
+  This display type provides support for an OpenGL accelerated
+  dis

[libvirt] [PATCH v4 02/10] qemu: qemuBuildHostdevCommandLine: Use a helper variable mdevsrc

2018-07-18 Thread Erik Skultety
Decrease the number of accessors we have to use.

Signed-off-by: Erik Skultety 
---
 src/qemu/qemu_command.c | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c
index 44ae8dcef7..c628ac3985 100644
--- a/src/qemu/qemu_command.c
+++ b/src/qemu/qemu_command.c
@@ -5424,7 +5424,9 @@ qemuBuildHostdevCommandLine(virCommandPtr cmd,
 
 /* MDEV */
 if (virHostdevIsMdevDevice(hostdev)) {
-switch ((virMediatedDeviceModelType) subsys->u.mdev.model) {
+virDomainHostdevSubsysMediatedDevPtr mdevsrc = &subsys->u.mdev;
+
+switch ((virMediatedDeviceModelType) mdevsrc->model) {
 case VIR_MDEV_MODEL_TYPE_VFIO_PCI:
 if (!virQEMUCapsGet(qemuCaps, QEMU_CAPS_DEVICE_VFIO_PCI)) {
 virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
@@ -5432,6 +5434,7 @@ qemuBuildHostdevCommandLine(virCommandPtr cmd,
  "supported by this version of QEMU"));
 return -1;
 }
+
 break;
 case VIR_MDEV_MODEL_TYPE_VFIO_CCW:
 if (!virQEMUCapsGet(qemuCaps, QEMU_CAPS_DEVICE_VFIO_CCW)) {
-- 
2.14.4

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list


[libvirt] [PATCH v4 08/10] conf: Introduce new attribute 'display'

2018-07-18 Thread Erik Skultety
QEMU 2.12 introduced a new type of display for mediated devices using
vfio-pci backend which allows a mediated device to be used as a VGA
compatible device as an alternative to an emulated video device. QEMU
exposes this feature via a vfio device property 'display' with supported
values 'on/off/auto' (libvirt will default to 'off').

This patch adds the necessary bits to domain config handling in order to
expose this feature. Since there's no convenient way for libvirt to come
up with usable defaults for the display setting, simply because libvirt
is not able to figure out which of the display implementations - dma-buf
which requires OpenGL support vs vfio regions which doesn't need OpenGL
(works with OpenGL enabled too) - the underlying mdev uses.

Signed-off-by: Erik Skultety 
Reviewed-by: Ján Tomko 
---
 docs/formatdomain.html.in | 20 -
 docs/schemas/domaincommon.rng |  5 ++
 src/conf/domain_conf.c| 19 -
 src/conf/domain_conf.h|  1 +
 src/qemu/qemu_domain.c| 94 ++-
 tests/qemuxml2argvdata/hostdev-mdev-display.xml   | 39 ++
 tests/qemuxml2xmloutdata/hostdev-mdev-display.xml | 47 
 tests/qemuxml2xmltest.c   |  1 +
 8 files changed, 220 insertions(+), 6 deletions(-)
 create mode 100644 tests/qemuxml2argvdata/hostdev-mdev-display.xml
 create mode 100644 tests/qemuxml2xmloutdata/hostdev-mdev-display.xml

diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in
index 091888c879..3783e786ad 100644
--- a/docs/formatdomain.html.in
+++ b/docs/formatdomain.html.in
@@ -4532,9 +4532,23 @@
   guest. Currently, model='vfio-pci' and
   model='vfio-ccw' (Since 
4.4.0)
   is supported. Refer MDEV to create
-  a mediated device on the host. There are also some implications on 
the
-  usage of guest's address type depending on the model
-  attribute, see the address element below.
+  a mediated device on the host.
+  Since 4.6.0 (QEMU 2.12) an optional
+  display attribute may be used to enable or disable
+  support for an accelerated remote desktop backed by a mediated
+  device (such as NVIDIA vGPU or Intel GVT-g) as an alternative to
+  emulated video devices. This attribute
+  is limited to model='vfio-pci' only. Supported values
+  are either on or off (default is 'off').
+  It is required to use a
+  graphical framebuffer in order to
+  use this attribute, currently only supported with VNC, Spice and
+  egl-headless graphics devices.
+  
+Note: There are also some implications on the usage of guest's
+address type depending on the model attribute,
+see the address element below.
+  
   
 
 
diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng
index 157726752c..be8430ab22 100644
--- a/docs/schemas/domaincommon.rng
+++ b/docs/schemas/domaincommon.rng
@@ -4579,6 +4579,11 @@
 vfio-ccw
   
 
+
+  
+
+  
+
 
   
 
diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
index 8deff5442f..bd1dc0780d 100644
--- a/src/conf/domain_conf.c
+++ b/src/conf/domain_conf.c
@@ -7646,6 +7646,7 @@ virDomainHostdevDefParseXMLSubsys(xmlNodePtr node,
 char *rawio = NULL;
 char *backendStr = NULL;
 char *model = NULL;
+char *display = NULL;
 int backend;
 int ret = -1;
 virDomainHostdevSubsysPCIPtr pcisrc = &def->source.subsys.u.pci;
@@ -7665,6 +7666,7 @@ virDomainHostdevDefParseXMLSubsys(xmlNodePtr node,
 sgio = virXMLPropString(node, "sgio");
 rawio = virXMLPropString(node, "rawio");
 model = virXMLPropString(node, "model");
+display = virXMLPropString(node, "display");
 
 /* @type is passed in from the caller rather than read from the
  * xml document, because it is specified in different places for
@@ -7752,6 +7754,15 @@ virDomainHostdevDefParseXMLSubsys(xmlNodePtr node,
model);
 goto cleanup;
 }
+
+if (display &&
+(mdevsrc->display = virTristateSwitchTypeFromString(display)) <= 
0) {
+virReportError(VIR_ERR_XML_ERROR,
+   _("unknown value '%s' for  attribute "
+ "'display'"),
+   display);
+goto cleanup;
+}
 }
 
 switch (def->source.subsys.type) {
@@ -7805,6 +7816,7 @@ virDomainHostdevDefParseXMLSubsys(xmlNodePtr node,
 VIR_FREE(rawio);
 VIR_FREE(backendStr);
 VIR_FREE(model);
+VIR_FREE(display);
 return ret;
 }
 
@@ -26558,9 +26570,14 @@ virDomainHostdevDefFormat(virBufferPtr buf,
   virTristateBoolTypeToString(scsisrc->rawio));
 }
 
-if (def

[libvirt] [PATCH v4 03/10] qemu: caps: Introduce a capability for egl-headless

2018-07-18 Thread Erik Skultety
Since QEMU 2.10, it's possible to use a new type of display -
egl-headless which uses drm nodes to provide OpenGL support. This patch
adds a capability for that. However, since QEMU doesn't provide a QMP
command to probe it, we have to base the capability on specific QEMU
version.

Signed-off-by: Erik Skultety 
Acked-by: Michal Privoznik 
Reviewed-by: Ján Tomko 
Signed-off-by: Erik Skultety 
---
 src/qemu/qemu_capabilities.c   | 6 ++
 src/qemu/qemu_capabilities.h   | 1 +
 tests/qemucapabilitiesdata/caps_2.10.0.aarch64.xml | 1 +
 tests/qemucapabilitiesdata/caps_2.10.0.ppc64.xml   | 1 +
 tests/qemucapabilitiesdata/caps_2.10.0.s390x.xml   | 1 +
 tests/qemucapabilitiesdata/caps_2.10.0.x86_64.xml  | 1 +
 tests/qemucapabilitiesdata/caps_2.11.0.s390x.xml   | 1 +
 tests/qemucapabilitiesdata/caps_2.11.0.x86_64.xml  | 1 +
 tests/qemucapabilitiesdata/caps_2.12.0.aarch64.xml | 1 +
 tests/qemucapabilitiesdata/caps_2.12.0.ppc64.xml   | 1 +
 tests/qemucapabilitiesdata/caps_2.12.0.s390x.xml   | 1 +
 tests/qemucapabilitiesdata/caps_2.12.0.x86_64.xml  | 1 +
 tests/qemucapabilitiesdata/caps_3.0.0.ppc64.xml| 1 +
 tests/qemucapabilitiesdata/caps_3.0.0.x86_64.xml   | 1 +
 14 files changed, 19 insertions(+)

diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c
index 23b483349f..df358f8967 100644
--- a/src/qemu/qemu_capabilities.c
+++ b/src/qemu/qemu_capabilities.c
@@ -503,6 +503,7 @@ VIR_ENUM_IMPL(virQEMUCaps, QEMU_CAPS_LAST,
   "machine.pseries.cap-hpt-max-page-size",
   "machine.pseries.cap-htm",
   "usb-storage.werror",
+  "egl-headless",
 );
 
 
@@ -4030,6 +4031,11 @@ virQEMUCapsInitQMPMonitor(virQEMUCapsPtr qemuCaps,
 virQEMUCapsSet(qemuCaps, QEMU_CAPS_MACHINE_PSERIES_RESIZE_HPT);
 }
 
+/* '-display egl-headless' cmdline option is supported since QEMU 2.10, but
+ * there's no way to probe it */
+if (qemuCaps->version >= 201)
+virQEMUCapsSet(qemuCaps, QEMU_CAPS_EGL_HEADLESS);
+
 /* no way to query for -numa dist */
 if (qemuCaps->version >= 201)
 virQEMUCapsSet(qemuCaps, QEMU_CAPS_NUMA_DIST);
diff --git a/src/qemu/qemu_capabilities.h b/src/qemu/qemu_capabilities.h
index 1fa0ebfea3..55221e7e57 100644
--- a/src/qemu/qemu_capabilities.h
+++ b/src/qemu/qemu_capabilities.h
@@ -487,6 +487,7 @@ typedef enum { /* virQEMUCapsFlags grouping marker for 
syntax-check */
 QEMU_CAPS_MACHINE_PSERIES_CAP_HPT_MAX_PAGE_SIZE, /* -machine 
pseries.cap-hpt-max-page-size */
 QEMU_CAPS_MACHINE_PSERIES_CAP_HTM, /* -machine pseries.cap-htm */
 QEMU_CAPS_USB_STORAGE_WERROR, /* -device usb-storage,werror=..,rerror=.. */
+QEMU_CAPS_EGL_HEADLESS, /* -display egl-headless */
 
 QEMU_CAPS_LAST /* this must always be the last item */
 } virQEMUCapsFlags;
diff --git a/tests/qemucapabilitiesdata/caps_2.10.0.aarch64.xml 
b/tests/qemucapabilitiesdata/caps_2.10.0.aarch64.xml
index 169641063c..a70e050765 100644
--- a/tests/qemucapabilitiesdata/caps_2.10.0.aarch64.xml
+++ b/tests/qemucapabilitiesdata/caps_2.10.0.aarch64.xml
@@ -159,6 +159,7 @@
   
   
   
+  
   201
   0
   307647
diff --git a/tests/qemucapabilitiesdata/caps_2.10.0.ppc64.xml 
b/tests/qemucapabilitiesdata/caps_2.10.0.ppc64.xml
index 92c095abd2..72709905d8 100644
--- a/tests/qemucapabilitiesdata/caps_2.10.0.ppc64.xml
+++ b/tests/qemucapabilitiesdata/caps_2.10.0.ppc64.xml
@@ -158,6 +158,7 @@
   
   
   
+  
   201
   0
   386992
diff --git a/tests/qemucapabilitiesdata/caps_2.10.0.s390x.xml 
b/tests/qemucapabilitiesdata/caps_2.10.0.s390x.xml
index 5e22e21224..7347f5683f 100644
--- a/tests/qemucapabilitiesdata/caps_2.10.0.s390x.xml
+++ b/tests/qemucapabilitiesdata/caps_2.10.0.s390x.xml
@@ -119,6 +119,7 @@
   
   
   
+  
   201
   0
   307899
diff --git a/tests/qemucapabilitiesdata/caps_2.10.0.x86_64.xml 
b/tests/qemucapabilitiesdata/caps_2.10.0.x86_64.xml
index 10b066bff1..d69a148cd2 100644
--- a/tests/qemucapabilitiesdata/caps_2.10.0.x86_64.xml
+++ b/tests/qemucapabilitiesdata/caps_2.10.0.x86_64.xml
@@ -203,6 +203,7 @@
   
   
   
+  
   201
   0
   367995
diff --git a/tests/qemucapabilitiesdata/caps_2.11.0.s390x.xml 
b/tests/qemucapabilitiesdata/caps_2.11.0.s390x.xml
index 6ca2e57ef8..b359f9a049 100644
--- a/tests/qemucapabilitiesdata/caps_2.11.0.s390x.xml
+++ b/tests/qemucapabilitiesdata/caps_2.11.0.s390x.xml
@@ -126,6 +126,7 @@
   
   
   
+  
   2011000
   0
   346751
diff --git a/tests/qemucapabilitiesdata/caps_2.11.0.x86_64.xml 
b/tests/qemucapabilitiesdata/caps_2.11.0.x86_64.xml
index c52e44a498..210f774c4e 100644
--- a/tests/qemucapabilitiesdata/caps_2.11.0.x86_64.xml
+++ b/tests/qemucapabilitiesdata/caps_2.11.0.x86_64.xml
@@ -207,6 +207,7 @@
   
   
   
+  
   2011000
   0
   371455
diff --git a/tests/qemucapabilitiesdata/caps_2.12.0.aarch64.xml 
b/tests/qemucapabilitiesdata/caps_2.12.0.aarch64.xml
index ecc029f403..80e7afec04 100644
--- a/tests/qemucapabilitiesdata/caps_2.12.0.aarc

[libvirt] [PATCH v4 09/10] qemu: command: Enable formatting vfio-pci.display option onto cmdline

2018-07-18 Thread Erik Skultety
Since QEMU 2.12, QEMU understands a new vfio-pci device option 'display'
which can be used to turn on display capabilities on vgpu-enabled
mediated devices, IOW emulated GPU devices like QXL will no longer be
needed with vgpu-enable mdevs.
QEMU defaults to 'auto' for the 'display' attribute, which is not
foolproof, so we need to play it safe here and explicitly format
display='off' if this attribute wasn't provided in the XML explicitly.

Signed-off-by: Erik Skultety 
---
 src/qemu/qemu_command.c|  4 +++
 .../hostdev-mdev-display-missing-graphics.xml  | 35 ++
 ...v-display-spice-egl-headless.x86_64-latest.args | 37 +++
 .../hostdev-mdev-display-spice-egl-headless.xml| 40 +
 ...ev-mdev-display-spice-opengl.x86_64-latest.args | 36 +++
 .../hostdev-mdev-display-spice-opengl.xml  | 41 ++
 ...dev-display-vnc-egl-headless.x86_64-latest.args | 37 +++
 .../hostdev-mdev-display-vnc-egl-headless.xml  | 40 +
 .../hostdev-mdev-display-vnc.x86_64-latest.args| 36 +++
 .../qemuxml2argvdata/hostdev-mdev-display-vnc.xml  | 39 
 tests/qemuxml2argvtest.c   |  7 
 11 files changed, 352 insertions(+)
 create mode 100644 
tests/qemuxml2argvdata/hostdev-mdev-display-missing-graphics.xml
 create mode 100644 
tests/qemuxml2argvdata/hostdev-mdev-display-spice-egl-headless.x86_64-latest.args
 create mode 100644 
tests/qemuxml2argvdata/hostdev-mdev-display-spice-egl-headless.xml
 create mode 100644 
tests/qemuxml2argvdata/hostdev-mdev-display-spice-opengl.x86_64-latest.args
 create mode 100644 tests/qemuxml2argvdata/hostdev-mdev-display-spice-opengl.xml
 create mode 100644 
tests/qemuxml2argvdata/hostdev-mdev-display-vnc-egl-headless.x86_64-latest.args
 create mode 100644 
tests/qemuxml2argvdata/hostdev-mdev-display-vnc-egl-headless.xml
 create mode 100644 
tests/qemuxml2argvdata/hostdev-mdev-display-vnc.x86_64-latest.args
 create mode 100644 tests/qemuxml2argvdata/hostdev-mdev-display-vnc.xml

diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c
index 7c44045d61..c61da13893 100644
--- a/src/qemu/qemu_command.c
+++ b/src/qemu/qemu_command.c
@@ -5207,6 +5207,10 @@ qemuBuildHostdevMediatedDevStr(const virDomainDef *def,
 virBufferAdd(&buf, dev_str, -1);
 virBufferAsprintf(&buf, ",id=%s,sysfsdev=%s", dev->info->alias, mdevPath);
 
+if (mdevsrc->display != VIR_TRISTATE_SWITCH_ABSENT)
+virBufferAsprintf(&buf, ",display=%s",
+  virTristateSwitchTypeToString(mdevsrc->display));
+
 if (qemuBuildDeviceAddressStr(&buf, def, dev->info, qemuCaps) < 0)
 goto cleanup;
 
diff --git a/tests/qemuxml2argvdata/hostdev-mdev-display-missing-graphics.xml 
b/tests/qemuxml2argvdata/hostdev-mdev-display-missing-graphics.xml
new file mode 100644
index 00..ea559a6444
--- /dev/null
+++ b/tests/qemuxml2argvdata/hostdev-mdev-display-missing-graphics.xml
@@ -0,0 +1,35 @@
+
+  QEMUGuest2
+  c7a5fdbd-edaf-9455-926a-d65c16db1809
+  219136
+  219136
+  1
+  
+hvm
+
+  
+  
+  destroy
+  restart
+  destroy
+  
+/usr/bin/qemu-system-i686
+
+  
+  
+  
+  
+
+
+
+
+
+
+
+  
+
+  
+
+
+  
+
diff --git 
a/tests/qemuxml2argvdata/hostdev-mdev-display-spice-egl-headless.x86_64-latest.args
 
b/tests/qemuxml2argvdata/hostdev-mdev-display-spice-egl-headless.x86_64-latest.args
new file mode 100644
index 00..0ac90c81d2
--- /dev/null
+++ 
b/tests/qemuxml2argvdata/hostdev-mdev-display-spice-egl-headless.x86_64-latest.args
@@ -0,0 +1,37 @@
+LC_ALL=C \
+PATH=/bin \
+HOME=/home/test \
+USER=test \
+LOGNAME=test \
+QEMU_AUDIO_DRV=spice \
+/usr/bin/qemu-system-i686 \
+-name guest=QEMUGuest2,debug-threads=on \
+-S \
+-object secret,id=masterKey0,format=raw,\
+file=/tmp/lib/domain--1-QEMUGuest2/master-key.aes \
+-machine pc,accel=tcg,usb=off,dump-guest-core=off \
+-m 214 \
+-realtime mlock=off \
+-smp 1,sockets=1,cores=1,threads=1 \
+-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \
+-no-user-config \
+-nodefaults \
+-chardev socket,id=charmonitor,fd=1729,server,nowait \
+-mon chardev=charmonitor,id=monitor,mode=control \
+-rtc base=utc \
+-no-shutdown \
+-no-acpi \
+-boot strict=on \
+-device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 \
+-drive file=/dev/HostVG/QEMUGuest2,format=raw,if=none,id=drive-ide0-0-0 \
+-device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 \
+-spice port=0,seamless-migration=on \
+-display egl-headless \
+-device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,\
+vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pci.0,addr=0x2 \
+-device vfio-pci,id=hostdev0,\
+sysfsdev=/sys/bus/mdev/devices/53764d0e-85a0-42b4-af5c-2046b460b1dc,display=on,\
+bus=pci.0,addr=0x3 \
+-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,\
+resourcecontrol=

[libvirt] [PATCH v4 10/10] docs: Update news about the VNC console enablement for mdevs

2018-07-18 Thread Erik Skultety
Signed-off-by: Erik Skultety 
Acked-by: Michal Privoznik 
---
 docs/news.xml | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/docs/news.xml b/docs/news.xml
index 773c95b0da..fd08c01aeb 100644
--- a/docs/news.xml
+++ b/docs/news.xml
@@ -44,6 +44,16 @@
   support should be available to the guest.
 
   
+  
+
+  qemu: Enable VNC console for mediated devices
+
+
+  Host devices now support a new atribute 'display' which can be used
+  to turn on frame buffer rendering on a vgpu mediated device instead 
of
+  on an emulated GPU, like QXL.
+
+  
 
 
 
-- 
2.14.4

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list


Re: [libvirt] [PATCH v2 3/3] docs: news: Provide an update about the video type 'none'

2018-07-18 Thread Erik Skultety
On Fri, Jul 13, 2018 at 02:02:10PM +0200, Ján Tomko wrote:
> On Thu, Jul 12, 2018 at 05:08:48PM +0200, Erik Skultety wrote:
> > Signed-off-by: Erik Skultety 
> > ---
> > docs/news.xml | 14 ++
> > 1 file changed, 14 insertions(+)
> >
> > diff --git a/docs/news.xml b/docs/news.xml
> > index 773c95b0da..93832acc4c 100644
> > --- a/docs/news.xml
> > +++ b/docs/news.xml
> > @@ -46,6 +46,20 @@
> >   
> > 
> > 
> > +  
> > +
> > +  qemu: Introduce a new video model of type 'none'
> > +
> > +
> > +  Historically, libvirt would add a default 'cirrus' video device 
> > to
> > +  a guest if the XML specified 'graphics' but lacked 'video'.
>
> Historical behavior is hardly news :P
>
> > This can
> > +  be incovenient with GPU mediated devices which can serve as the 
> > only
> > +  rendering devices within the guest, rather than still relying on 
> > an
> > +  emulated GPU which would also be the primary device. Having a 
> > 'none'
> > +  model is our only backwards compatible option how to turn this 
> > legacy
> > +  behaviour off when needed.
>
> How about:
>
> Introduce a new video model type that disables the automatic addition of
> a video device to domains with 'graphics' specified in their XML.
>
> This can be useful with GPU mediated devices which can serve as the only
> rendering devices within the guest.

Much better, thanks. I didn't get an explicit ACK or a RB, so may I presume one?

Erik

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH v4 03/10] qemu: caps: Introduce a capability for egl-headless

2018-07-18 Thread Erik Skultety
On Wed, Jul 18, 2018 at 02:31:39PM +0200, Erik Skultety wrote:
> Since QEMU 2.10, it's possible to use a new type of display -
> egl-headless which uses drm nodes to provide OpenGL support. This patch
> adds a capability for that. However, since QEMU doesn't provide a QMP
> command to probe it, we have to base the capability on specific QEMU
> version.
>
> Signed-off-by: Erik Skultety 
> Acked-by: Michal Privoznik 
> Reviewed-by: Ján Tomko 
> Signed-off-by: Erik Skultety 

Sigh, consider this extra SoB removed, that's probably just some interactive
rebase bit I forgot about.

Erik

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH v2 3/3] docs: news: Provide an update about the video type 'none'

2018-07-18 Thread Ján Tomko

On Wed, Jul 18, 2018 at 02:34:05PM +0200, Erik Skultety wrote:

On Fri, Jul 13, 2018 at 02:02:10PM +0200, Ján Tomko wrote:

On Thu, Jul 12, 2018 at 05:08:48PM +0200, Erik Skultety wrote:
> Signed-off-by: Erik Skultety 
> ---
> docs/news.xml | 14 ++
> 1 file changed, 14 insertions(+)
>
> diff --git a/docs/news.xml b/docs/news.xml
> index 773c95b0da..93832acc4c 100644
> --- a/docs/news.xml
> +++ b/docs/news.xml
> @@ -46,6 +46,20 @@
>   
> 
> 
> +  
> +
> +  qemu: Introduce a new video model of type 'none'
> +
> +
> +  Historically, libvirt would add a default 'cirrus' video device to
> +  a guest if the XML specified 'graphics' but lacked 'video'.

Historical behavior is hardly news :P

> This can
> +  be incovenient with GPU mediated devices which can serve as the 
only
> +  rendering devices within the guest, rather than still relying on an
> +  emulated GPU which would also be the primary device. Having a 
'none'
> +  model is our only backwards compatible option how to turn this 
legacy
> +  behaviour off when needed.

How about:

Introduce a new video model type that disables the automatic addition of
a video device to domains with 'graphics' specified in their XML.

This can be useful with GPU mediated devices which can serve as the only
rendering devices within the guest.


Much better, thanks. I didn't get an explicit ACK or a RB, so may I presume one?



Sure. If you don't want to:

Reviewed-by: Ján Tomko 

Jano


signature.asc
Description: Digital signature
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH v4 03/10] qemu: caps: Introduce a capability for egl-headless

2018-07-18 Thread Pavel Hrdina
On Wed, Jul 18, 2018 at 02:57:30PM +0200, Erik Skultety wrote:
> On Wed, Jul 18, 2018 at 02:31:39PM +0200, Erik Skultety wrote:
> > Since QEMU 2.10, it's possible to use a new type of display -
> > egl-headless which uses drm nodes to provide OpenGL support. This patch
> > adds a capability for that. However, since QEMU doesn't provide a QMP
> > command to probe it, we have to base the capability on specific QEMU
> > version.
> >
> > Signed-off-by: Erik Skultety 
> > Acked-by: Michal Privoznik 
> > Reviewed-by: Ján Tomko 
> > Signed-off-by: Erik Skultety 
> 
> Sigh, consider this extra SoB removed, that's probably just some interactive
> rebase bit I forgot about.

This happens if you use "-s" parameter, however, if you put your SoB
always on the last line of the commit message "-s" will not add second
one.

Pavel

> 
> Erik
> 
> --
> libvir-list mailing list
> libvir-list@redhat.com
> https://www.redhat.com/mailman/listinfo/libvir-list


signature.asc
Description: PGP signature
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH v4 03/10] qemu: caps: Introduce a capability for egl-headless

2018-07-18 Thread Erik Skultety
On Wed, Jul 18, 2018 at 03:08:39PM +0200, Pavel Hrdina wrote:
> On Wed, Jul 18, 2018 at 02:57:30PM +0200, Erik Skultety wrote:
> > On Wed, Jul 18, 2018 at 02:31:39PM +0200, Erik Skultety wrote:
> > > Since QEMU 2.10, it's possible to use a new type of display -
> > > egl-headless which uses drm nodes to provide OpenGL support. This patch
> > > adds a capability for that. However, since QEMU doesn't provide a QMP
> > > command to probe it, we have to base the capability on specific QEMU
> > > version.
> > >
> > > Signed-off-by: Erik Skultety 
> > > Acked-by: Michal Privoznik 
> > > Reviewed-by: Ján Tomko 
> > > Signed-off-by: Erik Skultety 
> >
> > Sigh, consider this extra SoB removed, that's probably just some interactive
> > rebase bit I forgot about.
>
> This happens if you use "-s" parameter, however, if you put your SoB
> always on the last line of the commit message "-s" will not add second
> one.

Yeah, I should start adding the RBs above the SoB to prevent that, I'm not
going to drop my auto-sign in the config, because I would just keep forgetting
to sign it.

Erik

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATCH] Add DOMAIN_MEMORY_STAT_DISK_CACHES constant

2018-07-18 Thread Daniel P . Berrangé
Signed-off-by: Daniel P. Berrangé 
---
 domain.go   | 1 +
 domain_compat.h | 6 ++
 2 files changed, 7 insertions(+)

Pushed as a build fix

diff --git a/domain.go b/domain.go
index cc0285b..6a9e2a1 100644
--- a/domain.go
+++ b/domain.go
@@ -377,6 +377,7 @@ const (
DOMAIN_MEMORY_STAT_RSS= 
DomainMemoryStatTags(C.VIR_DOMAIN_MEMORY_STAT_RSS)
DOMAIN_MEMORY_STAT_USABLE = 
DomainMemoryStatTags(C.VIR_DOMAIN_MEMORY_STAT_USABLE)
DOMAIN_MEMORY_STAT_LAST_UPDATE= 
DomainMemoryStatTags(C.VIR_DOMAIN_MEMORY_STAT_LAST_UPDATE)
+   DOMAIN_MEMORY_STAT_DISK_CACHES= 
DomainMemoryStatTags(C.VIR_DOMAIN_MEMORY_STAT_DISK_CACHES)
DOMAIN_MEMORY_STAT_NR = 
DomainMemoryStatTags(C.VIR_DOMAIN_MEMORY_STAT_NR)
 )
 
diff --git a/domain_compat.h b/domain_compat.h
index 345505c..f23670c 100644
--- a/domain_compat.h
+++ b/domain_compat.h
@@ -1046,4 +1046,10 @@ int virDomainGetLaunchSecurityInfoCompat(virDomainPtr 
domain,
 #define VIR_DOMAIN_LAUNCH_SECURITY_SEV_MEASUREMENT "sev-measurement"
 #endif
 
+/* 4.6.0 */
+
+#ifndef VIR_DOMAIN_MEMORY_STAT_DISK_CACHES
+#define VIR_DOMAIN_MEMORY_STAT_DISK_CACHES 10
+#endif
+
 #endif /* LIBVIRT_GO_DOMAIN_COMPAT_H__ */
-- 
2.17.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [perl PATCH] Add VIR_DOMAIN_MEMORY_STAT_DISK_CACHES constant

2018-07-18 Thread Daniel P . Berrangé
Signed-off-by: Daniel P. Berrangé 
---
 Changes| 2 +-
 lib/Sys/Virt.xs| 5 +
 lib/Sys/Virt/Domain.pm | 4 
 3 files changed, 10 insertions(+), 1 deletion(-)

Pushed as a build fix

diff --git a/Changes b/Changes
index 74bbb22..4c77425 100644
--- a/Changes
+++ b/Changes
@@ -2,7 +2,7 @@ Revision history for perl module Sys::Virt
 
 4.6.0 2018-00-00
 
- - XXX
+ - Add VIR_DOMAIN_MEMORY_STAT_DISK_CACHES
 
 4.5.0 2018-07-03
 
diff --git a/lib/Sys/Virt.xs b/lib/Sys/Virt.xs
index 40134d3..14967d7 100644
--- a/lib/Sys/Virt.xs
+++ b/lib/Sys/Virt.xs
@@ -5850,6 +5850,10 @@ memory_stats(dom, flags=0)
   case VIR_DOMAIN_MEMORY_STAT_USABLE:
   (void)hv_store (RETVAL, "usable", 6, virt_newSVll(stats[i].val), 
0);
   break;
+
+  case VIR_DOMAIN_MEMORY_STAT_DISK_CACHES:
+  (void)hv_store (RETVAL, "disk_caches", 11, 
virt_newSVll(stats[i].val), 0);
+  break;
   }
   }
   Safefree(stats);
@@ -9040,6 +9044,7 @@ BOOT:
   REGISTER_CONSTANT(VIR_DOMAIN_MEMORY_STAT_RSS, MEMORY_STAT_RSS);
   REGISTER_CONSTANT(VIR_DOMAIN_MEMORY_STAT_LAST_UPDATE, 
MEMORY_STAT_LAST_UPDATE);
   REGISTER_CONSTANT(VIR_DOMAIN_MEMORY_STAT_USABLE, MEMORY_STAT_USABLE);
+  REGISTER_CONSTANT(VIR_DOMAIN_MEMORY_STAT_DISK_CACHES, 
MEMORY_STAT_DISK_CACHES);
 
 
   REGISTER_CONSTANT_STR(VIR_DOMAIN_BLOCK_IOTUNE_TOTAL_BYTES_SEC, 
BLOCK_IOTUNE_TOTAL_BYTES_SEC);
diff --git a/lib/Sys/Virt/Domain.pm b/lib/Sys/Virt/Domain.pm
index 284c38c..8eee18c 100644
--- a/lib/Sys/Virt/Domain.pm
+++ b/lib/Sys/Virt/Domain.pm
@@ -3903,6 +3903,10 @@ Amount of usable memory
 
 Time of last stats refresh from guest
 
+=item Sys::Virt::Domain::MEMORY_STAT_DISK_CACHES
+
+Disk cache size
+
 =back
 
 =head2 DOMAIN LIST CONSTANTS
-- 
2.17.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [go PATCH] Add DOMAIN_MEMORY_STAT_DISK_CACHES constant

2018-07-18 Thread Daniel P . Berrangé
Sorry this checkout was missing the subject prefix config.
This is for the go binding.

On Wed, Jul 18, 2018 at 02:47:03PM +0100, Daniel P. Berrangé wrote:
> Signed-off-by: Daniel P. Berrangé 
> ---
>  domain.go   | 1 +
>  domain_compat.h | 6 ++
>  2 files changed, 7 insertions(+)
> 
> Pushed as a build fix
> 
> diff --git a/domain.go b/domain.go
> index cc0285b..6a9e2a1 100644
> --- a/domain.go
> +++ b/domain.go
> @@ -377,6 +377,7 @@ const (
>   DOMAIN_MEMORY_STAT_RSS= 
> DomainMemoryStatTags(C.VIR_DOMAIN_MEMORY_STAT_RSS)
>   DOMAIN_MEMORY_STAT_USABLE = 
> DomainMemoryStatTags(C.VIR_DOMAIN_MEMORY_STAT_USABLE)
>   DOMAIN_MEMORY_STAT_LAST_UPDATE= 
> DomainMemoryStatTags(C.VIR_DOMAIN_MEMORY_STAT_LAST_UPDATE)
> + DOMAIN_MEMORY_STAT_DISK_CACHES= 
> DomainMemoryStatTags(C.VIR_DOMAIN_MEMORY_STAT_DISK_CACHES)
>   DOMAIN_MEMORY_STAT_NR = 
> DomainMemoryStatTags(C.VIR_DOMAIN_MEMORY_STAT_NR)
>  )
>  
> diff --git a/domain_compat.h b/domain_compat.h
> index 345505c..f23670c 100644
> --- a/domain_compat.h
> +++ b/domain_compat.h
> @@ -1046,4 +1046,10 @@ int virDomainGetLaunchSecurityInfoCompat(virDomainPtr 
> domain,
>  #define VIR_DOMAIN_LAUNCH_SECURITY_SEV_MEASUREMENT "sev-measurement"
>  #endif
>  
> +/* 4.6.0 */
> +
> +#ifndef VIR_DOMAIN_MEMORY_STAT_DISK_CACHES
> +#define VIR_DOMAIN_MEMORY_STAT_DISK_CACHES 10
> +#endif
> +
>  #endif /* LIBVIRT_GO_DOMAIN_COMPAT_H__ */
> -- 
> 2.17.1
> 

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] CPU Support

2018-07-18 Thread Eduardo Habkost
CCing the AMD people who worked on this.

On Wed, Jul 18, 2018 at 12:18:45PM +0200, Pavel Hrdina wrote:
> On Wed, Jul 18, 2018 at 10:50:34AM +0100, Daniel P. Berrangé wrote:
> > On Wed, Jul 18, 2018 at 12:41:48PM +0300, Hetz Ben Hamo wrote:
> > > Hi,
> > > 
> > > I've been looking at the CPU list and although I see lots of CPU's, I
> > > cannot find 2 CPU families:
> > > 
> > > * AMD Ryzen
> > > * AMD Threadripper
> > > 
> > > Although EPYC has been added recently.
> > > 
> > > Are there any missing details which preventing adding those CPU's to the
> > > list?
> > 
> > Libvirt adds CPU models based on what QEMU supports. So from libvirt side 
> > the
> > answer is simply that QEMU doesn't expose any models for Ryzen/Threadripper,
> > but I'm not clear why it doesn't...
> > 
> > For a while I thought Ryzen/Threadripper would have same feature set as
> > EPYC, but I've seen bugs recently suggesting that is not in fact the
> > case. So it does look like having those models exposed by QEMU might
> > be useful.
> > 
> > Copy'ing QEMU devel & the CPU model maintainers for opinions.
> 
> I think that QEMU should figure out some pattern for naming CPU models
> because it's one big mess.  EPYC and Ryzen are bad names for QEMU as
> Core/Xeon would be for Intel CPUs.  It's the name of a model families
> and it will probably remain the same but with different
> microarchitecture.
> 
> Better name would be similarly like for the latest Inter CPUs,
> Skylake-Client and Skylake-Server.  Currently AMD has already two
> microarchitectures, Zen and Zen+ and there is third one Zen 2 planned.
> 
> Zen has AMD Ryzen, AMD Ryzen Threadripper and AMD Epyc.
> Zen+ has AMD Ryzen, AMD Ryzen Threadripper
> 
> And I bet that Zen 2 will follow the same model families.
> 
> We probably cannot rename EPYC now, but before we introduce Ryzen and
> Threadripper let's thing about it and come up with better names, for
> example Zen-Client/Zen-Server Zen+-Client or something like that.
> 
> Pavel

-- 
Eduardo

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list


Re: [libvirt] [RFC PATCHv2 00/10] x86 RDT Cache Monitoring Technology (CMT)

2018-07-18 Thread Martin Kletzander

On Wed, Jul 18, 2018 at 12:19:18PM +, Wang, Huaqiang wrote:




-Original Message-
From: Martin Kletzander [mailto:mklet...@redhat.com]
Sent: Wednesday, July 18, 2018 8:07 PM
To: Wang, Huaqiang 
Cc: libvir-list@redhat.com; Feng, Shaohe ; Niu, Bing
; Ding, Jian-feng ; Zang, Rui

Subject: Re: [libvirt] [RFC PATCHv2 00/10] x86 RDT Cache Monitoring
Technology (CMT)

On Wed, Jul 18, 2018 at 02:29:32AM +, Wang, Huaqiang wrote:
>
>
>> -Original Message-
>> From: Martin Kletzander [mailto:mklet...@redhat.com]
>> Sent: Tuesday, July 17, 2018 5:11 PM
>> To: Wang, Huaqiang 
>> Cc: libvir-list@redhat.com; Feng, Shaohe ;
>> Niu, Bing ; Ding, Jian-feng
>> ; Zang, Rui 
>> Subject: Re: [libvirt] [RFC PATCHv2 00/10] x86 RDT Cache Monitoring
>> Technology (CMT)
>>
>> On Tue, Jul 17, 2018 at 07:19:41AM +, Wang, Huaqiang wrote:
>> >Hi Martin,
>> >
>> >Thanks for your comments. Please see my reply inline.
>> >
>> >> -Original Message-
>> >> From: Martin Kletzander [mailto:mklet...@redhat.com]
>> >> Sent: Tuesday, July 17, 2018 2:27 PM
>> >> To: Wang, Huaqiang 
>> >> Cc: libvir-list@redhat.com; Feng, Shaohe ;
>> >> Niu, Bing ; Ding, Jian-feng
>> >> ; Zang, Rui 
>> >> Subject: Re: [libvirt] [RFC PATCHv2 00/10] x86 RDT Cache
>> >> Monitoring Technology (CMT)
>> >>
>> >> On Mon, Jul 09, 2018 at 03:00:48PM +0800, Wang Huaqiang wrote:
>> >> >
>> >> >This is the V2 of RFC and the POC source code for introducing x86
>> >> >RDT CMT feature, thanks Martin Kletzander for his review and
>> >> >constructive suggestion for V1.
>> >> >
>> >> >This series is trying to provide the similar functions of the
>> >> >perf event based CMT, MBMT and MBML features in reporting cache
>> >> >occupancy, total memory bandwidth utilization and local memory
>> >> >bandwidth utilization information in livirt. Firstly we focus on cmt.
>> >> >
>> >> >x86 RDT Cache Monitoring Technology (CMT) provides a medthod to
>> >> >track the cache occupancy information per CPU thread. We are
>> >> >leveraging the implementation of kernel resctrl filesystem and
>> >> >create our patches on top of that.
>> >> >
>> >> >Describing the functionality from a high level:
>> >> >
>> >> >1. Extend the output of 'domstats' and report CMT inforamtion.
>> >> >
>> >> >Comparing with perf event based CMT implementation in libvirt,
>> >> >this series extends the output of command 'domstat' and reports
>> >> >cache occupancy information like these:
>> >> >
>> >> >[root@dl-c200 libvirt]# virsh domstats vm3 --cpu-resource
>> >> >Domain: 'vm3'
>> >> >  cpu.cacheoccupancy.vcpus_2.value=4415488
>> >> >  cpu.cacheoccupancy.vcpus_2.vcpus=2
>> >> >  cpu.cacheoccupancy.vcpus_1.value=7839744
>> >> >  cpu.cacheoccupancy.vcpus_1.vcpus=1
>> >> >  cpu.cacheoccupancy.vcpus_0,3.value=53796864
>> >> >  cpu.cacheoccupancy.vcpus_0,3.vcpus=0,3
>> >> >
>> >> >The vcpus have been arragned into three monitoring groups, these
>> >> >three groups cover vcpu 1, vcpu 2 and vcpus 0,3 respectively.
>> >> >Take an example, the 'cpu.cacheoccupancy.vcpus_0,3.value' reports
>> >> >the cache occupancy information for vcpu 0 and vcpu 3, the
>> >> 'cpu.cacheoccupancy.vcpus_0,3.vcpus'
>> >> >represents the vcpu group information.
>> >> >
>> >> >To address Martin's suggestion "beware as 1-4 is something else
>> >> >than
>> >> >1,4 so you need to differentiate that.", the content of 'vcpus'
>> >> >(cpu.cacheoccupancy..vcpus=xxx) has been specially
>> >> >processed, if vcpus is a continous range, e.g. 0-2, then the
>> >> >output of cpu.cacheoccupancy.vcpus_0-2.vcpus will be like
>> >> >'cpu.cacheoccupancy.vcpus_0-2.vcpus=0,1,2'
>> >> >instead of
>> >> >'cpu.cacheoccupancy.vcpus_0-2.vcpus=0-2'.
>> >> >Please note that 'vcpus_0-2' is a name of this monitoring group,
>> >> >could be specified any other word from the XML configuration file
>> >> >or lively changed with the command introduced in following part.
>> >> >
>> >>
>> >> One small nit according to the naming (but it shouldn't block any
>> >> reviewers from reviewing, just keep this in mind for next version
>> >> for
>> >> example) is that this is still inconsistent.
>> >
>> >OK.  I'll try to use words such as 'cache', 'cpu resource' and avoid
>> >using 'RDT', 'CMT'.
>> >
>>
>> Oh, you misunderstood, I meant the naming in the domstats output =)
>>
>> >The way domstats are structured when there is something like an
>> >> array could shed some light into this.  What you suggested is
>> >> really kind of hard to parse (although looks better).  What would
>> >> you say to
>> something like this:
>> >>
>> >>   cpu.cacheoccupancy.count = 3
>> >>   cpu.cacheoccupancy.0.value=4415488
>> >>   cpu.cacheoccupancy.0.vcpus=2
>> >>   cpu.cacheoccupancy.0.name=vcpus_2
>> >>   cpu.cacheoccupancy.1.value=7839744
>> >>   cpu.cacheoccupancy.1.vcpus=1
>> >>   cpu.cacheoccupancy.1.name=vcpus_1
>> >>   cpu.cacheoccupancy.2.value=53796864
>> >>   cpu.cacheoccupancy.2.vcpus=0,3
>> >>   cpu.cacheoccupancy.2.name=0,3
>> >>
>> >
>> >Your arrangement looks more rea

Re: [libvirt] [PATCH 01/10] qemu: hotplug: Reuse qemuHotplugPrepareDiskAccess in qemuDomainRemoveDiskDevice

2018-07-18 Thread Ján Tomko

On Tue, Jul 17, 2018 at 02:14:21PM +0200, Peter Krempa wrote:

qemuHotplugPrepareDiskAccess can be used to tear down disk access so we
can replace the open-coded version collecting the same function calls.

Signed-off-by: Peter Krempa 
---
src/qemu/qemu_hotplug.c | 13 ++---
1 file changed, 2 insertions(+), 11 deletions(-)



Reviewed-by: Ján Tomko 

Jano


signature.asc
Description: Digital signature
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH 02/10] qemu: hotplug: Remove pointless variable

2018-07-18 Thread Ján Tomko

On Tue, Jul 17, 2018 at 02:14:22PM +0200, Peter Krempa wrote:

Now that there's only one use of it, replace it directly by the code
filling it.

Signed-off-by: Peter Krempa 
---
src/qemu/qemu_hotplug.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)



Reviewed-by: Ján Tomko 

Jano


signature.asc
Description: Digital signature
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH 03/10] qemu: hotplug: Don't format NULL in %s in qemuHotplugPrepareDiskAccess

2018-07-18 Thread Ján Tomko

On Tue, Jul 17, 2018 at 02:14:23PM +0200, Peter Krempa wrote:

The warning messages which include the disk source could potentially
format NULL using %s as virDomainDiskGetSource may return NULL for e.g.
NBD disks. As most of the APIs are NOOP for remote disks the usage of
the source string only should be fine for now.

Signed-off-by: Peter Krempa 
---
src/qemu/qemu_hotplug.c | 8 
1 file changed, 4 insertions(+), 4 deletions(-)



Reviewed-by: Ján Tomko 

Jano


signature.asc
Description: Digital signature
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATCHv4 00/15] Switch from yajl to Jansson

2018-07-18 Thread Ján Tomko
Per the discussion here:
https://www.redhat.com/archives/libvir-list/2017-November/msg00225.html
Switch from using yajl to Jansson.

v1:
https://www.redhat.com/archives/libvir-list/2018-March/msg01781.html
v2:
https://www.redhat.com/archives/libvir-list/2018-May/msg00695.html
v3:
https://www.redhat.com/archives/libvir-list/2018-May/msg00850.html

v4:
Introduce a separate check for Jansson >= 2.8, which preserves
ordering of object keys. For older platforms (e.g. Debian 8),
skip tests that depend on stable ordering.

All patches except for 12/15 (trivial) and 15/15 (new) were ACK'd
already by either Peter or Andrea.

Tested on Debian 8 with libjansson-dev 2.7.

Also available on my repo:
http://repo.or.cz/libvirt/jtomko.git/ jansson-skip

Ján Tomko (15):
  build: add --with-jansson
  build: undef WITH_JANSSON for SETUID_RPC_CLIENT
  Switch from yajl to Jansson
  FIXUP: fix tests
  FIXUP: Deprecate building --with-yajl
  FIXUP: make nss depend on Jansson instead of yajl
  FIXUP: compile and link with Jansson instead of yajl
  FIXUP: s/WITH_YAJL/WITH_JANSSON/
  FIXUP: libvirt.spec: use jansson instead of yajl
  Remove functions using yajl
  build: remove references to WITH_YAJL for SETUID_RPC_CLIENT
  Remove virJSONValueNewStringLen
  build: switch --with-qemu default from yes to check
  build: require Jansson if QEMU driver is enabled
  m4: Introduce STABLE_ORDERING_JANSSON

 config-post.h|   3 +-
 configure.ac |   3 +
 libvirt.spec.in  |   4 +-
 m4/virt-driver-qemu.m4   |   9 +-
 m4/virt-jansson.m4   |  32 ++
 m4/virt-nss.m4   |   4 +-
 m4/virt-yajl.m4  |  27 +-
 src/Makefile.am  |   8 +-
 src/libvirt_private.syms |   1 -
 src/qemu/qemu_driver.c   |   2 +-
 src/util/Makefile.inc.am |   4 +-
 src/util/virjson.c   | 616 +++
 src/util/virjson.h   |   1 -
 tests/Makefile.am|  12 +-
 tests/cputest.c  |  16 +-
 tests/libxlxml2domconfigtest.c   |   4 +-
 tests/qemuagenttest.c|   2 +-
 tests/qemublocktest.c|   6 +
 tests/qemucapabilitiestest.c |   7 +-
 tests/qemucaps2xmltest.c |   2 +-
 tests/qemucommandutiltest.c  |   7 +-
 tests/qemuhotplugtest.c  |   7 +-
 tests/qemumigparamsdata/empty.json   |   4 +-
 tests/qemumigparamsdata/unsupported.json |   4 +-
 tests/qemumigparamstest.c|   7 +-
 tests/qemumonitorjsontest.c  |   7 +-
 tests/virjsontest.c  |   5 +
 tests/virmacmaptest.c|   5 +
 tests/virmacmaptestdata/empty.json   |   4 +-
 tests/virmocklibxl.c |   4 +-
 tests/virnetdaemontest.c |   7 +-
 tests/virstoragetest.c   |   4 +-
 32 files changed, 274 insertions(+), 554 deletions(-)
 create mode 100644 m4/virt-jansson.m4

-- 
2.16.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATCHv4 05/15] FIXUP: Deprecate building --with-yajl

2018-07-18 Thread Ján Tomko
---
 m4/virt-yajl.m4 | 27 +++
 1 file changed, 3 insertions(+), 24 deletions(-)

diff --git a/m4/virt-yajl.m4 b/m4/virt-yajl.m4
index c4ea0102a3..8d4c43a6b2 100644
--- a/m4/virt-yajl.m4
+++ b/m4/virt-yajl.m4
@@ -23,31 +23,10 @@ AC_DEFUN([LIBVIRT_ARG_YAJL],[
 
 AC_DEFUN([LIBVIRT_CHECK_YAJL],[
   dnl YAJL JSON library http://lloyd.github.com/yajl/
-  if test "$with_qemu:$with_yajl" = yes:check; then
-dnl Some versions of qemu require the use of yajl; try to detect them
-dnl here, although we do not require qemu to exist in order to compile.
-dnl This check mirrors src/qemu/qemu_capabilities.c
-AC_PATH_PROGS([QEMU], [qemu-kvm qemu kvm qemu-system-x86_64],
-  [], [$PATH:/usr/bin:/usr/libexec])
-if test -x "$QEMU"; then
-  if $QEMU -help 2>/dev/null | grep -q libvirt; then
-with_yajl=yes
-  else
-[qemu_version_sed='s/.*ersion \([0-9.,]*\).*/\1/']
-qemu_version=`$QEMU -version | sed "$qemu_version_sed"`
-case $qemu_version in
-  [[1-9]].* | 0.15.* ) with_yajl=yes ;;
-  0.* | '' ) ;;
-  *) AC_MSG_ERROR([Unexpected qemu version string]) ;;
-esac
-  fi
-fi
+  if test "$with_yajl" = yes; then
+AC_MSG_ERROR([Compilation with YAJL is no longer supported])
   fi
-
-  LIBVIRT_CHECK_LIB_ALT([YAJL], [yajl],
-[yajl_parse_complete], [yajl/yajl_common.h],
-[YAJL2], [yajl],
-[yajl_tree_parse], [yajl/yajl_common.h])
+  with_yajl=no
 ])
 
 AC_DEFUN([LIBVIRT_RESULT_YAJL],[
-- 
2.16.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list


[libvirt] [PATCHv4 06/15] FIXUP: make nss depend on Jansson instead of yajl

2018-07-18 Thread Ján Tomko
---
 m4/virt-nss.m4 | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/m4/virt-nss.m4 b/m4/virt-nss.m4
index 951a74e835..082b7b14f6 100644
--- a/m4/virt-nss.m4
+++ b/m4/virt-nss.m4
@@ -27,9 +27,9 @@ AC_DEFUN([LIBVIRT_CHECK_NSS],[
   bsd_nss=no
   fail=0
   if test "x$with_nss_plugin" != "xno" ; then
-if test "x$with_yajl" != "xyes" ; then
+if test "x$with_jansson" != "xyes" ; then
   if test "x$with_nss_plugin" = "xyes" ; then
-AC_MSG_ERROR([Can't build nss plugin without yajl])
+AC_MSG_ERROR([Can't build nss plugin without JSON support])
   else
 with_nss_plugin=no
   fi
-- 
2.16.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list


[libvirt] [PATCHv4 07/15] FIXUP: compile and link with Jansson instead of yajl

2018-07-18 Thread Ján Tomko
---
 src/Makefile.am  | 8 
 src/util/Makefile.inc.am | 4 ++--
 tests/Makefile.am| 2 +-
 3 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/src/Makefile.am b/src/Makefile.am
index db8c8ebd1a..83263e69e5 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -544,7 +544,7 @@ libvirt_admin_la_CFLAGS = \
 libvirt_admin_la_CFLAGS += \
$(XDR_CFLAGS) \
$(CAPNG_CFLAGS) \
-   $(YAJL_CFLAGS) \
+   $(JANSSON_CFLAGS) \
$(SSH2_CFLAGS) \
$(SASL_CFLAGS) \
$(GNUTLS_CFLAGS) \
@@ -552,7 +552,7 @@ libvirt_admin_la_CFLAGS += \
 
 libvirt_admin_la_LIBADD += \
$(CAPNG_LIBS) \
-   $(YAJL_LIBS) \
+   $(JANSSON_LIBS) \
$(DEVMAPPER_LIBS) \
$(LIBXML_LIBS) \
$(SSH2_LIBS) \
@@ -994,14 +994,14 @@ libvirt_nss_la_SOURCES = \
 libvirt_nss_la_CFLAGS = \
-DLIBVIRT_NSS \
$(AM_CFLAGS) \
-   $(YAJL_CFLAGS) \
+   $(JANSSON_CFLAGS) \
$(NULL)
 libvirt_nss_la_LDFLAGS = \
$(AM_LDFLAGS) \
$(NULL)
 
 libvirt_nss_la_LIBADD = \
-   $(YAJL_LIBS) \
+   $(JANSSON_LIBS) \
$(NULL)
 endif WITH_NSS
 
diff --git a/src/util/Makefile.inc.am b/src/util/Makefile.inc.am
index a22265606c..71b2b93c2d 100644
--- a/src/util/Makefile.inc.am
+++ b/src/util/Makefile.inc.am
@@ -251,7 +251,7 @@ libvirt_util_la_SOURCES = \
$(NULL)
 libvirt_util_la_CFLAGS = \
$(CAPNG_CFLAGS) \
-   $(YAJL_CFLAGS) \
+   $(JANSSON_CFLAGS) \
$(LIBNL_CFLAGS) \
$(AM_CFLAGS) \
$(AUDIT_CFLAGS) \
@@ -264,7 +264,7 @@ libvirt_util_la_CFLAGS = \
$(NULL)
 libvirt_util_la_LIBADD = \
$(CAPNG_LIBS) \
-   $(YAJL_LIBS) \
+   $(JANSSON_LIBS) \
$(LIBNL_LIBS) \
$(THREAD_LIBS) \
$(AUDIT_LIBS) \
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 21a6c823d9..e3f0947c4b 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -46,7 +46,7 @@ AM_CFLAGS = \
$(SASL_CFLAGS) \
$(SELINUX_CFLAGS) \
$(APPARMOR_CFLAGS) \
-   $(YAJL_CFLAGS) \
+   $(JANSSON_CFLAGS) \
$(COVERAGE_CFLAGS) \
$(XDR_CFLAGS) \
$(WARN_CFLAGS)
-- 
2.16.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list


[libvirt] [PATCHv4 02/15] build: undef WITH_JANSSON for SETUID_RPC_CLIENT

2018-07-18 Thread Ján Tomko
There is no code using WITH_JANSSON yet, but once we add it,
it should not be compiled for the setuid_rpc_client.

Signed-off-by: Ján Tomko 
---
 config-post.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/config-post.h b/config-post.h
index 063e30fa37..24117bbbd4 100644
--- a/config-post.h
+++ b/config-post.h
@@ -36,6 +36,7 @@
 # undef WITH_DEVMAPPER
 # undef WITH_DTRACE_PROBES
 # undef WITH_GNUTLS
+# undef WITH_JANSSON
 # undef WITH_LIBSSH
 # undef WITH_MACVTAP
 # undef WITH_NUMACTL
-- 
2.16.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATCHv4 11/15] build: remove references to WITH_YAJL for SETUID_RPC_CLIENT

2018-07-18 Thread Ján Tomko
We no longer allow building WITH_YAJL, remove the remaining
uses of the macro.

Signed-off-by: Ján Tomko 
---
 config-post.h | 2 --
 1 file changed, 2 deletions(-)

diff --git a/config-post.h b/config-post.h
index 24117bbbd4..05672121f0 100644
--- a/config-post.h
+++ b/config-post.h
@@ -44,8 +44,6 @@
 # undef WITH_SSH2
 # undef WITH_SYSTEMD_DAEMON
 # undef WITH_VIRTUALPORT
-# undef WITH_YAJL
-# undef WITH_YAJL2
 #endif
 
 /*
-- 
2.16.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATCHv4 03/15] Switch from yajl to Jansson

2018-07-18 Thread Ján Tomko
Yajl has not seen much activity upstream recently.
Switch to using Jansson >= 2.5.

All the platforms we target on https://libvirt.org/platforms.html
have a version >= 2.7 listed on the sites below:
https://repology.org/metapackage/jansson/versions
https://build.opensuse.org/package/show/devel:libraries:c_c++/libjansson

Additionally, Ubuntu 14.04 on Travis-CI has 2.5. Set the requirement
to 2.5 since we don't use anything from newer versions.

Implement virJSONValue{From,To}String using Jansson, delete the yajl
code (and the related virJSONParser structure) and report an error
if someone explicitly specifies --with-yajl.

Also adjust the test data to account for Jansson's different whitespace
usage for empty arrays and tune up the specfile to keep 'make rpm'
working when bisecting.

Signed-off-by: Ján Tomko 
---
 src/util/virjson.c | 211 +
 1 file changed, 211 insertions(+)

diff --git a/src/util/virjson.c b/src/util/virjson.c
index 29530dcb15..608ba85d67 100644
--- a/src/util/virjson.c
+++ b/src/util/virjson.c
@@ -1985,6 +1985,217 @@ virJSONValueToString(virJSONValuePtr object,
 }
 
 
+#elif WITH_JANSSON
+# include 
+
+static virJSONValuePtr
+virJSONValueFromJansson(json_t *json)
+{
+virJSONValuePtr ret = NULL;
+const char *key;
+json_t *cur;
+size_t i;
+
+switch (json_typeof(json)) {
+case JSON_OBJECT:
+ret = virJSONValueNewObject();
+if (!ret)
+goto error;
+
+json_object_foreach(json, key, cur) {
+virJSONValuePtr val = virJSONValueFromJansson(cur);
+if (!val)
+goto error;
+
+if (virJSONValueObjectAppend(ret, key, val) < 0) {
+virJSONValueFree(val);
+goto error;
+}
+}
+
+break;
+
+case JSON_ARRAY:
+ret = virJSONValueNewArray();
+if (!ret)
+goto error;
+
+json_array_foreach(json, i, cur) {
+virJSONValuePtr val = virJSONValueFromJansson(cur);
+if (!val)
+goto error;
+
+if (virJSONValueArrayAppend(ret, val) < 0) {
+virJSONValueFree(val);
+goto error;
+}
+}
+break;
+
+case JSON_STRING:
+ret = virJSONValueNewString(json_string_value(json));
+break;
+
+case JSON_INTEGER:
+ret = virJSONValueNewNumberLong(json_integer_value(json));
+break;
+
+case JSON_REAL:
+ret = virJSONValueNewNumberDouble(json_real_value(json));
+break;
+
+case JSON_TRUE:
+ret = virJSONValueNewBoolean(true);
+break;
+
+case JSON_FALSE:
+ret = virJSONValueNewBoolean(false);
+break;
+
+case JSON_NULL:
+ret = virJSONValueNewNull();
+break;
+}
+
+return ret;
+
+ error:
+virJSONValueFree(ret);
+return NULL;
+}
+
+virJSONValuePtr
+virJSONValueFromString(const char *jsonstring)
+{
+virJSONValuePtr ret = NULL;
+json_t *json;
+json_error_t error;
+size_t flags = JSON_REJECT_DUPLICATES |
+   JSON_DECODE_ANY;
+
+if (!(json = json_loads(jsonstring, flags, &error))) {
+virReportError(VIR_ERR_INTERNAL_ERROR,
+   _("failed to parse JSON %d:%d: %s"),
+   error.line, error.column, error.text);
+return NULL;
+}
+
+ret = virJSONValueFromJansson(json);
+json_decref(json);
+return ret;
+}
+
+
+static json_t *
+virJSONValueToJansson(virJSONValuePtr object)
+{
+json_t *ret = NULL;
+size_t i;
+
+switch ((virJSONType)object->type) {
+case VIR_JSON_TYPE_OBJECT:
+ret = json_object();
+if (!ret)
+goto no_memory;
+for (i = 0; i < object->data.object.npairs; i++) {
+virJSONObjectPairPtr cur = object->data.object.pairs + i;
+json_t *val = virJSONValueToJansson(cur->value);
+
+if (!val)
+goto error;
+if (json_object_set_new(ret, cur->key, val) < 0) {
+json_decref(val);
+goto no_memory;
+}
+}
+break;
+
+case VIR_JSON_TYPE_ARRAY:
+ret = json_array();
+if (!ret)
+goto no_memory;
+for (i = 0; i < object->data.array.nvalues; i++) {
+virJSONValuePtr cur = object->data.array.values[i];
+json_t *val = virJSONValueToJansson(cur);
+
+if (!val)
+goto error;
+if (json_array_append_new(ret, val) < 0) {
+json_decref(val);
+goto no_memory;
+}
+}
+break;
+
+case VIR_JSON_TYPE_STRING:
+ret = json_string(object->data.string);
+break;
+
+case VIR_JSON_TYPE_NUMBER: {
+long long ll_val;
+double d_val;
+if (virStrToLong_ll(object->data.number, NULL, 10, &ll_val) < 0) {
+if (virStrToDouble(obje

[libvirt] [PATCHv4 10/15] Remove functions using yajl

2018-07-18 Thread Ján Tomko
We no longer support building WITH_YAJL, remove the dead code
as well as the virJSONParser structures that are no longer used.

Signed-off-by: Ján Tomko 
---
 src/util/virjson.c | 529 +
 1 file changed, 1 insertion(+), 528 deletions(-)

diff --git a/src/util/virjson.c b/src/util/virjson.c
index 608ba85d67..80274bc6c5 100644
--- a/src/util/virjson.c
+++ b/src/util/virjson.c
@@ -29,22 +29,6 @@
 #include "virstring.h"
 #include "virutil.h"
 
-#if WITH_YAJL
-# include 
-# include 
-
-# ifdef WITH_YAJL2
-#  define yajl_size_t size_t
-#  define VIR_YAJL_STATUS_OK(status) ((status) == yajl_status_ok)
-# else
-#  define yajl_size_t unsigned int
-#  define yajl_complete_parse yajl_parse_complete
-#  define VIR_YAJL_STATUS_OK(status) \
-((status) == yajl_status_ok || (status) == yajl_status_insufficient_data)
-# endif
-
-#endif
-
 /* XXX fixme */
 #define VIR_FROM_THIS VIR_FROM_NONE
 
@@ -88,23 +72,6 @@ struct _virJSONValue {
 };
 
 
-typedef struct _virJSONParserState virJSONParserState;
-typedef virJSONParserState *virJSONParserStatePtr;
-struct _virJSONParserState {
-virJSONValuePtr value;
-char *key;
-};
-
-typedef struct _virJSONParser virJSONParser;
-typedef virJSONParser *virJSONParserPtr;
-struct _virJSONParser {
-virJSONValuePtr head;
-virJSONParserStatePtr state;
-size_t nstate;
-int wrap;
-};
-
-
 virJSONType
 virJSONValueGetType(const virJSONValue *value)
 {
@@ -1491,501 +1458,7 @@ virJSONValueCopy(const virJSONValue *in)
 }
 
 
-#if WITH_YAJL
-static int
-virJSONParserInsertValue(virJSONParserPtr parser,
- virJSONValuePtr value)
-{
-if (!parser->head) {
-parser->head = value;
-} else {
-virJSONParserStatePtr state;
-if (!parser->nstate) {
-VIR_DEBUG("got a value to insert without a container");
-return -1;
-}
-
-state = &parser->state[parser->nstate-1];
-
-switch (state->value->type) {
-case VIR_JSON_TYPE_OBJECT: {
-if (!state->key) {
-VIR_DEBUG("missing key when inserting object value");
-return -1;
-}
-
-if (virJSONValueObjectAppend(state->value,
- state->key,
- value) < 0)
-return -1;
-
-VIR_FREE(state->key);
-}   break;
-
-case VIR_JSON_TYPE_ARRAY: {
-if (state->key) {
-VIR_DEBUG("unexpected key when inserting array value");
-return -1;
-}
-
-if (virJSONValueArrayAppend(state->value,
-value) < 0)
-return -1;
-}   break;
-
-default:
-VIR_DEBUG("unexpected value type, not a container");
-return -1;
-}
-}
-
-return 0;
-}
-
-
-static int
-virJSONParserHandleNull(void *ctx)
-{
-virJSONParserPtr parser = ctx;
-virJSONValuePtr value = virJSONValueNewNull();
-
-VIR_DEBUG("parser=%p", parser);
-
-if (!value)
-return 0;
-
-if (virJSONParserInsertValue(parser, value) < 0) {
-virJSONValueFree(value);
-return 0;
-}
-
-return 1;
-}
-
-
-static int
-virJSONParserHandleBoolean(void *ctx,
-   int boolean_)
-{
-virJSONParserPtr parser = ctx;
-virJSONValuePtr value = virJSONValueNewBoolean(boolean_);
-
-VIR_DEBUG("parser=%p boolean=%d", parser, boolean_);
-
-if (!value)
-return 0;
-
-if (virJSONParserInsertValue(parser, value) < 0) {
-virJSONValueFree(value);
-return 0;
-}
-
-return 1;
-}
-
-
-static int
-virJSONParserHandleNumber(void *ctx,
-  const char *s,
-  yajl_size_t l)
-{
-virJSONParserPtr parser = ctx;
-char *str;
-virJSONValuePtr value;
-
-if (VIR_STRNDUP(str, s, l) < 0)
-return -1;
-value = virJSONValueNewNumber(str);
-VIR_FREE(str);
-
-VIR_DEBUG("parser=%p str=%s", parser, str);
-
-if (!value)
-return 0;
-
-if (virJSONParserInsertValue(parser, value) < 0) {
-virJSONValueFree(value);
-return 0;
-}
-
-return 1;
-}
-
-
-static int
-virJSONParserHandleString(void *ctx,
-  const unsigned char *stringVal,
-  yajl_size_t stringLen)
-{
-virJSONParserPtr parser = ctx;
-virJSONValuePtr value = virJSONValueNewStringLen((const char *)stringVal,
- stringLen);
-
-VIR_DEBUG("parser=%p str=%p", parser, (const char *)stringVal);
-
-if (!value)
-return 0;
-
-if (virJSONParserInsertValue(parser, value) < 0) {
-virJSONValueFree(value);
-return 0;
-}
-
-return 1;
-}
-
-
-static int
-virJSONParserHandleMapKey(void *ctx,
-  const unsigned char *stringVal,

[libvirt] [PATCHv4 04/15] FIXUP: fix tests

2018-07-18 Thread Ján Tomko
Jansson does not add an extra newline after the output
and it formats empty elements differently.
---
 tests/qemublocktest.c| 1 +
 tests/qemumigparamsdata/empty.json   | 4 +---
 tests/qemumigparamsdata/unsupported.json | 4 +---
 tests/virmacmaptestdata/empty.json   | 4 +---
 4 files changed, 4 insertions(+), 9 deletions(-)

diff --git a/tests/qemublocktest.c b/tests/qemublocktest.c
index 0c335abc5b..9a387cf063 100644
--- a/tests/qemublocktest.c
+++ b/tests/qemublocktest.c
@@ -309,6 +309,7 @@ testQemuDiskXMLToPropsValidateFile(const void *opaque)
 goto cleanup;
 
 virBufferAdd(&buf, jsonstr, -1);
+virBufferAddLit(&buf, "\n");
 VIR_FREE(jsonstr);
 }
 
diff --git a/tests/qemumigparamsdata/empty.json 
b/tests/qemumigparamsdata/empty.json
index 0db3279e44..0967ef424b 100644
--- a/tests/qemumigparamsdata/empty.json
+++ b/tests/qemumigparamsdata/empty.json
@@ -1,3 +1 @@
-{
-
-}
+{}
diff --git a/tests/qemumigparamsdata/unsupported.json 
b/tests/qemumigparamsdata/unsupported.json
index 0db3279e44..0967ef424b 100644
--- a/tests/qemumigparamsdata/unsupported.json
+++ b/tests/qemumigparamsdata/unsupported.json
@@ -1,3 +1 @@
-{
-
-}
+{}
diff --git a/tests/virmacmaptestdata/empty.json 
b/tests/virmacmaptestdata/empty.json
index 41b42e677b..fe51488c70 100644
--- a/tests/virmacmaptestdata/empty.json
+++ b/tests/virmacmaptestdata/empty.json
@@ -1,3 +1 @@
-[
-
-]
+[]
-- 
2.16.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list


[libvirt] [PATCHv4 08/15] FIXUP: s/WITH_YAJL/WITH_JANSSON/

2018-07-18 Thread Ján Tomko
---
 src/qemu/qemu_driver.c |  2 +-
 tests/Makefile.am  | 10 +-
 tests/cputest.c| 16 
 tests/libxlxml2domconfigtest.c |  4 ++--
 tests/qemuagenttest.c  |  2 +-
 tests/qemucapabilitiestest.c   |  2 +-
 tests/qemucaps2xmltest.c   |  2 +-
 tests/qemucommandutiltest.c|  2 +-
 tests/qemuhotplugtest.c|  2 +-
 tests/qemumigparamstest.c  |  2 +-
 tests/qemumonitorjsontest.c|  2 +-
 tests/virmocklibxl.c   |  4 ++--
 tests/virnetdaemontest.c   |  2 +-
 tests/virstoragetest.c |  4 ++--
 14 files changed, 28 insertions(+), 28 deletions(-)

diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c
index 847dab2edc..6fb66516c2 100644
--- a/src/qemu/qemu_driver.c
+++ b/src/qemu/qemu_driver.c
@@ -2092,7 +2092,7 @@ qemuDomainReboot(virDomainPtr dom, unsigned int flags)
  */
 if ((!useAgent) ||
 (ret < 0 && (acpiRequested || !flags))) {
-#if !WITH_YAJL
+#if !WITH_JANSSON
 virReportError(VIR_ERR_OPERATION_INVALID, "%s",
_("ACPI reboot is not supported without the JSON 
monitor"));
 goto endjob;
diff --git a/tests/Makefile.am b/tests/Makefile.am
index e3f0947c4b..302b50e1cd 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -331,9 +331,9 @@ if WITH_CIL
 test_programs += objectlocking
 endif WITH_CIL
 
-if WITH_YAJL
+if WITH_JANSSON
 test_programs += virjsontest
-endif WITH_YAJL
+endif WITH_JANSSON
 
 test_programs += \
networkxml2xmltest \
@@ -1219,15 +1219,15 @@ virdeterministichashmock_la_LIBADD = $(MOCKLIBS_LIBS)
 
 test_libraries += virdeterministichashmock.la
 
-if WITH_YAJL
+if WITH_JANSSON
 virmacmaptest_SOURCES = \
virmacmaptest.c testutils.h testutils.c
 virmacmaptest_LDADD = $(LDADDS)
 
 test_programs += virmacmaptest
-else ! WITH_YAJL
+else ! WITH_JANSSON
 EXTRA_DIST +=  virmacmaptest.c
-endif ! WITH_YAJL
+endif ! WITH_JANSSON
 
 virnetdevtest_SOURCES = \
virnetdevtest.c testutils.h testutils.c
diff --git a/tests/cputest.c b/tests/cputest.c
index baf2b3c648..9cc361d125 100644
--- a/tests/cputest.c
+++ b/tests/cputest.c
@@ -40,7 +40,7 @@
 #include "cpu/cpu_map.h"
 #include "virstring.h"
 
-#if WITH_QEMU && WITH_YAJL
+#if WITH_QEMU && WITH_JANSSON
 # include "testutilsqemu.h"
 # include "qemumonitortestutils.h"
 # define __QEMU_CAPSPRIV_H_ALLOW__
@@ -67,7 +67,7 @@ struct data {
 int result;
 };
 
-#if WITH_QEMU && WITH_YAJL
+#if WITH_QEMU && WITH_JANSSON
 static virQEMUDriver driver;
 #endif
 
@@ -479,7 +479,7 @@ typedef enum {
 JSON_MODELS_REQUIRED,
 } cpuTestCPUIDJson;
 
-#if WITH_QEMU && WITH_YAJL
+#if WITH_QEMU && WITH_JANSSON
 static virQEMUCapsPtr
 cpuTestMakeQEMUCaps(const struct data *data)
 {
@@ -554,7 +554,7 @@ cpuTestGetCPUModels(const struct data *data,
 return 0;
 }
 
-#else /* if WITH_QEMU && WITH_YAJL */
+#else /* if WITH_QEMU && WITH_JANSSON */
 
 static int
 cpuTestGetCPUModels(const struct data *data,
@@ -834,7 +834,7 @@ cpuTestUpdateLive(const void *arg)
 }
 
 
-#if WITH_QEMU && WITH_YAJL
+#if WITH_QEMU && WITH_JANSSON
 static int
 cpuTestJSONCPUID(const void *arg)
 {
@@ -911,7 +911,7 @@ mymain(void)
 virDomainCapsCPUModelsPtr ppc_models = NULL;
 int ret = 0;
 
-#if WITH_QEMU && WITH_YAJL
+#if WITH_QEMU && WITH_JANSSON
 if (qemuTestDriverInit(&driver) < 0)
 return EXIT_FAILURE;
 
@@ -1004,7 +1004,7 @@ mymain(void)
 host "/" cpu " (" #models ")", \
 host, cpu, models, 0, result)
 
-#if WITH_QEMU && WITH_YAJL
+#if WITH_QEMU && WITH_JANSSON
 # define DO_TEST_JSON(arch, host, json) \
 do { \
 if (json == JSON_MODELS) { \
@@ -1205,7 +1205,7 @@ mymain(void)
 DO_TEST_CPUID(VIR_ARCH_X86_64, "Xeon-X5460", JSON_NONE);
 
  cleanup:
-#if WITH_QEMU && WITH_YAJL
+#if WITH_QEMU && WITH_JANSSON
 qemuTestDriverFree(&driver);
 #endif
 
diff --git a/tests/libxlxml2domconfigtest.c b/tests/libxlxml2domconfigtest.c
index 54a92cc959..34a63e22b5 100644
--- a/tests/libxlxml2domconfigtest.c
+++ b/tests/libxlxml2domconfigtest.c
@@ -33,7 +33,7 @@
 
 #include "testutils.h"
 
-#if defined(WITH_LIBXL) && defined(WITH_YAJL) && 
defined(HAVE_LIBXL_DOMAIN_CONFIG_FROM_JSON)
+#if defined(WITH_LIBXL) && defined(WITH_JANSSON) && 
defined(HAVE_LIBXL_DOMAIN_CONFIG_FROM_JSON)
 
 # include "internal.h"
 # include "viralloc.h"
@@ -227,4 +227,4 @@ int main(void)
 return EXIT_AM_SKIP;
 }
 
-#endif /* WITH_LIBXL && WITH_YAJL && HAVE_LIBXL_DOMAIN_CONFIG_FROM_JSON */
+#endif /* WITH_LIBXL && WITH_JANSSON && HAVE_LIBXL_DOMAIN_CONFIG_FROM_JSON */
diff --git a/tests/qemuagenttest.c b/tests/qemuagenttest.c
index 2f79986207..232b34f9cd 100644
--- a/tests/qemuagenttest.c
+++ b/tests/qemuagenttest.c
@@ -907,7 +907,7 @@ mymain(void)
 {
 int ret = 0;
 
-#if !WITH_YAJL
+#if !WITH_JANSSON
 fputs("libvirt not compiled with JSON support, skipping this test\n", 
stderr);
 return EXIT_AM_SKIP;
 #endif
diff --git a/tests/qemucapabilitiestest.c b/tests/qemucapabilit

[libvirt] [PATCHv4 13/15] build: switch --with-qemu default from yes to check

2018-07-18 Thread Ján Tomko
Unless explicitly requested, enable the QEMU driver
only if the Jansson library is present.

Signed-off-by: Ján Tomko 
---
 m4/virt-driver-qemu.m4 | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/m4/virt-driver-qemu.m4 b/m4/virt-driver-qemu.m4
index 80e1d3ad46..ddb2834705 100644
--- a/m4/virt-driver-qemu.m4
+++ b/m4/virt-driver-qemu.m4
@@ -18,7 +18,7 @@ dnl .
 dnl
 
 AC_DEFUN([LIBVIRT_DRIVER_ARG_QEMU], [
-  LIBVIRT_ARG_WITH_FEATURE([QEMU], [QEMU/KVM], [yes])
+  LIBVIRT_ARG_WITH_FEATURE([QEMU], [QEMU/KVM], [check])
   LIBVIRT_ARG_WITH([QEMU_USER], [username to run QEMU system instance as],
['platform dependent'])
   LIBVIRT_ARG_WITH([QEMU_GROUP], [groupname to run QEMU system instance as],
@@ -26,6 +26,10 @@ AC_DEFUN([LIBVIRT_DRIVER_ARG_QEMU], [
 ])
 
 AC_DEFUN([LIBVIRT_DRIVER_CHECK_QEMU], [
+  AC_REQUIRE([LIBVIRT_CHECK_JANSSON])
+  if test "$with_qemu" = "check"; then
+with_qemu=$with_jansson
+  fi
   if test "$with_qemu" = "yes" ; then
 AC_DEFINE_UNQUOTED([WITH_QEMU], 1, [whether QEMU driver is enabled])
   fi
-- 
2.16.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATCHv4 09/15] FIXUP: libvirt.spec: use jansson instead of yajl

2018-07-18 Thread Ján Tomko
---
 libvirt.spec.in | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libvirt.spec.in b/libvirt.spec.in
index 6f360c5579..f83e5f8601 100644
--- a/libvirt.spec.in
+++ b/libvirt.spec.in
@@ -330,7 +330,7 @@ BuildRequires: systemd-devel >= 185
 BuildRequires: libudev-devel >= 145
 %endif
 BuildRequires: libpciaccess-devel >= 0.10.9
-BuildRequires: yajl-devel
+BuildRequires: jansson-devel
 %if %{with_sanlock}
 BuildRequires: sanlock-devel >= 2.4
 %endif
@@ -1321,7 +1321,7 @@ rm -f po/stamp-po
--without-apparmor \
--without-hal \
--with-udev \
-   --with-yajl \
+   --with-jansson \
%{?arg_sanlock} \
--with-libpcap \
--with-macvtap \
-- 
2.16.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list


[libvirt] [PATCHv4 14/15] build: require Jansson if QEMU driver is enabled

2018-07-18 Thread Ján Tomko
If the QEMU driver was requested, require Jansson, since we need to use
the JSON monitor to probe capabilities for all QEMU version supported
by libvirt.

Signed-off-by: Ján Tomko 
---
 m4/virt-driver-qemu.m4 | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/m4/virt-driver-qemu.m4 b/m4/virt-driver-qemu.m4
index ddb2834705..2d9992d0dc 100644
--- a/m4/virt-driver-qemu.m4
+++ b/m4/virt-driver-qemu.m4
@@ -27,6 +27,9 @@ AC_DEFUN([LIBVIRT_DRIVER_ARG_QEMU], [
 
 AC_DEFUN([LIBVIRT_DRIVER_CHECK_QEMU], [
   AC_REQUIRE([LIBVIRT_CHECK_JANSSON])
+  if test "$with_qemu:$with_jansson" = "yes:no"; then
+AC_MSG_ERROR([Jansson >= 2.5 is required to build QEMU driver])
+  fi
   if test "$with_qemu" = "check"; then
 with_qemu=$with_jansson
   fi
-- 
2.16.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATCHv4 01/15] build: add --with-jansson

2018-07-18 Thread Ján Tomko
Introduce the configure argument and check for Jansson >= 2.5

Signed-off-by: Ján Tomko 
---
 configure.ac   |  3 +++
 m4/virt-jansson.m4 | 29 +
 2 files changed, 32 insertions(+)
 create mode 100644 m4/virt-jansson.m4

diff --git a/configure.ac b/configure.ac
index a87ca06854..c6287f656a 100644
--- a/configure.ac
+++ b/configure.ac
@@ -250,6 +250,7 @@ LIBVIRT_ARG_FIREWALLD
 LIBVIRT_ARG_FUSE
 LIBVIRT_ARG_GLUSTER
 LIBVIRT_ARG_HAL
+LIBVIRT_ARG_JANSSON
 LIBVIRT_ARG_LIBPCAP
 LIBVIRT_ARG_LIBSSH
 LIBVIRT_ARG_LIBXML
@@ -290,6 +291,7 @@ LIBVIRT_CHECK_FUSE
 LIBVIRT_CHECK_GLUSTER
 LIBVIRT_CHECK_GNUTLS
 LIBVIRT_CHECK_HAL
+LIBVIRT_CHECK_JANSSON
 LIBVIRT_CHECK_LIBNL
 LIBVIRT_CHECK_LIBPARTED
 LIBVIRT_CHECK_LIBPCAP
@@ -970,6 +972,7 @@ LIBVIRT_RESULT_FUSE
 LIBVIRT_RESULT_GLUSTER
 LIBVIRT_RESULT_GNUTLS
 LIBVIRT_RESULT_HAL
+LIBVIRT_RESULT_JANSSON
 LIBVIRT_RESULT_LIBNL
 LIBVIRT_RESULT_LIBPCAP
 LIBVIRT_RESULT_LIBSSH
diff --git a/m4/virt-jansson.m4 b/m4/virt-jansson.m4
new file mode 100644
index 00..206d6a5ced
--- /dev/null
+++ b/m4/virt-jansson.m4
@@ -0,0 +1,29 @@
+dnl The jansson library
+dnl
+dnl This library is free software; you can redistribute it and/or
+dnl modify it under the terms of the GNU Lesser General Public
+dnl License as published by the Free Software Foundation; either
+dnl version 2.1 of the License, or (at your option) any later version.
+dnl
+dnl This library is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+dnl Lesser General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU Lesser General Public
+dnl License along with this library.  If not, see
+dnl .
+dnl
+
+AC_DEFUN([LIBVIRT_ARG_JANSSON],[
+  LIBVIRT_ARG_WITH_FEATURE([JANSSON], [jansson], [check])
+])
+
+AC_DEFUN([LIBVIRT_CHECK_JANSSON],[
+  dnl Jansson http://www.digip.org/jansson/
+  LIBVIRT_CHECK_PKG([JANSSON], [jansson], [2.5])
+])
+
+AC_DEFUN([LIBVIRT_RESULT_JANSSON],[
+  LIBVIRT_RESULT_LIB([JANSSON])
+])
-- 
2.16.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATCHv4 15/15] m4: Introduce STABLE_ORDERING_JANSSON

2018-07-18 Thread Ján Tomko
Add a second check for Jansson >= 2.8, which includes
fixes to preserve ordering of object keys.

Use this constant to guard tests that depend on stable ordering.

Signed-off-by: Ján Tomko 
---
 m4/virt-jansson.m4   | 3 +++
 tests/qemublocktest.c| 5 +
 tests/qemucapabilitiestest.c | 5 +
 tests/qemucommandutiltest.c  | 5 +
 tests/qemuhotplugtest.c  | 5 +
 tests/qemumigparamstest.c| 5 +
 tests/qemumonitorjsontest.c  | 5 +
 tests/virjsontest.c  | 5 +
 tests/virmacmaptest.c| 5 +
 tests/virnetdaemontest.c | 5 +
 10 files changed, 48 insertions(+)

diff --git a/m4/virt-jansson.m4 b/m4/virt-jansson.m4
index 206d6a5ced..ab4c020f62 100644
--- a/m4/virt-jansson.m4
+++ b/m4/virt-jansson.m4
@@ -22,6 +22,9 @@ AC_DEFUN([LIBVIRT_ARG_JANSSON],[
 AC_DEFUN([LIBVIRT_CHECK_JANSSON],[
   dnl Jansson http://www.digip.org/jansson/
   LIBVIRT_CHECK_PKG([JANSSON], [jansson], [2.5])
+  dnl Older versions of Jansson did not preserve the order of object keys
+  dnl use this check to guard the tests that are sensitive to this
+  LIBVIRT_CHECK_PKG([STABLE_ORDERING_JANSSON], [jansson], [2.8], [true])
 ])
 
 AC_DEFUN([LIBVIRT_RESULT_JANSSON],[
diff --git a/tests/qemublocktest.c b/tests/qemublocktest.c
index 9a387cf063..d22b4b754e 100644
--- a/tests/qemublocktest.c
+++ b/tests/qemublocktest.c
@@ -337,6 +337,11 @@ mymain(void)
 char *capslatest_x86_64 = NULL;
 virQEMUCapsPtr caps_x86_64 = NULL;
 
+#if !WITH_STABLE_ORDERING_JANSSON
+fputs("libvirt not compiled with recent enough Jansson, skipping this 
test\n", stderr);
+return EXIT_AM_SKIP;
+#endif
+
 if (qemuTestDriverInit(&driver) < 0)
 return EXIT_FAILURE;
 
diff --git a/tests/qemucapabilitiestest.c b/tests/qemucapabilitiestest.c
index 641ec4f597..28f903a88c 100644
--- a/tests/qemucapabilitiestest.c
+++ b/tests/qemucapabilitiestest.c
@@ -141,6 +141,11 @@ mymain(void)
 int ret = 0;
 testQemuData data;
 
+#if !WITH_STABLE_ORDERING_JANSSON
+fputs("libvirt not compiled with recent enough Jansson, skipping this 
test\n", stderr);
+return EXIT_AM_SKIP;
+#endif
+
 #if !WITH_JANSSON
 fputs("libvirt not compiled with JSON support, skipping this test\n", 
stderr);
 return EXIT_AM_SKIP;
diff --git a/tests/qemucommandutiltest.c b/tests/qemucommandutiltest.c
index 8e57a1b79d..9b13dde63f 100644
--- a/tests/qemucommandutiltest.c
+++ b/tests/qemucommandutiltest.c
@@ -76,6 +76,11 @@ mymain(void)
 int ret = 0;
 testQemuCommandBuildObjectFromJSONData data1;
 
+#if !WITH_STABLE_ORDERING_JANSSON
+fputs("libvirt not compiled with recent enough Jansson, skipping this 
test\n", stderr);
+return EXIT_AM_SKIP;
+#endif
+
 #if !WITH_JANSSON
 fputs("libvirt not compiled with JSON support, skipping this test\n", 
stderr);
 return EXIT_AM_SKIP;
diff --git a/tests/qemuhotplugtest.c b/tests/qemuhotplugtest.c
index c7bc69ab32..674ba92b27 100644
--- a/tests/qemuhotplugtest.c
+++ b/tests/qemuhotplugtest.c
@@ -589,6 +589,11 @@ mymain(void)
 struct qemuHotplugTestData data = {0};
 struct testQemuHotplugCpuParams cpudata;
 
+#if !WITH_STABLE_ORDERING_JANSSON
+fputs("libvirt not compiled with recent enough Jansson, skipping this 
test\n", stderr);
+return EXIT_AM_SKIP;
+#endif
+
 #if !WITH_JANSSON
 fputs("libvirt not compiled with JSON support, skipping this test\n", 
stderr);
 return EXIT_AM_SKIP;
diff --git a/tests/qemumigparamstest.c b/tests/qemumigparamstest.c
index b8af68211b..5e12430991 100644
--- a/tests/qemumigparamstest.c
+++ b/tests/qemumigparamstest.c
@@ -203,6 +203,11 @@ mymain(void)
 virQEMUDriver driver;
 int ret = 0;
 
+#if !WITH_STABLE_ORDERING_JANSSON
+fputs("libvirt not compiled with recent enough Jansson, skipping this 
test\n", stderr);
+return EXIT_AM_SKIP;
+#endif
+
 #if !WITH_JANSSON
 fputs("libvirt not compiled with JSON support, skipping this test\n", 
stderr);
 return EXIT_AM_SKIP;
diff --git a/tests/qemumonitorjsontest.c b/tests/qemumonitorjsontest.c
index c11615f7ac..1826c4f297 100644
--- a/tests/qemumonitorjsontest.c
+++ b/tests/qemumonitorjsontest.c
@@ -2863,6 +2863,11 @@ mymain(void)
 virJSONValuePtr metaschema = NULL;
 char *metaschemastr = NULL;
 
+#if !WITH_STABLE_ORDERING_JANSSON
+fputs("libvirt not compiled with recent enough Jansson, skipping this 
test\n", stderr);
+return EXIT_AM_SKIP;
+#endif
+
 #if !WITH_JANSSON
 fputs("libvirt not compiled with JSON support, skipping this test\n", 
stderr);
 return EXIT_AM_SKIP;
diff --git a/tests/virjsontest.c b/tests/virjsontest.c
index d42413d11d..d352d370fd 100644
--- a/tests/virjsontest.c
+++ b/tests/virjsontest.c
@@ -479,6 +479,11 @@ mymain(void)
 {
 int ret = 0;
 
+#if !WITH_STABLE_ORDERING_JANSSON
+fputs("libvirt not compiled with recent enough Jansson, skipping this 
test\n", stderr);
+return EXIT_AM_SKIP;
+#endif
+
 #define DO_TEST_FULL(name, cmd, doc, expect, pass) \
 do { \
 struct testInfo i

[libvirt] [PATCHv4 12/15] Remove virJSONValueNewStringLen

2018-07-18 Thread Ján Tomko
It is no longer used.

Signed-off-by: Ján Tomko 
---
 src/libvirt_private.syms |  1 -
 src/util/virjson.c   | 22 --
 src/util/virjson.h   |  1 -
 3 files changed, 24 deletions(-)

diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms
index 1caecb96b6..c27c224e93 100644
--- a/src/libvirt_private.syms
+++ b/src/libvirt_private.syms
@@ -2096,7 +2096,6 @@ virJSONValueNewNumberUint;
 virJSONValueNewNumberUlong;
 virJSONValueNewObject;
 virJSONValueNewString;
-virJSONValueNewStringLen;
 virJSONValueObjectAdd;
 virJSONValueObjectAddVArgs;
 virJSONValueObjectAppend;
diff --git a/src/util/virjson.c b/src/util/virjson.c
index 80274bc6c5..01a387b2f7 100644
--- a/src/util/virjson.c
+++ b/src/util/virjson.c
@@ -420,28 +420,6 @@ virJSONValueNewString(const char *data)
 }
 
 
-virJSONValuePtr
-virJSONValueNewStringLen(const char *data,
- size_t length)
-{
-virJSONValuePtr val;
-
-if (!data)
-return virJSONValueNewNull();
-
-if (VIR_ALLOC(val) < 0)
-return NULL;
-
-val->type = VIR_JSON_TYPE_STRING;
-if (VIR_STRNDUP(val->data.string, data, length) < 0) {
-VIR_FREE(val);
-return NULL;
-}
-
-return val;
-}
-
-
 static virJSONValuePtr
 virJSONValueNewNumber(const char *data)
 {
diff --git a/src/util/virjson.h b/src/util/virjson.h
index 75f7f17b44..0d5a7ef753 100644
--- a/src/util/virjson.h
+++ b/src/util/virjson.h
@@ -59,7 +59,6 @@ int virJSONValueObjectAddVArgs(virJSONValuePtr obj, va_list 
args)
 
 
 virJSONValuePtr virJSONValueNewString(const char *data);
-virJSONValuePtr virJSONValueNewStringLen(const char *data, size_t length);
 virJSONValuePtr virJSONValueNewNumberInt(int data);
 virJSONValuePtr virJSONValueNewNumberUint(unsigned int data);
 virJSONValuePtr virJSONValueNewNumberLong(long long data);
-- 
2.16.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH v4 02/10] qemu: qemuBuildHostdevCommandLine: Use a helper variable mdevsrc

2018-07-18 Thread Ján Tomko

On Wed, Jul 18, 2018 at 02:31:38PM +0200, Erik Skultety wrote:

Decrease the number of accessors we have to use.

Signed-off-by: Erik Skultety 
---
src/qemu/qemu_command.c | 5 -
1 file changed, 4 insertions(+), 1 deletion(-)



Reviewed-by: Ján Tomko 

Jano


signature.asc
Description: Digital signature
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH v4 04/10] qemu: Introduce a new graphics display type 'headless'

2018-07-18 Thread Ján Tomko

On Wed, Jul 18, 2018 at 02:31:40PM +0200, Erik Skultety wrote:

Since 2.10 QEMU supports a new display type egl-headless which uses the
drm nodes for OpenGL rendering copying back the rendered bits back to
QEMU into a dma-buf which can be accessed by standard "display" apps
like VNC or SPICE. Although this display type can be used on its own,
for any practical use case it makes sense to pair it with either VNC or
SPICE display. The clear benefit of this display is that VNC gains
OpenGL support, which it natively doesn't have, and SPICE gains remote
OpenGL support (native OpenGL support only works locally through a UNIX
socket, i.e. listen type=socket/none)

Signed-off-by: Erik Skultety 
---
docs/formatdomain.html.in  | 33 +++-
docs/schemas/domaincommon.rng  |  3 ++
src/conf/domain_conf.c |  6 ++-
src/conf/domain_conf.h |  1 +
src/libxl/libxl_conf.c |  1 +
src/qemu/qemu_command.c| 14 -
src/qemu/qemu_domain.c | 60 +-
src/qemu/qemu_driver.c |  2 +
src/qemu/qemu_hotplug.c|  1 +
src/qemu/qemu_process.c|  4 ++
src/vmx/vmx.c  |  1 +
tests/domaincapsschemadata/full.xml|  1 +
tests/qemuxml2argvdata/graphics-egl-headless.args  | 26 ++
tests/qemuxml2argvdata/graphics-egl-headless.xml   | 31 +++
.../qemuxml2argvdata/graphics-sdl-egl-headless.xml | 35 +
.../graphics-spice-egl-headless.args   | 31 +++
.../graphics-spice-egl-headless.xml| 36 +
.../graphics-spice-invalid-egl-headless.xml| 37 +
.../graphics-vnc-egl-headless.args | 28 ++
.../qemuxml2argvdata/graphics-vnc-egl-headless.xml | 37 +
tests/qemuxml2argvtest.c   | 17 ++
.../graphics-spice-egl-headless.xml| 44 
.../graphics-vnc-egl-headless.xml  | 42 +++
tests/qemuxml2xmltest.c|  2 +
24 files changed, 488 insertions(+), 5 deletions(-)
create mode 100644 tests/qemuxml2argvdata/graphics-egl-headless.args
create mode 100644 tests/qemuxml2argvdata/graphics-egl-headless.xml
create mode 100644 tests/qemuxml2argvdata/graphics-sdl-egl-headless.xml
create mode 100644 tests/qemuxml2argvdata/graphics-spice-egl-headless.args
create mode 100644 tests/qemuxml2argvdata/graphics-spice-egl-headless.xml
create mode 100644 
tests/qemuxml2argvdata/graphics-spice-invalid-egl-headless.xml
create mode 100644 tests/qemuxml2argvdata/graphics-vnc-egl-headless.args
create mode 100644 tests/qemuxml2argvdata/graphics-vnc-egl-headless.xml
create mode 100644 tests/qemuxml2xmloutdata/graphics-spice-egl-headless.xml
create mode 100644 tests/qemuxml2xmloutdata/graphics-vnc-egl-headless.xml



Reviewed-by: Ján Tomko 

Jano


signature.asc
Description: Digital signature
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH v4 08/10] conf: Introduce new attribute 'display'

2018-07-18 Thread Ján Tomko

On Wed, Jul 18, 2018 at 02:31:44PM +0200, Erik Skultety wrote:

QEMU 2.12 introduced a new type of display for mediated devices using
vfio-pci backend which allows a mediated device to be used as a VGA
compatible device as an alternative to an emulated video device. QEMU
exposes this feature via a vfio device property 'display' with supported
values 'on/off/auto' (libvirt will default to 'off').

This patch adds the necessary bits to domain config handling in order to
expose this feature. Since there's no convenient way for libvirt to come
up with usable defaults for the display setting, simply because libvirt
is not able to figure out which of the display implementations - dma-buf
which requires OpenGL support vs vfio regions which doesn't need OpenGL
(works with OpenGL enabled too) - the underlying mdev uses.

Signed-off-by: Erik Skultety 
Reviewed-by: Ján Tomko 
---
docs/formatdomain.html.in | 20 -
docs/schemas/domaincommon.rng |  5 ++
src/conf/domain_conf.c| 19 -
src/conf/domain_conf.h|  1 +
src/qemu/qemu_domain.c| 94 ++-
tests/qemuxml2argvdata/hostdev-mdev-display.xml   | 39 ++
tests/qemuxml2xmloutdata/hostdev-mdev-display.xml | 47 
tests/qemuxml2xmltest.c   |  1 +
8 files changed, 220 insertions(+), 6 deletions(-)
create mode 100644 tests/qemuxml2argvdata/hostdev-mdev-display.xml
create mode 100644 tests/qemuxml2xmloutdata/hostdev-mdev-display.xml




@@ -6286,6 +6373,9 @@ qemuDomainDeviceDefPostParse(virDomainDeviceDefPtr dev,
case VIR_DOMAIN_DEVICE_INPUT:
case VIR_DOMAIN_DEVICE_SOUND:
case VIR_DOMAIN_DEVICE_HOSTDEV:
+ret = qemuDomainHostdevDefPostParse(dev->data.hostdev, qemuCaps);
+break;


This should only be called for VIR_DOMAIN_DEVICE_HOSTDEV, not the other
device types listed above.

Jano


+
case VIR_DOMAIN_DEVICE_WATCHDOG:
case VIR_DOMAIN_DEVICE_GRAPHICS:
case VIR_DOMAIN_DEVICE_HUB:


signature.asc
Description: Digital signature
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCHv4 12/15] Remove virJSONValueNewStringLen

2018-07-18 Thread Peter Krempa
On Wed, Jul 18, 2018 at 16:44:37 +0200, Ján Tomko wrote:
> It is no longer used.
> 
> Signed-off-by: Ján Tomko 
> ---

ACK


signature.asc
Description: PGP signature
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH v4 09/10] qemu: command: Enable formatting vfio-pci.display option onto cmdline

2018-07-18 Thread Ján Tomko

On Wed, Jul 18, 2018 at 02:31:45PM +0200, Erik Skultety wrote:

Since QEMU 2.12, QEMU understands a new vfio-pci device option 'display'
which can be used to turn on display capabilities on vgpu-enabled
mediated devices, IOW emulated GPU devices like QXL will no longer be
needed with vgpu-enable mdevs.
QEMU defaults to 'auto' for the 'display' attribute, which is not
foolproof, so we need to play it safe here and explicitly format
display='off' if this attribute wasn't provided in the XML explicitly.


Outdated comment, now that we set it in postParse.



Signed-off-by: Erik Skultety 
---
src/qemu/qemu_command.c|  4 +++
.../hostdev-mdev-display-missing-graphics.xml  | 35 ++
...v-display-spice-egl-headless.x86_64-latest.args | 37 +++
.../hostdev-mdev-display-spice-egl-headless.xml| 40 +
...ev-mdev-display-spice-opengl.x86_64-latest.args | 36 +++
.../hostdev-mdev-display-spice-opengl.xml  | 41 ++
...dev-display-vnc-egl-headless.x86_64-latest.args | 37 +++
.../hostdev-mdev-display-vnc-egl-headless.xml  | 40 +
.../hostdev-mdev-display-vnc.x86_64-latest.args| 36 +++
.../qemuxml2argvdata/hostdev-mdev-display-vnc.xml  | 39 
tests/qemuxml2argvtest.c   |  7 
11 files changed, 352 insertions(+)
create mode 100644 
tests/qemuxml2argvdata/hostdev-mdev-display-missing-graphics.xml
create mode 100644 
tests/qemuxml2argvdata/hostdev-mdev-display-spice-egl-headless.x86_64-latest.args
create mode 100644 
tests/qemuxml2argvdata/hostdev-mdev-display-spice-egl-headless.xml
create mode 100644 
tests/qemuxml2argvdata/hostdev-mdev-display-spice-opengl.x86_64-latest.args
create mode 100644 tests/qemuxml2argvdata/hostdev-mdev-display-spice-opengl.xml
create mode 100644 
tests/qemuxml2argvdata/hostdev-mdev-display-vnc-egl-headless.x86_64-latest.args
create mode 100644 
tests/qemuxml2argvdata/hostdev-mdev-display-vnc-egl-headless.xml
create mode 100644 
tests/qemuxml2argvdata/hostdev-mdev-display-vnc.x86_64-latest.args
create mode 100644 tests/qemuxml2argvdata/hostdev-mdev-display-vnc.xml

diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c
index 7c44045d61..c61da13893 100644
--- a/src/qemu/qemu_command.c
+++ b/src/qemu/qemu_command.c
@@ -5207,6 +5207,10 @@ qemuBuildHostdevMediatedDevStr(const virDomainDef *def,
virBufferAdd(&buf, dev_str, -1);
virBufferAsprintf(&buf, ",id=%s,sysfsdev=%s", dev->info->alias, mdevPath);

+if (mdevsrc->display != VIR_TRISTATE_SWITCH_ABSENT)


To play it completely safe, we could add
 && qemuCapsGet(QEMU_CAPS_VFIO_PCI_DISPLAY)
here, and allow having it set to VIR_TRISTATE_SWITCH_OFF in 
qemuDomainMdevDefValidate.

But I'm not sure whether it's worth the trouble.

Regardless
Reviewed-by: Ján Tomko 

Jano


+  virTristateSwitchTypeToString(mdevsrc->display));
+
if (qemuBuildDeviceAddressStr(&buf, def, dev->info, qemuCaps) < 0)
goto cleanup;



signature.asc
Description: Digital signature
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCHv4 15/15] m4: Introduce STABLE_ORDERING_JANSSON

2018-07-18 Thread Peter Krempa
On Wed, Jul 18, 2018 at 16:44:40 +0200, Ján Tomko wrote:
> Add a second check for Jansson >= 2.8, which includes
> fixes to preserve ordering of object keys.
> 
> Use this constant to guard tests that depend on stable ordering.
> 
> Signed-off-by: Ján Tomko 
> ---
>  m4/virt-jansson.m4   | 3 +++
>  tests/qemublocktest.c| 5 +
>  tests/qemucapabilitiestest.c | 5 +
>  tests/qemucommandutiltest.c  | 5 +
>  tests/qemuhotplugtest.c  | 5 +
>  tests/qemumigparamstest.c| 5 +
>  tests/qemumonitorjsontest.c  | 5 +
>  tests/virjsontest.c  | 5 +
>  tests/virmacmaptest.c| 5 +
>  tests/virnetdaemontest.c | 5 +
>  10 files changed, 48 insertions(+)

Disclamer: I don't have old enough jansson to test this.

ACK, way better than messing with all the data files.


signature.asc
Description: PGP signature
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATCH 6/9] qemu: json: Extract gathering of block statistics

2018-07-18 Thread Peter Krempa
The code is useful also when gathering statistics per node name, so
extract it to a separate functions.

Signed-off-by: Peter Krempa 
---
 src/qemu/qemu_monitor_json.c | 51 ++--
 1 file changed, 35 insertions(+), 16 deletions(-)

diff --git a/src/qemu/qemu_monitor_json.c b/src/qemu/qemu_monitor_json.c
index e41c1d47aa..37a4e59189 100644
--- a/src/qemu/qemu_monitor_json.c
+++ b/src/qemu/qemu_monitor_json.c
@@ -2253,26 +2253,15 @@ int qemuMonitorJSONGetBlockInfo(qemuMonitorPtr mon,
 }


-static int
-qemuMonitorJSONGetOneBlockStatsInfo(virJSONValuePtr dev,
-const char *dev_name,
-int depth,
-virHashTablePtr hash,
-bool backingChain)
+static qemuBlockStatsPtr
+qemuMonitorJSONBlockStatsCollectData(virJSONValuePtr dev,
+ int *nstats)
 {
 qemuBlockStatsPtr bstats = NULL;
-virJSONValuePtr stats;
+qemuBlockStatsPtr ret = NULL;
 virJSONValuePtr parent;
 virJSONValuePtr parentstats;
-int ret = -1;
-int nstats = 0;
-char *entry_name = qemuDomainStorageAlias(dev_name, depth);
-virJSONValuePtr backing;
-
-if (!entry_name)
-goto cleanup;
-if (VIR_ALLOC(bstats) < 0)
-goto cleanup;
+virJSONValuePtr stats;

 if ((stats = virJSONValueObjectGetObject(dev, "stats")) == NULL) {
 virReportError(VIR_ERR_INTERNAL_ERROR, "%s",
@@ -2281,6 +2270,9 @@ qemuMonitorJSONGetOneBlockStatsInfo(virJSONValuePtr dev,
 goto cleanup;
 }

+if (VIR_ALLOC(bstats) < 0)
+goto cleanup;
+
 #define QEMU_MONITOR_BLOCK_STAT_GET(NAME, VAR, MANDATORY) \
 if (MANDATORY || virJSONValueObjectHasKey(stats, NAME)) { \
 nstats++; \
@@ -2307,6 +2299,33 @@ qemuMonitorJSONGetOneBlockStatsInfo(virJSONValuePtr dev,
 bstats->wr_highest_offset_valid = true;
 }

+VIR_STEAL_PTR(ret, bstats);
+
+ cleanup:
+VIR_FREE(bstats);
+return ret;
+}
+
+
+static int
+qemuMonitorJSONGetOneBlockStatsInfo(virJSONValuePtr dev,
+const char *dev_name,
+int depth,
+virHashTablePtr hash,
+bool backingChain)
+{
+qemuBlockStatsPtr bstats = NULL;
+int ret = -1;
+int nstats = 0;
+char *entry_name = qemuDomainStorageAlias(dev_name, depth);
+virJSONValuePtr backing;
+
+if (!entry_name)
+goto cleanup;
+
+if (!(bstats = qemuMonitorJSONBlockStatsCollectData(dev, &nstats)))
+goto cleanup;
+
 if (virHashAddEntry(hash, entry_name, bstats) < 0)
 goto cleanup;
 bstats = NULL;
-- 
2.16.2

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list


[libvirt] [PATCH 1/9] qemu: driver: Reuse qemuDomainBlocksStatsGather in qemuDomainGetBlockInfo

2018-07-18 Thread Peter Krempa
Allow updating capacity for the block devices returned by
qemuDomainBlocksStatsGather and replace the open-coded call to
qemuMonitorGetAllBlockStatsInfo by the helper.

Signed-off-by: Peter Krempa 
---
 src/qemu/qemu_driver.c | 41 -
 1 file changed, 12 insertions(+), 29 deletions(-)

diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c
index 847dab2edc..4f2855a26b 100644
--- a/src/qemu/qemu_driver.c
+++ b/src/qemu/qemu_driver.c
@@ -11063,6 +11063,7 @@ qemuDomainBlockStatsGatherTotals(void *payload,
  * @driver: driver object
  * @vm: domain object
  * @path: to gather the statistics for
+ * @capacity: refresh capacity of the backing image
  * @retstats: returns pointer to structure holding the stats
  *
  * Gathers the block statistics for use in qemuDomainBlockStats* APIs.
@@ -11073,6 +11074,7 @@ static int
 qemuDomainBlocksStatsGather(virQEMUDriverPtr driver,
 virDomainObjPtr vm,
 const char *path,
+bool capacity,
 qemuBlockStatsPtr *retstats)
 {
 qemuDomainObjPrivatePtr priv = vm->privateData;
@@ -11101,6 +11103,11 @@ qemuDomainBlocksStatsGather(virQEMUDriverPtr driver,

 qemuDomainObjEnterMonitor(driver, vm);
 nstats = qemuMonitorGetAllBlockStatsInfo(priv->mon, &blockstats, false);
+
+if (capacity && nstats >= 0 &&
+qemuMonitorBlockStatsUpdateCapacity(priv->mon, blockstats, false) < 0)
+nstats = -1;
+
 if (qemuDomainObjExitMonitor(driver, vm) < 0 || nstats < 0)
 goto cleanup;

@@ -11154,7 +11161,7 @@ qemuDomainBlockStats(virDomainPtr dom,
 if (virDomainObjCheckActive(vm) < 0)
 goto endjob;

-if (qemuDomainBlocksStatsGather(driver, vm, path, &blockstats) < 0)
+if (qemuDomainBlocksStatsGather(driver, vm, path, false, &blockstats) < 0)
 goto endjob;

 stats->rd_req = blockstats->rd_req;
@@ -11208,7 +11215,7 @@ qemuDomainBlockStatsFlags(virDomainPtr dom,
 if (virDomainObjCheckActive(vm) < 0)
 goto endjob;

-if ((nstats = qemuDomainBlocksStatsGather(driver, vm, path,
+if ((nstats = qemuDomainBlocksStatsGather(driver, vm, path, false,
   &blockstats)) < 0)
 goto endjob;

@@ -12021,10 +12028,7 @@ qemuDomainGetBlockInfo(virDomainPtr dom,
 int ret = -1;
 virDomainDiskDefPtr disk;
 virQEMUDriverConfigPtr cfg = NULL;
-int rc;
-virHashTablePtr stats = NULL;
-qemuBlockStats *entry;
-char *alias = NULL;
+qemuBlockStatsPtr entry = NULL;

 virCheckFlags(0, -1);

@@ -12065,28 +12069,8 @@ qemuDomainGetBlockInfo(virDomainPtr dom,
 goto endjob;
 }

-if (!disk->info.alias ||
-!(alias = qemuDomainStorageAlias(disk->info.alias, 0))) {
-virReportError(VIR_ERR_INTERNAL_ERROR,
-   _("missing disk device alias name for %s"), disk->dst);
+if (qemuDomainBlocksStatsGather(driver, vm, path, true, &entry) < 0)
 goto endjob;
-}
-
-qemuDomainObjEnterMonitor(driver, vm);
-rc = qemuMonitorGetAllBlockStatsInfo(qemuDomainGetMonitor(vm),
- &stats, false);
-if (rc >= 0)
-rc = qemuMonitorBlockStatsUpdateCapacity(qemuDomainGetMonitor(vm),
- stats, false);
-
-if (qemuDomainObjExitMonitor(driver, vm) < 0 || rc < 0)
-goto endjob;
-
-if (!(entry = virHashLookup(stats, alias))) {
-virReportError(VIR_ERR_INTERNAL_ERROR,
-   _("failed to gather stats for disk '%s'"), disk->dst);
-goto endjob;
-}

 if (!entry->wr_highest_offset_valid) {
 info->allocation = entry->physical;
@@ -12131,8 +12115,7 @@ qemuDomainGetBlockInfo(virDomainPtr dom,
  endjob:
 qemuDomainObjEndJob(driver, vm);
  cleanup:
-VIR_FREE(alias);
-virHashFree(stats);
+VIR_FREE(entry);
 virDomainObjEndAPI(&vm);
 virObjectUnref(cfg);
 return ret;
-- 
2.16.2

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list


[libvirt] [PATCH 0/9] qemu: stats/misc refactors (blockdev-add saga)

2018-07-18 Thread Peter Krempa
Last lot of refactors which could be split of prior to starting it for
real.

Peter Krempa (9):
  qemu: driver: Reuse qemuDomainBlocksStatsGather in
qemuDomainGetBlockInfo
  qemu: domain: Move out clearing of backing chain in
qemuDomainDetermineDiskChain
  qemu: domain: Add helper for getting the disk backend alias
  qemu: command: use qemuDomainDiskGetBackendAlias in commandline
building
  qemu: monitor: Add the 'query-nodes' argument for query-blockstats
  qemu: json: Extract gathering of block statistics
  qemu: monitor: Split out code to gather data from 'query-block'
  utils: storage: Add helper for checking if storage source is the same
  qemu: Replace qemuDomainDiskSourceDiffers by
virStorageSourceIsSameLocation

 src/libvirt_private.syms |   1 +
 src/qemu/qemu_block.c|   2 +-
 src/qemu/qemu_blockjob.c |   4 +-
 src/qemu/qemu_command.c  |  24 ++
 src/qemu/qemu_domain.c   |  66 ---
 src/qemu/qemu_domain.h   |  10 +++--
 src/qemu/qemu_driver.c   |  47 +++-
 src/qemu/qemu_hotplug.c  |   2 +-
 src/qemu/qemu_monitor.c  |   8 +++-
 src/qemu/qemu_monitor.h  |   3 +-
 src/qemu/qemu_monitor_json.c | 103 ---
 src/qemu/qemu_monitor_json.h |   3 +-
 src/qemu/qemu_process.c  |   7 ++-
 src/util/virstoragefile.c|  43 ++
 src/util/virstoragefile.h|   3 ++
 15 files changed, 198 insertions(+), 128 deletions(-)

-- 
2.16.2

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list


[libvirt] [PATCH 4/9] qemu: command: use qemuDomainDiskGetBackendAlias in commandline building

2018-07-18 Thread Peter Krempa
Use the proper backend for the block device both when using -drive and
when using -blockdev for disk drives and floppy disks.

Signed-off-by: Peter Krempa 
---
 src/qemu/qemu_command.c | 24 +++-
 1 file changed, 15 insertions(+), 9 deletions(-)

diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c
index 44ae8dcef7..6e550cf951 100644
--- a/src/qemu/qemu_command.c
+++ b/src/qemu/qemu_command.c
@@ -1818,7 +1818,7 @@ qemuBuildDiskDeviceStr(const virDomainDef *def,
 virBuffer opt = VIR_BUFFER_INITIALIZER;
 const char *bus = virDomainDiskQEMUBusTypeToString(disk->bus);
 const char *contAlias;
-char *drivealias;
+char *backendAlias = NULL;
 int controllerModel;

 if (qemuCheckDiskConfig(disk, qemuCaps) < 0)
@@ -2077,10 +2077,14 @@ qemuBuildDiskDeviceStr(const virDomainDef *def,
 virQEMUCapsGet(qemuCaps, QEMU_CAPS_DISK_SHARE_RW))
 virBufferAddLit(&opt, ",share-rw=on");

-if (!(drivealias = qemuAliasDiskDriveFromDisk(disk)))
+if (qemuDomainDiskGetBackendAlias(disk, qemuCaps, &backendAlias) < 0)
 goto error;
-virBufferAsprintf(&opt, ",drive=%s,id=%s", drivealias, disk->info.alias);
-VIR_FREE(drivealias);
+
+if (backendAlias)
+virBufferAsprintf(&opt, ",drive=%s", backendAlias);
+VIR_FREE(backendAlias);
+
+virBufferAsprintf(&opt, ",id=%s", disk->info.alias);
 if (bootindex && virQEMUCapsGet(qemuCaps, QEMU_CAPS_BOOTINDEX))
 virBufferAsprintf(&opt, ",bootindex=%u", bootindex);
 if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_BLOCKIO)) {
@@ -2139,6 +2143,7 @@ qemuBuildDiskDeviceStr(const virDomainDef *def,
 return virBufferContentAndReset(&opt);

  error:
+VIR_FREE(backendAlias);
 virBufferFreeAndReset(&opt);
 return NULL;
 }
@@ -2148,8 +2153,8 @@ static int
 qemuBuildFloppyCommandLineOptions(virCommandPtr cmd,
   const virDomainDef *def,
   virDomainDiskDefPtr disk,
+  virQEMUCapsPtr qemuCaps,
   unsigned int bootindex)
-
 {
 virBuffer fdc_opts = VIR_BUFFER_INITIALIZER;
 char driveLetter;
@@ -2163,10 +2168,11 @@ qemuBuildFloppyCommandLineOptions(virCommandPtr cmd,
 else
 driveLetter = 'A';

-if (!(backendAlias = qemuAliasDiskDriveFromDisk(disk)))
-return -1;
+if (qemuDomainDiskGetBackendAlias(disk, qemuCaps, &backendAlias) < 0)
+goto cleanup;

-if (virAsprintf(&backendStr, "drive%c=%s", driveLetter, backendAlias) < 0)
+if (backendAlias &&
+virAsprintf(&backendStr, "drive%c=%s", driveLetter, backendAlias) < 0)
 goto cleanup;

 if (bootindex &&
@@ -2281,7 +2287,7 @@ qemuBuildDiskCommandLine(virCommandPtr cmd,

 if (!qemuDiskBusNeedsDriveArg(disk->bus)) {
 if (disk->bus == VIR_DOMAIN_DISK_BUS_FDC) {
-if (qemuBuildFloppyCommandLineOptions(cmd, def, disk,
+if (qemuBuildFloppyCommandLineOptions(cmd, def, disk, qemuCaps,
   bootindex) < 0)
 return -1;
 } else {
-- 
2.16.2

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list


[libvirt] [PATCH 5/9] qemu: monitor: Add the 'query-nodes' argument for query-blockstats

2018-07-18 Thread Peter Krempa
The 'query-blockstats' command does not return statistics for the
explicitly named nodes unless the new argument is specified. Add
infrastrucuture that will allow us to use the new approach if desired.

Signed-off-by: Peter Krempa 
---
 src/qemu/qemu_block.c| 2 +-
 src/qemu/qemu_monitor.c  | 8 ++--
 src/qemu/qemu_monitor.h  | 3 ++-
 src/qemu/qemu_monitor_json.c | 9 ++---
 src/qemu/qemu_monitor_json.h | 3 ++-
 5 files changed, 17 insertions(+), 8 deletions(-)

diff --git a/src/qemu/qemu_block.c b/src/qemu/qemu_block.c
index 7ad79c7e7d..a633bfa0b0 100644
--- a/src/qemu/qemu_block.c
+++ b/src/qemu/qemu_block.c
@@ -356,7 +356,7 @@ qemuBlockNodeNamesDetect(virQEMUDriverPtr driver,
 return -1;

 data = qemuMonitorQueryNamedBlockNodes(qemuDomainGetMonitor(vm));
-blockstats = qemuMonitorQueryBlockstats(qemuDomainGetMonitor(vm));
+blockstats = qemuMonitorQueryBlockstats(qemuDomainGetMonitor(vm), false);

 if (qemuDomainObjExitMonitor(driver, vm) < 0 || !data || !blockstats)
 goto cleanup;
diff --git a/src/qemu/qemu_monitor.c b/src/qemu/qemu_monitor.c
index bc116e4e2d..8618a44f32 100644
--- a/src/qemu/qemu_monitor.c
+++ b/src/qemu/qemu_monitor.c
@@ -2243,15 +2243,19 @@ qemuMonitorGetBlockInfo(qemuMonitorPtr mon)
 /**
  * qemuMonitorQueryBlockstats:
  * @mon: monitor object
+ * @nodenames: include backing chain nodes with explicitly specified name
  *
  * Returns data from a call to 'query-blockstats'.
  */
 virJSONValuePtr
-qemuMonitorQueryBlockstats(qemuMonitorPtr mon)
+qemuMonitorQueryBlockstats(qemuMonitorPtr mon,
+   bool nodenames)
 {
 QEMU_CHECK_MONITOR_NULL(mon);

-return qemuMonitorJSONQueryBlockstats(mon);
+VIR_DEBUG("nodenames: %d", nodenames);
+
+return qemuMonitorJSONQueryBlockstats(mon, nodenames);
 }


diff --git a/src/qemu/qemu_monitor.h b/src/qemu/qemu_monitor.h
index 81474a04f6..b4b9cf91eb 100644
--- a/src/qemu/qemu_monitor.h
+++ b/src/qemu/qemu_monitor.h
@@ -563,7 +563,8 @@ int qemuMonitorSetMemoryStatsPeriod(qemuMonitorPtr mon,
 int qemuMonitorBlockIOStatusToError(const char *status);
 virHashTablePtr qemuMonitorGetBlockInfo(qemuMonitorPtr mon);

-virJSONValuePtr qemuMonitorQueryBlockstats(qemuMonitorPtr mon);
+virJSONValuePtr qemuMonitorQueryBlockstats(qemuMonitorPtr mon,
+   bool nodenames);

 typedef struct _qemuBlockStats qemuBlockStats;
 typedef qemuBlockStats *qemuBlockStatsPtr;
diff --git a/src/qemu/qemu_monitor_json.c b/src/qemu/qemu_monitor_json.c
index cc3c8f2dd6..e41c1d47aa 100644
--- a/src/qemu/qemu_monitor_json.c
+++ b/src/qemu/qemu_monitor_json.c
@@ -2326,13 +2326,16 @@ qemuMonitorJSONGetOneBlockStatsInfo(virJSONValuePtr dev,


 virJSONValuePtr
-qemuMonitorJSONQueryBlockstats(qemuMonitorPtr mon)
+qemuMonitorJSONQueryBlockstats(qemuMonitorPtr mon,
+   bool nodenames)
 {
 virJSONValuePtr cmd;
 virJSONValuePtr reply = NULL;
 virJSONValuePtr ret = NULL;

-if (!(cmd = qemuMonitorJSONMakeCommand("query-blockstats", NULL)))
+if (!(cmd = qemuMonitorJSONMakeCommand("query-blockstats",
+   "B:query-nodes", nodenames,
+   NULL)))
 return NULL;

 if (qemuMonitorJSONCommand(mon, cmd, &reply) < 0)
@@ -2361,7 +2364,7 @@ qemuMonitorJSONGetAllBlockStatsInfo(qemuMonitorPtr mon,
 size_t i;
 virJSONValuePtr devices;

-if (!(devices = qemuMonitorJSONQueryBlockstats(mon)))
+if (!(devices = qemuMonitorJSONQueryBlockstats(mon, false)))
 return -1;

 for (i = 0; i < virJSONValueArraySize(devices); i++) {
diff --git a/src/qemu/qemu_monitor_json.h b/src/qemu/qemu_monitor_json.h
index 66536ceb97..a6b6579849 100644
--- a/src/qemu/qemu_monitor_json.h
+++ b/src/qemu/qemu_monitor_json.h
@@ -82,7 +82,8 @@ int qemuMonitorJSONSetMemoryStatsPeriod(qemuMonitorPtr mon,
 int qemuMonitorJSONGetBlockInfo(qemuMonitorPtr mon,
 virHashTablePtr table);

-virJSONValuePtr qemuMonitorJSONQueryBlockstats(qemuMonitorPtr mon);
+virJSONValuePtr qemuMonitorJSONQueryBlockstats(qemuMonitorPtr mon,
+   bool nodenames);
 int qemuMonitorJSONGetAllBlockStatsInfo(qemuMonitorPtr mon,
 virHashTablePtr hash,
 bool backingChain);
-- 
2.16.2

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list


[libvirt] [PATCH 2/9] qemu: domain: Move out clearing of backing chain in qemuDomainDetermineDiskChain

2018-07-18 Thread Peter Krempa
In some cases backing chain needs to be cleared prior to re-detection.
Move this step out of qemuDomainDetermineDiskChain as only certain
places need it and the function itself is able to skip to the end of the
chain to perform detection.

Signed-off-by: Peter Krempa 
---
 src/qemu/qemu_blockjob.c | 4 ++--
 src/qemu/qemu_domain.c   | 4 
 src/qemu/qemu_domain.h   | 1 -
 src/qemu/qemu_driver.c   | 4 ++--
 src/qemu/qemu_hotplug.c  | 2 +-
 src/qemu/qemu_process.c  | 7 +--
 6 files changed, 10 insertions(+), 12 deletions(-)

diff --git a/src/qemu/qemu_blockjob.c b/src/qemu/qemu_blockjob.c
index b08e047490..0f52996ade 100644
--- a/src/qemu/qemu_blockjob.c
+++ b/src/qemu/qemu_blockjob.c
@@ -175,8 +175,8 @@ qemuBlockJobEventProcess(virQEMUDriverPtr driver,
 disk->mirrorState = VIR_DOMAIN_DISK_MIRROR_STATE_NONE;
 disk->mirrorJob = VIR_DOMAIN_BLOCK_JOB_TYPE_UNKNOWN;
 disk->src->id = 0;
-ignore_value(qemuDomainDetermineDiskChain(driver, vm, disk,
-  true, true));
+virStorageSourceBackingStoreClear(disk->src);
+ignore_value(qemuDomainDetermineDiskChain(driver, vm, disk, true));
 ignore_value(qemuBlockNodeNamesDetect(driver, vm, asyncJob));
 diskPriv->blockjob = false;
 break;
diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c
index ed76495309..c92ac8c926 100644
--- a/src/qemu/qemu_domain.c
+++ b/src/qemu/qemu_domain.c
@@ -8352,7 +8352,6 @@ int
 qemuDomainDetermineDiskChain(virQEMUDriverPtr driver,
  virDomainObjPtr vm,
  virDomainDiskDefPtr disk,
- bool force_probe,
  bool report_broken)
 {
 virQEMUDriverConfigPtr cfg = virQEMUDriverGetConfig(driver);
@@ -8368,9 +8367,6 @@ qemuDomainDetermineDiskChain(virQEMUDriverPtr driver,
 goto cleanup;
 }

-if (force_probe)
-virStorageSourceBackingStoreClear(src);
-
 /* There is no need to check the backing chain for disks without backing
  * support */
 if (virStorageSourceIsLocalStorage(src) &&
diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h
index e748d78adb..1692fa9838 100644
--- a/src/qemu/qemu_domain.h
+++ b/src/qemu/qemu_domain.h
@@ -718,7 +718,6 @@ int qemuDomainCheckDiskPresence(virQEMUDriverPtr driver,
 int qemuDomainDetermineDiskChain(virQEMUDriverPtr driver,
  virDomainObjPtr vm,
  virDomainDiskDefPtr disk,
- bool force_probe,
  bool report_broken);

 bool qemuDomainDiskSourceDiffers(virDomainDiskDefPtr disk,
diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c
index 4f2855a26b..64e2c274c2 100644
--- a/src/qemu/qemu_driver.c
+++ b/src/qemu/qemu_driver.c
@@ -7850,7 +7850,7 @@ qemuDomainChangeDiskLive(virDomainObjPtr vm,
 if (virDomainDiskTranslateSourcePool(disk) < 0)
 goto cleanup;

-if (qemuDomainDetermineDiskChain(driver, vm, disk, false, true) < 0)
+if (qemuDomainDetermineDiskChain(driver, vm, disk, true) < 0)
 goto cleanup;

 if (!(orig_disk = virDomainDiskFindByBusAndDst(vm->def,
@@ -16931,7 +16931,7 @@ qemuDomainBlockPivot(virQEMUDriverPtr driver,
 oldsrc = disk->src;
 disk->src = disk->mirror;

-if (qemuDomainDetermineDiskChain(driver, vm, disk, false, true) < 0)
+if (qemuDomainDetermineDiskChain(driver, vm, disk, true) < 0)
 goto cleanup;

 if (disk->mirror->format &&
diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c
index 2b6633a998..98f7a28826 100644
--- a/src/qemu/qemu_hotplug.c
+++ b/src/qemu/qemu_hotplug.c
@@ -669,7 +669,7 @@ qemuDomainAttachDeviceDiskLive(virQEMUDriverPtr driver,
 if (qemuSetUnprivSGIO(dev) < 0)
 goto cleanup;

-if (qemuDomainDetermineDiskChain(driver, vm, disk, false, true) < 0)
+if (qemuDomainDetermineDiskChain(driver, vm, disk, true) < 0)
 goto cleanup;

 switch ((virDomainDiskDevice) disk->device)  {
diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c
index c903a8e5c8..138e128807 100644
--- a/src/qemu/qemu_process.c
+++ b/src/qemu/qemu_process.c
@@ -6015,7 +6015,9 @@ qemuProcessPrepareHostStorage(virQEMUDriverPtr driver,
 if (virStorageSourceIsEmpty(disk->src))
 continue;

-if (qemuDomainDetermineDiskChain(driver, vm, disk, true, true) >= 0)
+virStorageSourceBackingStoreClear(disk->src);
+
+if (qemuDomainDetermineDiskChain(driver, vm, disk, true) >= 0)
 continue;

 if (qemuDomainCheckDiskStartupPolicy(driver, vm, idx, cold_boot) >= 0)
@@ -7691,7 +7693,8 @@ qemuProcessReconnect(void *opaque)
 /* This should be the only place that calls
  * qemuDomainDetermineDiskChain with @report_broken == false
  * to guarantee best-effort domain reconnect */
-if (qemuDoma

[libvirt] [PATCH 9/9] qemu: Replace qemuDomainDiskSourceDiffers by virStorageSourceIsSameLocation

2018-07-18 Thread Peter Krempa
Now that we have a saner replacement for checking if the disk source is
the same use it instead of formatting qemu command-line chunks.

Signed-off-by: Peter Krempa 
---
 src/qemu/qemu_domain.c | 34 --
 src/qemu/qemu_domain.h |  3 ---
 src/qemu/qemu_driver.c |  2 +-
 3 files changed, 1 insertion(+), 38 deletions(-)

diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c
index f8a621a5c9..acf5cc8dd2 100644
--- a/src/qemu/qemu_domain.c
+++ b/src/qemu/qemu_domain.c
@@ -8555,40 +8555,6 @@ qemuDomainDiskChainElementPrepare(virQEMUDriverPtr 
driver,
 }


-bool
-qemuDomainDiskSourceDiffers(virDomainDiskDefPtr disk,
-virDomainDiskDefPtr origDisk)
-{
-char *diskSrc = NULL, *origDiskSrc = NULL;
-bool diskEmpty, origDiskEmpty;
-bool ret = true;
-
-diskEmpty = virStorageSourceIsEmpty(disk->src);
-origDiskEmpty = virStorageSourceIsEmpty(origDisk->src);
-
-if (diskEmpty && origDiskEmpty)
-return false;
-
-if (diskEmpty ^ origDiskEmpty)
-return true;
-
-/* This won't be a network storage, so no need to get the diskPriv
- * in order to fetch the secret, thus NULL for param2 */
-if (qemuGetDriveSourceString(disk->src, NULL, &diskSrc) < 0 ||
-qemuGetDriveSourceString(origDisk->src, NULL, &origDiskSrc) < 0)
-goto cleanup;
-
-/* So far in qemu disk sources are considered different
- * if either path to disk or its format changes. */
-ret = virDomainDiskGetFormat(disk) != virDomainDiskGetFormat(origDisk) ||
-  STRNEQ_NULLABLE(diskSrc, origDiskSrc);
- cleanup:
-VIR_FREE(diskSrc);
-VIR_FREE(origDiskSrc);
-return ret;
-}
-
-
 /*
  * Makes sure the @disk differs from @orig_disk only by the source
  * path and nothing else.  Fields that are being checked and the
diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h
index 22c3a51354..bff293fc0a 100644
--- a/src/qemu/qemu_domain.h
+++ b/src/qemu/qemu_domain.h
@@ -720,9 +720,6 @@ int qemuDomainDetermineDiskChain(virQEMUDriverPtr driver,
  virDomainDiskDefPtr disk,
  bool report_broken);

-bool qemuDomainDiskSourceDiffers(virDomainDiskDefPtr disk,
- virDomainDiskDefPtr origDisk);
-
 bool qemuDomainDiskChangeSupported(virDomainDiskDefPtr disk,
virDomainDiskDefPtr orig_disk);

diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c
index 64e2c274c2..dea548f982 100644
--- a/src/qemu/qemu_driver.c
+++ b/src/qemu/qemu_driver.c
@@ -7871,7 +7871,7 @@ qemuDomainChangeDiskLive(virDomainObjPtr vm,
 if (!qemuDomainDiskChangeSupported(disk, orig_disk))
 goto cleanup;

-if (qemuDomainDiskSourceDiffers(disk, orig_disk)) {
+if (!virStorageSourceIsSameLocation(disk->src, orig_disk->src)) {
 /* Disk source can be changed only for removable devices */
 if (disk->device != VIR_DOMAIN_DISK_DEVICE_CDROM &&
 disk->device != VIR_DOMAIN_DISK_DEVICE_FLOPPY) {
-- 
2.16.2

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list


[libvirt] [PATCH 3/9] qemu: domain: Add helper for getting the disk backend alias

2018-07-18 Thread Peter Krempa
The disk backend alias was historically the alias of the -drive backing
the storage. For setups with -blockdev this will become more complex as
it will depend on other configs and generally will differ.
---
 src/qemu/qemu_domain.c | 28 
 src/qemu/qemu_domain.h |  6 ++
 2 files changed, 34 insertions(+)

diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c
index c92ac8c926..f8a621a5c9 100644
--- a/src/qemu/qemu_domain.c
+++ b/src/qemu/qemu_domain.c
@@ -8449,6 +8449,34 @@ qemuDomainDetermineDiskChain(virQEMUDriverPtr driver,
 }


+/**
+ * qemuDomainDiskGetBackendAlias:
+ * @disk: disk definition
+ * @qemuCaps: emulator capabilities
+ * @backendAlias: filled with the alias of the disk storage backend
+ *
+ * Returns the correct alias for the disk backend. This may be the alias of
+ * -drive for legacy setup or the correct node name for -blockdev setups.
+ *
+ * @backendAlias may be NULL on success if the backend does not exist
+ * (disk is empty). Caller is responsible for freeing @backendAlias.
+ *
+ * Returns 0 on success, -1 on error with libvirt error reported.
+ */
+int
+qemuDomainDiskGetBackendAlias(virDomainDiskDefPtr disk,
+  virQEMUCapsPtr qemuCaps ATTRIBUTE_UNUSED,
+  char **backendAlias)
+{
+*backendAlias = NULL;
+
+if (!(*backendAlias = qemuAliasDiskDriveFromDisk(disk)))
+return -1;
+
+return 0;
+}
+
+
 /**
  * qemuDomainDiskChainElementRevoke:
  *
diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h
index 1692fa9838..22c3a51354 100644
--- a/src/qemu/qemu_domain.h
+++ b/src/qemu/qemu_domain.h
@@ -732,6 +732,12 @@ int qemuDomainStorageFileInit(virQEMUDriverPtr driver,
   virStorageSourcePtr parent);
 char *qemuDomainStorageAlias(const char *device, int depth);

+int qemuDomainDiskGetBackendAlias(virDomainDiskDefPtr disk,
+  virQEMUCapsPtr qemuCaps,
+  char **backendAlias)
+ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2)
+ATTRIBUTE_NONNULL(3) ATTRIBUTE_RETURN_CHECK;
+
 void qemuDomainDiskChainElementRevoke(virQEMUDriverPtr driver,
   virDomainObjPtr vm,
   virStorageSourcePtr elem);
-- 
2.16.2

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list


[libvirt] [PATCH 7/9] qemu: monitor: Split out code to gather data from 'query-block'

2018-07-18 Thread Peter Krempa
Extract the code for future reuse.

Signed-off-by: Peter Krempa 
---
 src/qemu/qemu_monitor_json.c | 43 +++
 1 file changed, 27 insertions(+), 16 deletions(-)

diff --git a/src/qemu/qemu_monitor_json.c b/src/qemu/qemu_monitor_json.c
index 37a4e59189..81043e2af5 100644
--- a/src/qemu/qemu_monitor_json.c
+++ b/src/qemu/qemu_monitor_json.c
@@ -2423,39 +2423,50 @@ qemuMonitorJSONGetAllBlockStatsInfo(qemuMonitorPtr mon,


 static int
-qemuMonitorJSONBlockStatsUpdateCapacityOne(virJSONValuePtr image,
-   const char *dev_name,
-   int depth,
-   virHashTablePtr stats,
-   bool backingChain)
+qemuMonitorJSONBlockStatsUpdateCapacityData(virJSONValuePtr image,
+const char *name,
+virHashTablePtr stats)
 {
 qemuBlockStatsPtr bstats;
-int ret = -1;
-char *entry_name = qemuDomainStorageAlias(dev_name, depth);
-virJSONValuePtr backing;

-if (!(bstats = virHashLookup(stats, entry_name))) {
+if (!(bstats = virHashLookup(stats, name))) {
 if (VIR_ALLOC(bstats) < 0)
-goto cleanup;
+return -1;

-if (virHashAddEntry(stats, entry_name, bstats) < 0) {
+if (virHashAddEntry(stats, name, bstats) < 0) {
 VIR_FREE(bstats);
-goto cleanup;
+return -1;
 }
 }

-/* After this point, we ignore failures; the stats were
- * zero-initialized when created which is a sane fallback.  */
-ret = 0;
+/* failures can be ignored after this point */
 if (virJSONValueObjectGetNumberUlong(image, "virtual-size",
  &bstats->capacity) < 0)
-goto cleanup;
+return 0;

 /* if actual-size is missing, image is not thin provisioned */
 if (virJSONValueObjectGetNumberUlong(image, "actual-size",
  &bstats->physical) < 0)
 bstats->physical = bstats->capacity;

+return 0;
+}
+
+
+static int
+qemuMonitorJSONBlockStatsUpdateCapacityOne(virJSONValuePtr image,
+   const char *dev_name,
+   int depth,
+   virHashTablePtr stats,
+   bool backingChain)
+{
+int ret = -1;
+char *entry_name = qemuDomainStorageAlias(dev_name, depth);
+virJSONValuePtr backing;
+
+if (qemuMonitorJSONBlockStatsUpdateCapacityData(image, entry_name, stats) 
< 0)
+goto cleanup;
+
 if (backingChain &&
 (backing = virJSONValueObjectGetObject(image, "backing-image"))) {
 ret = qemuMonitorJSONBlockStatsUpdateCapacityOne(backing,
-- 
2.16.2

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list


[libvirt] [PATCH 8/9] utils: storage: Add helper for checking if storage source is the same

2018-07-18 Thread Peter Krempa
To allow checking whether a storage source points to the same location
add a helper which checks the relevant fields. This will allow replacing
a similar check done by formatting the command line arguments for
qemu-like syntax.

Signed-off-by: Peter Krempa 
---
 src/libvirt_private.syms  |  1 +
 src/util/virstoragefile.c | 43 +++
 src/util/virstoragefile.h |  3 +++
 3 files changed, 47 insertions(+)

diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms
index 1caecb96b6..fb7c8c724d 100644
--- a/src/libvirt_private.syms
+++ b/src/libvirt_private.syms
@@ -2855,6 +2855,7 @@ virStorageSourceIsBlockLocal;
 virStorageSourceIsEmpty;
 virStorageSourceIsLocalStorage;
 virStorageSourceIsRelative;
+virStorageSourceIsSameLocation;
 virStorageSourceNetworkAssignDefaultPorts;
 virStorageSourceNewFromBacking;
 virStorageSourceNewFromBackingAbsolute;
diff --git a/src/util/virstoragefile.c b/src/util/virstoragefile.c
index 58f67278da..96ed4b1489 100644
--- a/src/util/virstoragefile.c
+++ b/src/util/virstoragefile.c
@@ -2259,6 +2259,49 @@ virStorageSourceCopy(const virStorageSource *src,
 }


+/**
+ * virStorageSourceIsSameLocation:
+ *
+ * Returns true if the sources @a and @b point to the same storage location.
+ * This does not compare any other configuration option
+ */
+bool
+virStorageSourceIsSameLocation(virStorageSourcePtr a,
+   virStorageSourcePtr b)
+{
+size_t i;
+
+/* there are multiple possibilities to define an empty source */
+if (virStorageSourceIsEmpty(a) &&
+virStorageSourceIsEmpty(b))
+return true;
+
+if (virStorageSourceGetActualType(a) != virStorageSourceGetActualType(b))
+return false;
+
+if (STRNEQ_NULLABLE(a->path, b->path) ||
+STRNEQ_NULLABLE(a->volume, b->volume) ||
+STRNEQ_NULLABLE(a->snapshot, b->snapshot))
+return false;
+
+if (a->type == VIR_STORAGE_TYPE_NETWORK) {
+if (a->protocol != b->protocol ||
+a->nhosts != b->nhosts)
+return false;
+
+for (i = 0; i < a->nhosts; i++) {
+if (a->hosts[i].transport != b->hosts[i].transport ||
+a->hosts[i].port != b->hosts[i].port ||
+STRNEQ_NULLABLE(a->hosts[i].name, b->hosts[i].name) ||
+STRNEQ_NULLABLE(a->hosts[i].socket, b->hosts[i].socket))
+return false;
+}
+}
+
+return true;
+}
+
+
 /**
  * virStorageSourceInitChainElement:
  * @newelem: New backing chain element disk source
diff --git a/src/util/virstoragefile.h b/src/util/virstoragefile.h
index 991098e6c6..c2c40edf68 100644
--- a/src/util/virstoragefile.h
+++ b/src/util/virstoragefile.h
@@ -436,6 +436,9 @@ virStorageSourcePtr 
virStorageSourceNewFromBacking(virStorageSourcePtr parent);
 virStorageSourcePtr virStorageSourceCopy(const virStorageSource *src,
  bool backingChain)
 ATTRIBUTE_NONNULL(1);
+bool virStorageSourceIsSameLocation(virStorageSourcePtr a,
+virStorageSourcePtr b)
+ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2);

 int virStorageSourceParseRBDColonString(const char *rbdstr,
 virStorageSourcePtr src)
-- 
2.16.2

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list


  1   2   >