from:"Eli Qiao"

Re: [libvirt] [PATCH 0/9] Yet another version of CAT stuff (no idea about the version number)

2017-12-13 Thread Eli Qiao

hi Martin,

Thanks for reaching me, sure, I will try to testing your patch.

BTW, do you have a github repo/branch for these  which I can directly pull?

Best regards
- Eli

a leaf duckweed belongs to the sea, where not to meet in life

2017-12-13 23:39 GMT+08:00 Martin Kletzander :

> Added stuff that Pavel wanted, removed some testcases that I still have in
> my
> repo, but it's way to complicated to add properly, so for now I would just
> go
> with this and manual testing of starting some domains.  I got no hardware
> to
> test this on, currently, so some testing is needed before pushing this.
>
> @Eli: Can you help with the testing?
>
> Martin Kletzander (9):
>   Rename virResctrlInfo to virResctrlInfoPerCache
>   util: Add virResctrlInfo
>   conf: Use virResctrlInfo in capabilities
>   util: Remove now-unneeded resctrl functions
>   resctrl: Add functions to work with resctrl allocations
>   conf: Add support for cputune/cachetune
>   tests: Add virresctrltest
>   qemu: Add support for resctrl
>   docs: Add CAT (resctrl) support into news.xml
>
>  docs/formatdomain.html.in  |   54 +
>  docs/news.xml  |9 +
>  docs/schemas/domaincommon.rng  |   32 +
>  po/POTFILES.in |1 +
>  src/Makefile.am|2 +-
>  src/conf/capabilities.c|   55 +-
>  src/conf/capabilities.h|4 +-
>  src/conf/domain_conf.c |  251 
>  src/conf/domain_conf.h |   13 +
>  src/libvirt_private.syms   |   16 +-
>  src/qemu/qemu_process.c|   61 +-
>  src/util/virresctrl.c  | 1380
> ++--
>  src/util/virresctrl.h  |   86 +-
>  src/util/virresctrlpriv.h  |   27 +
>  tests/Makefile.am  |8 +-
>  tests/genericxml2xmlindata/cachetune-cdp.xml   |   36 +
>  .../cachetune-colliding-allocs.xml |   30 +
>  .../cachetune-colliding-tunes.xml  |   32 +
>  .../cachetune-colliding-types.xml  |   30 +
>  tests/genericxml2xmlindata/cachetune-small.xml |   29 +
>  tests/genericxml2xmlindata/cachetune.xml   |   33 +
>  tests/genericxml2xmltest.c |   10 +
>  tests/virresctrldata/resctrl-cdp.schemata  |2 +
>  .../virresctrldata/resctrl-skx-twocaches.schemata  |1 +
>  tests/virresctrldata/resctrl-skx.schemata  |1 +
>  tests/virresctrldata/resctrl.schemata  |1 +
>  tests/virresctrltest.c |  102 ++
>  27 files changed, 2174 insertions(+), 132 deletions(-)
>  create mode 100644 src/util/virresctrlpriv.h
>  create mode 100644 tests/genericxml2xmlindata/cachetune-cdp.xml
>  create mode 100644 tests/genericxml2xmlindata/
> cachetune-colliding-allocs.xml
>  create mode 100644 tests/genericxml2xmlindata/
> cachetune-colliding-tunes.xml
>  create mode 100644 tests/genericxml2xmlindata/
> cachetune-colliding-types.xml
>  create mode 100644 tests/genericxml2xmlindata/cachetune-small.xml
>  create mode 100644 tests/genericxml2xmlindata/cachetune.xml
>  create mode 100644 tests/virresctrldata/resctrl-cdp.schemata
>  create mode 100644 tests/virresctrldata/resctrl-skx-twocaches.schemata
>  create mode 100644 tests/virresctrldata/resctrl-skx.schemata
>  create mode 100644 tests/virresctrldata/resctrl.schemata
>  create mode 100644 tests/virresctrltest.c
>
> --
> 2.15.1
>
>
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] Yet another RFC for CAT

2017-09-12 Thread Eli Qiao

>
>
>
> We didn't want to exec external python programs because that certainly
> *does* have bad scalability, terrible error reporting facilities and
> need to parse ill defined data formats from stdout, etc. It doesn't
> magically solve the complexity, just moves it elsewhere where we have
> less ability to tailor it to fit into libvirt's model.
>
>
>
BTW, to clarify, RMD is not wroten by python, it's golang, and it's not
just a tool,
it's a running service(agent) on the host, provided RESTful API by unix/TCP
socket.
and much more smarter (policy based) then static allocation. Support
re-enforcement
based on monitoring data (cache usage).

It aimed to be the only one interface for all who want to operation
/sys/fs/resctrl, or
even early kernel (4.10) which has no /sys/fs/resctrl (thought MSR).

It not only provide VM but for all kinds of workload/cpu/containers.

BR - Eli


> Regards,
> Daniel
> --
> |: https://berrange.com  -o-https://www.flickr.com/photos/
> dberrange :|
> |: https://libvirt.org -o-
> https://fstop138.berrange.com :|
> |: https://entangle-photo.org-o-https://www.instagram.com/
> dberrange :|
>
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] Yet another RFC for CAT

2017-09-06 Thread Eli Qiao

2017-09-04 23:57 GMT+08:00 Daniel P. Berrange :

> On Mon, Sep 04, 2017 at 04:14:00PM +0200, Martin Kletzander wrote:
> > * The current design (finally something libvirt-related, right?)
> >
> > The discussion ended with a conclusion of the following (with my best
> > knowledge, there were so many discussions about so many things that I
> > would spend too much time looking up all of them):
> >
> > - Users should not need to specify bit masks, such complexity should be
> >   abstracted.  We'll use sizes (e.g. 4MB)
> >
> > - Multiple vCPUs might need to share the same allocation.
> >
> > - Exclusivity of allocations is to be assumed, that is only unoccupied
> >   cache should be used for new allocations.
> >
> > The last point seems trivial but it's actually very specific condition
> > that, if removed, can cause several problems.  If it's hard to grasp the
> > last point together with the second one, you're on the right track.  If
> > not, then I'll try to make a point for why the last point should be
> > removed in 3... 2... 1...
> >
> > * Design flaws
>
>
> > 1) Users cannot specify any allocation that would share only part with
> >some other allocation of the domain or the default group.
> >
>

yep, There's no share cache ways support.

I was thinking that create a cache resource group in libvirt, and user can
add vms into that resource group, this is good for those who would like to
have share cache resource, maybe NFV case.

but for case:

VM1: fff00
VM2: 00fff
which have a `f` (4 cache ways) share, seems have not really meanful.
at least, I don't heart that we have that case. This was mentioned by
*Marcelo Tosatti *before too.

> 2) It was not specified what to do with the default resource group.
> >There might be several ways to approach this, with varying pros and
> >cons:
> >
> > a) Treat it as any other group.  That is any bit set for this group
> >will be excluded from usable bits when creating new allocation
> >for a domain.
> >
> > - Very predictable behaviour
> >
> > - You will not be able to allocate any amount of cache without
> >   previous setting for the default group as that will have all
> >   the bits set which will make all the cache unusable
> >
> > b) Automatically remove the appropriate amount of bits that are
> >needed for new domains.
> >
> > - No need to do any change to the system settings in order to
> >   use this new feature
> >
> > - We would have to change system settings, which is generally
> >   frowned upon when done "automatically" as a side effect of
> >   starting a domain, especially for such scarce resource as
> >   cache
> >
> > - The change to system settings would not be entirely
> >   predictable
> >
> > c) Act like it doesn't exist and don't remove its allocations from
> >consideration
> >
> > - Doesn't really make sense as system processes might be
> >   trashing the cache as any VM, moreover when all VM processes
> >   without allocations will be based in the default group as
> >   well
> >
> > 3) There is no way for users to know what the particular settings are
> >for any running domain.
>

I think you are going to expose what the current CBM looks like for
a given VM? That's fair enough.


> >
> > The first point was deemed a corner case.  Fair enough on its own, but
> > considering point 2 and its solutions, it is rather difficult for me to
> > justify it.  Also, let's say you have domain with 4 vCPUs out of which
> > you know 1 might be trashing the cache, but you don't want to restrict
> > it completely, but others will utilize it very nicely.  Sensible
> > allocations for such domain's vCPUs might be:
> >
> >  vCPU  0:   000f
> >  vCPUs 1-3: 
> >
> > as you want vCPUs 1-3 to utilize even the part of cache that might get
> > trashed by vCPU 0.  Or they might share some data (especially
> > guest-memory-related).
> >
> > The case above is not possible to set up with only per-vcpu(s) scalar
> > setting.  And there are more as you might imagine now.  For example how
> > do we behave with iothreads and emulator threads?
>
> This is kinds of hard to implement, but possible.

is 1:1 mapping of resource group to VM?

if you want to have iothreads and emulator threads to have separated
cache allocation, you may need to create resource group to associated with
VM's vcpus and iothreads and emulator thread.

but COS number is limited, does it worth to have so fine granularity
control?


> Ok, I see what you're getting at.  I've actually forgotten what
> our current design looks like though :-)
>
> What level of granularity were we allowing within a guest ?
> All vCPUs use separate cache regions from each other, or all
> vCPUs use a share cached region, but separate from other guests,
> or a mix ?
>
> > * My suggestion:
> >
> > - Provide an API for

Re: [libvirt] [PATCH RESEND RFC V2 0/2] Implement l3 CAT

2017-07-10 Thread Eli Qiao

ping 


On Wednesday, 5 July 2017 at 10:39 AM, Eli Qiao wrote:

> Allow user to define cachetune in domain xml.
> 
> RESEND:
> * Fix stupid complile problem pointed out by Martin, check and syntax-check
> passed.
> 
> V2 -> V1 changes:
> * Redefine cachetune xml in domain xml.
> * Create a struct for driver to talk with util/virresctrl.*
> * Nit fixes
> 
> 
> --
> libvir-list mailing list
> libvir-list@redhat.com (mailto:libvir-list@redhat.com)
> https://www.redhat.com/mailman/listinfo/libvir-list
> 
> 


--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATCH RESEND RFC V2 1/2] Resctrl: Add new xml element to support cache tune

2017-07-04 Thread Eli Qiao

This patch adds new xml element to support cache tune as:


  ...
  
  ...


cacheId: reference of the host's cache banks id, it's from capabilities
  xml.
type:cache bank type, it could be both, code, data.
sizeKiB: must be multiple of granularity, must be greater than or equal
 to minimum.
vcpus:   cache allocation on vcpu set, if empty, will apply the allocation
 on all vcpus.

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 docs/schemas/domaincommon.rng |  29 +++
 src/conf/domain_conf.c| 112 ++
 src/conf/domain_conf.h|  19 +++
 3 files changed, 160 insertions(+)

diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng
index e259e3e..6e0262e 100644
--- a/docs/schemas/domaincommon.rng
+++ b/docs/schemas/domaincommon.rng
@@ -834,6 +834,25 @@
 
   
 
+
+  
+
+  
+
+
+  
+
+ 
+
+  
+
+
+  
+
+  
+
+  
+
 
   
 
@@ -5693,6 +5712,16 @@
   -1
 
   
+  
+
+  [0-9]+
+
+  
+  
+
+  (both|code|data)
+
+  
   
   
 
diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
index 0409c62..65a9486 100644
--- a/src/conf/domain_conf.c
+++ b/src/conf/domain_conf.c
@@ -2927,6 +2927,11 @@ void virDomainDefFree(virDomainDefPtr def)
 
 xmlFreeNode(def->metadata);
 
+for (i = 0; i < def->cachetune.n_banks; i++)
+virBitmapFree(def->cachetune.cache_banks[i].vcpus);
+
+VIR_FREE(def->cachetune.cache_banks);
+
 VIR_FREE(def);
 }
 
@@ -16318,6 +16323,81 @@ virDomainVcpuPinDefParseXML(virDomainDefPtr def,
 return ret;
 }
 
+/* Parse the XML definition for cachetune
+ * and a cachetune has the form
+ *

[libvirt] [PATCH RESEND RFC V2 0/2] Implement l3 CAT

2017-07-04 Thread Eli Qiao

Allow user to define cachetune in domain xml.

RESEND:
  * Fix stupid complile problem pointed out by Martin, check and syntax-check
passed.

V2 -> V1 changes:
  * Redefine cachetune xml in domain xml.
  * Create a struct for driver to talk with util/virresctrl.*
  * Nit fixes


Eli Qiao (2):
  Resctrl: Add new xml element to support cache tune
  Resctrl: Do cache allocation while boot a qemu VM

 docs/schemas/domaincommon.rng |  29 ++
 include/libvirt/virterror.h   |   1 +
 src/Makefile.am   |   1 +
 src/conf/domain_conf.c| 112 
 src/conf/domain_conf.h|  19 +
 src/libvirt_private.syms  |   9 +
 src/qemu/qemu_process.c   |  81 +++
 src/util/virerror.c   |   1 +
 src/util/virresctrl.c | 822 ++
 src/util/virresctrl.h |  88 
 tests/Makefile.am |   8 +-
 tests/virresctrldata/L3-free.schemata |   1 +
 tests/virresctrldata/L3CODE-free.schemata |   1 +
 tests/virresctrldata/L3DATA-free.schemata |   1 +
 tests/virresctrldata/linux-resctrl|   1 +
 tests/virresctrldata/linux-resctrl-cdp|   1 +
 tests/virresctrltest.c| 119 +
 17 files changed, 1294 insertions(+), 1 deletion(-)
 create mode 100644 src/util/virresctrl.c
 create mode 100644 src/util/virresctrl.h
 create mode 100644 tests/virresctrldata/L3-free.schemata
 create mode 100644 tests/virresctrldata/L3CODE-free.schemata
 create mode 100644 tests/virresctrldata/L3DATA-free.schemata
 create mode 12 tests/virresctrldata/linux-resctrl
 create mode 12 tests/virresctrldata/linux-resctrl-cdp
 create mode 100644 tests/virresctrltest.c

--
1.9.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATCH RESEND RFC V2 2/2] Resctrl: Do cache allocation while boot a qemu VM

2017-07-04 Thread Eli Qiao

Set cachetune if user define cachetune in domain xml when using qemu
driver.

This patch adds 3 major private interface.

virResctrlGetFreeCache: return free cache, default cache substract cache
allocated.
virResctrlSetCachetunes: set cache banks which defined in a domain.
virResctrlRemoveCachetunes: remove cache allocation group from the
host.

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 include/libvirt/virterror.h   |   1 +
 src/Makefile.am   |   1 +
 src/libvirt_private.syms  |   9 +
 src/qemu/qemu_process.c   |  81 +++
 src/util/virerror.c   |   1 +
 src/util/virresctrl.c | 822 ++
 src/util/virresctrl.h |  88 
 tests/Makefile.am |   8 +-
 tests/virresctrldata/L3-free.schemata |   1 +
 tests/virresctrldata/L3CODE-free.schemata |   1 +
 tests/virresctrldata/L3DATA-free.schemata |   1 +
 tests/virresctrldata/linux-resctrl|   1 +
 tests/virresctrldata/linux-resctrl-cdp|   1 +
 tests/virresctrltest.c| 119 +
 14 files changed, 1134 insertions(+), 1 deletion(-)
 create mode 100644 src/util/virresctrl.c
 create mode 100644 src/util/virresctrl.h
 create mode 100644 tests/virresctrldata/L3-free.schemata
 create mode 100644 tests/virresctrldata/L3CODE-free.schemata
 create mode 100644 tests/virresctrldata/L3DATA-free.schemata
 create mode 12 tests/virresctrldata/linux-resctrl
 create mode 12 tests/virresctrldata/linux-resctrl-cdp
 create mode 100644 tests/virresctrltest.c

diff --git a/include/libvirt/virterror.h b/include/libvirt/virterror.h
index 2efee8f..4bc0c74 100644
--- a/include/libvirt/virterror.h
+++ b/include/libvirt/virterror.h
@@ -132,6 +132,7 @@ typedef enum {
 
 VIR_FROM_PERF = 65, /* Error from perf */
 VIR_FROM_LIBSSH = 66,   /* Error from libssh connection transport */
+VIR_FROM_RESCTRL = 67,  /* Error from resctrl */
 
 # ifdef VIR_ENUM_SENTINELS
 VIR_ERR_DOMAIN_LAST
diff --git a/src/Makefile.am b/src/Makefile.am
index eae32dc..8dbb778 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -167,6 +167,7 @@ UTIL_SOURCES =  
\
util/virprocess.c util/virprocess.h \
util/virqemu.c util/virqemu.h   \
util/virrandom.h util/virrandom.c   \
+   util/virresctrl.h util/virresctrl.c \
util/virrotatingfile.h util/virrotatingfile.c   \
util/virscsi.c util/virscsi.h   \
util/virscsihost.c util/virscsihost.h   \
diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms
index c1e9471..72e5096 100644
--- a/src/libvirt_private.syms
+++ b/src/libvirt_private.syms
@@ -2439,6 +2439,15 @@ virRandomGenerateWWN;
 virRandomInt;
 
 
+# util/virresctrl.h
+virResctrlBitmap2String;
+virResctrlFreeSchemata;
+virResctrlGetFreeCache;
+virResctrlRemoveCachetunes;
+virResctrlSetCachetunes;
+virResctrlTypeToString;
+
+
 # util/virrotatingfile.h
 virRotatingFileReaderConsume;
 virRotatingFileReaderFree;
diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c
index fa9990e..3b11caa 100644
--- a/src/qemu/qemu_process.c
+++ b/src/qemu/qemu_process.c
@@ -70,6 +70,7 @@
 #include "virbitmap.h"
 #include "viratomic.h"
 #include "virnuma.h"
+#include "virresctrl.h"
 #include "virstring.h"
 #include "virhostdev.h"
 #include "secret_util.h"
@@ -5130,6 +5131,77 @@ qemuProcessSetupVcpus(virDomainObjPtr vm)
 return 0;
 }
 
+static int
+qemuProcessSetCacheBanks(virCapsHostPtr caps, virDomainObjPtr vm)
+{
+size_t i, j;
+virDomainCachetunePtr cachetune;
+virResctrlCachetunePtr resctrl_cachetune;
+unsigned int max_vcpus = virDomainDefGetVcpusMax(vm->def);
+pid_t *pids = NULL;
+virDomainVcpuDefPtr vcpu;
+size_t npids = 0;
+size_t count = 0;
+int ret = -1;
+
+cachetune = &(vm->def->cachetune);
+
+if (VIR_ALLOC_N(resctrl_cachetune, cachetune->n_banks) < 0)
+goto cleanup;
+
+/* construct resctrl_cachetune array */
+for (i = 0; i < cachetune->n_banks; i++) {
+
+resctrl_cachetune[i].cache_id = cachetune->cache_banks[i].cache_id;
+resctrl_cachetune[i].type = cachetune->cache_banks[i].type;
+resctrl_cachetune[i].size = cachetune->cache_banks[i].size;
+
+/* get granularity from host's capabilities */
+for (j = 0; j < caps->ncaches; j++) {
+/* even enable CDP, granularity for code and data are same */
+if (caps->caches[j]->id == resctrl_cachetune[i].cache_id &&
+caps->caches[j

Re: [libvirt] [PATCH RFC V2 1/2] Resctrl: Add new xml element to support cache tune

2017-07-02 Thread Eli Qiao



On Friday, 30 June 2017 at 7:06 PM, Martin Kletzander wrote:

> On Mon, Jun 26, 2017 at 06:33:39PM +0800, Eli Qiao wrote:
> > This patch adds new xml element to support cache tune as:
> >  
> > 
> > ...
> > 
> > ...
> > 
> >  
> > cacheId: reference of the host's cache banks id, it's from capabilities
> > xml.
> > type: cache bank type, it could be both, code, data.
> > sizeKiB: must be multiple of granularity, must be greater than or equal
> > to minimum.
> >  
>  
>  
> Why would you come up with something like 'sizeKib'? That's so
> inconvenient.
>  
>  

Sure,   
>  
> > vcpus: cache allocation on vcpu set, if empty, will apply the allocation
> > on all vcpus.
> >  
> > Signed-off-by: Eli Qiao <liyong.q...@intel.com 
> > (mailto:liyong.q...@intel.com)>
> > ---
> > docs/schemas/domaincommon.rng | 29 +++
> > src/conf/domain_conf.c | 113 +-
> > src/conf/domain_conf.h | 19 +++
> > 3 files changed, 160 insertions(+), 1 deletion(-)
> >  
> > diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
> > index 0409c62..fa8d03e 100644
> > --- a/src/conf/domain_conf.c
> > +++ b/src/conf/domain_conf.c
> > @@ -17642,7 +17730,6 @@ virDomainDefParseXML(xmlDocPtr xml,
> > virReportError(VIR_ERR_INTERNAL_ERROR, "%s",
> > _("cannot extract vcpusched nodes"));
> > goto error;
> > - }
> >  
> > for (i = 0; i < n; i++) {
> > if (virDomainVcpuThreadSchedParse(nodes[i], def) < 0)
> >  
>  
>  
> And this hunk breaks the whole compilation, of course.
>  
ops, stupid me.  
>  
> I'm not at work for few days plus there's a public holiday in my
> country, so I won't be here for a while. That shouldn't stop anyone
> else reviewing your code, but I'm giving you a heads-up that I'll
> probably work on a separate solution if I have some extra time on my
> hands during vacation.
>  
Sure, that’s great if you can have your version, I would like to help reviewing 
and testing  
>  
> Have a nice day,
> Martin
>  
> --
> libvir-list mailing list
> libvir-list@redhat.com (mailto:libvir-list@redhat.com)
> https://www.redhat.com/mailman/listinfo/libvir-list
>  
>  


--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATCH RFC V2 0/2] Implement l3 CAT

2017-06-26 Thread Eli Qiao

Allow user to define cachetune in domain xml.

V2 -> V1 changes:
  * Redefine cachetune xml in domain xml.
  * Create a struct for driver to talk with util/virresctrl.*
  * Nit fixes

Eli Qiao (2):
  Resctrl: Add new xml element to support cache tune
  Resctrl: Do cache allocation while boot a qemu VM

 docs/schemas/domaincommon.rng |  29 ++
 include/libvirt/virterror.h   |   1 +
 src/Makefile.am   |   1 +
 src/conf/domain_conf.c| 112 
 src/conf/domain_conf.h|  19 +
 src/libvirt_private.syms  |   9 +
 src/qemu/qemu_process.c   |  81 +++
 src/util/virerror.c   |   1 +
 src/util/virresctrl.c | 822 ++
 src/util/virresctrl.h |  88 
 tests/Makefile.am |   8 +-
 tests/virresctrldata/L3-free.schemata |   1 +
 tests/virresctrldata/L3CODE-free.schemata |   1 +
 tests/virresctrldata/L3DATA-free.schemata |   1 +
 tests/virresctrldata/linux-resctrl|   1 +
 tests/virresctrldata/linux-resctrl-cdp|   1 +
 tests/virresctrltest.c| 119 +
 17 files changed, 1294 insertions(+), 1 deletion(-)
 create mode 100644 src/util/virresctrl.c
 create mode 100644 src/util/virresctrl.h
 create mode 100644 tests/virresctrldata/L3-free.schemata
 create mode 100644 tests/virresctrldata/L3CODE-free.schemata
 create mode 100644 tests/virresctrldata/L3DATA-free.schemata
 create mode 12 tests/virresctrldata/linux-resctrl
 create mode 12 tests/virresctrldata/linux-resctrl-cdp
 create mode 100644 tests/virresctrltest.c

--
1.9.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATCH RFC V2 2/2] Resctrl: Do cache allocation while boot a qemu VM

2017-06-26 Thread Eli Qiao

Set cachetune if user define cachetune in domain xml when using qemu
driver.

This patch adds 3 major private interface.

virResctrlGetFreeCache: return free cache, default cache substract cache
allocated.
virResctrlSetCachetunes: set cache banks which defined in a domain.
virResctrlRemoveCachetunes: remove cache allocation group from the
host.

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 include/libvirt/virterror.h   |   1 +
 src/Makefile.am   |   1 +
 src/libvirt_private.syms  |   9 +
 src/qemu/qemu_process.c   |  81 +++
 src/util/virerror.c   |   1 +
 src/util/virresctrl.c | 822 ++
 src/util/virresctrl.h |  88 
 tests/Makefile.am |   8 +-
 tests/virresctrldata/L3-free.schemata |   1 +
 tests/virresctrldata/L3CODE-free.schemata |   1 +
 tests/virresctrldata/L3DATA-free.schemata |   1 +
 tests/virresctrldata/linux-resctrl|   1 +
 tests/virresctrldata/linux-resctrl-cdp|   1 +
 tests/virresctrltest.c| 119 +
 14 files changed, 1134 insertions(+), 1 deletion(-)
 create mode 100644 src/util/virresctrl.c
 create mode 100644 src/util/virresctrl.h
 create mode 100644 tests/virresctrldata/L3-free.schemata
 create mode 100644 tests/virresctrldata/L3CODE-free.schemata
 create mode 100644 tests/virresctrldata/L3DATA-free.schemata
 create mode 12 tests/virresctrldata/linux-resctrl
 create mode 12 tests/virresctrldata/linux-resctrl-cdp
 create mode 100644 tests/virresctrltest.c

diff --git a/include/libvirt/virterror.h b/include/libvirt/virterror.h
index 2efee8f..4bc0c74 100644
--- a/include/libvirt/virterror.h
+++ b/include/libvirt/virterror.h
@@ -132,6 +132,7 @@ typedef enum {
 
 VIR_FROM_PERF = 65, /* Error from perf */
 VIR_FROM_LIBSSH = 66,   /* Error from libssh connection transport */
+VIR_FROM_RESCTRL = 67,  /* Error from resctrl */
 
 # ifdef VIR_ENUM_SENTINELS
 VIR_ERR_DOMAIN_LAST
diff --git a/src/Makefile.am b/src/Makefile.am
index eae32dc..8dbb778 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -167,6 +167,7 @@ UTIL_SOURCES =  
\
util/virprocess.c util/virprocess.h \
util/virqemu.c util/virqemu.h   \
util/virrandom.h util/virrandom.c   \
+   util/virresctrl.h util/virresctrl.c \
util/virrotatingfile.h util/virrotatingfile.c   \
util/virscsi.c util/virscsi.h   \
util/virscsihost.c util/virscsihost.h   \
diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms
index c1e9471..72e5096 100644
--- a/src/libvirt_private.syms
+++ b/src/libvirt_private.syms
@@ -2439,6 +2439,15 @@ virRandomGenerateWWN;
 virRandomInt;
 
 
+# util/virresctrl.h
+virResctrlBitmap2String;
+virResctrlFreeSchemata;
+virResctrlGetFreeCache;
+virResctrlRemoveCachetunes;
+virResctrlSetCachetunes;
+virResctrlTypeToString;
+
+
 # util/virrotatingfile.h
 virRotatingFileReaderConsume;
 virRotatingFileReaderFree;
diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c
index fa9990e..3b11caa 100644
--- a/src/qemu/qemu_process.c
+++ b/src/qemu/qemu_process.c
@@ -70,6 +70,7 @@
 #include "virbitmap.h"
 #include "viratomic.h"
 #include "virnuma.h"
+#include "virresctrl.h"
 #include "virstring.h"
 #include "virhostdev.h"
 #include "secret_util.h"
@@ -5130,6 +5131,77 @@ qemuProcessSetupVcpus(virDomainObjPtr vm)
 return 0;
 }
 
+static int
+qemuProcessSetCacheBanks(virCapsHostPtr caps, virDomainObjPtr vm)
+{
+size_t i, j;
+virDomainCachetunePtr cachetune;
+virResctrlCachetunePtr resctrl_cachetune;
+unsigned int max_vcpus = virDomainDefGetVcpusMax(vm->def);
+pid_t *pids = NULL;
+virDomainVcpuDefPtr vcpu;
+size_t npids = 0;
+size_t count = 0;
+int ret = -1;
+
+cachetune = &(vm->def->cachetune);
+
+if (VIR_ALLOC_N(resctrl_cachetune, cachetune->n_banks) < 0)
+goto cleanup;
+
+/* construct resctrl_cachetune array */
+for (i = 0; i < cachetune->n_banks; i++) {
+
+resctrl_cachetune[i].cache_id = cachetune->cache_banks[i].cache_id;
+resctrl_cachetune[i].type = cachetune->cache_banks[i].type;
+resctrl_cachetune[i].size = cachetune->cache_banks[i].size;
+
+/* get granularity from host's capabilities */
+for (j = 0; j < caps->ncaches; j++) {
+/* even enable CDP, granularity for code and data are same */
+if (caps->caches[j]->id == resctrl_cachetune[i].cache_id &&
+caps->caches[j

[libvirt] [PATCH RFC V2 1/2] Resctrl: Add new xml element to support cache tune

2017-06-26 Thread Eli Qiao

This patch adds new xml element to support cache tune as:


  ...
  
  ...


cacheId: reference of the host's cache banks id, it's from capabilities
  xml.
type:cache bank type, it could be both, code, data.
sizeKiB: must be multiple of granularity, must be greater than or equal
 to minimum.
vcpus:   cache allocation on vcpu set, if empty, will apply the allocation
 on all vcpus.

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 docs/schemas/domaincommon.rng |  29 +++
 src/conf/domain_conf.c| 113 +-
 src/conf/domain_conf.h|  19 +++
 3 files changed, 160 insertions(+), 1 deletion(-)

diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng
index e259e3e..6e0262e 100644
--- a/docs/schemas/domaincommon.rng
+++ b/docs/schemas/domaincommon.rng
@@ -834,6 +834,25 @@
 
   
 
+
+  
+
+  
+
+
+  
+
+ 
+
+  
+
+
+  
+
+  
+
+  
+
 
   
 
@@ -5693,6 +5712,16 @@
   -1
 
   
+  
+
+  [0-9]+
+
+  
+  
+
+  (both|code|data)
+
+  
   
   
 
diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
index 0409c62..fa8d03e 100644
--- a/src/conf/domain_conf.c
+++ b/src/conf/domain_conf.c
@@ -2927,6 +2927,11 @@ void virDomainDefFree(virDomainDefPtr def)
 
 xmlFreeNode(def->metadata);
 
+for (i = 0; i < def->cachetune.n_banks; i++)
+virBitmapFree(def->cachetune.cache_banks[i].vcpus);
+
+VIR_FREE(def->cachetune.cache_banks);
+
 VIR_FREE(def);
 }
 
@@ -16318,6 +16323,81 @@ virDomainVcpuPinDefParseXML(virDomainDefPtr def,
 return ret;
 }
 
+/* Parse the XML definition for cachetune
+ * and a cachetune has the form
+ *

Re: [libvirt] [PATCH RFC 1/2] Resctrl: Add new xml element to support cache tune

2017-06-22 Thread Eli Qiao



On Thursday, 22 June 2017 at 8:41 PM, Martin Kletzander wrote:

> I missed this. Ye, you're right, thanks for showing that with a use
> case. We could then require only the cache_id and automatically fill
> in level and type. Migration (or rather any start) should then fail if
> that cache_id doesn't correspond to the level and type requested. I
> still can't find a reason for 'id', though.

I agree that id, level is useless, but also type is required, the case is when
host’s enabled CDP like this:

  


  


the cachetune need to be defined as:







Thought?

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH RFC 2/2] Resctrl: Add uitls functions to operate sysfs resctrl

2017-06-22 Thread Eli Qiao

On Wednesday, 21 June 2017 at 10:44 PM, Martin Kletzander wrote:

> n 21, 2017 at 02:07:15PM +0800, Eli Qiao wrote:
>  
> You don't need to pass the whole structure of all the data. Can't the
> qemu function just do something like:
>  
> virResctrlLock()
> foreach cachetune:
> region = virResctrlGetFreeRegion(size, type)
> foreach cachetune.vcpu:
> virResctrlSetRegion(vcpu.pid, region)
>  
> Or like with some other tuning, you can have a function that determines
> the region when given vcpu and just call it for all vcpus. You can
> save the regions in the status XML, so that not only users can see it,
> but you can also reference them from that aforementioned function. Or
> you could have saved pairs of id: region, but I think that's not needed.
>  
> That reminds me, unless referred to from somewhere, the cachetune
> doesn't even need the id.
>  
> But basically, you don't need to pass the whole cachetune or any other
> structure. The code is very messy, check your pointers and don't
> compare references to NULLs. Read the diffs after yourself. I know it
> works for you, but the code needs to be readable as well.
>  

Forgive me to spam it again.  

I agree that not to pass whole cachetune , but it seems I need to define new
struct or passing more paramter while setting a cachetune.

1. It’s not just so simple we only get the Region, and setRegion.

We need to consume the llc cache from the “default” group on the host and
then write back it to /sys/fs/resctrl/ and it’s better that we need to compute 
all
the mask at same time because for multiple socket host, if user don’t specify
cache allocation on all sockets, we need to complete the mask by the min bits.

So I don’t think its a good solution to setcachetune in a look.

Better to pass all domain cache tune to virrresctrl and let it calculate it at 
a time
then write it back to /sys/fs/resctrl

2. for region = virResctrlGetFreeRegion(size, type)
if host has multiple sockets, we can not decide which region only be size, type.
Which is to say, we need the cache_id.

3. while doing cache allocation, we need to pass cache control information:

min, granularity, maxAlloc. beyond that , we need cache_id (we can find by 
vcups list)
type, size..

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH RFC 2/2] Resctrl: Add uitls functions to operate sysfs resctrl

2017-06-21 Thread Eli Qiao



On Wednesday, 21 June 2017 at 10:44 PM, Martin Kletzander wrote:

> You don't need to pass the whole structure of all the data. Can't the
> qemu function just do something like:
> 
> virResctrlLock()
> foreach cachetune:
> region = virResctrlGetFreeRegion(size, type)
> foreach cachetune.vcpu:
> virResctrlSetRegion(vcpu.pid, region)
> 
Sure, good suggestion, I will try to read what other tune does. 
> 
> Or like with some other tuning, you can have a function that determines
> the region when given vcpu and just call it for all vcpus. You can
> save the regions in the status XML, so that not only users can see it,
> but you can also reference them from that aforementioned function. Or
> you could have saved pairs of id: region, but I think that's not needed.
> 
> That reminds me, unless referred to from somewhere, the cachetune
> doesn't even need the id.
> 
> But basically, you don't need to pass the whole cachetune or any other
> structure. The code is very messy, check your pointers and don't
> compare references to NULLs. Read the diffs after yourself. I know it
> works for you, but the code needs to be readable as well.


Forgive me I am not a qualified c programmer for years :(, I will try to 
refine them.
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH RFC 1/2] Resctrl: Add new xml element to support cache tune

2017-06-21 Thread Eli Qiao

hi Martin  

It’s really nice of you to help reviewing the mass code. Thanks.

I don’t find a better way to split patch.  


On Wednesday, 21 June 2017 at 9:53 PM, Martin Kletzander wrote:

> On Mon, Jun 12, 2017 at 05:48:40PM +0800, Eli Qiao wrote:
> > This patch adds new xml element to support cache tune as:
> >  
> > 
> > ...
> >  > vcpus='1,2'/>
> >  
>  
>  
> The cache_id automatically implies level and type. Either have one or
> the other. I know we talked about this already (maybe multiple times),
> but without any clear outcome. For me the sensible thing is to have
> level and type as that doesn't need to be changed when moving between
> hosts, and if it cannot be migrated, then it's properly checked.
>  
Think about this case, if the VM has numa setting, the VM has multiple vcpu
running across sockets, if we don’t specify cache_id (cache id stand for
on which Socket/Cache), how can we know on which Socket we allocation for the 
VM?

I can image there’s 2 cases:

1. if we don’t specify vcpus, and our host have 2 or more Socket, we have this 
xml define

 

We allocate 2816 KiB cache on all of the Socket/Cache.

2. if we specify vcpus
 

 

We need to make sure we vcpu 1, 2 are mapped to Socket/Cache 0 and 3,4 on 
Socket/Cache 1.
So that vcpus running on Socket/Cache 0 has 2816 KiB cache allocated and vcpus 
running on
Socket/Cache 1 has 5632 KiB cache allocated.

Does it make sense?
  
…


> >  
> > virDomainCputune cputune;
> >  
> > + virDomainCachetune cachetune;
> > +
> >  
>  
>  
> It is part of cputune in the XML, why not here?

Oh yes, I will rethink how to simple the domain cache tune.  
>  
> > virDomainNumaPtr numa;
> > virDomainResourceDefPtr resource;
> > virDomainIdMapDef idmap;
> > --
> > 1.9.1
> >  
> > --
> > libvir-list mailing list
> > libvir-list@redhat.com (mailto:libvir-list@redhat.com)
> > https://www.redhat.com/mailman/listinfo/libvir-list
> >  
>  
>  
> --
> libvir-list mailing list
> libvir-list@redhat.com (mailto:libvir-list@redhat.com)
> https://www.redhat.com/mailman/listinfo/libvir-list
>  
>  


--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH RFC 2/2] Resctrl: Add uitls functions to operate sysfs resctrl

2017-06-21 Thread Eli Qiao



On Tuesday, 20 June 2017 at 8:39 PM, Martin Kletzander wrote:

> On Mon, Jun 12, 2017 at 05:48:41PM +0800, Eli Qiao wrote:
> > This patch adds 3 major private interface.
> >  
> > virResctrlGetFreeCache: return free cache, default cache substract cache
> > allocated.
> > virResctrlSetCachetunes: set cache banks which defined in a domain.
> > virResctrlRemoveCachetunes: remove cache allocation group from the
> > host.
> >  
> > There's some existed issue when do syntax-check as I reference the cache
> > tune and cachebank definition (from conf/domain_conf.h) in
> > util/virresctrl.h.  
> >  
>  
>  
>  

>  
> Yes, util/ cannot depend on conf/ in libvirt due to various reasons.
> All the data you want to use in util/ need to be defined there. If that
> corresponds to some XML, the parsers and formatters must be in conf/.
> In rare cases, there might be need for two data structures, one in util/
> and one in conf/. I don't think that's needed in this case.
>  
>  


I can move the virDomainCacheBank definition to util/virresctrl.h

but what about the virCapsHostPtr, we need the host cache information and 
resctrl information
it’s defined in src/conf/capabilities.h”

Do we need to define another copy in virresctrl.h ?
  
> --
> libvir-list mailing list
> libvir-list@redhat.com (mailto:libvir-list@redhat.com)
> https://www.redhat.com/mailman/listinfo/libvir-list
>  
>  


--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH RFC 2/2] Resctrl: Add uitls functions to operate sysfs resctrl

2017-06-20 Thread Eli Qiao

hello,
ping 


On Monday, 12 June 2017 at 5:48 PM, Eli Qiao wrote:

> This patch adds 3 major private interface.
> 
> virResctrlGetFreeCache: return free cache, default cache substract cache
> allocated.
> virResctrlSetCachetunes: set cache banks which defined in a domain.
> virResctrlRemoveCachetunes: remove cache allocation group from the
> host.
> 
> There's some existed issue when do syntax-check as I reference the cache
> tune and cachebank definition (from conf/domain_conf.h) in
> util/virresctrl.h.
> ---
> include/libvirt/virterror.h | 1 +
> src/Makefile.am (http://Makefile.am) | 1 +
> src/libvirt_private.syms | 9 +
> src/qemu/qemu_process.c | 54 ++
> src/util/virerror.c | 1 +
> src/util/virresctrl.c | 851 ++
> src/util/virresctrl.h | 79 +++
> tests/Makefile.am (http://Makefile.am) | 8 +-
> tests/virresctrldata/L3-free.schemata | 1 +
> tests/virresctrldata/L3CODE-free.schemata | 1 +
> tests/virresctrldata/L3DATA-free.schemata | 1 +
> tests/virresctrldata/linux-resctrl | 1 +
> tests/virresctrldata/linux-resctrl-cdp | 1 +
> tests/virresctrltest.c | 119 +
> 14 files changed, 1127 insertions(+), 1 deletion(-)
> create mode 100644 src/util/virresctrl.c
> create mode 100644 src/util/virresctrl.h
> create mode 100644 tests/virresctrldata/L3-free.schemata
> create mode 100644 tests/virresctrldata/L3CODE-free.schemata
> create mode 100644 tests/virresctrldata/L3DATA-free.schemata
> create mode 12 tests/virresctrldata/linux-resctrl
> create mode 12 tests/virresctrldata/linux-resctrl-cdp
> create mode 100644 tests/virresctrltest.c
> 
> diff --git a/include/libvirt/virterror.h b/include/libvirt/virterror.h
> index 2efee8f..4bc0c74 100644
> --- a/include/libvirt/virterror.h
> +++ b/include/libvirt/virterror.h
> @@ -132,6 +132,7 @@ typedef enum {
> 
> VIR_FROM_PERF = 65, /* Error from perf */
> VIR_FROM_LIBSSH = 66, /* Error from libssh connection transport */
> + VIR_FROM_RESCTRL = 67, /* Error from resctrl */
> 
> # ifdef VIR_ENUM_SENTINELS
> VIR_ERR_DOMAIN_LAST
> diff --git a/src/Makefile.am (http://Makefile.am) b/src/Makefile.am 
> (http://Makefile.am)
> index eae32dc..8dbb778 100644
> --- a/src/Makefile.am (http://Makefile.am)
> +++ b/src/Makefile.am (http://Makefile.am)
> @@ -167,6 +167,7 @@ UTIL_SOURCES = \
> util/virprocess.c util/virprocess.h \
> util/virqemu.c util/virqemu.h \
> util/virrandom.h util/virrandom.c \
> + util/virresctrl.h util/virresctrl.c \
> util/virrotatingfile.h util/virrotatingfile.c \
> util/virscsi.c util/virscsi.h \
> util/virscsihost.c util/virscsihost.h \
> diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms
> index b6c828f..7392cfa 100644
> --- a/src/libvirt_private.syms
> +++ b/src/libvirt_private.syms
> @@ -2440,6 +2440,15 @@ virRandomGenerateWWN;
> virRandomInt;
> 
> 
> +# util/virresctrl.h
> +virResctrlBitmap2String;
> +virResctrlFreeSchemata;
> +virResctrlGetFreeCache;
> +virResctrlRemoveCachetunes;
> +virResctrlSetCachetunes;
> +virResctrlTypeToString;
> +
> +
> # util/virrotatingfile.h
> virRotatingFileReaderConsume;
> virRotatingFileReaderFree;
> diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c
> index 4a66f0d..8efeb19 100644
> --- a/src/qemu/qemu_process.c
> +++ b/src/qemu/qemu_process.c
> @@ -70,6 +70,7 @@
> #include "virbitmap.h"
> #include "viratomic.h"
> #include "virnuma.h"
> +#include "virresctrl.h"
> #include "virstring.h"
> #include "virhostdev.h"
> #include "secret_util.h"
> @@ -5088,6 +5089,51 @@ qemuProcessSetupVcpus(virDomainObjPtr vm)
> return 0;
> }
> 
> +static int
> +qemuProcessSetCacheBanks(virCapsHostPtr caps, virDomainObjPtr vm)
> +{
> + size_t i, j;
> + virDomainCachetunePtr cachetune;
> + unsigned int max_vcpus = virDomainDefGetVcpusMax(vm->def);
> + pid_t *pids = NULL;
> + virDomainVcpuDefPtr vcpu;
> + size_t npid = 0;
> + size_t count = 0;
> + int ret = -1;
> +
> + cachetune = &(vm->def->cachetune);
> +
> + for (i = 0; i < cachetune->n_banks; i++) {
> + if (cachetune->cache_banks[i].vcpus) {
> + for (j = 0; j < max_vcpus; j++) {
> + if (virBitmapIsBitSet(cachetune->cache_banks[i].vcpus, j)) {
> +
> + vcpu = virDomainDefGetVcpu(vm->def, j);
> + if (!vcpu->online)
> + continue;
> +
> + if (VIR_RESIZE_N(pids, npid, count, 1) < 0)
> + goto cleanup;
> + pids[count ++] = qemuDomainGetVcpuPid(vm, j);
> + }
> + }
> + }
> + }
> +
> + /* If not specify vcpus in cachetune, add vm->pid */
> + if (pids == NULL) {
> + if (VIR_ALLOC_N(pids, 1) < 0)
> + goto cleanup;
> + pids[0] = vm->pid;

[libvirt] [PATCH RFC 1/2] Resctrl: Add new xml element to support cache tune

2017-06-12 Thread Eli Qiao

This patch adds new xml element to support cache tune as:


  ...
  
  ...


id: any non-minus number
cache_id: reference of the host's cache banks id, it's from capabilities
level: cache level
type: cache bank type
size: should be multiples of the min_size of the bank on host.
vcpus: cache allocation on vcpu set, if empty, will apply the allocation
on all vcpus

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 docs/schemas/domaincommon.rng |  54 +
 src/conf/domain_conf.c| 131 ++
 src/conf/domain_conf.h|  21 +++
 3 files changed, 206 insertions(+)

diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng
index 4950ddc..11dbb55 100644
--- a/docs/schemas/domaincommon.rng
+++ b/docs/schemas/domaincommon.rng
@@ -834,6 +834,35 @@
 
   
 
+
+  
+
+  
+
+
+  
+
+
+  
+
+
+  
+
+
+  
+
+
+  
+
+  
+
+
+  
+
+  
+
+  
+
 
   
 
@@ -5686,6 +5715,31 @@
   -1
 
   
+  
+
+  [0-9]+
+
+  
+  
+
+  [0-9]+
+
+  
+  
+
+  (3)
+
+  
+  
+
+  (both|code|data)
+
+  
+  
+
+  KiB
+
+  
   
   
 
diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
index 5c32f93..93984bc 100644
--- a/src/conf/domain_conf.c
+++ b/src/conf/domain_conf.c
@@ -16250,6 +16250,104 @@ virDomainVcpuPinDefParseXML(virDomainDefPtr def,
 return ret;
 }
 
+/* Parse the XML definition for cachetune
+ * and a cachetune has the form
+ * 
+ */
+static int
+virDomainCacheTuneDefParseXML(virDomainDefPtr def,
+  int n,
+  xmlNodePtr* nodes)
+{
+char* tmp = NULL;
+size_t i;
+int type = -1;
+virDomainCacheBankPtr bank = NULL;
+
+if (VIR_ALLOC_N(bank, n) < 0)
+goto cleanup;
+
+for (i = 0; i < n; i++) {
+if (!(tmp = virXMLPropString(nodes[i], "id"))) {
+virReportError(VIR_ERR_XML_ERROR, "%s", _("missing id in cache 
tune"));
+goto cleanup;
+}
+if (virStrToLong_uip(tmp, NULL, 10, &(bank[i].id)) < 0) {
+virReportError(VIR_ERR_XML_ERROR,
+   _("invalid setting for cache id '%s'"), tmp);
+goto cleanup;
+}
+VIR_FREE(tmp);
+
+if (!(tmp = virXMLPropString(nodes[i], "cache_id"))) {
+virReportError(VIR_ERR_XML_ERROR, "%s", _("missing cache id in 
cache tune"));
+goto cleanup;
+}
+if (virStrToLong_uip(tmp, NULL, 10, &(bank[i].cache_id)) < 0) {
+virReportError(VIR_ERR_XML_ERROR,
+   _("invalid setting for cache host id '%s'"), tmp);
+goto cleanup;
+}
+VIR_FREE(tmp);
+
+if (!(tmp = virXMLPropString(nodes[i], "level"))) {
+virReportError(VIR_ERR_XML_ERROR, "%s", _("missing cache level in 
cache tune"));
+goto cleanup;
+}
+if (virStrToLong_uip(tmp, NULL, 10, &(bank[i].level)) < 0) {
+virReportError(VIR_ERR_XML_ERROR,
+   _("invalid setting for cache level '%s'"), tmp);
+goto cleanup;
+}
+VIR_FREE(tmp);
+
+
+if (!(tmp = virXMLPropString(nodes[i], "size"))) {
+virReportError(VIR_ERR_XML_ERROR, "%s", _("missing size in cache 
tune"));
+goto cleanup;
+}
+if (virStrToLong_ull(tmp, NULL, 10, &(bank[i].size)) < 0) {
+virReportError(VIR_ERR_XML_ERROR,
+   _("invalid setting for cache size '%s'"), tmp);
+goto cleanup;
+}
+/* convert to B */
+bank[i].size *= 1024;
+VIR_FREE(tmp);
+
+if (!(tmp = virXMLPropString(nodes[i], "type"))) {
+virReportError(VIR_ERR_INTERNAL_ERROR, "%s",
+   _("missing cache type"));
+goto cleanup;
+}
+if ((type = virCacheTypeFromString(tmp)) < 0) {
+virReportError(VIR_ERR_XML_ERROR,
+_("'unsupported cache type '%s'"), tmp);
+goto cleanup;
+}
+
+if ((tmp = virXMLPropString(nodes[i], "vcpus"))) {
+if (virBitmapParse(tmp, [i].vcpus,
+VIR_DOMAIN_CPUMASK_LEN) < 0)
+goto cleanup;
+
+if (virBitmapIsAllClear(ba

[libvirt] [PATCH RFC 0/2] Implement l3 CAT

2017-06-12 Thread Eli Qiao

This is a RFC patch to implement l3 CAT. There's old RFC V3 [1] patch, but don't
get any attentions, reason maybe that it's is not a workable one.

I address some of the comments for RFC V2 version from Martin.

1. Add file lock while access /sys/fs/resctrl base on kernel documents [3].
2. Variable renaming.
3. Tested locally to create vm with cachetune defined and works well.

Issues want to get some comments:

I can not pass syntax-check as I reference the cache tune and cachebank
definition (from conf/domain_conf.h). Since I need a complex struct to
pass cachetune and host's cachetune information to resctrl.

I can split patch if it's hard to review. Sorry for inconvient.

Patch was pushed to github [4] for easing read.

[1] https://www.redhat.com/archives/libvir-list/2017-June/msg00069.html
[2] https://www.redhat.com/archives/libvir-list/2017-April/msg01466.html
[3] https://www.kernel.org/doc/Documentation/x86/intel_rdt_ui.txt
[4] https://github.com/taget/libvirt/commits/cache

Eli Qiao (2):
  Resctrl: Add new xml element to support cache tune
  Resctrl: Add uitls functions to operate sysfs resctrl

 docs/schemas/domaincommon.rng |  54 ++
 include/libvirt/virterror.h   |   1 +
 src/Makefile.am   |   1 +
 src/conf/domain_conf.c| 131 +
 src/conf/domain_conf.h|  21 +
 src/libvirt_private.syms  |   9 +
 src/qemu/qemu_process.c   |  54 ++
 src/util/virerror.c   |   1 +
 src/util/virresctrl.c | 851 ++
 src/util/virresctrl.h |  79 +++
 tests/Makefile.am |   8 +-
 tests/virresctrldata/L3-free.schemata |   1 +
 tests/virresctrldata/L3CODE-free.schemata |   1 +
 tests/virresctrldata/L3DATA-free.schemata |   1 +
 tests/virresctrldata/linux-resctrl|   1 +
 tests/virresctrldata/linux-resctrl-cdp|   1 +
 tests/virresctrltest.c| 119 +
 17 files changed, 1333 insertions(+), 1 deletion(-)
 create mode 100644 src/util/virresctrl.c
 create mode 100644 src/util/virresctrl.h
 create mode 100644 tests/virresctrldata/L3-free.schemata
 create mode 100644 tests/virresctrldata/L3CODE-free.schemata
 create mode 100644 tests/virresctrldata/L3DATA-free.schemata
 create mode 12 tests/virresctrldata/linux-resctrl
 create mode 12 tests/virresctrldata/linux-resctrl-cdp
 create mode 100644 tests/virresctrltest.c

--
1.9.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATCH RFC 2/2] Resctrl: Add uitls functions to operate sysfs resctrl

2017-06-12 Thread Eli Qiao

  count = 1;
+}
+ret = virResctrlSetCachetunes(caps, cachetune, vm->def->uuid, pids, count);
+
+ cleanup:
+VIR_FREE(pids);
+return ret;
+}
+
 
 int
 qemuProcessSetupIOThread(virDomainObjPtr vm,
@@ -5914,6 +5960,11 @@ qemuProcessLaunch(virConnectPtr conn,
 qemuProcessAutoDestroyAdd(driver, vm, conn) < 0)
 goto cleanup;
 
+VIR_WARN("Cache allocation");
+if (qemuProcessSetCacheBanks(&(driver->caps->host),
+ vm) < 0)
+goto cleanup;
+
 ret = 0;
 
  cleanup:
@@ -6419,6 +6470,9 @@ void qemuProcessStop(virQEMUDriverPtr driver,
 virPerfFree(priv->perf);
 priv->perf = NULL;
 
+if (&(vm->def->cachetune) != NULL)
+virResctrlRemoveCachetunes(vm->def->uuid);
+
 qemuProcessRemoveDomainStatus(driver, vm);
 
 /* Remove VNC and Spice ports from port reservation bitmap, but only if
diff --git a/src/util/virerror.c b/src/util/virerror.c
index ef17fb5..02fabcc 100644
--- a/src/util/virerror.c
+++ b/src/util/virerror.c
@@ -139,6 +139,7 @@ VIR_ENUM_IMPL(virErrorDomain, VIR_ERR_DOMAIN_LAST,
 
   "Perf", /* 65 */
   "Libssh transport layer",
+  "Resource Control",
 )
 
 
diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c
new file mode 100644
index 000..89bc43e
--- /dev/null
+++ b/src/util/virresctrl.c
@@ -0,0 +1,851 @@
+/*
+ * virresctrl.c: methods for managing resource control
+ *
+ * Copyright (C) 2017 Intel, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library.  If not, see
+ * <http://www.gnu.org/licenses/>.
+ *
+ * Authors:
+ *  Eli Qiao <liyong.q...@intel.com>
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "virresctrl.h"
+#include "virerror.h"
+#include "virlog.h"
+#include "viralloc.h"
+#include "virstring.h"
+#include "virfile.h"
+
+VIR_LOG_INIT("util.resctrl");
+
+#define VIR_FROM_THIS VIR_FROM_RESCTRL
+#define SYSFS_RESCTRL_PATH "/sys/fs/resctrl"
+#define MAX_CBM_LEN 20
+#define VIR_RESCTRL_LOCK(fd, op) flock(fd, op)
+#define VIR_RESCTRL_UNLOCK(fd) flock(fd, LOCK_UN)
+#define CONSTRUCT_RESCTRL_PATH(domain_name, item_name) \
+do { \
+if (NULL == domain_name) { \
+if (virAsprintf(, "%s/%s", SYSFS_RESCTRL_PATH, item_name) < 0) \
+return -1; \
+} else { \
+if (virAsprintf(, "%s/%s/%s", SYSFS_RESCTRL_PATH, domain_name, \
+item_name) < 0) \
+return -1;  \
+} \
+} while (0)
+
+VIR_ENUM_IMPL(virResctrl, VIR_RESCTRL_TYPE_LAST,
+  "L3",
+  "L3CODE",
+  "L3DATA")
+
+/**
+ * a virResctrlGroup represents a resource control group, it's a directory
+ * under /sys/fs/resctrl.
+ * e.g. /sys/fs/resctrl/CG1
+ * |-- cpus
+ * |-- schemata
+ * `-- tasks
+ * # cat schemata
+ * L3DATA:0=f;1=f
+ * L3CODE:0=f;1=f
+ *
+ * Besides, it can also represent the default resource control group of the
+ * host.
+ */
+
+typedef struct _virResctrlGroup virResctrlGroup;
+typedef virResctrlGroup *virResctrlGroupPtr;
+struct _virResctrlGroup {
+char *name; /* resource group name, NULL for default host group */
+size_t n_tasks; /* number of tasks assigned to the resource group */
+char **tasks; /* task id list */
+virResctrlSchemataPtr schemata[VIR_RESCTRL_TYPE_LAST]; /* Array for 
schemata */
+};
+
+/* All resource control groups on this host, including default resource group 
*/
+typedef struct _virResctrlHost virResctrlHost;
+typedef virResctrlHost *virResctrlHostPtr;
+struct _virResctrlHost {
+size_t n_groups; /* number of resource control group */
+virResctrlGroupPtr *groups; /* list of resource control group */
+};
+
+void
+virResctrlFreeSchemata(virResctrlSchemataPtr ptr)
+{
+size_t i;
+
+if (!ptr)
+return;
+
+for (i = 0; i < ptr->n_masks; i++) {
+virBitmapFree(ptr->masks[i]->mask);
+VIR_FREE(ptr->masks[i]);
+}
+
+VIR_FREE(ptr);
+ptr = NULL;
+}
+
+static void
+virResctrlFreeGroup(virResctrlGroupPtr ptr)
+{
+size_t i;
+
+if (!ptr)
+return;
+
+for (i = 0; i < ptr->n_tasks; i++)
+VIR_FREE(ptr->tasks[i]

Re: [libvirt] [PATCH v2] Report more correct information for cache control

2017-06-11 Thread Eli Qiao

Thanks martin, 

Looks good to me. 


On Friday, 9 June 2017 at 4:52 PM, Martin Kletzander wrote:

> On some platforms the number of bits in the cbm_mask might not be
> divisible by 4 (and not even by 2), so we need to properly count the
> bits. Similar file, min_cbm_bits, is properly parsed and used, but if
> the number is greater than one, we lose the information about
> granularity when reporting the data in capabilities. For that matter
> always report granularity, but if it is not the same as the minimum,
> add that information in there as well.
> 
> Signed-off-by: Martin Kletzander  (mailto:mklet...@redhat.com)>
> ---
> v2:
> - More expressive comments for structure members
> - Cache size adjusted to match the SKX machine (33M)
> 
> v1:
> - https://www.redhat.com/archives/libvir-list/2017-June/msg00229.html
> 
ACK 

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [V4 RESEND PATCH] Expose resource control capabilites on cache bank

2017-06-06 Thread Eli Qiao

> >  
> > This could be wrong on new platform in Intel’s SKX CPU after check with 
> > platform guys.
> >  
> > The cbm_mask is “7ff” (11 bits) on SKX. I will refine this by counting the 
> > bits.
> >  
> > We can virFileReadValueString() then convert it to unsigned int, then count 
> > the bits of ‘1’.
> >  
> > thought ? or a new utils function in virfile is required?
>  
> I missed this patch, sorry. I was thinking about changing it to bit
> counting, but I thought it will always be divisible by 4. I will send a
> trivial update to this, but it would be nice to have test cases for it.
> Could you get some of that data from such a machine n the meantime? If
> not, I can just coy what we have and change the mask.
>  
I don’t have a SKX machine by hand, I ask from other guys, the only different 
is  

cat /sys/fs/resctrl/info/L3/cbm_mask

7ff

it’s 11 bits, which can not divided by 4.  :( I don’t understand why the 
hardware designed as this.

// below are the /sys/fs/resctrl contents on SKX 
/

# cd /sys/fs/resctrl

# mkdir p1

# tree /sys/fs/resctrl/

/sys/fs/resctrl/

├── cpus

├── info

│   └── L3

│   ├── cbm_mask

│   ├── min_cbm_bits

│   └── num_closids

├── p1

│   ├── cpus

│   ├── schemata

│   └── tasks

├── schemata

└── tasks

  

# cat /sys/fs/resctrl/info/L3/cbm_mask

7ff

  

# cat /sys/fs/resctrl/info/L3/min_cbm_bits

1

  

# cat /sys/fs/resctrl/info/L3/num_closids

16

  

# cat /sys/fs/resctrl/cpus

,,,,,,,,,,,,,

  

# cat /sys/fs/resctrl/schemata

L3:0=7ff;1=7ff
  
>  
> > I am not sure min_cbm_bits should be used.
> >  
> > Besides, on old platform (haswell), the min_cbm_bits is 2, but we still can 
> > specify a cbm like
> > “0x7” ) 3bit while do the cache allocation.
> >  
>  
>  
> Yes, you can. min_cbm_bits is not a granularity. it is the minimum
> number of consecutive bits that you need to have when allocating. We
> should also add the granularity there. I'll fix that as well then.
>  

Great,
for “haswell" platform, which is old and has bad support for CAT,  
 
https://github.com/torvalds/linux/blob/master/arch/x86/kernel/cpu/intel_rdt.c#L166

I think we can deal with this in libvirt as a special case ?
  

  
>  
> > Later we need this on doing cache allocation(instead of read these value 
> > from the host again), we will
> > need
> > 1. total cache size
> > 2. what the cache size of 1 (CBM) bit stand for, so that we can calculate 
> > how many bits we want.
> >  
> > Maybe we need to add another filed like ’step’ (to indicate 1 bits stand 
> > for) ?
> >  
> > > > control->scope = scope;
> > > >  
> > > > @@ -1732,10 +1743,10 @@ virCapabilitiesInitCaches(virCapsPtr caps)
> > > > VIR_CACHE_TYPE_BOTH) < 0)
> > > > goto cleanup;
> > > > } else if (typeret == 1) {
> > > > - if ((virCapabilitiesGetCacheControl(bank,
> > > > - VIR_CACHE_TYPE_CODE) < 0) ||
> > > > - (virCapabilitiesGetCacheControl(bank,
> > > > - VIR_CACHE_TYPE_DATA) < 0))
> > > > + if (virCapabilitiesGetCacheControl(bank,
> > > > + VIR_CACHE_TYPE_CODE) < 0 ||
> > > > + virCapabilitiesGetCacheControl(bank,
> > > > + VIR_CACHE_TYPE_DATA) < 0)
> > > > goto cleanup;
> > > > }
> > > >  
> > > > diff --git i/tests/vircaps2xmldata/vircaps-x86_64-resctrl-cdp.xml 
> > > > w/tests/vircaps2xmldata/vircaps-x86_64-resctrl-cdp.xml
> > > > index c9f460d8a227..49aa0b98ca88 100644
> > > > --- i/tests/vircaps2xmldata/vircaps-x86_64-resctrl-cdp.xml
> > > > +++ w/tests/vircaps2xmldata/vircaps-x86_64-resctrl-cdp.xml
> > > > @@ -42,12 +42,12 @@
> > > > 
> > > > 
> > > > 
> > > > - 
> > > > - 
> > > > + 
> > > > + 
> > > > 
> > > > 
> > > > - 
> > > > - 
> > > > + 
> > > > + 
> > > > 
> > > > 
> > > > 
> > > > diff --git i/tests/vircaps2xmldata/vircaps-x86_64-resctrl.xml 
> > > > w/tests/vircaps2xmldata/vircaps-x86_64-resctrl.xml
> > > > index 04a5eb895727..cb78b4ab788d 100644
> > > > --- i/tests/vircaps2xmldata/vircaps-x86_64-resctrl.xml
> > > > +++ w/tests/vircaps2xmldata/vircaps-x86_64-resctrl.xml
> > > > @@ -42,10 +42,10 @@
> > > > 
> > > > 
> > > > 
> > > > - 
> > > > + 
> > > > 
> > > > 
> > > > - 
> > > > + 
> > > >  
> > >  
> >  
> >  
> >  
> > > > 
> > > > 
> > > > 
> > > > --
> > > > Martin
> > > >  
> > >  
> >  
> >  
>  
>  
>  


--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH] Report more correct information for cache control

2017-06-05 Thread Eli Qiao

hi Martin  

The code looks good to me, I have 1 comment for the granularity’s comment, and 
another one for the test data.

Otherwise, looks perfect, good to learn how to deal with hex   


On Monday, 5 June 2017 at 9:14 PM, Martin Kletzander wrote:

> On some platforms the number of bits in the cbm_mask might not be
> divisible by 4 (and not even by 2), so we need to properly count the
> bits. Similar file, min_cbm_bits, is properly parsed and used, but if
> the number is greater than one, we lose the information about
> granularity when reporting the data in capabilities. For that matter
> always report granularity, but if it is not the same as the minimum,
> add that information in there as well.
>  
> Signed-off-by: Martin Kletzander  (mailto:mklet...@redhat.com)>
> ---
> docs/schemas/capability.rng | 7 -
> src/conf/capabilities.c | 34 ++
> src/conf/capabilities.h | 1 +
> .../linux-resctrl-skx/resctrl/info/L3/cbm_mask | 1 +
> .../linux-resctrl-skx/resctrl/info/L3/min_cbm_bits | 1 +
> .../linux-resctrl-skx/resctrl/info/L3/num_closids | 1 +
> .../linux-resctrl-skx/resctrl/schemata | 1 +
> .../system/cpu/cpu0/cache/index0/id | 1 +
> .../system/cpu/cpu0/cache/index0/level | 1 +
> .../system/cpu/cpu0/cache/index0/shared_cpu_list | 1 +
> .../system/cpu/cpu0/cache/index0/shared_cpu_map | 1 +
> .../system/cpu/cpu0/cache/index0/size | 1 +
> .../system/cpu/cpu0/cache/index0/type | 1 +
> .../linux-resctrl-skx/system/cpu/cpu0/online | 1 +
> .../system/cpu/cpu0/topology/core_id | 1 +
> .../system/cpu/cpu0/topology/core_siblings | 1 +
> .../system/cpu/cpu0/topology/core_siblings_list | 1 +
> .../system/cpu/cpu0/topology/physical_package_id | 1 +
> .../system/cpu/cpu0/topology/thread_siblings | 1 +
> .../system/cpu/cpu0/topology/thread_siblings_list | 1 +
> .../linux-resctrl-skx/system/cpu/online | 1 +
> .../linux-resctrl-skx/system/cpu/present | 1 +
> .../linux-resctrl-skx/system/node/node0/cpu0 | 1 +
> .../linux-resctrl-skx/system/node/node0/cpulist | 1 +
> .../linux-resctrl-skx/system/node/node0/cpumap | 1 +
> .../linux-resctrl-skx/system/node/node0/distance | 1 +
> .../linux-resctrl-skx/system/node/online | 1 +
> .../vircaps2xmldata/vircaps-x86_64-resctrl-cdp.xml | 8 ++---
> .../vircaps2xmldata/vircaps-x86_64-resctrl-skx.xml | 31 
> tests/vircaps2xmldata/vircaps-x86_64-resctrl.xml | 4 +--
> tests/vircaps2xmltest.c | 1 +
> 31 files changed, 97 insertions(+), 13 deletions(-)
> create mode 100644 
> tests/vircaps2xmldata/linux-resctrl-skx/resctrl/info/L3/cbm_mask
> create mode 100644 
> tests/vircaps2xmldata/linux-resctrl-skx/resctrl/info/L3/min_cbm_bits
> create mode 100644 
> tests/vircaps2xmldata/linux-resctrl-skx/resctrl/info/L3/num_closids
> create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx/resctrl/schemata
> create mode 100644 
> tests/vircaps2xmldata/linux-resctrl-skx/system/cpu/cpu0/cache/index0/id
> create mode 100644 
> tests/vircaps2xmldata/linux-resctrl-skx/system/cpu/cpu0/cache/index0/level
> create mode 100644 
> tests/vircaps2xmldata/linux-resctrl-skx/system/cpu/cpu0/cache/index0/shared_cpu_list
> create mode 100644 
> tests/vircaps2xmldata/linux-resctrl-skx/system/cpu/cpu0/cache/index0/shared_cpu_map
> create mode 100644 
> tests/vircaps2xmldata/linux-resctrl-skx/system/cpu/cpu0/cache/index0/size
> create mode 100644 
> tests/vircaps2xmldata/linux-resctrl-skx/system/cpu/cpu0/cache/index0/type
>  
>  

It seems you miss cache/index3 ?
  
> create mode 100644 
> tests/vircaps2xmldata/linux-resctrl-skx/system/cpu/cpu0/online
> create mode 100644 
> tests/vircaps2xmldata/linux-resctrl-skx/system/cpu/cpu0/topology/core_id
> create mode 100644 
> tests/vircaps2xmldata/linux-resctrl-skx/system/cpu/cpu0/topology/core_siblings
> create mode 100644 
> tests/vircaps2xmldata/linux-resctrl-skx/system/cpu/cpu0/topology/core_siblings_list
> create mode 100644 
> tests/vircaps2xmldata/linux-resctrl-skx/system/cpu/cpu0/topology/physical_package_id
> create mode 100644 
> tests/vircaps2xmldata/linux-resctrl-skx/system/cpu/cpu0/topology/thread_siblings
> create mode 100644 
> tests/vircaps2xmldata/linux-resctrl-skx/system/cpu/cpu0/topology/thread_siblings_list
> create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx/system/cpu/online
> create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx/system/cpu/present
> create mode 12 
> tests/vircaps2xmldata/linux-resctrl-skx/system/node/node0/cpu0
> create mode 100644 
> tests/vircaps2xmldata/linux-resctrl-skx/system/node/node0/cpulist
> create mode 100644 
> tests/vircaps2xmldata/linux-resctrl-skx/system/node/node0/cpumap
> create mode 100644 
> tests/vircaps2xmldata/linux-resctrl-skx/system/node/node0/distance
> create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx/system/node/online
> create mode 100644 tests/vircaps2xmldata/vircaps-x86_64-resctrl-skx.xml
>  
> diff --git a/docs/schemas/capability.rng b/docs/schemas/capability.rng
> index e5cbfa362ec0..071090c2d991 100644
> ---

[libvirt] [PATCH] Capabilities: add step to cache control

2017-06-05 Thread Eli Qiao

This patch adds cache control step size under control tag.

step is calculated by total_cache_size / cbm_length. This will be needed
while later doing cache allocation to calculate cbm bits.

Besides, count host's cbm length by counting bit instead of checking
cbm_mask length, along with test case updated.

Signed-off-by: Eli Qiao <qiaoliy...@gmail.com>
---
 docs/schemas/capability.rng   |  3 +++
 src/conf/capabilities.c   | 19 ---
 src/conf/capabilities.h   |  1 +
 .../linux-resctrl/resctrl/info/L3/cbm_mask|  2 +-
 .../linux-resctrl/resctrl/info/L3/min_cbm_bits|  2 +-
 .../linux-resctrl/resctrl/info/L3/num_closids |  2 +-
 tests/vircaps2xmldata/vircaps-x86_64-resctrl-cdp.xml  |  8 
 tests/vircaps2xmldata/vircaps-x86_64-resctrl.xml  |  4 ++--
 8 files changed, 29 insertions(+), 12 deletions(-)

diff --git a/docs/schemas/capability.rng b/docs/schemas/capability.rng
index e5cbfa3..6be7ac9 100644
--- a/docs/schemas/capability.rng
+++ b/docs/schemas/capability.rng
@@ -276,6 +276,9 @@
   
 
   
+  
+
+  
   
 
   
diff --git a/src/conf/capabilities.c b/src/conf/capabilities.c
index 3becc7e..d1266e6 100644
--- a/src/conf/capabilities.c
+++ b/src/conf/capabilities.c
@@ -908,9 +908,10 @@ virCapabilitiesFormatCaches(virBufferPtr buf,
 for (j = 0; j < bank->ncontrols; j++) {
 bool min_kilos = !(bank->controls[j]->min % 1024);
 virBufferAsprintf(,
-  "\n",
   bank->controls[j]->min >> (min_kilos * 10),
+  bank->controls[j]->step >> (min_kilos * 10),
   min_kilos ? "KiB" : "B",
   virCacheTypeToString(bank->controls[j]->scope),
   bank->controls[j]->max_allocation);
@@ -1602,6 +1603,8 @@ virCapabilitiesGetCacheControl(virCapsHostCacheBankPtr 
bank,
 char *cbm_mask = NULL;
 char *type_upper = NULL;
 unsigned int min_cbm_bits = 0;
+unsigned int cbm_mask_val = 0;
+unsigned int cbm_mask_len = 0;
 virCapsHostCacheControlPtr control;
 
 if (VIR_ALLOC(control) < 0)
@@ -1632,8 +1635,18 @@ virCapabilitiesGetCacheControl(virCapsHostCacheBankPtr 
bank,
 
 virStringTrimOptionalNewline(cbm_mask);
 
-/* cbm_mask: cache bit mask, it's in hex, eg: f */
-control->min = min_cbm_bits * bank->size / (strlen(cbm_mask) * 4);
+/* cbm_mask: cache bit mask, it's in hex, e.g. f or 7ff */
+if (virStrToLong_uip(cbm_mask, NULL, 16, _mask_val) < 0)
+goto cleanup;
+
+while (cbm_mask_val & 0x1) {
+cbm_mask_val = cbm_mask_val >> 1;
+cbm_mask_len ++;
+}
+
+control->step = bank->size / cbm_mask_len;
+
+control->min = min_cbm_bits * control->step;
 
 control->scope = scope;
 
diff --git a/src/conf/capabilities.h b/src/conf/capabilities.h
index ee87d59..2cc9bdc 100644
--- a/src/conf/capabilities.h
+++ b/src/conf/capabilities.h
@@ -152,6 +152,7 @@ typedef struct _virCapsHostCacheControl 
virCapsHostCacheControl;
 typedef virCapsHostCacheControl *virCapsHostCacheControlPtr;
 struct _virCapsHostCacheControl {
 unsigned long long min; /* minimum cache control size in B */
+unsigned long long step; /* cache control step size in B */
 virCacheType scope;  /* data, code or both */
 unsigned int max_allocation; /* max number of supported allocations */
 };
diff --git a/tests/vircaps2xmldata/linux-resctrl/resctrl/info/L3/cbm_mask 
b/tests/vircaps2xmldata/linux-resctrl/resctrl/info/L3/cbm_mask
index 78031da..d482bbb 100644
--- a/tests/vircaps2xmldata/linux-resctrl/resctrl/info/L3/cbm_mask
+++ b/tests/vircaps2xmldata/linux-resctrl/resctrl/info/L3/cbm_mask
@@ -1 +1 @@
-f
+7ff
diff --git a/tests/vircaps2xmldata/linux-resctrl/resctrl/info/L3/min_cbm_bits 
b/tests/vircaps2xmldata/linux-resctrl/resctrl/info/L3/min_cbm_bits
index 0cfbf08..d00491f 100644
--- a/tests/vircaps2xmldata/linux-resctrl/resctrl/info/L3/min_cbm_bits
+++ b/tests/vircaps2xmldata/linux-resctrl/resctrl/info/L3/min_cbm_bits
@@ -1 +1 @@
-2
+1
diff --git a/tests/vircaps2xmldata/linux-resctrl/resctrl/info/L3/num_closids 
b/tests/vircaps2xmldata/linux-resctrl/resctrl/info/L3/num_closids
index b8626c4..b6a7d89 100644
--- a/tests/vircaps2xmldata/linux-resctrl/resctrl/info/L3/num_closids
+++ b/tests/vircaps2xmldata/linux-resctrl/resctrl/info/L3/num_closids
@@ -1 +1 @@
-4
+16
diff --git a/tests/vircaps2xmldata/vircaps-x86_64-resctrl-cdp.xml 
b/tests/vircaps2xmldata/vircaps-x86_64-resctrl-cdp.xml
index 49aa0b9..98893b6 100644
--- a/tests/v

Re: [libvirt] [V4 RESEND PATCH] Expose resource control capabilites on cache bank

2017-06-05 Thread Eli Qiao

Martin:

Please hold on merge this first.

I found we still need some modification.  

> >  
> > Again, wrong indentation and unnecessary parentheses.
> >  
> > Otherwise it looks good, so ACK with those differences and reworded
> > commit message. Let me know if you agree and I can push it immediately.
> > The suggested diff follows:
> >  
> > diff --git i/docs/schemas/capability.rng w/docs/schemas/capability.rng
> > index 927fc184de41..e5cbfa362ec0 100644
> > --- i/docs/schemas/capability.rng
> > +++ w/docs/schemas/capability.rng
> > @@ -261,13 +261,7 @@
> > 
> > 
> > 
> > - 
> > - 
> > - both
> > - code
> > - data
> > - 
> > - 
> > + 
> > 
> > 
> > 
> > @@ -285,14 +279,8 @@
> > 
> > 
> > 
> > - 
> > - 
> > - both
> > - code
> > - data
> > - 
> > - 
> > - 
> > + 
> > + 
> > 
> > 
> > 
> > @@ -302,6 +290,16 @@
> > 
> > 
> >  
> > + 
> > + 
> > + 
> > + both
> > + code
> > + data
> > + 
> > + 
> > + 
> > +
> > 
> > 
> > 
> > diff --git i/src/conf/capabilities.c w/src/conf/capabilities.c
> > index 8cd2957e9c88..3becc7e18c62 100644
> > --- i/src/conf/capabilities.c
> > +++ w/src/conf/capabilities.c
> > @@ -909,7 +909,7 @@ virCapabilitiesFormatCaches(virBufferPtr buf,
> > bool min_kilos = !(bank->controls[j]->min % 1024);
> > virBufferAsprintf(,
> > " > - "scope='%s' max_allocation='%u'/>\n",
> > + "type='%s' maxAllocs='%u'/>\n",
> > bank->controls[j]->min >> (min_kilos * 10),
> > min_kilos ? "KiB" : "B",
> > virCacheTypeToString(bank->controls[j]->scope),
> > @@ -920,12 +920,10 @@ virCapabilitiesFormatCaches(virBufferPtr buf,
> > virBufferAddLit(buf, ">\n");
> > virBufferAddBuffer(buf, );
> > virBufferAddLit(buf, "\n");
> > -
> > } else {
> > virBufferAddLit(buf, "/>\n");
> > }
> >  
> > - virBufferFreeAndReset();
> > VIR_FREE(cpus_str);
> > }
> >  
> > @@ -1557,16 +1555,19 @@ virCapsHostCacheBankFree(virCapsHostCacheBankPtr 
> > ptr)
> > VIR_FREE(ptr);
> > }
> >  
> > -/* test which TYPE of cache control supported
> > - * -1: don't support
> > - * 0: cat
> > - * 1: cdp
> > +/*
> > + * This function tests which TYPE of cache control is supported
> > + * Return values are:
> > + * -1: not supported
> > + * 0: CAT
> > + * 1: CDP
> > */
> > static int
> > virCapabilitiesGetCacheControlType(virCapsHostCacheBankPtr bank)
> > {
> > int ret = -1;
> > char *path = NULL;
> > +
> > if (virAsprintf(,
> > SYSFS_RESCTRL_PATH "/info/L%u",
> > bank->level) < 0)
> > @@ -1576,7 +1577,10 @@ 
> > virCapabilitiesGetCacheControlType(virCapsHostCacheBankPtr bank)
> > ret = 0;
> > } else {
> > VIR_FREE(path);
> > - /* for CDP enabled case, CODE and DATA will show together */
> > + /*
> > + * If CDP is enabled, there will be both CODE and DATA, but it's enough
> > + * to check one of those only.
> > + */
> > if (virAsprintf(,
> > SYSFS_RESCTRL_PATH "/info/L%uCODE",
> > bank->level) < 0)
> > @@ -1597,13 +1601,14 @@ 
> > virCapabilitiesGetCacheControl(virCapsHostCacheBankPtr bank,
> > char *path = NULL;
> > char *cbm_mask = NULL;
> > char *type_upper = NULL;
> > + unsigned int min_cbm_bits = 0;
> > virCapsHostCacheControlPtr control;
> >  
> > if (VIR_ALLOC(control) < 0)
> > goto cleanup;
> >  
> > - if ((scope > VIR_CACHE_TYPE_BOTH)
> > - && (virStringToUpper(_upper, virCacheTypeToString(scope)) < 0))
> > + if (scope != VIR_CACHE_TYPE_BOTH &&
> > + virStringToUpper(_upper, virCacheTypeToString(scope)) < 0)
> > goto cleanup;
> >  
> > if (virFileReadValueUint(>max_allocation,
> > @@ -1619,10 +1624,16 @@ 
> > virCapabilitiesGetCacheControl(virCapsHostCacheBankPtr bank,
> > type_upper ? type_upper: "") < 0)
> > goto cleanup;
> >  
> > + if (virFileReadValueUint(_cbm_bits,
> > + SYSFS_RESCTRL_PATH "/info/L%u%s/min_cbm_bits",
> > + bank->level,
> > + type_upper ? type_upper : "") < 0)
> > + goto cleanup;
> > +
> > virStringTrimOptionalNewline(cbm_mask);
> >  
> > /* cbm_mask: cache bit mask, it's in hex, eg: f */
> > - control->min = bank->size / (strlen(cbm_mask) * 4);
> > + control->min = min_cbm_bits * bank->size / (strlen(cbm_mask) * 4);
> >  

This could be wrong on new platform in Intel’s SKX CPU after check with 
platform guys.

The cbm_mask is “7ff” (11 bits) on SKX. I will refine this by counting the bits.

We can virFileReadValueString() then convert it to unsigned int, then count the 
bits of ‘1’.

thought ? or a new utils function in virfile is required?

I am not sure min_cbm_bits should be used.

Besides, on old platform (haswell), the min_cbm_bits is 2, but we still can 
specify a cbm like
“0x7” ) 3bit while do the cache allocation.

Later we need this on doing cache allocation(instead of read these value from 
the host again), we will
need
1. total cache size
2. what the cache size of 1 (CBM) bit stand for, so that we can calculate how 
many bits we want.

Maybe we need to add another filed like ’step’ (to indicate 1 bits stand for) ?

> >  control->scope = scope;
> >  
> > @@ -1732,10 +1743,10 @@ virCapabilitiesInitCaches(virCapsPtr caps)
> > VIR_CACHE_TYPE_BOTH) < 0)
> > goto cleanup;
> > } else if (typeret ==

Re: [libvirt] [V4 RESEND PATCH] Expose resource control capabilites on cache bank

2017-06-04 Thread Eli Qiao

hi Martin 
Thanks for you reviewing and I am okay with the diff as  you suggested.

Please help to push this patch.

Eli. 


On Sunday, 4 June 2017 at 5:39 PM, Martin Kletzander wrote:

> On Wed, May 17, 2017 at 05:08:33PM +0800, taget wrote:
> > From: Eli Qiao <liyong.q...@intel.com (mailto:liyong.q...@intel.com)>
> > 
> > * This patch amends the cache bank capability as follow:
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > For CDP enabled on x86 arch, we will have:
> > 
> > 
> > 
> > 
> > 
> > ...
> > 
> > * Added a new testdata directory `linux-resctrl-cdp` to test CDP enabled
> > case.
> > 
> > * Along with vircaps2xmltest case updated.
> > 
> > P.S. here the scope is from /sys/fs/resctrl/info/L3{"" "code" "data"}, I
> > keep it capital upper as I need to use a macro to convert from enum to
> > string easily.
> > 
> > Signed-off-by: Eli Qiao <liyong.q...@intel.com 
> > (mailto:liyong.q...@intel.com)>
> > ---
> > docs/schemas/capability.rng | 20 
> > src/conf/capabilities.c | 133 -
> > src/conf/capabilities.h | 10 ++
> > .../vircaps2xmldata/linux-resctrl-cdp/resctrl/cpus | 1 +
> > .../linux-resctrl-cdp/resctrl/info/L3CODE/cbm_mask | 1 +
> > .../resctrl/info/L3CODE/min_cbm_bits | 1 +
> > .../resctrl/info/L3CODE/num_closids | 1 +
> > .../linux-resctrl-cdp/resctrl/info/L3DATA/cbm_mask | 1 +
> > .../resctrl/info/L3DATA/min_cbm_bits | 1 +
> > .../resctrl/info/L3DATA/num_closids | 1 +
> > .../linux-resctrl-cdp/resctrl/manualres/cpus | 1 +
> > .../linux-resctrl-cdp/resctrl/manualres/schemata | 2 +
> > .../linux-resctrl-cdp/resctrl/manualres/tasks | 0
> > .../linux-resctrl-cdp/resctrl/schemata | 2 +
> > .../linux-resctrl-cdp/resctrl/tasks | 0
> > tests/vircaps2xmldata/linux-resctrl-cdp/system | 1 +
> > .../vircaps2xmldata/vircaps-x86_64-resctrl-cdp.xml | 55 +
> > tests/vircaps2xmldata/vircaps-x86_64-resctrl.xml | 8 +-
> > tests/vircaps2xmltest.c | 8 ++
> > 19 files changed, 244 insertions(+), 3 deletions(-)
> > create mode 100644 tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/cpus
> > create mode 100755 
> > tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3CODE/cbm_mask
> > create mode 100755 
> > tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3CODE/min_cbm_bits
> > create mode 100755 
> > tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3CODE/num_closids
> > create mode 100755 
> > tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3DATA/cbm_mask
> > create mode 100755 
> > tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3DATA/min_cbm_bits
> > create mode 100755 
> > tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3DATA/num_closids
> > create mode 100644 
> > tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/manualres/cpus
> > create mode 100644 
> > tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/manualres/schemata
> > create mode 100644 
> > tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/manualres/tasks
> > create mode 100644 tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/schemata
> > create mode 100644 tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/tasks
> > create mode 12 tests/vircaps2xmldata/linux-resctrl-cdp/system
> > create mode 100644 tests/vircaps2xmldata/vircaps-x86_64-resctrl-cdp.xml
> > 
> > diff --git a/docs/schemas/capability.rng b/docs/schemas/capability.rng
> > index 26f0aa2..927fc18 100644
> > --- a/docs/schemas/capability.rng
> > +++ b/docs/schemas/capability.rng
> > @@ -277,6 +277,26 @@
> > 
> > 
> > 
> > + 
> > + 
> > + 
> > + 
> > + 
> > + 
> > + 
> > + 
> > + 
> > 
> 
> 
> This should be 'type', as discussed before.
> 
> > + 
> > + both
> > + code
> > + data
> > + 
> > + 
> > 
> 
> 
> And hence this could be its own new definition, since it is already used
> in the bank definition.
> 
> > + 
> 
> Since we are trying to prefer camelCase, even in the XML, and this
> should be plural, I would suggest 'maxAllocs'.
> 
> > + 
> > + 
> > + 
> > + 
> > 
> > 
> > 
> > diff --git a/src/conf/capabilities.c b/src/conf/capabilities.c
> > index d699b08..c4a1fdf 100644
> > --- a/src/conf/capabilities.c
> > +++ b/src/conf/capabilities.c
> > @@ -51,6 +51,7 @@
> > #define VIR_FROM_THIS VIR_FROM_CAPABILITIES
> > 
> > #define SYSFS_SYSTEM_PAT

[libvirt] [V3 PATCH] RFC: Reimplement cache allocation for a VM

2017-06-02 Thread Eli Qiao

From: Eli Qiao <liyong.q...@intel.com>

This is a RFC patch for the reimplement of `support cache tune(CAT) in
libvirt`[1].

This patch defines some structs to represent data struct in linux
resctrl fs which will be used later to do cache allocation.

The patch expose a private interface `virResctrlFreeSchemata`, which
will be used to query the cache allocation on the host.

Also added unit test cases to test this interface can works well.

There are already patch sets[2] to address it, and functional
works, but people doesn't like it cause it has global variable, and
missing unit test case for new added capabilites, etc.

Martin has proposed a test infra to do vircaps2xmltest, and I extened it
on top of it to extend resctrl control[3], this is kinds of new desiged
apart from [2], so I propose this RFC patch to do some rework on it.

[1] https://www.redhat.com/archives/libvir-list/2017-January/msg00683.html
[2] https://www.redhat.com/archives/libvir-list/2017-March/msg00181.html
[3] https://www.redhat.com/archives/libvir-list/2017-April/msg00516.html
---
 include/libvirt/virterror.h   |   1 +
 src/Makefile.am   |   1 +
 src/libvirt_private.syms  |   7 +
 src/util/virerror.c   |   1 +
 src/util/virresctrl.c | 390 ++
 src/util/virresctrl.h |  72 ++
 tests/Makefile.am |   8 +-
 tests/virresctrldata/L3-free.schemata |   1 +
 tests/virresctrldata/L3CODE-free.schemata |   1 +
 tests/virresctrldata/L3DATA-free.schemata |   1 +
 tests/virresctrldata/linux-resctrl|   1 +
 tests/virresctrldata/linux-resctrl-cdp|   1 +
 tests/virresctrltest.c| 119 +
 13 files changed, 603 insertions(+), 1 deletion(-)
 create mode 100644 src/util/virresctrl.c
 create mode 100644 src/util/virresctrl.h
 create mode 100644 tests/virresctrldata/L3-free.schemata
 create mode 100644 tests/virresctrldata/L3CODE-free.schemata
 create mode 100644 tests/virresctrldata/L3DATA-free.schemata
 create mode 12 tests/virresctrldata/linux-resctrl
 create mode 12 tests/virresctrldata/linux-resctrl-cdp
 create mode 100644 tests/virresctrltest.c

diff --git a/include/libvirt/virterror.h b/include/libvirt/virterror.h
index 2efee8f..4bc0c74 100644
--- a/include/libvirt/virterror.h
+++ b/include/libvirt/virterror.h
@@ -132,6 +132,7 @@ typedef enum {
 
 VIR_FROM_PERF = 65, /* Error from perf */
 VIR_FROM_LIBSSH = 66,   /* Error from libssh connection transport */
+VIR_FROM_RESCTRL = 67,  /* Error from resctrl */
 
 # ifdef VIR_ENUM_SENTINELS
 VIR_ERR_DOMAIN_LAST
diff --git a/src/Makefile.am b/src/Makefile.am
index 0b7cfc8..08ad5b4 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -167,6 +167,7 @@ UTIL_SOURCES =  
\
util/virprocess.c util/virprocess.h \
util/virqemu.c util/virqemu.h   \
util/virrandom.h util/virrandom.c   \
+   util/virresctrl.h util/virresctrl.c \
util/virrotatingfile.h util/virrotatingfile.c   \
util/virscsi.c util/virscsi.h   \
util/virscsihost.c util/virscsihost.h   \
diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms
index 429b095..cdd3467 100644
--- a/src/libvirt_private.syms
+++ b/src/libvirt_private.syms
@@ -2424,6 +2424,13 @@ virRandomGenerateWWN;
 virRandomInt;
 
 
+# util/virresctrl.h
+virResctrlBitmap2String;
+virResctrlFreeSchemata;
+virResctrlGetFreeCache;
+virResctrlTypeToString;
+
+
 # util/virrotatingfile.h
 virRotatingFileReaderConsume;
 virRotatingFileReaderFree;
diff --git a/src/util/virerror.c b/src/util/virerror.c
index ef17fb5..02fabcc 100644
--- a/src/util/virerror.c
+++ b/src/util/virerror.c
@@ -139,6 +139,7 @@ VIR_ENUM_IMPL(virErrorDomain, VIR_ERR_DOMAIN_LAST,
 
   "Perf", /* 65 */
   "Libssh transport layer",
+  "Resource Control",
 )
 
 
diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c
new file mode 100644
index 000..d3822ed
--- /dev/null
+++ b/src/util/virresctrl.c
@@ -0,0 +1,390 @@
+/*
+ * virresctrl.c: methods for managing resource control
+ *
+ * Copyright (C) 2017 Intel, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have recei

Re: [libvirt] [V4 RESEND PATCH] Expose resource control capabilites on cache bank

2017-05-30 Thread Eli Qiao

ping 


On Wednesday, 17 May 2017 at 5:08 PM, taget wrote:

> From: Eli Qiao <liyong.q...@intel.com (mailto:liyong.q...@intel.com)>
> 
> * This patch amends the cache bank capability as follow:
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> For CDP enabled on x86 arch, we will have:
> 
> 
> 
> 
> 
> ...
> 
> * Added a new testdata directory `linux-resctrl-cdp` to test CDP enabled
> case.
> 
> * Along with vircaps2xmltest case updated.
> 
> P.S. here the scope is from /sys/fs/resctrl/info/L3{"" "code" "data"}, I
> keep it capital upper as I need to use a macro to convert from enum to
> string easily.
> 
> Signed-off-by: Eli Qiao <liyong.q...@intel.com (mailto:liyong.q...@intel.com)>
> ---
> docs/schemas/capability.rng | 20 
> src/conf/capabilities.c | 133 -
> src/conf/capabilities.h | 10 ++
> .../vircaps2xmldata/linux-resctrl-cdp/resctrl/cpus | 1 +
> .../linux-resctrl-cdp/resctrl/info/L3CODE/cbm_mask | 1 +
> .../resctrl/info/L3CODE/min_cbm_bits | 1 +
> .../resctrl/info/L3CODE/num_closids | 1 +
> .../linux-resctrl-cdp/resctrl/info/L3DATA/cbm_mask | 1 +
> .../resctrl/info/L3DATA/min_cbm_bits | 1 +
> .../resctrl/info/L3DATA/num_closids | 1 +
> .../linux-resctrl-cdp/resctrl/manualres/cpus | 1 +
> .../linux-resctrl-cdp/resctrl/manualres/schemata | 2 +
> .../linux-resctrl-cdp/resctrl/manualres/tasks | 0
> .../linux-resctrl-cdp/resctrl/schemata | 2 +
> .../linux-resctrl-cdp/resctrl/tasks | 0
> tests/vircaps2xmldata/linux-resctrl-cdp/system | 1 +
> .../vircaps2xmldata/vircaps-x86_64-resctrl-cdp.xml | 55 +
> tests/vircaps2xmldata/vircaps-x86_64-resctrl.xml | 8 +-
> tests/vircaps2xmltest.c | 8 ++
> 19 files changed, 244 insertions(+), 3 deletions(-)
> create mode 100644 tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/cpus
> create mode 100755 
> tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3CODE/cbm_mask
> create mode 100755 
> tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3CODE/min_cbm_bits
> create mode 100755 
> tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3CODE/num_closids
> create mode 100755 
> tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3DATA/cbm_mask
> create mode 100755 
> tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3DATA/min_cbm_bits
> create mode 100755 
> tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3DATA/num_closids
> create mode 100644 
> tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/manualres/cpus
> create mode 100644 
> tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/manualres/schemata
> create mode 100644 
> tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/manualres/tasks
> create mode 100644 tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/schemata
> create mode 100644 tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/tasks
> create mode 12 tests/vircaps2xmldata/linux-resctrl-cdp/system
> create mode 100644 tests/vircaps2xmldata/vircaps-x86_64-resctrl-cdp.xml
> 
> diff --git a/docs/schemas/capability.rng b/docs/schemas/capability.rng
> index 26f0aa2..927fc18 100644
> --- a/docs/schemas/capability.rng
> +++ b/docs/schemas/capability.rng
> @@ -277,6 +277,26 @@
> 
> 
> 
> + 
> + 
> + 
> + 
> + 
> + 
> + 
> + 
> + 
> + 
> + both
> + code
> + data
> + 
> + 
> + 
> + 
> + 
> + 
> + 
> 
> 
> 
> diff --git a/src/conf/capabilities.c b/src/conf/capabilities.c
> index d699b08..c4a1fdf 100644
> --- a/src/conf/capabilities.c
> +++ b/src/conf/capabilities.c
> @@ -51,6 +51,7 @@
> #define VIR_FROM_THIS VIR_FROM_CAPABILITIES
> 
> #define SYSFS_SYSTEM_PATH "/sys/devices/system"
> +#define SYSFS_RESCTRL_PATH "/sys/fs/resctrl"
> 
> VIR_LOG_INIT("conf.capabilities")
> 
> @@ -872,6 +873,9 @@ virCapabilitiesFormatCaches(virBufferPtr buf,
> virCapsHostCacheBankPtr *caches)
> {
> size_t i = 0;
> + size_t j = 0;
> + int indent = virBufferGetIndent(buf, false);
> + virBuffer controlBuf = VIR_BUFFER_INITIALIZER;
> 
> if (!ncaches)
> return 0;
> @@ -893,13 +897,35 @@ virCapabilitiesFormatCaches(virBufferPtr buf,
> */
> virBufferAsprintf(buf,
> " - "size='%llu' unit='%s' cpus='%s'/>\n",
> + "size='%llu' unit='%s' cpus='%s'",
> bank->id, bank->level,
> virCacheTypeToString(bank->type),
> bank->size >> (kilos * 10),
> kilos ? "KiB" : "B",
> cpus_str);
> 
> + virBufferAdjustIndent(, indent + 4);
> + for (j = 0; j < bank->ncontrols; j++) {
> + bool min_kilos = !(bank->controls[j]->min % 1024);
> + virBufferAsprintf(,
> + " + "scope='%s' max_allocation='%u'/>\n",
>

[libvirt] [V4 RESEND PATCH] Expose resource control capabilites on cache bank

2017-05-25 Thread Eli Qiao

* This patch amends the cache bank capability as follow:


  

  
  

  


For CDP enabled on x86 arch, we will have:

  


  
...

* Added a new testdata directory `linux-resctrl-cdp` to test CDP enabled
case.

* Along with vircaps2xmltest case updated.

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 docs/schemas/capability.rng|  20 +++
 src/conf/capabilities.c| 138 -
 src/conf/capabilities.h|  12 ++
 .../vircaps2xmldata/linux-resctrl-cdp/resctrl/cpus |   1 +
 .../linux-resctrl-cdp/resctrl/info/L3CODE/cbm_mask |   1 +
 .../resctrl/info/L3CODE/min_cbm_bits   |   1 +
 .../resctrl/info/L3CODE/num_closids|   1 +
 .../linux-resctrl-cdp/resctrl/info/L3DATA/cbm_mask |   1 +
 .../resctrl/info/L3DATA/min_cbm_bits   |   1 +
 .../resctrl/info/L3DATA/num_closids|   1 +
 .../linux-resctrl-cdp/resctrl/manualres/cpus   |   1 +
 .../linux-resctrl-cdp/resctrl/manualres/schemata   |   2 +
 .../linux-resctrl-cdp/resctrl/manualres/tasks  |   0
 .../linux-resctrl-cdp/resctrl/schemata |   2 +
 .../linux-resctrl-cdp/resctrl/tasks|   0
 tests/vircaps2xmldata/linux-resctrl-cdp/system |   1 +
 .../vircaps2xmldata/vircaps-x86_64-resctrl-cdp.xml |  55 
 tests/vircaps2xmldata/vircaps-x86_64-resctrl.xml   |   8 +-
 tests/vircaps2xmltest.c|  10 +-
 19 files changed, 252 insertions(+), 4 deletions(-)
 create mode 100644 tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/cpus
 create mode 100755 
tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3CODE/cbm_mask
 create mode 100755 
tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3CODE/min_cbm_bits
 create mode 100755 
tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3CODE/num_closids
 create mode 100755 
tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3DATA/cbm_mask
 create mode 100755 
tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3DATA/min_cbm_bits
 create mode 100755 
tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3DATA/num_closids
 create mode 100644 
tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/manualres/cpus
 create mode 100644 
tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/manualres/schemata
 create mode 100644 
tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/manualres/tasks
 create mode 100644 tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/schemata
 create mode 100644 tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/tasks
 create mode 12 tests/vircaps2xmldata/linux-resctrl-cdp/system
 create mode 100644 tests/vircaps2xmldata/vircaps-x86_64-resctrl-cdp.xml

diff --git a/docs/schemas/capability.rng b/docs/schemas/capability.rng
index 26f0aa2..927fc18 100644
--- a/docs/schemas/capability.rng
+++ b/docs/schemas/capability.rng
@@ -277,6 +277,26 @@
   
 
   
+  
+
+  
+
+  
+  
+
+  
+  
+
+  both
+  code
+  data
+
+  
+  
+
+  
+
+  
 
   
 
diff --git a/src/conf/capabilities.c b/src/conf/capabilities.c
index a91a72a..0888605 100644
--- a/src/conf/capabilities.c
+++ b/src/conf/capabilities.c
@@ -51,6 +51,7 @@
 #define VIR_FROM_THIS VIR_FROM_CAPABILITIES
 
 #define SYSFS_SYSTEM_PATH "/sys/devices/system"
+#define SYSFS_RESCTRL_PATH "/sys/fs/resctrl"
 
 VIR_LOG_INIT("conf.capabilities")
 
@@ -872,6 +873,9 @@ virCapabilitiesFormatCaches(virBufferPtr buf,
 virCapsHostCacheBankPtr *caches)
 {
 size_t i = 0;
+size_t j = 0;
+int indent = virBufferGetIndent(buf, false);
+virBuffer controlBuf = VIR_BUFFER_INITIALIZER;
 
 if (!ncaches)
 return 0;
@@ -893,13 +897,35 @@ virCapabilitiesFormatCaches(virBufferPtr buf,
  */
 virBufferAsprintf(buf,
   "\n",
+  "size='%llu' unit='%s' cpus='%s'",
   bank->id, bank->level,
   virCacheTypeToString(bank->type),
   bank->size >> (kilos * 10),
   kilos ? "KiB" : "B",
   cpus_str);
 
+virBufferAdjustIndent(, indent + 4);
+for (j = 0; j < bank->ncontrols; j++) {
+bool min_kilos = !(bank->controls[j]->min % 1024);
+virBufferAsprintf(,
+  "\n",
+  bank->controls[j]->min >> (min_kilos * 10),
+  min_kilos ? "KiB" : "B",
+  virCa

Re: [libvirt] Libvirt CAT support plan

2017-05-03 Thread Eli Qiao



On Wednesday, 3 May 2017 at 4:34 PM, Daniel P. Berrange wrote:

> On Wed, May 03, 2017 at 08:30:31AM +0800, Eli Qiao wrote:
> > hi all,
> >  
> > Thanks for helping reviewing for CAT support in the past days.
> >  
> > I writing this email to ask for the plan in libvirt support.
> >  
> > I think we’v discussed this early this year, and I’v proposed a patch set 
> > [1].
> > But don’t get merged because of some performance reason ?
> >  
> > Then I proposed a redesign RFC[2] based on Martin’s cache branch,
> > thans Martin for the reviewing, I can address them but the question here
> > is it depends on Martin’s `cache` branch, which if for exposing host’s 
> > `cache`
> > information in capabilities xml, and it doesn’t get merged ether, I feel 
> > helpless.
> >  
>  
>  
> Martin posted review comments on your latest patches just a couple of days
> ago, and you've not posted any newer version of the patches since then
> that address those comments.
>  
Yes, I see that and response, I will refine my RFC patch.

I just worry about the dependencies. It’s long time and don’t get reviewed 
after the last post.
https://www.redhat.com/archives/libvir-list/2017-April/msg00541.html

And Martin’s `cache` branch was not merged yet, that’s not under my control, 
this
is my concern.  
>  
> > As CAT is a key feature which would be required by many customers and
> > especially the OpenStack integration.
> >  
>  
>  
> Regardless of what/who needs a feature, we're not going to rush to
> merge patches if there are still outstanding issues that need fixing.
>  
Agree.  
I would like to get some advice on which direction should go, with the new 
implementation,  
the dependency is a problem.
>  
> > Would like to know the plan and get the some suggestions.
>  
>  
> Carry on addressing the feedback provided & posting new versions of the
> patches for review.
>  
>  

Sure, I will continue working on that.  

BR,
Eli Qiao  

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] Libvirt CAT support plan

2017-05-02 Thread Eli Qiao

hi all,

Thanks for helping reviewing for CAT support in the past days.

I writing this email to ask for the plan in libvirt support.

I think we’v discussed this early this year, and I’v proposed a patch set [1].
But don’t get merged because of some performance reason ?

Then I proposed a redesign RFC[2] based on Martin’s cache branch,
thans Martin for the reviewing, I can address them but the question here
is it depends on Martin’s `cache` branch, which if for exposing host’s `cache`
information in capabilities xml, and it doesn’t get merged ether, I feel 
helpless.

As CAT is a key feature which would be required by many customers and
especially the OpenStack integration.

Would like to know the plan and get the some suggestions.

Shall I keep refine the old design [1] or woking on new design [2] (with 
dependencies) ?

[1] https://www.redhat.com/archives/libvir-list/2017-March/msg00181.html
[2] https://www.redhat.com/archives/libvir-list/2017-April/msg00747.html  

BR,
Eli
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH RFC V2] RFC: Reimplement cache allocation for a VM

2017-05-02 Thread Eli Qiao

hi Martin,  

Thanks for the reviewing, this’s the RFC for the reimplement cache allocation, 
so I don’t have the
code very cleanup yet,
I have a quick summaries:

1. We can use flock while read/writing /sys/fs//resctrl
2. I will do some renaming for the variables (plural or singular)
3. Will consider to refine the memory  

For the struct design, I am not sure if it’s the best way to have the schema a 
list, it doesn’t support  
‘indexing’ by cache type (L3/L3DATA/L3CODE), how about we pre-create a struct 
with the supported
cache type (we know what we support), instead of reading them from 
/sys/fs/resctrl ?


On Friday, 28 April 2017 at 8:16 PM, Martin Kletzander wrote:

> On Fri, Apr 14, 2017 at 06:01:46PM +0800, Eli Qiao wrote:
> > This is a RFC patch for the reimplement of `support cache tune(CAT) in
> > libvirt`[1].
> >  
>  
>  
> I searched for 'lock' in this mail and found nothing...
>  
No, I don’t add lock for now, this is the initial patch, aimed to avoid global 
variable.

We can use flock when access /sys/fs/resctrl/ (write/read). I don’t think 
that’s a big deal for now.   
>  
> > This patch defines some structs to represent data struct in linux
> > resctrl fs which will be used later to do cache allocation.
> >  
> > The patch expose a private interface `virResctrlFreeSchemata`, which
> > will be used to query the cache allocation on the host.
> >  
> > Also added unit test cases to test this interface can works well.
> >  
> > There are already patch sets[2] to address it, and functional
> > works, but people doesn't like it cause it has global variable, and
> > missing unit test case for new added capabilites, etc.
> >  
> > Martin has proposed a test infra to do vircaps2xmltest, and I extened it
> > on top of it to extend resctrl control[3], this is kinds of new desiged
> > apart from [2], so I propose this RFC patch to do some rework on it.
> >  
> > [1] https://www.redhat.com/archives/libvir-list/2017-January/msg00683.html
> > [2] https://www.redhat.com/archives/libvir-list/2017-March/msg00181.html
> > [3] https://www.redhat.com/archives/libvir-list/2017-April/msg00516.html
> > ---
> >  
> > + "L2")
>  
> Is there CAT for L2 currently?

No for now.  
>  
> > +
> > +/**
> > + * a virResctrlDomain represents a resource control group, it's a directory
> > + * under /sys/fs/resctrl.
> > + * eg: /sys/fs/resctrl/CG1
> > + * |-- cpus
> > + * |-- schemata
> > + * `-- tasks
> > + * # cat schemata
> > + * L3DATA:0=f;1=f
> > + * L3CODE:0=f;1=f
> > + *
> > + * Besides, it can also represent the default resource control group of the
> > + * host.
> > + */
> > +
> > +typedef struct _virResctrlGroup virResctrlGroup;
> > +typedef virResctrlGroup *virResctrlGroupPtr;
> > +struct _virResctrlGroup {
> > + char *name; /* resource group name, eg: CG1. If it represent host's
> >  
>  
>  
> s/eg/e.g./, but I feel the example is unnecessary
>  
> > + default resource group name, should be a NULL pointer */
>  
> Just "NULL for the default host group" is more than enough.
>  

Make sense.
  
>  
> > + size_t n_tasks; /* number of task assigned to the resource group */
>  
>  
> s/task /tasks/
>  
> > + char **tasks; /* task list which contains task id eg: 77454 */
> > +
> >  
>  
>  
> Why is this list of strings? Hopefully I'll see that after I burrow
> down a few more lines.
>  
I tried in my old design, we need to write tasks in to sysfile one task one 
time, maybe I am wrong, I am okay to modify it as a string with ‘\n'
as separator, not big deal.  
>  
> > + size_t n_schematas; /* number of schemata the resource group contains,
> > + eg: 2 */
> >  
>  
>  
> again, no need for
>  
Sure  
>  
> > + virResctrlSchemataPtr *schematas; /* scheamta list */
>  
>  
> 'schemata' is plural, there is no such thing as 'schematas'.
>  
>  

Ops, good to know,( not native speaker :( )  
>   
>  

> > +};
> > +
> > +/* All resource control groups on this host, including default resource 
> > group */
> > +typedef struct _virResctrlDomain virResctrlDomain;
> > +typedef virResctrlDomain *virResctrlDomainPtr;
> >  
>  
>  
> If they are on the host, why is "Domain" in the name?
>  
Prefer Host then Domain? okay, will change.  
>  
> > +struct _virResctrlDomain {
> > + size_t n_groups; /* number of resource control group */
> > + virResctrlGroupPtr *groups; /* list of resource control group */
> > +};
> > +
> > +void
> > +virResctrlF

Re: [libvirt] [PATCH RFC V2] RFC: Reimplement cache allocation for a VM

2017-04-23 Thread Eli Qiao

Ping

Please let me know which is the direction.

Thx.
  


On Friday, 14 April 2017 at 6:01 PM, Eli Qiao wrote:

> This is a RFC patch for the reimplement of `support cache tune(CAT) in
> libvirt`[1].
> 
> This patch defines some structs to represent data struct in linux
> resctrl fs which will be used later to do cache allocation.
> 
> The patch expose a private interface `virResctrlFreeSchemata`, which
> will be used to query the cache allocation on the host.
> 
> Also added unit test cases to test this interface can works well.
> 
> There are already patch sets[2] to address it, and functional
> works, but people doesn't like it cause it has global variable, and
> missing unit test case for new added capabilites, etc.
> 
> Martin has proposed a test infra to do vircaps2xmltest, and I extened it
> on top of it to extend resctrl control[3], this is kinds of new desiged
> apart from [2], so I propose this RFC patch to do some rework on it.
> 
> [1] https://www.redhat.com/archives/libvir-list/2017-January/msg00683.html
> [2] https://www.redhat.com/archives/libvir-list/2017-March/msg00181.html
> [3] https://www.redhat.com/archives/libvir-list/2017-April/msg00516.html
> ---
> include/libvirt/virterror.h | 1 +
> src/Makefile.am (http://Makefile.am) | 1 +
> src/libvirt_private.syms | 6 +
> src/util/virerror.c | 1 +
> src/util/virresctrl.c | 423 ++
> src/util/virresctrl.h | 86 ++
> tests/Makefile.am (http://Makefile.am) | 7 +-
> tests/virresctrldata/L3-free.schemata | 1 +
> tests/virresctrldata/L3CODE-free.schemata | 1 +
> tests/virresctrldata/L3DATA-free.schemata | 1 +
> tests/virresctrltest.c | 117 +
> 11 files changed, 644 insertions(+), 1 deletion(-)
> create mode 100644 src/util/virresctrl.c
> create mode 100644 src/util/virresctrl.h
> create mode 100644 tests/virresctrldata/L3-free.schemata
> create mode 100644 tests/virresctrldata/L3CODE-free.schemata
> create mode 100644 tests/virresctrldata/L3DATA-free.schemata
> create mode 100644 tests/virresctrltest.c
> 
> diff --git a/include/libvirt/virterror.h b/include/libvirt/virterror.h
> index 2efee8f..4bc0c74 100644
> --- a/include/libvirt/virterror.h
> +++ b/include/libvirt/virterror.h
> @@ -132,6 +132,7 @@ typedef enum {
> 
> VIR_FROM_PERF = 65, /* Error from perf */
> VIR_FROM_LIBSSH = 66, /* Error from libssh connection transport */
> + VIR_FROM_RESCTRL = 67, /* Error from resctrl */
> 
> # ifdef VIR_ENUM_SENTINELS
> VIR_ERR_DOMAIN_LAST
> diff --git a/src/Makefile.am (http://Makefile.am) b/src/Makefile.am 
> (http://Makefile.am)
> index 60eba37..0ae2af5 100644
> --- a/src/Makefile.am (http://Makefile.am)
> +++ b/src/Makefile.am (http://Makefile.am)
> @@ -165,6 +165,7 @@ UTIL_SOURCES = \
> util/virprocess.c util/virprocess.h \
> util/virqemu.c util/virqemu.h \
> util/virrandom.h util/virrandom.c \
> + util/virresctrl.h util/virresctrl.c \
> util/virrotatingfile.h util/virrotatingfile.c \
> util/virscsi.c util/virscsi.h \
> util/virscsihost.c util/virscsihost.h \
> diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms
> index 9d7760d..b7225fe 100644
> --- a/src/libvirt_private.syms
> +++ b/src/libvirt_private.syms
> @@ -2396,6 +2396,12 @@ virRandomGenerateWWN;
> virRandomInt;
> 
> 
> +# util/virresctrl.h
> +virResctrlFreeSchemata;
> +virResctrlGetFreeCache;
> +virResctrlTypeToString;
> +
> +
> # util/virrotatingfile.h
> virRotatingFileReaderConsume;
> virRotatingFileReaderFree;
> diff --git a/src/util/virerror.c b/src/util/virerror.c
> index ef17fb5..02fabcc 100644
> --- a/src/util/virerror.c
> +++ b/src/util/virerror.c
> @@ -139,6 +139,7 @@ VIR_ENUM_IMPL(virErrorDomain, VIR_ERR_DOMAIN_LAST,
> 
> "Perf", /* 65 */
> "Libssh transport layer",
> + "Resource Control",
> )
> 
> 
> diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c
> new file mode 100644
> index 000..778c2ec
> --- /dev/null
> +++ b/src/util/virresctrl.c
> @@ -0,0 +1,423 @@
> +/*
> + * virresctrl.c: methods for managing resource control
> + *
> + * Copyright (C) 2017 Intel, Inc.
> + *
> + * This library is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * This library is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a

[libvirt] [PATCH RFC V2] RFC: Reimplement cache allocation for a VM

2017-04-14 Thread Eli Qiao

This is a RFC patch for the reimplement of `support cache tune(CAT) in
libvirt`[1].

This patch defines some structs to represent data struct in linux
resctrl fs which will be used later to do cache allocation.

The patch expose a private interface `virResctrlFreeSchemata`, which
will be used to query the cache allocation on the host.

Also added unit test cases to test this interface can works well.

There are already patch sets[2] to address it, and functional
works, but people doesn't like it cause it has global variable, and
missing unit test case for new added capabilites, etc.

Martin has proposed a test infra to do vircaps2xmltest, and I extened it
on top of it to extend resctrl control[3], this is kinds of new desiged
apart from [2], so I propose this RFC patch to do some rework on it.

[1] https://www.redhat.com/archives/libvir-list/2017-January/msg00683.html
[2] https://www.redhat.com/archives/libvir-list/2017-March/msg00181.html
[3] https://www.redhat.com/archives/libvir-list/2017-April/msg00516.html
---
 include/libvirt/virterror.h   |   1 +
 src/Makefile.am   |   1 +
 src/libvirt_private.syms  |   6 +
 src/util/virerror.c   |   1 +
 src/util/virresctrl.c | 423 ++
 src/util/virresctrl.h |  86 ++
 tests/Makefile.am |   7 +-
 tests/virresctrldata/L3-free.schemata |   1 +
 tests/virresctrldata/L3CODE-free.schemata |   1 +
 tests/virresctrldata/L3DATA-free.schemata |   1 +
 tests/virresctrltest.c| 117 +
 11 files changed, 644 insertions(+), 1 deletion(-)
 create mode 100644 src/util/virresctrl.c
 create mode 100644 src/util/virresctrl.h
 create mode 100644 tests/virresctrldata/L3-free.schemata
 create mode 100644 tests/virresctrldata/L3CODE-free.schemata
 create mode 100644 tests/virresctrldata/L3DATA-free.schemata
 create mode 100644 tests/virresctrltest.c

diff --git a/include/libvirt/virterror.h b/include/libvirt/virterror.h
index 2efee8f..4bc0c74 100644
--- a/include/libvirt/virterror.h
+++ b/include/libvirt/virterror.h
@@ -132,6 +132,7 @@ typedef enum {

 VIR_FROM_PERF = 65, /* Error from perf */
 VIR_FROM_LIBSSH = 66,   /* Error from libssh connection transport */
+VIR_FROM_RESCTRL = 67,  /* Error from resctrl */

 # ifdef VIR_ENUM_SENTINELS
 VIR_ERR_DOMAIN_LAST
diff --git a/src/Makefile.am b/src/Makefile.am
index 60eba37..0ae2af5 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -165,6 +165,7 @@ UTIL_SOURCES =  
\
util/virprocess.c util/virprocess.h \
util/virqemu.c util/virqemu.h   \
util/virrandom.h util/virrandom.c   \
+   util/virresctrl.h util/virresctrl.c \
util/virrotatingfile.h util/virrotatingfile.c   \
util/virscsi.c util/virscsi.h   \
util/virscsihost.c util/virscsihost.h   \
diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms
index 9d7760d..b7225fe 100644
--- a/src/libvirt_private.syms
+++ b/src/libvirt_private.syms
@@ -2396,6 +2396,12 @@ virRandomGenerateWWN;
 virRandomInt;


+# util/virresctrl.h
+virResctrlFreeSchemata;
+virResctrlGetFreeCache;
+virResctrlTypeToString;
+
+
 # util/virrotatingfile.h
 virRotatingFileReaderConsume;
 virRotatingFileReaderFree;
diff --git a/src/util/virerror.c b/src/util/virerror.c
index ef17fb5..02fabcc 100644
--- a/src/util/virerror.c
+++ b/src/util/virerror.c
@@ -139,6 +139,7 @@ VIR_ENUM_IMPL(virErrorDomain, VIR_ERR_DOMAIN_LAST,

   "Perf", /* 65 */
   "Libssh transport layer",
+  "Resource Control",
 )


diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c
new file mode 100644
index 000..778c2ec
--- /dev/null
+++ b/src/util/virresctrl.c
@@ -0,0 +1,423 @@
+/*
+ * virresctrl.c: methods for managing resource control
+ *
+ * Copyright (C) 2017 Intel, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library.  If not, see
+ * <http://www.gnu.org/licenses/>.
+ *
+ * Authors:
+ *  Eli Qiao <liyong.q...@intel.com>
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "virresctr

[libvirt] [RFC] RFC: Reimplement cache allocation for a VM

2017-04-11 Thread Eli Qiao

This is a RFC patch for the reimplement of `support cache tune(CAT) in
libvirt`[1].

There are already patch sets[2] to address it, and functional
works, but people doesn't like it cause it has global variable, and
missing unit test case for new added capabilites, etc.

Martin has proposed a test infra to do vircaps2xmltest, and I extened it
on top of it to extend resctrl control[3], this is kinds of new desiged
apart from [2], so I propose this RFC patch to do some rework on it.

[] https://www.redhat.com/archives/libvir-list/2017-January/msg00683.html
[] https://www.redhat.com/archives/libvir-list/2017-March/msg00181.html
[] https://www.redhat.com/archives/libvir-list/2017-April/msg00516.html
---
 src/util/virresctrl.c | 81 
 src/util/virresctrl.h | 86 +++
 2 files changed, 167 insertions(+)
 create mode 100644 src/util/virresctrl.c
 create mode 100644 src/util/virresctrl.h

diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c
new file mode 100644
index 000..f555eb8
--- /dev/null
+++ b/src/util/virresctrl.c
@@ -0,0 +1,81 @@
+/*
+ * virresctrl.c: methods for managing resource control
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library.  If not, see
+ * <http://www.gnu.org/licenses/>.
+ *
+ * Authors:
+ *  Eli Qiao <liyong.q...@intel.com>
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "virresctrl.h"
+#include "viralloc.h"
+#include "virerror.h"
+#include "virfile.h"
+#include "virhostcpu.h"
+#include "virlog.h"
+#include "virstring.h"
+#include "virarch.h"
+
+VIR_LOG_INIT("util.resctrl");
+
+#define VIR_FROM_THIS VIR_FROM_RESCTRL
+
+VIR_ENUM_IMPL(virResctrl, VIR_RESCTRL_TYPE_LAST,
+  "L3",
+  "L3CODE",
+  "L3DATA",
+  "L2")
+
+/**
+ * a virResctrlDomain represents a resource control group, it's a directory
+ * under /sys/fs/resctrl.
+ * eg: /sys/fs/resctrl/CG1
+ * |-- cpus
+ * |-- schemata
+ * `-- tasks
+ * # cat schemata
+ * L3DATA:0=f;1=f
+ * L3CODE:0=f;1=f
+ *
+ * Besides, it can also represent the default resource control group of the
+ * host.
+ */
+
+typedef struct _virResctrlGroup virResctrlGroup;
+typedef virResctrlGroup *virResctrlGroupPtr;
+struct _virResctrlGroup {
+char *name; /* resource group name, eg: CG1. If it represent host's
+   default resource group name, should be a NULL pointer */
+size_t n_tasks; /* number of task assigned to the resource group */
+char **tasks; /* task list which contains task id eg: 77454 */
+
+size_t n_schematas; /* number of schemata the resource group contains,
+ eg: 2 */
+virResctrlSchemataPtr *schematas; /* scheamta list */
+};
+
+/* All resource control groups on this host, including default resource group 
*/
+typedef struct _virResCtrlDomain virResCtrlDomain;
+typedef virResCtrlDomain *virResCtrlDomainPtr;
+struct _virResCtrlDomain {
+size_t n_groups; /* number of resource control group */
+virResctrlGroupPtr groups; /* list of resource control group */
+};
diff --git a/src/util/virresctrl.h b/src/util/virresctrl.h
new file mode 100644
index 000..da89542
--- /dev/null
+++ b/src/util/virresctrl.h
@@ -0,0 +1,86 @@
+/*
+ * virresctrl.h: header for managing resctrl control
+ *
+ * Copyright (C) 2016 Intel, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library.  If not, see
+ * <http://www.gnu.org/licenses/>.
+ *
+ * Authors:
+ * Eli Qiao <liyong.q...@intel.com>
+ */
+
+#ifndef __VIR_RESCTRL_H__
+# define __VIR_RESCTRL_H__
+
+typedef enum {
+VIR_RESCTRL_TYPE_L3,
+VIR_

[libvirt] [PATCH V5] Expose resource control capabilites on cache bank

2017-04-11 Thread Eli Qiao

This patch is based on Martin's cache branch.

* This patch amends the cache bank capability as follow:


  

  
  

  


For CDP enabled on x86 arch, we will have:

  


  
...

* Added a new testdata directory `linux-resctrl-cdp` to test CDP enabled
case.

* Along with vircaps2xmltest case updated.

P.S. here the scope is from /sys/fs/resctrl/info/L3{"" "code" "data"}, I
keep it capital upper as I need to use a macro to convert from enum to
string easily.

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 docs/schemas/capability.rng|  20 +++
 src/conf/capabilities.c| 139 -
 src/conf/capabilities.h|  20 +++
 .../vircaps2xmldata/linux-resctrl-cdp/resctrl/cpus |   1 +
 .../linux-resctrl-cdp/resctrl/info/L3CODE/cbm_mask |   1 +
 .../resctrl/info/L3CODE/min_cbm_bits   |   1 +
 .../resctrl/info/L3CODE/num_closids|   1 +
 .../linux-resctrl-cdp/resctrl/info/L3DATA/cbm_mask |   1 +
 .../resctrl/info/L3DATA/min_cbm_bits   |   1 +
 .../resctrl/info/L3DATA/num_closids|   1 +
 .../linux-resctrl-cdp/resctrl/manualres/cpus   |   1 +
 .../linux-resctrl-cdp/resctrl/manualres/schemata   |   2 +
 .../linux-resctrl-cdp/resctrl/manualres/tasks  |   0
 .../linux-resctrl-cdp/resctrl/schemata |   2 +
 .../linux-resctrl-cdp/resctrl/tasks|   0
 tests/vircaps2xmldata/linux-resctrl-cdp/system |   1 +
 .../vircaps2xmldata/vircaps-x86_64-resctrl-cdp.xml |  55 
 tests/vircaps2xmldata/vircaps-x86_64-resctrl.xml   |   8 +-
 tests/vircaps2xmltest.c|   8 ++
 19 files changed, 259 insertions(+), 4 deletions(-)
 create mode 100644 tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/cpus
 create mode 100755 
tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3CODE/cbm_mask
 create mode 100755 
tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3CODE/min_cbm_bits
 create mode 100755 
tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3CODE/num_closids
 create mode 100755 
tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3DATA/cbm_mask
 create mode 100755 
tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3DATA/min_cbm_bits
 create mode 100755 
tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3DATA/num_closids
 create mode 100644 
tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/manualres/cpus
 create mode 100644 
tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/manualres/schemata
 create mode 100644 
tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/manualres/tasks
 create mode 100644 tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/schemata
 create mode 100644 tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/tasks
 create mode 12 tests/vircaps2xmldata/linux-resctrl-cdp/system
 create mode 100644 tests/vircaps2xmldata/vircaps-x86_64-resctrl-cdp.xml

diff --git a/docs/schemas/capability.rng b/docs/schemas/capability.rng
index 2080953..f371a1c 100644
--- a/docs/schemas/capability.rng
+++ b/docs/schemas/capability.rng
@@ -277,6 +277,26 @@
   
 
   
+  
+
+  
+
+  
+  
+
+  
+  
+
+  both
+  code
+  data
+
+  
+  
+
+  
+
+  
 
   
 
diff --git a/src/conf/capabilities.c b/src/conf/capabilities.c
index 416dd1a..c7a2e73 100644
--- a/src/conf/capabilities.c
+++ b/src/conf/capabilities.c
@@ -52,6 +52,7 @@
 #define VIR_FROM_THIS VIR_FROM_CAPABILITIES
 
 #define SYSFS_SYSTEM_PATH "/sys/devices/system/"
+#define SYSFS_RESCTRL_PATH "/sys/fs/resctrl/"
 
 VIR_LOG_INIT("conf.capabilities")
 
@@ -873,6 +874,9 @@ virCapabilitiesFormatCaches(virBufferPtr buf,
 virCapsHostCacheBankPtr *caches)
 {
 size_t i = 0;
+size_t j = 0;
+int indent = virBufferGetIndent(buf, false);
+virBuffer controlBuf = VIR_BUFFER_INITIALIZER;
 
 if (!ncaches)
 return 0;
@@ -894,13 +898,35 @@ virCapabilitiesFormatCaches(virBufferPtr buf,
  */
 virBufferAsprintf(buf,
   "\n",
+  "size='%llu' unit='%s' cpus='%s'",
   bank->id, bank->level,
   virCacheTypeToString(bank->type),
   bank->size >> (kilos * 10),
   kilos ? "KiB" : "B",
   cpus_str);
 
+virBufferAdjustIndent(, indent + 4);
+for (j = 0; j < bank->ncontrols; j++) {
+bool min_kilos = !(bank->controls[j]->min % 1024);
+virBufferAsprintf(,
+

Re: [libvirt] [PATCH V4] Expose resource control capabilites on cache bank

2017-04-11 Thread Eli Qiao



On Tuesday, 11 April 2017 at 4:10 PM, Martin Kletzander wrote:

> This should just be:
> 
> + if (virAsprintf(, "%s/vircaps2xmldata/linux-%s/resctrl",
> + abs_srcdir, data->filename) < 0)
> 
yep, stupid me. I get it now. --
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH V4] Expose resource control capabilites on cache bank

2017-04-11 Thread Eli Qiao



On Tuesday, 11 April 2017 at 3:48 PM, Peter Krempa wrote:

> On Tue, Apr 11, 2017 at 13:44:26 +0800, Eli Qiao wrote:
> > This patch is based on Martin's cache branch.
> >  
> > * This patch amends the cache bank capability as follow:
> >  
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> >  
> > For CDP enabled on x86 arch, we will have:
> > 
> > 
> > 
> > 
> > 
> > ...
> >  
> > * Added a new testdata directory `linux-resctrl-cdp` to test CDP enabled
> > case.
> >  
> > * Along with vircaps2xmltest case updated.
> >  
> > P.S. here the scope is from /sys/fs/resctrl/info/L3{"" "CODE" "DATA"}, I
> > keep it capital upper as I need to use a macro to convert from enum to
> > string easily.
> >  
>  
>  
> We dont need to do that. The attributes should be lower case. The code
> can convert it to anything it needs.
>  
>  

what I did is that expose what the host’s sys/fs/resctrl/info looks like, if 
people
think we should use lower case, I am okay to change.  
>  
> >  
> > Signed-off-by: Eli Qiao <liyong.q...@intel.com 
> > (mailto:liyong.q...@intel.com)>
> > ---
> >  
>  
>  
> [...]
>  
> > diff --git a/docs/schemas/capability.rng b/docs/schemas/capability.rng
> > index 2080953..bfed4f8 100644
> > --- a/docs/schemas/capability.rng
> > +++ b/docs/schemas/capability.rng
> > @@ -277,6 +277,26 @@
> > 
> > 
> > 
> > + 
> > + 
> > + 
> > + 
> > + 
> > + 
> > + 
> > + 
> > + 
> > + 
> > + BOTH
> > + CODE
> > + DATA
> >  
>  
>  
> Why are the values all caps? We prefer lowercase attributes in the XML.
>  
Answer in previous reply.  
>  
> > + 
> > + 
> > + 
> > + 
> > + 
> > + 
> > + 
> > 
> > 
> > 
> > diff --git a/src/conf/capabilities.c b/src/conf/capabilities.c
> > index 416dd1a..c6641d1 100644
> > --- a/src/conf/capabilities.c
> > +++ b/src/conf/capabilities.c
> > @@ -52,6 +52,7 @@
> > #define VIR_FROM_THIS VIR_FROM_CAPABILITIES
> >  
> > #define SYSFS_SYSTEM_PATH "/sys/devices/system/"
> > +#define SYSFS_RESCTRL_PATH "/sys/fs/resctrl/"
> >  
> > VIR_LOG_INIT("conf.capabilities")
> >  
> > @@ -873,6 +874,9 @@ virCapabilitiesFormatCaches(virBufferPtr buf,
> > virCapsHostCacheBankPtr *caches)
> > {
> > size_t i = 0;
> > + size_t j = 0;
> > + int indent = virBufferGetIndent(buf, false);
> > + virBuffer controlBuf = VIR_BUFFER_INITIALIZER;
> >  
> > if (!ncaches)
> > return 0;
> > @@ -894,13 +898,38 @@ virCapabilitiesFormatCaches(virBufferPtr buf,
> > */
> > virBufferAsprintf(buf,
> > " > - "size='%llu' unit='%s' cpus='%s'/>\n",
> > + "size='%llu' unit='%s' cpus='%s'",
> > bank->id, bank->level,
> > virCacheTypeToString(bank->type),
> > bank->size >> (kilos * 10),
> > kilos ? "KiB" : "B",
> > cpus_str);
> >  
> > + /* Format control */
> > + virBufferAdjustIndent(, indent + 4);
> >  
>  
>  
> This looks wrong. You should increase/decrease the indent by some
> number. The old value should not be used.
>  
I have test case covered, please check 
tests/vircaps2xmldata/vircaps-x86_64-resctrl-cdp.xml  
>  
> > + for (j = 0; j < bank->ncontrols; j++) {
> > + bool min_kilos = !(bank->controls[j]->min % 1024);
> > + virBufferAsprintf(,
> > + " > + "scope='%s' max_allocation='%u'/>\n",
> > + bank->controls[j]->min >> (min_kilos * 10),
> > + min_kilos ? "KiB" : "B",
> > + virResctrlCacheTypeToString(bank->controls[j]->scope),
> > + bank->controls[j]->max_allocation);
> > + }
> > +
> > + /* Put it all together */
> >  
>  
>  
> You don't need to point out obvious things.

Just copy from other place, I am okay to remove it.  
> > + if (virBufferUse()) {
>  
>  
> This logic looks weird and opaque. Can you decide by any other means
> than the filling of the buffer?
>  
>  

It’s same meaning with  bank->ncontrols > 0
this testfile will help you to easy understand what’s doing here.
tests/vircaps2xmldata/vircaps-x86_64-resctrl-cdp.xml  

I am okay to change if you feel it’s hard to get understand.
>  
> > + virBufferAddLit(buf, ">\n");
> > + virBufferAddBuffer(buf, );
> > + virBufferAddLit(buf, "\n

Re: [libvirt] [PATCH V3] Expose resource control capabilites on cache bank

2017-04-10 Thread Eli Qiao


> >  
> > 
> > 
> >  
> >  
> > @Daniel,  
> >  
> > the enum values are same with `type`
> >  
> > unified: 0
> > instruction: 1
> > data: 2
> >  
> > but scope should be both(0)/code(1)/data(2), so the attribute name will be
>  
> 'both' and 'unified' mean the same thing.
>  
> 'instruction' and 'code' mean the same thing to.
>  
> So we should use the same terminology for both attributes. IOW, I
> suggest we use 'both', 'code', and 'data' everywhere.
>  

Hi Daniel/Martin

I update V4 patch of “extend resctrl capabilites” [1]

I would like to request for comment, this is something different for your 
suggestion
I added the reason in P.S section of the comment message.

Looking for your comments.


[1] https://www.redhat.com/archives/libvir-list/2017-April/msg00516.html

Thanks.

Eli.
  
>  
>  
> Regards,
> Daniel
> --  
> |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
> |: http://libvirt.org -o- http://virt-manager.org :|
> |: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :|
>  
>  


--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATCH V4] Expose resource control capabilites on cache bank

2017-04-10 Thread Eli Qiao

This patch is based on Martin's cache branch.

* This patch amends the cache bank capability as follow:


  

  
  

  


For CDP enabled on x86 arch, we will have:

  


  
...

* Added a new testdata directory `linux-resctrl-cdp` to test CDP enabled
case.

* Along with vircaps2xmltest case updated.

P.S. here the scope is from /sys/fs/resctrl/info/L3{"" "CODE" "DATA"}, I
keep it capital upper as I need to use a macro to convert from enum to
string easily.

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 docs/schemas/capability.rng|  20 +++
 src/conf/capabilities.c| 135 -
 src/conf/capabilities.h|  26 +++-
 .../vircaps2xmldata/linux-resctrl-cdp/resctrl/cpus |   1 +
 .../linux-resctrl-cdp/resctrl/info/L3CODE/cbm_mask |   1 +
 .../resctrl/info/L3CODE/min_cbm_bits   |   1 +
 .../resctrl/info/L3CODE/num_closids|   1 +
 .../linux-resctrl-cdp/resctrl/info/L3DATA/cbm_mask |   1 +
 .../resctrl/info/L3DATA/min_cbm_bits   |   1 +
 .../resctrl/info/L3DATA/num_closids|   1 +
 .../linux-resctrl-cdp/resctrl/manualres/cpus   |   1 +
 .../linux-resctrl-cdp/resctrl/manualres/schemata   |   2 +
 .../linux-resctrl-cdp/resctrl/manualres/tasks  |   0
 .../linux-resctrl-cdp/resctrl/schemata |   2 +
 .../linux-resctrl-cdp/resctrl/tasks|   0
 tests/vircaps2xmldata/linux-resctrl-cdp/system |   1 +
 .../vircaps2xmldata/vircaps-x86_64-resctrl-cdp.xml |  55 +
 tests/vircaps2xmldata/vircaps-x86_64-resctrl.xml   |   8 +-
 tests/vircaps2xmltest.c|  10 ++
 19 files changed, 261 insertions(+), 6 deletions(-)
 create mode 100644 tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/cpus
 create mode 100755 
tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3CODE/cbm_mask
 create mode 100755 
tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3CODE/min_cbm_bits
 create mode 100755 
tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3CODE/num_closids
 create mode 100755 
tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3DATA/cbm_mask
 create mode 100755 
tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3DATA/min_cbm_bits
 create mode 100755 
tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3DATA/num_closids
 create mode 100644 
tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/manualres/cpus
 create mode 100644 
tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/manualres/schemata
 create mode 100644 
tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/manualres/tasks
 create mode 100644 tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/schemata
 create mode 100644 tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/tasks
 create mode 12 tests/vircaps2xmldata/linux-resctrl-cdp/system
 create mode 100644 tests/vircaps2xmldata/vircaps-x86_64-resctrl-cdp.xml

diff --git a/docs/schemas/capability.rng b/docs/schemas/capability.rng
index 2080953..bfed4f8 100644
--- a/docs/schemas/capability.rng
+++ b/docs/schemas/capability.rng
@@ -277,6 +277,26 @@
   
 
   
+  
+
+  
+
+  
+  
+
+  
+  
+
+  BOTH
+  CODE
+  DATA
+
+  
+  
+
+  
+
+  
 
   
 
diff --git a/src/conf/capabilities.c b/src/conf/capabilities.c
index 416dd1a..c6641d1 100644
--- a/src/conf/capabilities.c
+++ b/src/conf/capabilities.c
@@ -52,6 +52,7 @@
 #define VIR_FROM_THIS VIR_FROM_CAPABILITIES
 
 #define SYSFS_SYSTEM_PATH "/sys/devices/system/"
+#define SYSFS_RESCTRL_PATH "/sys/fs/resctrl/"
 
 VIR_LOG_INIT("conf.capabilities")
 
@@ -873,6 +874,9 @@ virCapabilitiesFormatCaches(virBufferPtr buf,
 virCapsHostCacheBankPtr *caches)
 {
 size_t i = 0;
+size_t j = 0;
+int indent = virBufferGetIndent(buf, false);
+virBuffer controlBuf = VIR_BUFFER_INITIALIZER;
 
 if (!ncaches)
 return 0;
@@ -894,13 +898,38 @@ virCapabilitiesFormatCaches(virBufferPtr buf,
  */
 virBufferAsprintf(buf,
   "\n",
+  "size='%llu' unit='%s' cpus='%s'",
   bank->id, bank->level,
   virCacheTypeToString(bank->type),
   bank->size >> (kilos * 10),
   kilos ? "KiB" : "B",
   cpus_str);
 
+/* Format control */
+virBufferAdjustIndent(, indent + 4);
+for (j = 0; j < bank->ncontrols; j++) {
+bool min_kilos = !(bank->controls[j]->min % 1024);
+

Re: [libvirt] [PATCH V3] Expose resource control capabilites on cache bank

2017-04-07 Thread Eli Qiao



On Friday, 7 April 2017 at 7:06 PM, Daniel P. Berrange wrote:

> On Fri, Apr 07, 2017 at 01:04:57PM +0200, Martin Kletzander wrote:
> > On Fri, Apr 07, 2017 at 05:47:54PM +0800, Eli Qiao wrote:
> > > >  
> > > > The name doesn't really matter that much, 'scope' makes a bit more
> > > > sense, 'type' is consistent with the cache bank specification, I'm fine
> > > > with any. The big question here was if it is possible to have:
> > > >  
> > > > 
> > > > 
> > > > 
> > > > 
> > > >  
> > > > And from what you say, the simple answer is "yes". So we need to have
> > > > the attribute there in the control element as well.
> > > >  
> > >  
> > >  
> > >  
> > > Dan/Martin
> > >  
> > > Could you please advice which should be changed ? LoL
> > >  
> > > This is the output if I enabled CDP
> > >  
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > >  
> > >  
> > > 1. change nallocations to allocations/max_allocation?
> >  
> > Dan said he's fine with both, I'd probably go for max_allocations
> >  
> > > 2. change type to scope ?
> >  
> > I don't care, pros for both in the previous mail.
>  
> Both attributes take the same enum values, so it is best to be consistent
> with the attribute name.
>  
Thanks very much for Daniel & Martin

Forgive me to ping you again, just make sure I am in correct place :

If I read your comments correctly, we can go with:  


  


  
  



  



@Daniel,  

the enum values are same with `type`

unified: 0
instruction: 1
data: 2

but scope should be both(0)/code(1)/data(2), so the attribute name will be
`scope`. As we need create another Type{to from}String marco for easy read from 
/sys/fs/resctr/LxDATA/CODE .


Regards,

Eli

> Regards,
> Daniel
> --  
> |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
> |: http://libvirt.org -o- http://virt-manager.org :|
> |: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :|
>  
>  


--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH V3] Expose resource control capabilites on cache bank

2017-04-07 Thread Eli Qiao

> 
> The name doesn't really matter that much, 'scope' makes a bit more
> sense, 'type' is consistent with the cache bank specification, I'm fine
> with any. The big question here was if it is possible to have:
> 
> 
> 
> 
> 
> 
> And from what you say, the simple answer is "yes". So we need to have
> the attribute there in the control element as well.
> 
> 


Dan/Martin

Could you please advice which should be changed ? LoL

This is the output if I enabled CDP


  


  
  


  



1. change nallocations to allocations/max_allocation?
2. change type to scope ? 
3. change `instruction` to `code` (with CDP enabled, it called DATA/CODE which 
is somewhat
different from /sys/fs/system/cpu/cpu*/cache/type, and I am now reuse 
virCacheType defined
by Martin, should I define another Type)?


> 
> P.S.: It would be clearly visible if you added the test case ;) 

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH V3] Expose resource control capabilites on cache bank

2017-04-06 Thread Eli Qiao



On Thursday, 6 April 2017 at 9:04 PM, Martin Kletzander wrote:

> On Thu, Apr 06, 2017 at 01:25:35PM +0100, Daniel P. Berrange wrote:
> > On Thu, Apr 06, 2017 at 08:20:56PM +0800, Eli Qiao wrote:
> > > This patch is based on Martin's cache branch.
> > >  
> > > This patch amends the cache bank capability as follow:
> > >  
> > > 
> > > 
> > > 
> > > 
> > >  
> >  
> >  
> > Why do we need to report 'type' on both bank & control elements. Are they
> > really expected to have different values ?
> >  
>  
>  
>  
>  

There’s a discussion from 
https://www.redhat.com/archives/libvir-list/2017-March/msg01689.html

I think I made a mistake here, it should be ’scope’ instead of ’type’ here.

No matter what’s it should be, at least we need to identify if the host enabled 
CDP for L3 cache or not.

If enabled CDP on host, the resctrl looks like this:

root@s2600wt:/sys/fs/resctrl# cat schemata
L3DATA:0=f;1=f
L3CODE:0=f;1=f
root@s2600wt:/sys/fs/resctrl# cat info/L3
L3CODE/ L3DATA/
root@s2600wt:/sys/fs/resctrl# cat info/L3DATA/num_closids
8
root@s2600wt:/sys/fs/resctrl# cat info/L3CODE/num_closids
8


as you can see that num_closids are divided into 2 part (compared to 16 without 
CDP),
 but other info are still same

root@s2600wt:/sys/fs/resctrl# cat info/L3CODE/cbm_mask
f
root@s2600wt:/sys/fs/resctrl# cat info/L3CODE/min_cbm_bits
1
root@s2600wt:/sys/fs/resctrl# cat info/L3DATA/cbm_mask
f
root@s2600wt:/sys/fs/resctrl# cat info/L3DATA/min_cbm_bits
1


--
below output are from disable CDP:

root@s2600wt:/sys/fs/resctrl# cat schemata
L3:0=f;1=f
root@s2600wt:/sys/fs/resctrl# cat info/L3/
cbm_mask  min_cbm_bits  num_closids
root@s2600wt:/sys/fs/resctrl# cat info/L3/cbm_mask
f
root@s2600wt:/sys/fs/resctrl# cat info/L3/min_cbm_bits
1
root@s2600wt:/sys/fs/resctrl# cat info/L3/num_closids
16

>   
>  
>  

>  
> That's one of my questions I had way back in some of the previous
> discussions. Did not get the answer. I suspect there is a reason to
> whether CDP is enabled or not and it is not just the type of the cache
> bank itself. If it is, then CDP makes no sense for us at all, actually
> for anyone who has access to cache information and that would mean bad
> design of that thing. And that's not something I would expect from this
> functionality.
>  
> >  
> > Regards,
> > Daniel
> > --
> > |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
> > |: http://libvirt.org -o- http://virt-manager.org :|
> > |: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :|
> >  
>  
>  
> --
> libvir-list mailing list
> libvir-list@redhat.com (mailto:libvir-list@redhat.com)
> https://www.redhat.com/mailman/listinfo/libvir-list
>  
>  


--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH V3] Expose resource control capabilites on cache bank

2017-04-06 Thread Eli Qiao



On Thursday, 6 April 2017 at 8:46 PM, Martin Kletzander wrote:

> On Thu, Apr 06, 2017 at 08:20:56PM +0800, Eli Qiao wrote:
> > This patch is based on Martin's cache branch.
> >  
> > This patch amends the cache bank capability as follow:
>  
> It helps a lot if you wait for a conclusion on a patch before sending
> another version as soon as you can change one line.
>  
Okay, I will keep my patience next time, this time is just because I was 
working over time yesterday.
>  
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> >  
>  
>  
> I know Dan proposed "nallocations", but it sounds like one word. I
> would rather use "allocations" or "max_allocs" or something
> understandable. The reason for it? We have no documentation for our
> capabilities XML. And nobody is trying to create one as far as I know.
> So at least the naming should be more intuitive.
>  
yep, I will wait for the final decision.  
>  
> > Along with vircaps2xmltest case updated.
>  
>  
> I'm sensing you haven't ran the tests. Am I right?
>  
Why ?

taget@s2600wt:~/libvirt$ ./tests/vircaps2xmltest
TEST: vircaps2xmltest
   4   OK  
>  
> > Signed-off-by: Eli Qiao <liyong.q...@intel.com 
> > (mailto:liyong.q...@intel.com)>
> > ---
> > src/conf/capabilities.c | 120 ++-
> > src/conf/capabilities.h | 13 ++-
> > tests/vircaps2xmldata/vircaps-x86_64-resctrl.xml | 8 +-
> > tests/vircaps2xmltest.c | 13 +++
> > 4 files changed, 148 insertions(+), 6 deletions(-)
> >  
> > diff --git a/src/conf/capabilities.c b/src/conf/capabilities.c
> > index 416dd1a..3094e48 100644
> > --- a/src/conf/capabilities.c
> > +++ b/src/conf/capabilities.c
> > @@ -52,6 +52,7 @@
> > #define VIR_FROM_THIS VIR_FROM_CAPABILITIES
> >  
> > #define SYSFS_SYSTEM_PATH "/sys/devices/system/"
> > +#define SYSFS_RESCTRL_PATH "/sys/fs/resctrl/"
> >  
> > VIR_LOG_INIT("conf.capabilities")
> >  
> > @@ -873,6 +874,7 @@ virCapabilitiesFormatCaches(virBufferPtr buf,
> > virCapsHostCacheBankPtr *caches)
> > {
> > size_t i = 0;
> > + size_t j = 0;
> >  
> > if (!ncaches)
> > return 0;
> > @@ -894,13 +896,32 @@ virCapabilitiesFormatCaches(virBufferPtr buf,
> > */
> > virBufferAsprintf(buf,
> > " > - "size='%llu' unit='%s' cpus='%s'/>\n",
> > + "size='%llu' unit='%s' cpus='%s'",
> > bank->id, bank->level,
> > virCacheTypeToString(bank->type),
> > bank->size >> (kilos * 10),
> > kilos ? "KiB" : "B",
> > cpus_str);
> >  
> > + if (bank->ncontrol > 0) {
> > + virBufferAddLit(buf, ">\n");
> > + virBufferAdjustIndent(buf, 2);
> > + for (j = 0; j < bank->ncontrol; j++) {
> > + bool min_kilos = !(bank->controls[j]->min % 1024);
> > + virBufferAsprintf(buf,
> > + " > + "type='%s' nallocations='%u'/>\n",
> > + bank->controls[j]->min >> (min_kilos * 10),
> > + min_kilos ? "KiB" : "B",
> > + virCacheTypeToString(bank->controls[j]->type),
> > + bank->controls[j]->nallocations);
> > + }
> > + virBufferAdjustIndent(buf, -2);
> > + virBufferAddLit(buf, "\n");
> > + } else {
> > + virBufferAddLit(buf, "/>\n");
> > + }
> > +
> >  
>  
>  
> There's virBufferAddBuffer() for easier usage of this. Just grep for
> childrenBuf and you'll see the usage.
>  
Good to know that , thx.  
>  
> > VIR_FREE(cpus_str);
> > }
> >  
> > @@ -1513,13 +1534,97 @@ virCapsHostCacheBankEquals(virCapsHostCacheBankPtr 
> > a,
> > void
> > virCapsHostCacheBankFree(virCapsHostCacheBankPtr ptr)
> > {
> > + size_t i;
> > +
> > if (!ptr)
> > return;
> >  
> > virBitmapFree(ptr->cpus);
> > + for (i = 0; i < ptr->ncontrol; i++)
> > + VIR_FREE(ptr->controls[i]);
> > + VIR_FREE(ptr->controls);
> > VIR_FREE(ptr);
> > }
> >  
> > +/* test which kinds of cache control supported
> > + * -1: don't support
> > + * 0: cat
> > + * 1: cdp
> > + */
> > +static int
> > +virCapabilitiesGetCacheControlType(virCapsHostCacheBankPtr bank)
> > +{
> > + int ret = -1;
> > + char *path = NULL;
> > + if (virAsprintf(, SYSFS_RESCTRL_PATH "info/L%u", bank->level) < 0)
> > + return -1;
> > +
> > + if (virF

[libvirt] [PATCH V3] Expose resource control capabilites on cache bank

2017-04-06 Thread Eli Qiao

This patch is based on Martin's cache branch.

This patch amends the cache bank capability as follow:


  

  
  

  


Along with vircaps2xmltest case updated.

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 src/conf/capabilities.c  | 120 ++-
 src/conf/capabilities.h  |  13 ++-
 tests/vircaps2xmldata/vircaps-x86_64-resctrl.xml |   8 +-
 tests/vircaps2xmltest.c  |  13 +++
 4 files changed, 148 insertions(+), 6 deletions(-)

diff --git a/src/conf/capabilities.c b/src/conf/capabilities.c
index 416dd1a..3094e48 100644
--- a/src/conf/capabilities.c
+++ b/src/conf/capabilities.c
@@ -52,6 +52,7 @@
 #define VIR_FROM_THIS VIR_FROM_CAPABILITIES
 
 #define SYSFS_SYSTEM_PATH "/sys/devices/system/"
+#define SYSFS_RESCTRL_PATH "/sys/fs/resctrl/"
 
 VIR_LOG_INIT("conf.capabilities")
 
@@ -873,6 +874,7 @@ virCapabilitiesFormatCaches(virBufferPtr buf,
 virCapsHostCacheBankPtr *caches)
 {
 size_t i = 0;
+size_t j = 0;
 
 if (!ncaches)
 return 0;
@@ -894,13 +896,32 @@ virCapabilitiesFormatCaches(virBufferPtr buf,
  */
 virBufferAsprintf(buf,
   "\n",
+  "size='%llu' unit='%s' cpus='%s'",
   bank->id, bank->level,
   virCacheTypeToString(bank->type),
   bank->size >> (kilos * 10),
   kilos ? "KiB" : "B",
   cpus_str);
 
+if (bank->ncontrol > 0) {
+virBufferAddLit(buf, ">\n");
+virBufferAdjustIndent(buf, 2);
+for (j = 0; j < bank->ncontrol; j++) {
+bool min_kilos = !(bank->controls[j]->min % 1024);
+virBufferAsprintf(buf,
+  "\n",
+  bank->controls[j]->min >> (min_kilos * 10),
+  min_kilos ? "KiB" : "B",
+  
virCacheTypeToString(bank->controls[j]->type),
+  bank->controls[j]->nallocations);
+}
+virBufferAdjustIndent(buf, -2);
+virBufferAddLit(buf, "\n");
+} else {
+virBufferAddLit(buf, "/>\n");
+}
+
 VIR_FREE(cpus_str);
 }
 
@@ -1513,13 +1534,97 @@ virCapsHostCacheBankEquals(virCapsHostCacheBankPtr a,
 void
 virCapsHostCacheBankFree(virCapsHostCacheBankPtr ptr)
 {
+size_t i;
+
 if (!ptr)
 return;
 
 virBitmapFree(ptr->cpus);
+for (i = 0; i < ptr->ncontrol; i++)
+VIR_FREE(ptr->controls[i]);
+VIR_FREE(ptr->controls);
 VIR_FREE(ptr);
 }
 
+/* test which kinds of cache control supported
+ * -1: don't support
+ *  0: cat
+ *  1: cdp
+ */
+static int
+virCapabilitiesGetCacheControlType(virCapsHostCacheBankPtr bank)
+{
+int ret = -1;
+char *path = NULL;
+if (virAsprintf(, SYSFS_RESCTRL_PATH "info/L%u", bank->level) < 0)
+return -1;
+
+if (virFileExists(path)) {
+ret = 0;
+} else {
+VIR_FREE(path);
+if (virAsprintf(, SYSFS_RESCTRL_PATH "info/L%uCODE", bank->level) 
< 0)
+return -1;
+if (virFileExists(path))
+ret = 1;
+}
+
+VIR_FREE(path);
+return ret;
+}
+
+static int
+virCapabilitiesGetCacheControl(virCapsHostCacheBankPtr bank, const char* type)
+{
+int ret = -1;
+char *path = NULL;
+char *cbm_mask = NULL;
+virCapsHostCacheControlPtr control;
+
+if (VIR_ALLOC(control) < 0)
+goto cleanup;
+
+if (virFileReadValueUint(>nallocations,
+ SYSFS_RESCTRL_PATH "info/L%u%s/num_closids",
+ bank->level,
+ type) < 0)
+goto cleanup;
+
+if (virFileReadValueString(_mask,
+   SYSFS_RESCTRL_PATH
+   "info/L%u%s/cbm_mask",
+   bank->level,
+   type) < 0)
+goto cleanup;
+
+virStringTrimOptionalNewline(cbm_mask);
+
+control->min = bank->size / (strlen(cbm_mask) * 4);
+
+if (STREQ("", type))
+control->type = VIR_CACHE_TYPE_UNIFIED;
+else if (STREQ("CODE", type))
+control->type = VIR_CACHE_TYPE_INSTRUCTION;
+else if (STREQ("DATA", type))
+control->type = VIR_CACHE_TYPE_DATA;
+else
+goto cleanup;
+
+if (VIR_APPEND_ELEMENT(bank->controls,
+   bank->ncontrol,
+   control) < 0)
+go

Re: [libvirt] [PATCH V2] Expose resource control capabilites on cache bank

2017-04-06 Thread Eli Qiao



On Thursday, 6 April 2017 at 7:56 PM, Daniel P. Berrange wrote:

> On Thu, Apr 06, 2017 at 07:32:59PM +0800, Eli Qiao wrote:
> > This patch is based on Martin's cache branch.
> >  
> > This patch amends the cache bank capability as follow:
> >  
> > 
> > 
> > 
> > 
> > 
> > 
> >  
>  
>  
> This is still wrong per my previous comments.
>  
>  
I will repost V3 to change it as your comments.

one question, if there’s no `control`, what’s the bank should be like?

[libvirt] [PATCH V2] Expose resource control capabilites on cache bank

2017-04-06 Thread Eli Qiao

This patch is based on Martin's cache branch.

This patch amends the cache bank capability as follow:


  

  



Along with vircaps2xmltest case updated.

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 src/conf/capabilities.c  | 110 +++
 src/conf/capabilities.h  |  13 ++-
 tests/vircaps2xmldata/vircaps-x86_64-resctrl.xml |   2 +
 tests/vircaps2xmltest.c  |   9 ++
 4 files changed, 132 insertions(+), 2 deletions(-)

diff --git a/src/conf/capabilities.c b/src/conf/capabilities.c
index 416dd1a..3ea518e 100644
--- a/src/conf/capabilities.c
+++ b/src/conf/capabilities.c
@@ -52,6 +52,7 @@
 #define VIR_FROM_THIS VIR_FROM_CAPABILITIES

 #define SYSFS_SYSTEM_PATH "/sys/devices/system/"
+#define SYSFS_RESCTRL_PATH "/sys/fs/resctrl/"

 VIR_LOG_INIT("conf.capabilities")

@@ -873,6 +874,7 @@ virCapabilitiesFormatCaches(virBufferPtr buf,
 virCapsHostCacheBankPtr *caches)
 {
 size_t i = 0;
+size_t j = 0;

 if (!ncaches)
 return 0;
@@ -900,6 +902,18 @@ virCapabilitiesFormatCaches(virBufferPtr buf,
   bank->size >> (kilos * 10),
   kilos ? "KiB" : "B",
   cpus_str);
+virBufferAdjustIndent(buf, 2);
+for (j = 0; j < bank->ncontrol; j++) {
+bool min_kilos = !(bank->controls[j]->min % 1024);
+virBufferAsprintf(buf,
+  "\n",
+  bank->controls[j]->min >> (min_kilos * 10),
+  min_kilos ? "KiB" : "B",
+  virCacheTypeToString(bank->controls[j]->type),
+  bank->controls[j]->nallocations);
+}
+virBufferAdjustIndent(buf, -2);

 VIR_FREE(cpus_str);
 }
@@ -1513,13 +1527,97 @@ virCapsHostCacheBankEquals(virCapsHostCacheBankPtr a,
 void
 virCapsHostCacheBankFree(virCapsHostCacheBankPtr ptr)
 {
+size_t i;
+
 if (!ptr)
 return;

 virBitmapFree(ptr->cpus);
+for (i = 0; i < ptr->ncontrol; i++)
+VIR_FREE(ptr->controls[i]);
+VIR_FREE(ptr->controls);
 VIR_FREE(ptr);
 }

+/* test which kinds of cache control supported
+ * -1: don't support
+ *  0: cat
+ *  1: cdp
+ */
+static int
+virCapabilitiesGetCacheControlType(virCapsHostCacheBankPtr bank)
+{
+int ret = -1;
+char *path = NULL;
+if (virAsprintf(, SYSFS_RESCTRL_PATH "info/L%u", bank->level) < 0)
+return -1;
+
+if (virFileExists(path)) {
+ret = 0;
+} else {
+VIR_FREE(path);
+if (virAsprintf(, SYSFS_RESCTRL_PATH "info/L%uCODE", bank->level) 
< 0)
+return -1;
+if (virFileExists(path))
+ret = 1;
+}
+
+VIR_FREE(path);
+return ret;
+}
+
+static int
+virCapabilitiesGetCacheControl(virCapsHostCacheBankPtr bank, const char* type)
+{
+int ret = -1;
+char *path = NULL;
+char *cbm_mask = NULL;
+virCapsHostCacheControlPtr control;
+
+if (VIR_ALLOC(control) < 0)
+goto cleanup;
+
+if (virFileReadValueUint(>nallocations,
+ SYSFS_RESCTRL_PATH "info/L%u%s/num_closids",
+ bank->level,
+ type) < 0)
+goto cleanup;
+
+if (virFileReadValueString(_mask,
+   SYSFS_RESCTRL_PATH
+   "info/L%u%s/cbm_mask",
+   bank->level,
+   type) < 0)
+goto cleanup;
+
+virStringTrimOptionalNewline(cbm_mask);
+
+control->min = bank->size / (strlen(cbm_mask) * 4);
+
+if (STREQ("", type))
+control->type = VIR_CACHE_TYPE_UNIFIED;
+else if (STREQ("CODE", type))
+control->type = VIR_CACHE_TYPE_INSTRUCTION;
+else if (STREQ("DATA", type))
+control->type = VIR_CACHE_TYPE_DATA;
+else
+goto cleanup;
+
+if (VIR_APPEND_ELEMENT(bank->controls,
+   bank->ncontrol,
+   control) < 0)
+goto error;
+
+ret = 0;
+
+ cleanup:
+VIR_FREE(path);
+return ret;
+ error:
+VIR_FREE(control);
+return -1;
+}
+
 int
 virCapabilitiesInitCaches(virCapsPtr caps)
 {
@@ -1601,6 +1699,17 @@ virCapabilitiesInitCaches(virCapsPtr caps)
 continue;
 }

+ret = virCapabilitiesGetCacheControlType(bank);
+
+if (ret == 0) {
+if (virCapabilitiesGetCacheControl(bank, "") < 0)
+goto cleanup;
+} else if (ret == 1) {
+if ((virCapabilitiesGetCa

Re: [libvirt] [PATCH] Expose resource control capabilites on cache bank

2017-04-06 Thread Eli Qiao



On Thursday, 6 April 2017 at 5:51 PM, Daniel P. Berrange wrote:

> On Thu, Apr 06, 2017 at 11:49:14AM +0200, Martin Kletzander wrote:
> > On Thu, Apr 06, 2017 at 04:20:06PM +0800, Eli Qiao wrote:
> > > This patch is based on Martin's cache branch.
> > > 
> > > This patch amends the cache bank capability as follow:
> > > 
> > > 
> > > 
> > >  > > cpus='6-11'/>
> > > 
> > > 
> > 
> > 
> 
> 
> Either the XML is malformed, or the indentation is wrong. The indentation
> suggests you want nested XML elements, but the parent element is an empty
> tag, so you've actually got a flat namespace here.
> 


  

  




I validate it online, no errors, the result is:
 
http://www.xmlvalidation.com/index.php?id=1=0=/xmlvalidation/start.jsp;jsessionid=5B8746DEE269CE0ADFB1B8F88CF5E94F
 
> 
> > > 
> > 
> > 
> > Were we exposing the number of CLoS IDs before? Was there a discussion
> > about it? Do we want to expose them? Probably yes, I'm just wondering.
> > 
> 
> 
> What are CLoS IDs and what are they used for ?
> 
> 
> 
> Regards,
> Daniel
> -- 
> |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
> |: http://libvirt.org -o- http://virt-manager.org :|
> |: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :|
> 
> --
> libvir-list mailing list
> libvir-list@redhat.com (mailto:libvir-list@redhat.com)
> https://www.redhat.com/mailman/listinfo/libvir-list
> 
> 


--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH] Expose resource control capabilites on cache bank

2017-04-06 Thread Eli Qiao



On Thursday, 6 April 2017 at 5:49 PM, Martin Kletzander wrote:

> On Thu, Apr 06, 2017 at 04:20:06PM +0800, Eli Qiao wrote:
> > This patch is based on Martin's cache branch.
> >  
> > This patch amends the cache bank capability as follow:
> >  
> > 
> > 
> > 
> > 
> >  
>  
>  
> Were we exposing the number of CLoS IDs before? Was there a discussion
> about it? Do we want to expose them? Probably yes, I'm just wondering.
>  

Just saw Daniel’s comments, we may need it, with ‘nallocations’, is that okay?

Do you want me to do a reversion now?

  
>  
> > Along with vircaps2xmltest case updated.
> >  
> > Signed-off-by: Eli Qiao <liyong.q...@intel.com 
> > (mailto:liyong.q...@intel.com)>
> > ---
> > src/conf/capabilities.c | 112 +--
> > src/conf/capabilities.h | 13 ++-
> > tests/vircaps2xmldata/vircaps-x86_64-caches.xml | 2 +
> > tests/vircaps2xmldata/vircaps-x86_64-resctrl.xml | 2 +
> > tests/vircaps2xmltest.c | 11 +++
> > 5 files changed, 132 insertions(+), 8 deletions(-)
> >  
> > diff --git a/src/conf/capabilities.c b/src/conf/capabilities.c
> > index 416dd1a..75c0bec 100644
> > --- a/src/conf/capabilities.c
> > +++ b/src/conf/capabilities.c
> > @@ -52,6 +52,7 @@
> > #define VIR_FROM_THIS VIR_FROM_CAPABILITIES
> >  
> > #define SYSFS_SYSTEM_PATH "/sys/devices/system/"
> > +#define SYSFS_RESCTRL_PATH "/sys/fs/resctrl/"
> >  
> > VIR_LOG_INIT("conf.capabilities")
> >  
> > @@ -873,6 +874,7 @@ virCapabilitiesFormatCaches(virBufferPtr buf,
> > virCapsHostCacheBankPtr *caches)
> > {
> > size_t i = 0;
> > + size_t j = 0;
> >  
> > if (!ncaches)
> > return 0;
> > @@ -900,6 +902,18 @@ virCapabilitiesFormatCaches(virBufferPtr buf,
> > bank->size >> (kilos * 10),
> > kilos ? "KiB" : "B",
> > cpus_str);
> > + virBufferAdjustIndent(buf, 2);
> > + for (j = 0; j < bank->ncontrol; j++) {
> > + bool min_kilos = !(bank->controls[j]->min % 1024);
> > + virBufferAsprintf(buf,
> > + " > + "type='%s' nclos='%u'/>\n",
> > + bank->controls[j]->min >> (min_kilos * 10),
> > + min_kilos ? "KiB" : "B",
> > + virCacheTypeToString(bank->controls[j]->type),
> > + bank->controls[j]->nclos);
> > + }
> > + virBufferAdjustIndent(buf, -2);
> >  
> > VIR_FREE(cpus_str);
> > }
> > @@ -1516,10 +1530,93 @@ virCapsHostCacheBankFree(virCapsHostCacheBankPtr 
> > ptr)
> > if (!ptr)
> > return;
> >  
> > + size_t i;
>  
> Upstream frowns upon C99 initialization in the middle of the code.
>  
Okay, I can more it before return.  
>  
> > virBitmapFree(ptr->cpus);
> > + for (i = 0; i < ptr->ncontrol; i++)
> > + VIR_FREE(ptr->controls[i]);
> > + VIR_FREE(ptr->controls);
> > VIR_FREE(ptr);
> > }
> >  
> > +/* test which kinds of cache control supported
> > + * -1: don't support
> > + * 0: cat
> > + * 1: cdp
> > + */
> > +static int
> > +virCapabilitiesGetCacheControlType(virCapsHostCacheBankPtr bank)
> > +{
> > + int ret = -1;
> > + char *path = NULL;
> > + if (virAsprintf(, SYSFS_RESCTRL_PATH "info/L%u", bank->level) < 0)
> > + return -1;
> > +
> > + if (virFileExists(path)) {
> > + ret = 0;
> > + } else {
> > + VIR_FREE(path);
> > + if (virAsprintf(, SYSFS_RESCTRL_PATH "info/L%uCODE", bank->level) < 
> > 0)
> > + return -1;
> > + if (virFileExists(path))
> > + ret = 1;
> > + }
> > +
> > + VIR_FREE(path);
> > + return ret;
> > +}
> > +
> > +static int
> > +virCapabilitiesGetCacheControl(virCapsHostCacheBankPtr bank, const char* 
> > type)
> > +{
> > + int ret = -1;
> > + char *path = NULL;
> > + char *cbm_mask = NULL;
> > + virCapsHostCacheControlPtr control;
> > +
> > + if (VIR_ALLOC(control) < 0)
> > + goto cleanup;
> > +
> > + if (virFileReadValueUint(>nclos,
> > + SYSFS_RESCTRL_PATH "info/L%u%s/num_closids",
> > + bank->level,
> > + type) < 0)
> > + goto cleanup;
> > +
> > + if (virFileReadValueString(_mask,
> > + SYSFS_RESCTRL_PATH
> > + "info/L%u%s/cbm_mask",
> > + bank->level,
> > + type) < 0)
> > + goto cleanup;
> > +
> > + virStringTrimOptionalNewline(cbm_mask);
> > +
> > + control->min = bank->size / (st

Re: [libvirt] [PATCH] Expose resource control capabilites on cache bank

2017-04-06 Thread Eli Qiao



On Thursday, 6 April 2017 at 5:49 PM, Martin Kletzander wrote:

> On Thu, Apr 06, 2017 at 04:20:06PM +0800, Eli Qiao wrote:
> > This patch is based on Martin's cache branch.
> >  
> > This patch amends the cache bank capability as follow:
> >  
> > 
> > 
> > 
> > 
> >  
>  
>  
> Were we exposing the number of CLoS IDs before? Was there a discussion
> about it? Do we want to expose them? Probably yes, I'm just wondering.
>  
> >  
We don’t expose it in previous discussion as in my old design, it is saved in a 
global internal struct, and we will use
it to limit the cache allocation number.

For now as you expose all these to capabilities XML, I want expose it out which 
can be reference by resctrl utils later.

Thought?

> > --
> > libvir-list mailing list
> > libvir-list@redhat.com (mailto:libvir-list@redhat.com)
> > https://www.redhat.com/mailman/listinfo/libvir-list
> >  
>  
>  
> --
> libvir-list mailing list
> libvir-list@redhat.com (mailto:libvir-list@redhat.com)
> https://www.redhat.com/mailman/listinfo/libvir-list
>  
>  


--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH] Expose resource control capabilites on cache bank

2017-04-06 Thread Eli Qiao



On Thursday, 6 April 2017 at 6:10 PM, Daniel P. Berrange wrote:

> On Thu, Apr 06, 2017 at 12:05:54PM +0200, Martin Kletzander wrote:
> > On Thu, Apr 06, 2017 at 10:51:44AM +0100, Daniel P. Berrange wrote:
> > > On Thu, Apr 06, 2017 at 11:49:14AM +0200, Martin Kletzander wrote:
> > > > On Thu, Apr 06, 2017 at 04:20:06PM +0800, Eli Qiao wrote:
> > > > > This patch is based on Martin's cache branch.
> > > > >  
> > > > > This patch amends the cache bank capability as follow:
> > > > >  
> > > > >  > > > > cpus='0-5'/>
> > > > > 
> > > > >  > > > > cpus='6-11'/>
> > > > > 
> > > > >  
> > > >  
> > > >  
> > >  
> > >  
> > > Either the XML is malformed, or the indentation is wrong. The indentation
> > > suggests you want nested XML elements, but the parent element is an empty
> > > tag, so you've actually got a flat namespace here.
> > >  
> > > >  
> > > > Were we exposing the number of CLoS IDs before? Was there a discussion
> > > > about it? Do we want to expose them? Probably yes, I'm just wondering.
> > > >  
> > >  
> > >  
> > > What are CLoS IDs and what are they used for ?
> >  
> > Effectively an ID for the allocation. The hardware has a limited number
> > of them, in this case 4. I can't remember whether that number is
> > per-bank, but it would not make much sense otherwise.
> >  
>  
>  
> So, if guests are requesting a private cache allocation, and cos id == 4,
> then we can only run 4 guests ?
>  
>  


I think yes, but it’s per bank resource (the bank here is equal to cache id), 
if you have 2 banks , you can create 8 guests (each has 1 bank allocation)

As far as I know, the number of clos id is 16 on most of Intel xeon CPUs
>  
> Regards,
> Daniel
> --  
> |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
> |: http://libvirt.org -o- http://virt-manager.org :|
> |: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :|
>  
> --
> libvir-list mailing list
> libvir-list@redhat.com (mailto:libvir-list@redhat.com)
> https://www.redhat.com/mailman/listinfo/libvir-list
>  
>  


--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATCH] Expose resource control capabilites on cache bank

2017-04-06 Thread Eli Qiao

This patch is based on Martin's cache branch.

This patch amends the cache bank capability as follow:


  

  

Along with vircaps2xmltest case updated.

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 src/conf/capabilities.c  | 112 +--
 src/conf/capabilities.h  |  13 ++-
 tests/vircaps2xmldata/vircaps-x86_64-caches.xml  |   2 +
 tests/vircaps2xmldata/vircaps-x86_64-resctrl.xml |   2 +
 tests/vircaps2xmltest.c  |  11 +++
 5 files changed, 132 insertions(+), 8 deletions(-)

diff --git a/src/conf/capabilities.c b/src/conf/capabilities.c
index 416dd1a..75c0bec 100644
--- a/src/conf/capabilities.c
+++ b/src/conf/capabilities.c
@@ -52,6 +52,7 @@
 #define VIR_FROM_THIS VIR_FROM_CAPABILITIES
 
 #define SYSFS_SYSTEM_PATH "/sys/devices/system/"
+#define SYSFS_RESCTRL_PATH "/sys/fs/resctrl/"
 
 VIR_LOG_INIT("conf.capabilities")
 
@@ -873,6 +874,7 @@ virCapabilitiesFormatCaches(virBufferPtr buf,
 virCapsHostCacheBankPtr *caches)
 {
 size_t i = 0;
+size_t j = 0;
 
 if (!ncaches)
 return 0;
@@ -900,6 +902,18 @@ virCapabilitiesFormatCaches(virBufferPtr buf,
   bank->size >> (kilos * 10),
   kilos ? "KiB" : "B",
   cpus_str);
+virBufferAdjustIndent(buf, 2);
+for (j = 0; j < bank->ncontrol; j++) {
+bool min_kilos = !(bank->controls[j]->min % 1024);
+virBufferAsprintf(buf,
+  "\n",
+  bank->controls[j]->min >> (min_kilos * 10),
+  min_kilos ? "KiB" : "B",
+  virCacheTypeToString(bank->controls[j]->type),
+  bank->controls[j]->nclos);
+}
+virBufferAdjustIndent(buf, -2);
 
 VIR_FREE(cpus_str);
 }
@@ -1516,10 +1530,93 @@ virCapsHostCacheBankFree(virCapsHostCacheBankPtr ptr)
 if (!ptr)
 return;
 
+size_t i;
 virBitmapFree(ptr->cpus);
+for (i = 0; i < ptr->ncontrol; i++)
+VIR_FREE(ptr->controls[i]);
+VIR_FREE(ptr->controls);
 VIR_FREE(ptr);
 }
 
+/* test which kinds of cache control supported
+ * -1: don't support
+ *  0: cat
+ *  1: cdp
+ */
+static int
+virCapabilitiesGetCacheControlType(virCapsHostCacheBankPtr bank)
+{
+int ret = -1;
+char *path = NULL;
+if (virAsprintf(, SYSFS_RESCTRL_PATH "info/L%u", bank->level) < 0)
+return -1;
+
+if (virFileExists(path)) {
+ret = 0;
+} else {
+VIR_FREE(path);
+if (virAsprintf(, SYSFS_RESCTRL_PATH "info/L%uCODE", bank->level) 
< 0)
+return -1;
+if (virFileExists(path))
+ret = 1;
+}
+
+VIR_FREE(path);
+return ret;
+}
+
+static int
+virCapabilitiesGetCacheControl(virCapsHostCacheBankPtr bank, const char* type)
+{
+int ret = -1;
+char *path = NULL;
+char *cbm_mask = NULL;
+virCapsHostCacheControlPtr control;
+
+if (VIR_ALLOC(control) < 0)
+goto cleanup;
+
+if (virFileReadValueUint(>nclos,
+ SYSFS_RESCTRL_PATH "info/L%u%s/num_closids",
+ bank->level,
+ type) < 0)
+goto cleanup;
+
+if (virFileReadValueString(_mask,
+   SYSFS_RESCTRL_PATH
+   "info/L%u%s/cbm_mask",
+   bank->level,
+   type) < 0)
+goto cleanup;
+
+virStringTrimOptionalNewline(cbm_mask);
+
+control->min = bank->size / (strlen(cbm_mask) * 4);
+
+if (STREQ("", type))
+control->type = VIR_CACHE_TYPE_UNIFIED;
+else if (STREQ("CODE", type))
+control->type = VIR_CACHE_TYPE_INSTRUCTION;
+else if (STREQ("DATA", type))
+control->type = VIR_CACHE_TYPE_DATA;
+else
+goto cleanup;
+
+if (VIR_APPEND_ELEMENT(bank->controls,
+   bank->ncontrol,
+   control) < 0)
+goto error;
+
+ret = 0;
+
+ cleanup:
+VIR_FREE(path);
+return ret;
+ error:
+VIR_FREE(control);
+return -1;
+}
+
 int
 virCapabilitiesInitCaches(virCapsPtr caps)
 {
@@ -1533,11 +1630,6 @@ virCapabilitiesInitCaches(virCapsPtr caps)
 struct dirent *ent = NULL;
 virCapsHostCacheBankPtr bank = NULL;
 
-/* Minimum level to expose in capabilities.  Can be lowered or removed 
(with
- * the appropriate code below), but should not be increased, because we'd
- * lose information. */
-const int cache_min_level = 3;
-
 /* offline CPUs don't provide cache info */
 if (vir

Re: [libvirt] [PATCH v2 09/12] Add host cache information into capabilities

2017-04-06 Thread Eli Qiao


> +
> +VIR_ENUM_IMPL(virCache, VIR_CACHE_TYPE_LAST,
> + "unified",
> + "instruction",
> + "data")
> +


> };
> 
> +typedef enum {
> + VIR_CACHE_TYPE_DATA,
> + VIR_CACHE_TYPE_INSTRUCTION,
> + VIR_CACHE_TYPE_UNIFIED,
> +
> + VIR_CACHE_TYPE_LAST
> +} virCacheType;
> +

The sequence is wrong, it should be :

VIR_CACHE_TYPE_UNIFIED,
VIR_CACHE_TYPE_INSTRUCTION,
VIR_CACHE_TYPE_DATA,


see the above VIR_ENUM_IMPL.
> +VIR_ENUM_DECL(virCache);
> +
> +typedef struct _virCapsHostCacheBank virCapsHostCacheBank;
> 
> 


--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH v2 12/12] tests: Add resctrl test for vircaps2xmltest

2017-04-05 Thread Eli Qiao



On Wednesday, 5 April 2017 at 10:36 PM, Martin Kletzander wrote:

> Add info from yet another machine, this time with resctrl data so that
> we can extend tests easily in a test-driven way.
> 
> Signed-off-by: Martin Kletzander  (mailto:mklet...@redhat.com)>
> ---
> tests/vircaps2xmldata/linux-resctrl/resctrl/cpus | 1 +
> .../linux-resctrl/resctrl/info/L3/cbm_mask | 1 +
> .../linux-resctrl/resctrl/info/L3/min_cbm_bits | 1 +
> .../linux-resctrl/resctrl/info/L3/num_closids | 1 +
> .../linux-resctrl/resctrl/manualres/cpus | 1 +
> .../linux-resctrl/resctrl/manualres/schemata | 1 +
> .../linux-resctrl/resctrl/manualres/tasks | 0
> .../vircaps2xmldata/linux-resctrl/resctrl/schemata | 1 +
> tests/vircaps2xmldata/linux-resctrl/resctrl/tasks | 0

This is great and it is what we want before enable CAT in libvirt, but seems 
that if we want to test CDP enabled, we need another linux-resctrl directory?  

how about separate resctrl from linux-resctrl?

Eli.--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH 15/16] tests: Add tests for caches into vircaps2xmltest

2017-03-31 Thread Eli Qiao


> >  
> > How about for l3:
> >  > reserved=“2816"/>
> >  
>  
>  
> Well, yes, kind of what you had in your patches. Wasn't it without the
> 'cbm_len' and 'avail'? The 'cbm_len' is avail/min, so it's redundant
> and avail is the same as the size of the whole cache, right? Also
> 'reserved' should not be there as that would have to be refreshed every
> time the info is gathered and that's not what capabilities are for.
> Also, if we say 'unified' instead of 'both', it sounds little more
> consistent.
>  
> So basically, I'm thinking we were somewhere along the lines of:
>  
> 
>  
> Or do I remember it wrong?
oh yeah, right!

for scope, it’s okay to use 'unified' to instead of ‘both’
for CDP enabled case would it be ?

1)  
 
 

or  

2)  



or  
3)



?

A correction, that would be 

the unit B is kinds of small for l3 cache.

Thx Eli.--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH 10/16] util: Add virSysfsGetCPUCache* functions

2017-03-31 Thread Eli Qiao



On Friday, 31 March 2017 at 7:39 PM, Martin Kletzander wrote:

> On Fri, Mar 31, 2017 at 05:32:16PM +0800, Eli Qiao wrote:
> >  
> >  
> > On Thursday, 30 March 2017 at 10:03 PM, Martin Kletzander wrote:
> >  
> > > Signed-off-by: Martin Kletzander <mklet...@redhat.com 
> > > (mailto:mklet...@redhat.com)>
> > > ---
> > > src/libvirt_private.syms | 5 +++
> > > src/util/virsysfs.c | 102 +++
> > > src/util/virsysfs.h | 34 
> > > 3 files changed, 141 insertions(+)
> > >  
> > > diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms
> > > index bcd2506ef7c9..0b3b41516fe6 100644
> > > --- a/src/libvirt_private.syms
> > > +++ b/src/libvirt_private.syms
> > > @@ -2623,6 +2623,11 @@ virVasprintfInternal;
> > > # util/virsysfs.h
> > > virSysfsCpuDirOpen;
> > > virSysfsDirOpen;
> > > +virSysfsGetCpuCacheValueBitmap;
> > > +virSysfsGetCpuCacheValueInt;
> > > +virSysfsGetCpuCacheValueScaledInt;
> > > +virSysfsGetCpuCacheValueString;
> > > +virSysfsGetCpuCacheValueUint;
> > > virSysfsGetCpuValueBitmap;
> > > virSysfsGetCpuValueInt;
> > > virSysfsGetCpuValueString;
> > > diff --git a/src/util/virsysfs.c b/src/util/virsysfs.c
> > > index a8550bbfbc26..2a64be4f5f73 100644
> > > --- a/src/util/virsysfs.c
> > > +++ b/src/util/virsysfs.c
> > > @@ -221,6 +221,108 @@ virSysfsCpuDirOpen(unsigned int cpu,
> > > }
> > >  
> > >  
> > > +/*
> > > + * Per-CPU/cache getters
> > > + */
> > > +int
> > > +virSysfsGetCpuCacheValueInt(unsigned int cpu,
> > > + const char *cache,
> > > + const char *file,
> > > + int *value)
> > >  
> >  
> >  
> >  
> > It will be helpful that we describe what cache, file looks like
> > even it’s straight enough to reading code.
> >  
>  
>  
> What do you mean? The files look just like they look in the system.
> It's a copy of what actually is under /sys/devices/system. The
> description is in the kernel.
>  
Okay, never mind.  
>  
> > > +{
> > > + char *path = NULL;
> > > + int ret = -1;
> > > +
> > > + if (virAsprintf(, "%s/cpu/cpu%u/cache/%s/%s",
> > >  
> >  
> >  
> >  
> > cpu/cpu0/cache/index3/size
>  
> ???
>  
ignore this.  
>  
> > > + sysfs_system_path, cpu, cache, file) < 0)
> > > + return -1;
> > > +
> > > + ret = virFileReadValueInt(path, value);
> > > +
> > > + VIR_FREE(path);
> > > + return ret;
> > > +}
> > > +
> > > +
> > > +int
> > > +virSysfsGetCpuCacheValueUint(unsigned int cpu,
> > > + const char *cache,
> > > + const char *file,
> > > + unsigned int *value)
> > > +{
> > > + char *path = NULL;
> > > + int ret = -1;
> > > +
> > > + if (virAsprintf(, "%s/cpu/cpu%u/cache/%s/%s",
> > > + sysfs_system_path, cpu, cache, file) < 0)
> > > + return -1;
> > > +
> > > + ret = virFileReadValueUint(path, value);
> > > +
> > > + VIR_FREE(path);
> > > + return ret;
> > > +}
> > > +
> > > +int
> > > +virSysfsGetCpuCacheValueScaledInt(unsigned int cpu,
> > > + const char *cache,
> > > + const char *file,
> > > + unsigned long long *value)
> > >  
> >  
> >  
> >  
> > Can we add notes here to tell the value is in unite KiB ?
>  
> Well what if you want to use this for another file in the future? This
> function will be able to read it even if the file has "1M" in it. Or
> did you mean something else?
>  
>  

My mistake, it’s should B (not KiB), It’s okay.  
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH 15/16] tests: Add tests for caches into vircaps2xmltest

2017-03-31 Thread Eli Qiao



On Friday, 31 March 2017 at 7:19 PM, Martin Kletzander wrote:

> On Fri, Mar 31, 2017 at 09:56:32AM +0800, Eli Qiao wrote:
> >  
> > > 
> > Okay, cool, this comes better than my patches and have some differences.
> > I am open with this as long as that it can meet cache allocation requires 
> > and
> > everyone will be happy.
> >  
> > I am ++ for this.
> >  
> > But I am not sure expose all of cache information in the capabilities XML.
>  
> ??? Are you not sure we "should expose" all of cache information? Or
> are you afraid we're not exposing enough information?
>  
Well, I was saying not to expose all of the information, but after your reply, 
it's okay for me.

  
>  
> > > 
> > > + 
> > > +  > > cpus='0-7'/>
> > >  
> >  
> >  
> > eg: if enabled CDP feature on the host, what the type of level=3 cache 
> > should be like?
>  
> This has nothing to do with resctrl yet. I'm just exposing the caches
> that exist on the host.
>  
> > > + 
> > for the bank id, it’s per cache level unique right (data/instruction shares 
> > same id)?
> >  
>  
>  
> It looks like it's per cache level/type unique. But it's precisely just
> what the kernel exposes to us. I'm not doing anything on top of that.
>  
> > > +  > > cpus='0-1'/>
> > > + 
> > > + 
> > > +  > > cpus='2-3'/>
> > > + 
> > > + 
> > > +  > > cpus='4-5'/>
> > > + 
> > > + 
> > > +  > > cpus='6-7'/>
> > > + 
> > > + 
> > >  
> >  
> >  
> >  
> > This’s really good that you have work this out by expose all these out to 
> > capabilities,
> > and it will be much easy to let resctrl keep focus on cache allocation.
> >  
> > So if util/virresctrl.c would like to access some cache abilities, it will 
> > first get virCapsPtr.host.caches,
> > right?
> >  
>  
>  
> Well yeah, it'll probably extend the CacheBank struct.
+1  
>  
> > but I am not sure if that’s be okay to expose all cache information which 
> > we can not
> > do the allocation yet.
> >  
>  
>  
> What's the harm?
>  

think about I have a host has 2 Socket 22 core and 2 thread per core, I will 
have 88 cpus

so the cache bank will be a large list  

2 for l3 , 44 for l2, 44 for l1d and 44 for l1c, not all of them are useful to 
users, and the capabilities XML are boring.

Even though I am Okay to expose all of them.
  
I remember that Daniel has some comments for this in the RFC to not expose all 
caches of the host if no cache allocation support yet.

Any way, no harm.


> > How can a user/admin to know from capabilities?
>  
> Easily. The XML you see above just says what cache is on the host. If
> any of the banks are allocatable, then it will have a sub-element. Is
> there any problem with that?
>  
>  

No problem, +1 to extend a sub-element .

Any suggestions for what’s it should be?  

How about for l3:  



> > > 
> > >  
> > > 
> > > diff --git a/tests/vircaps2xmltest.c b/tests/vircaps2xmltest.c
> > > index ffbe9a783811..dda0757766a8 100644
> > > --- a/tests/vircaps2xmltest.c
> > > +++ b/tests/vircaps2xmltest.c
> > > @@ -58,7 +58,8 @@ test_virCapabilities(const void *opaque)
> > > if (!caps)
> > > goto cleanup;
> > >  
> > > - if (virCapabilitiesInitNUMA(caps) < 0)
> > > + if (virCapabilitiesInitNUMA(caps) < 0 ||
> > > + virCapabilitiesInitCaches(caps) < 0)
> > > goto cleanup;
> > >  
> > > virSysfsSetSystemPath(NULL);
> > > --
> > > 2.12.2
> > >  
> > > --
> > > libvir-list mailing list
> > > libvir-list@redhat.com (mailto:libvir-list@redhat.com)
> > > https://www.redhat.com/mailman/listinfo/libvir-list
> > >  
> >  
>  
>  
>  


--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATCH V3] util: Add more virsysfs functions for handling resctrl sysfs

2017-03-31 Thread Eli Qiao

Extended /sys/fs/resctrl sysfs handling such as:

Read string/uint.
Write string.
Create/remove directory.

All these operations will be while we are enabled CAT feature later.

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 src/libvirt_private.syms |   9 +++
 src/util/virsysfs.c  | 143 +++
 src/util/virsysfs.h  |  26 +
 src/util/virsysfspriv.h  |   1 +
 4 files changed, 179 insertions(+)

diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms
index b551cb8..f6644cb 100644
--- a/src/libvirt_private.syms
+++ b/src/libvirt_private.syms
@@ -2620,17 +2620,26 @@ virVasprintfInternal;


 # util/virsysfs.h
+virSysfsCreateResCtrlDir;
 virSysfsGetCpuValueBitmap;
 virSysfsGetCpuValueInt;
 virSysfsGetCpuValueString;
 virSysfsGetCpuValueUint;
 virSysfsGetNodeValueBitmap;
 virSysfsGetNodeValueString;
+virSysfsGetResctrlInfoString;
+virSysfsGetResctrlInfoUint;
+virSysfsGetResctrlPath;
+virSysfsGetResctrlString;
+virSysfsGetResctrlUint;
 virSysfsGetSystemPath;
 virSysfsGetValueBitmap;
 virSysfsGetValueInt;
 virSysfsGetValueString;
+virSysfsRemoveResCtrlDir;
+virSysfsSetResctrlPath;
 virSysfsSetSystemPath;
+virSysfsWriteResctrlString;


 # util/virsysinfo.h
diff --git a/src/util/virsysfs.c b/src/util/virsysfs.c
index 7a98b48..a17c391 100644
--- a/src/util/virsysfs.c
+++ b/src/util/virsysfs.c
@@ -19,6 +19,7 @@
  */

 #include 
+#include 

 #include "internal.h"

@@ -36,8 +37,10 @@ VIR_LOG_INIT("util.sysfs");

 #define VIR_SYSFS_VALUE_MAXLEN 8192
 #define SYSFS_SYSTEM_PATH "/sys/devices/system"
+#define SYSFS_RESCTRL_PATH "/sys/fs/resctrl"

 static const char *sysfs_system_path = SYSFS_SYSTEM_PATH;
+static const char *sysfs_resctrl_path = SYSFS_RESCTRL_PATH;


 void virSysfsSetSystemPath(const char *path)
@@ -55,6 +58,20 @@ virSysfsGetSystemPath(void)
 return sysfs_system_path;
 }

+void virSysfsSetResctrlPath(const char *path)
+{
+if (path)
+sysfs_resctrl_path  = path;
+else
+sysfs_resctrl_path = SYSFS_RESCTRL_PATH;
+}
+
+const char *
+virSysfsGetResctrlPath(void)
+{
+return sysfs_resctrl_path;
+}
+
 int
 virSysfsGetValueInt(const char *file,
 int *value)
@@ -227,3 +244,129 @@ virSysfsGetNodeValueBitmap(unsigned int node,
 VIR_FREE(path);
 return ret;
 }
+
+int
+virSysfsGetResctrlString(const char* file,
+ char **value)
+{
+char *path = NULL;
+int ret = -1;
+if (virAsprintf(, "%s/%s", sysfs_resctrl_path, file) < 0)
+return -1;
+
+if (virFileReadAll(path, VIR_SYSFS_VALUE_MAXLEN, value) < 0)
+goto cleanup;
+
+ret = 0;
+
+ cleanup:
+VIR_FREE(path);
+return ret;
+}
+
+int
+virSysfsGetResctrlUint(const char *file,
+   unsigned int *value)
+{
+char *path = NULL;
+int ret = -1;
+if (virAsprintf(, "%s/%s", sysfs_resctrl_path, file) < 0)
+return -1;
+
+ret = virFileReadValueUint(path, value);
+
+VIR_FREE(path);
+return ret;
+}
+
+int
+virSysfsGetResctrlInfoString(const char* file,
+ char **value)
+{
+char *path = NULL;
+int ret = -1;
+if (virAsprintf(, "%s/info/%s", sysfs_resctrl_path, file) < 0)
+return -1;
+
+if (virFileReadAll(path, VIR_SYSFS_VALUE_MAXLEN, value) < 0)
+goto cleanup;
+
+ret = 0;
+
+ cleanup:
+VIR_FREE(path);
+return ret;
+}
+
+int
+virSysfsGetResctrlInfoUint(const char *file,
+   unsigned int *value)
+{
+char *path = NULL;
+int ret;
+
+if (virAsprintf(, "%s/info/%s", sysfs_resctrl_path, file) < 0)
+return -1;
+
+ret = virFileReadValueUint(path, value);
+
+VIR_FREE(path);
+return ret;
+}
+
+int
+virSysfsWriteResctrlString(const char *file,
+   const char *content)
+{
+char *path = NULL;
+int ret = -1;
+int writefd;
+
+if (virAsprintf(, "%s/%s", sysfs_resctrl_path, file) < 0)
+return -1;
+
+/* We can not use virFileWriteStr because resctrl requires oflag should be 
as
+ O_WRONLY | O_APPEND */
+if ((writefd = open(path, O_WRONLY | O_APPEND, S_IRUSR | S_IWUSR)) < 0)
+goto cleanup;
+
+if (safewrite(writefd, content, strlen(content)) < 0)
+goto cleanup;
+
+ret = 0;
+
+ cleanup:
+VIR_FORCE_CLOSE(writefd);
+VIR_FREE(path);
+return ret;
+}
+
+int
+virSysfsCreateResCtrlDir(const char *file)
+{
+char *path = NULL;
+int ret = -1;
+
+if (virAsprintf(, "%s/%s", sysfs_resctrl_path, file) < 0)
+return -1;
+
+ret = virDirCreate(path, 0755, 0, 0, 0);
+
+VIR_FREE(path);
+return ret;
+}
+
+int
+virSysfsRemoveResCtrlDir(const char *file)
+{
+char *path = NULL;
+int ret = -1;
+
+if (virAsprintf(, "%s/%s", sysfs_resctrl_path, file) < 0)
+return -1;

Re: [libvirt] [PATCH 10/16] util: Add virSysfsGetCPUCache* functions

2017-03-31 Thread Eli Qiao



On Thursday, 30 March 2017 at 10:03 PM, Martin Kletzander wrote:

> Signed-off-by: Martin Kletzander  (mailto:mklet...@redhat.com)>
> ---
> src/libvirt_private.syms | 5 +++
> src/util/virsysfs.c | 102 +++
> src/util/virsysfs.h | 34 
> 3 files changed, 141 insertions(+)
>  
> diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms
> index bcd2506ef7c9..0b3b41516fe6 100644
> --- a/src/libvirt_private.syms
> +++ b/src/libvirt_private.syms
> @@ -2623,6 +2623,11 @@ virVasprintfInternal;
> # util/virsysfs.h
> virSysfsCpuDirOpen;
> virSysfsDirOpen;
> +virSysfsGetCpuCacheValueBitmap;
> +virSysfsGetCpuCacheValueInt;
> +virSysfsGetCpuCacheValueScaledInt;
> +virSysfsGetCpuCacheValueString;
> +virSysfsGetCpuCacheValueUint;
> virSysfsGetCpuValueBitmap;
> virSysfsGetCpuValueInt;
> virSysfsGetCpuValueString;
> diff --git a/src/util/virsysfs.c b/src/util/virsysfs.c
> index a8550bbfbc26..2a64be4f5f73 100644
> --- a/src/util/virsysfs.c
> +++ b/src/util/virsysfs.c
> @@ -221,6 +221,108 @@ virSysfsCpuDirOpen(unsigned int cpu,
> }
>  
>  
> +/*
> + * Per-CPU/cache getters
> + */
> +int
> +virSysfsGetCpuCacheValueInt(unsigned int cpu,
> + const char *cache,
> + const char *file,
> + int *value)
>  
>  


It will be helpful that we describe what cache, file looks like
even it’s straight enough to reading code.
> +{
> + char *path = NULL;
> + int ret = -1;
> +
> + if (virAsprintf(, "%s/cpu/cpu%u/cache/%s/%s",
>  
>  


cpu/cpu0/cache/index3/size
  
> + sysfs_system_path, cpu, cache, file) < 0)
> + return -1;
> +
> + ret = virFileReadValueInt(path, value);
> +
> + VIR_FREE(path);
> + return ret;
> +}
> +
> +
> +int
> +virSysfsGetCpuCacheValueUint(unsigned int cpu,
> + const char *cache,
> + const char *file,
> + unsigned int *value)
> +{
> + char *path = NULL;
> + int ret = -1;
> +
> + if (virAsprintf(, "%s/cpu/cpu%u/cache/%s/%s",
> + sysfs_system_path, cpu, cache, file) < 0)
> + return -1;
> +
> + ret = virFileReadValueUint(path, value);
> +
> + VIR_FREE(path);
> + return ret;
> +}
> +
> +int
> +virSysfsGetCpuCacheValueScaledInt(unsigned int cpu,
> + const char *cache,
> + const char *file,
> + unsigned long long *value)
>  
>  


Can we add notes here to tell the value is in unite KiB ?
  
> +{
> + char *path = NULL;
> + int ret = -1;
> +
> + if (virAsprintf(, "%s/cpu/cpu%u/cache/%s/%s",
> + sysfs_system_path, cpu, cache, file) < 0)
> + return -1;
> +
> + ret = virFileReadValueScaledInt(path, value);
> +
> + VIR_FREE(path);
> + return ret;
> +}
> +
> +
> +int
> +virSysfsGetCpuCacheValueString(unsigned int cpu,
> + const char *cache,
> + const char *file,
> + char **value)
> +{
> + char *path = NULL;
> + int ret = -1;
> +
> + if (virAsprintf(, "cpu/cpu%u/cache/%s/%s", cpu, cache, file) < 0)
> + return -1;
> +
> + ret = virSysfsGetValueString(path, value);
> +
> + VIR_FREE(path);
> + return ret;
> +}
> +
> +int
> +virSysfsGetCpuCacheValueBitmap(unsigned int cpu,
> + const char *cache,
> + const char *file,
> + virBitmapPtr *value)
> +{
> + char *path = NULL;
> + int ret = -1;
> +
> + if (virAsprintf(, "%s/cpu/cpu%u/cache/%s/%s",
> + sysfs_system_path, cpu, cache, file) < 0)
> + return -1;
> +
> + ret = virFileReadValueBitmap(path, VIR_SYSFS_VALUE_MAXLEN, value);
> + VIR_FREE(path);
> + return ret;
> +}
> +
> +
> +/*
> + * Per-NUMA node getters
> + */
> int
> virSysfsGetNodeValueString(unsigned int node,
> const char *file,
> diff --git a/src/util/virsysfs.h b/src/util/virsysfs.h
> index 25bd100ea9cb..92f9111b069f 100644
> --- a/src/util/virsysfs.h
> +++ b/src/util/virsysfs.h
> @@ -77,6 +77,40 @@ virSysfsCpuDirOpen(unsigned int cpu,
>  
>  
> /*
> + * Per-CPU/cache getters
> + */
> +int
> +virSysfsGetCpuCacheValueInt(unsigned int cpu,
> + const char *cache,
> + const char *file,
> + int *value);
> +
> +int
> +virSysfsGetCpuCacheValueUint(unsigned int cpu,
> + const char *cache,
> + const char *file,
> + unsigned int *value);
> +
> +int
> +virSysfsGetCpuCacheValueScaledInt(unsigned int cpu,
> + const char *cache,
> + const char *file,
> + unsigned long long *value);
> +
> +int
> +virSysfsGetCpuCacheValueString(unsigned int cpu,
> + const char *cache,
> + const char *file,
> + char **value);
> +
> +int
> +virSysfsGetCpuCacheValueBitmap(unsigned int cpu,
> + const char *cache,
> + const char *file,
> + virBitmapPtr *value);
> +
> +
> +/*
> * Per-NUMA node getters
> */
> int
> --  
> 2.12.2
>  
> --
> libvir-list mailing list
> libvir-list@redhat.com (mailto:libvir-list@redhat.com)
> https://www.redhat.com/mailman/listinfo/libvir-list
>  
>  


--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH 15/16] tests: Add tests for caches into vircaps2xmltest

2017-03-30 Thread Eli Qiao


> 
Okay, cool, this comes better than my patches and have some differences.
I am open with this as long as that it can meet cache allocation requires and
everyone will be happy.  

I am ++ for this.

But I am not sure expose all of cache information in the capabilities XML.
> 
> + 
> + 
>  
>  

eg: if enabled CDP feature on the host, what the type of level=3 cache should 
be like?

> + 
for the bank id, it’s per cache level unique right (data/instruction shares 
same id)?  
> + 
> + 
> + 
> + 
> + 
> + 
> + 
> + 
> + 
> + 
> + 
> + 
>  
>  


This’s really good that you have work this out by expose all these out to 
capabilities,
and it will be much easy to let resctrl keep focus on cache allocation.

So if util/virresctrl.c would like to access some cache abilities, it will 
first get virCapsPtr.host.caches,
right?

but I am not sure if that’s be okay to expose all cache information which we 
can not
do the allocation yet.

How can a user/admin to know from capabilities?
> 
>  
> 
> diff --git a/tests/vircaps2xmltest.c b/tests/vircaps2xmltest.c
> index ffbe9a783811..dda0757766a8 100644
> --- a/tests/vircaps2xmltest.c
> +++ b/tests/vircaps2xmltest.c
> @@ -58,7 +58,8 @@ test_virCapabilities(const void *opaque)
> if (!caps)
> goto cleanup;
>  
> - if (virCapabilitiesInitNUMA(caps) < 0)
> + if (virCapabilitiesInitNUMA(caps) < 0 ||
> + virCapabilitiesInitCaches(caps) < 0)
> goto cleanup;
>  
> virSysfsSetSystemPath(NULL);
> --  
> 2.12.2
>  
> --
> libvir-list mailing list
> libvir-list@redhat.com (mailto:libvir-list@redhat.com)
> https://www.redhat.com/mailman/listinfo/libvir-list
>  
>  


--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH 06/16] util: Add virSysfsDirOpen

2017-03-30 Thread Eli Qiao



On Thursday, 30 March 2017 at 10:03 PM, Martin Kletzander wrote:

> Signed-off-by: Martin Kletzander  (mailto:mklet...@redhat.com)>
> ---
> src/libvirt_private.syms | 1 +
> src/util/virsysfs.c | 17 -
> src/util/virsysfs.h | 6 ++
> 3 files changed, 23 insertions(+), 1 deletion(-)
> 
> diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms
> index 9e61accda3fc..3965ef3f89c9 100644
> --- a/src/libvirt_private.syms
> +++ b/src/libvirt_private.syms
> @@ -2621,6 +2621,7 @@ virVasprintfInternal;
> 
> 
> # util/virsysfs.h
> +virSysfsDirOpen;
> virSysfsGetCpuValueBitmap;
> virSysfsGetCpuValueInt;
> virSysfsGetCpuValueString;
> diff --git a/src/util/virsysfs.c b/src/util/virsysfs.c
> index c482e188a301..6686d8ddbfcb 100644
> --- a/src/util/virsysfs.c
> +++ b/src/util/virsysfs.c
> @@ -25,7 +25,6 @@
> #include "virsysfspriv.h"
> 
> #include "viralloc.h"
> -#include "virfile.h"
> #include "virlog.h"
> #include "virstring.h"
> 
> @@ -120,6 +119,22 @@ virSysfsGetValueBitmap(const char *file,
> * Per-CPU getters
> */
> int
> +virSysfsDirOpen(const char *file,
> + DIR **dirp)
> +{
> + char *path = NULL;
> +
> + if (virAsprintf(, "%s/%s", sysfs_system_path, file) < 0)
> + return -1;
> +
> + if (!virFileIsDir(path))
> + return -2;
> +
> + return virDirOpen(dirp, path);
> +}
> +
> 
> 

what if I need another util function like:
 virSysfsDirResctrlOpen(const char *file, DIR **dirp)

do another copy and just modify path as:

if (virAsprintf(, "%s/%s", sysfs_resctrl_path, file) < 0)

is that okay for you to duplicated code like this?

This is not so common a way to adding utils functions.

If I need another `sysfs_system_path`, for /sys/fs/resctrl
I need to add a new variable sysfs_resctrl_path, then do a copy
of this function with some modification.

Eli.
> + DIR **dirp)

> +
> +int
> virSysfsGetCpuValueInt(unsigned int cpu,
> const char *file,
> int *value)
> diff --git a/src/util/virsysfs.h b/src/util/virsysfs.h
> index 1b24fc193a16..ff5012d62747 100644
> --- a/src/util/virsysfs.h
> +++ b/src/util/virsysfs.h
> @@ -23,6 +23,8 @@
> 
> # include "internal.h"
> # include "virbitmap.h"
> +# include "virfile.h"
> +
> 
> /*
> * Generic getters
> @@ -41,6 +43,10 @@ int
> virSysfsGetValueBitmap(const char *file,
> virBitmapPtr *value);
> 
> +int
> +virSysfsDirOpen(const char *file,
> + DIR **dirp);
> +
> 
> /*
> * Per-CPU getters
> -- 
> 2.12.2
> 
> --
> libvir-list mailing list
> libvir-list@redhat.com (mailto:libvir-list@redhat.com)
> https://www.redhat.com/mailman/listinfo/libvir-list
> 
> 


--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH V2 1/2] util: Add more virsysfs functions for handling resctrl sysfs

2017-03-30 Thread Eli Qiao


> > +# util/virresctrl.h
> > +virResCtrlAvailable;
> > +virResCtrlInit;
> > +
> >  
>  
>  
> This has nothing to do in the patch
>  
Okay, this should be involved by mistake while rebasing.  
>  
> > # util/virrotatingfile.h
> > virRotatingFileReaderConsume;
> > virRotatingFileReaderFree;
> > @@ -2626,13 +2630,18 @@ virSysfsGetCpuValueString;
> > virSysfsGetCpuValueUint;
> > virSysfsGetNodeValueBitmap;
> > virSysfsGetNodeValueString;
> > +virSysfsGetResctrlInfoString;
> > +virSysfsGetResctrlInfoUint;
> > +virSysfsGetResctrlPath;
> > +virSysfsGetResctrlString;
> > +virSysfsGetResctrlUint;
> > virSysfsGetSystemPath;
> > virSysfsGetValueBitmap;
> > virSysfsGetValueInt;
> > virSysfsGetValueString;
> > +virSysfsSetResctrlPath;
> > virSysfsSetSystemPath;
> >  
> > -
>  
> Don't remove the line. Every time you are doing a change in the code
> (and I mean any code), try being consistent. If the code follows some
> style, don't fight it, but go with it. Not everything can be written in
> the coding style. As you can see here, all files are separated by two
> lines. When you remove this, you make the file inconsistent.
>  
> Also, I mentioned many times before that you should run make check and
> make syntax-check between patches, and I'm sure our contribution
> guidelines mention that make check must pass between patches. This
> change should not pass the checks. It's fine every now and then, we all
> forget, but I'm trying to run make check and make syntax-check after
> each patch before sending it. Useful command to use from your topic
> branch is something along the lines of:
>  
> git rebase -ix 'make -j9 check && make -j9 syntax-check' master
> (written by hand from memory, there might be typo, I have a script and
> bunch of aliases for that)
>  
Sure, Daniel had reminded me before, I missed it this time, will keep in mind 
next path.  
>  
> > # util/virsysinfo.h
> > virSysinfoBaseBoardDefClear;
> > virSysinfoBIOSDefFree;
> > diff --git a/src/util/virsysfs.c b/src/util/virsysfs.c
> > index 7a98b48..97808be 100644
> > --- a/src/util/virsysfs.c
> > +++ b/src/util/virsysfs.c
> >  
>  
>  
> [...]
>  
> > @@ -227,3 +242,88 @@ virSysfsGetNodeValueBitmap(unsigned int node,
> > VIR_FREE(path);
> > return ret;
> > }
> > +
> > +int
> > +virSysfsGetResctrlString(const char* file,
> >  
>  
>  
> 'char *file' instead of 'char* file'
>  
> > + char **value)
> > +{
> > + char *path = NULL;
> > + int ret = -1;
> > + if (virAsprintf(, "%s/%s", sysfs_resctrl_path, file) < 0)
> > + return -1;
> > +
> > + if (!virFileExists(path)) {
> > + ret = -2;
> > + goto cleanup;
> > + }
> > +
> >  
>  
>  
> Now the question is; is it possible for some file to be missing there?
> I mean some file we'd expect? For the system path, the -2 return value
> is there because we need to work on older systems with older kernels and
> the files were being added in multiple releases. If there is no need
> for distinguishing that, then there is no point in explicitly checking
> for the file to be existing.
>  
No, we won’t expect file doesn’t existed. I will remove this checking  
>  
> And that leads me to another point. Since this patch is by itself, it's
> not visible how it is used. It looks good (not considering the things
> pointed out above), but there is no point in merging it until there is a
> value added. It's just added dead code. But it's something that we'll
> use, surely.
>  
Okay, I can add this patch to CAT supporting patch set as the first patch.  
>  
> Martin
>  
> P.S.: I just sent [1] the next couple of patches (again, first ones are
> just fixing some bullocks I found out that were left in the code
> -- that just happens when you're working on a codebase with lots
> of legacy code), but it adds host cache information to the
> capabilities. If you have a minute or two, feel free to check it
> out and let me know what you think.
>  
>  

Sure, I see you have added fake cache id, that’s good, I can drop it.  
>  
> [1] https://www.redhat.com/archives/libvir-list/2017-March/msg01592.html

I have one propose, not sure it’s a good or bad one:

I would like to suggest that let the developer keep focus on adding features,
then refactor some uitls functions later, developers won’t have all knowledge
for what’s the function should be go into which utils, and this will bring time
wasting on keeping rebasing rebasing ……

- Eli
  
> --
> libvir-list mailing list
> libvir-list@redhat.com (mailto:libvir-list@redhat.com)
> https://www.redhat.com/mailman/listinfo/libvir-list
>  
>  


--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATCH V2 2/2] test: Add fake cpu id file

2017-03-30 Thread Eli Qiao

Added fake cpu cache id and resctrl file

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 tests/vircaps2xmldata/linux-caches/cpu/cpu0/cache/index0/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu0/cache/index1/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu0/cache/index2/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu0/cache/index3/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu1/cache/index0/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu1/cache/index1/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu1/cache/index2/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu1/cache/index3/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu2/cache/index0/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu2/cache/index1/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu2/cache/index2/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu2/cache/index3/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu3/cache/index0/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu3/cache/index1/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu3/cache/index2/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu3/cache/index3/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu4/cache/index0/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu4/cache/index1/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu4/cache/index2/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu4/cache/index3/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu5/cache/index0/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu5/cache/index1/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu5/cache/index2/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu5/cache/index3/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu6/cache/index0/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu6/cache/index1/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu6/cache/index2/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu6/cache/index3/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu7/cache/index0/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu7/cache/index1/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu7/cache/index2/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu7/cache/index3/id | 1 +
 tests/vircaps2xmldata/linux-resctrl/cpus| 1 +
 tests/vircaps2xmldata/linux-resctrl/info/L3/cbm_mask| 1 +
 tests/vircaps2xmldata/linux-resctrl/info/L3/min_cbm_bits| 1 +
 tests/vircaps2xmldata/linux-resctrl/info/L3/num_closids | 1 +
 tests/vircaps2xmldata/linux-resctrl/schemata| 1 +
 tests/vircaps2xmldata/linux-resctrl/tasks   | 2 ++
 38 files changed, 39 insertions(+)
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu0/cache/index0/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu0/cache/index1/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu0/cache/index2/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu0/cache/index3/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu1/cache/index0/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu1/cache/index1/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu1/cache/index2/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu1/cache/index3/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu2/cache/index0/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu2/cache/index1/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu2/cache/index2/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu2/cache/index3/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu3/cache/index0/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu3/cache/index1/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu3/cache/index2/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu3/cache/index3/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu4/cache/index0/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu4/cache/index1/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu4/cache/index2/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu4/cache/index3/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu5/cache/index0/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu5/cache/index1/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu5/cache/index2/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu5/cache/index3/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu6/cache/index0/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu6/cache/index1/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu6/cache/index2/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu6/cache/index3/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu7/cache/ind

[libvirt] [PATCH V2 1/2] util: Add more virsysfs functions for handling resctrl sysfs

2017-03-30 Thread Eli Qiao

Extended /sys/fs/resctrl sysfs handling.

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 src/libvirt_private.syms |  11 -
 src/util/virsysfs.c  | 102 ++-
 src/util/virsysfs.h  |  16 
 src/util/virsysfspriv.h  |   1 +
 4 files changed, 128 insertions(+), 2 deletions(-)

diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms
index b551cb8..b03ad5b 100644
--- a/src/libvirt_private.syms
+++ b/src/libvirt_private.syms
@@ -2388,6 +2388,10 @@ virRandomGenerateWWN;
 virRandomInt;
 
 
+# util/virresctrl.h
+virResCtrlAvailable;
+virResCtrlInit;
+
 # util/virrotatingfile.h
 virRotatingFileReaderConsume;
 virRotatingFileReaderFree;
@@ -2626,13 +2630,18 @@ virSysfsGetCpuValueString;
 virSysfsGetCpuValueUint;
 virSysfsGetNodeValueBitmap;
 virSysfsGetNodeValueString;
+virSysfsGetResctrlInfoString;
+virSysfsGetResctrlInfoUint;
+virSysfsGetResctrlPath;
+virSysfsGetResctrlString;
+virSysfsGetResctrlUint;
 virSysfsGetSystemPath;
 virSysfsGetValueBitmap;
 virSysfsGetValueInt;
 virSysfsGetValueString;
+virSysfsSetResctrlPath;
 virSysfsSetSystemPath;
 
-
 # util/virsysinfo.h
 virSysinfoBaseBoardDefClear;
 virSysinfoBIOSDefFree;
diff --git a/src/util/virsysfs.c b/src/util/virsysfs.c
index 7a98b48..97808be 100644
--- a/src/util/virsysfs.c
+++ b/src/util/virsysfs.c
@@ -36,8 +36,10 @@ VIR_LOG_INIT("util.sysfs");
 
 #define VIR_SYSFS_VALUE_MAXLEN 8192
 #define SYSFS_SYSTEM_PATH "/sys/devices/system"
+#define SYSFS_RESCTRL_PATH "/sys/fs/resctrl"
 
 static const char *sysfs_system_path = SYSFS_SYSTEM_PATH;
+static const char *sysfs_resctrl_path = SYSFS_RESCTRL_PATH;
 
 
 void virSysfsSetSystemPath(const char *path)
@@ -48,13 +50,26 @@ void virSysfsSetSystemPath(const char *path)
 sysfs_system_path = SYSFS_SYSTEM_PATH;
 }
 
-
 const char *
 virSysfsGetSystemPath(void)
 {
 return sysfs_system_path;
 }
 
+void virSysfsSetResctrlPath(const char *path)
+{
+if (path)
+sysfs_resctrl_path  = path;
+else
+sysfs_resctrl_path = SYSFS_RESCTRL_PATH;
+}
+
+const char *
+virSysfsGetResctrlPath(void)
+{
+return sysfs_resctrl_path;
+}
+
 int
 virSysfsGetValueInt(const char *file,
 int *value)
@@ -227,3 +242,88 @@ virSysfsGetNodeValueBitmap(unsigned int node,
 VIR_FREE(path);
 return ret;
 }
+
+int
+virSysfsGetResctrlString(const char* file,
+ char **value)
+{
+char *path = NULL;
+int ret = -1;
+if (virAsprintf(, "%s/%s", sysfs_resctrl_path, file) < 0)
+return -1;
+
+if (!virFileExists(path)) {
+ret = -2;
+goto cleanup;
+}
+
+if (virFileReadAll(path, VIR_SYSFS_VALUE_MAXLEN, value) < 0)
+goto cleanup;
+
+ret = 0;
+
+ cleanup:
+VIR_FREE(path);
+return ret;
+}
+
+int
+virSysfsGetResctrlUint(const char* file,
+   unsigned int *value)
+{
+char *path = NULL;
+int ret = -1;
+if (virAsprintf(, "%s/%s", sysfs_resctrl_path, file) < 0)
+return -1;
+
+if (!virFileExists(path)) {
+ret = -2;
+goto cleanup;
+}
+
+ret = virFileReadValueUint(path, value);
+
+ cleanup:
+VIR_FREE(path);
+return ret;
+}
+
+int
+virSysfsGetResctrlInfoString(const char* file,
+ char **value)
+{
+char *path = NULL;
+int ret = -1;
+if (virAsprintf(, "%s/info/%s", sysfs_resctrl_path, file) < 0)
+return -1;
+
+if (!virFileExists(path)) {
+ret = -2;
+goto cleanup;
+}
+
+if (virFileReadAll(path, VIR_SYSFS_VALUE_MAXLEN, value) < 0)
+goto cleanup;
+
+ret = 0;
+
+ cleanup:
+VIR_FREE(path);
+return ret;
+}
+
+int
+virSysfsGetResctrlInfoUint(const char *file,
+  unsigned int *value)
+{
+char *path = NULL;
+int ret = -1;
+
+if (virAsprintf(, "%s/info/%s", sysfs_resctrl_path, file) < 0)
+return -1;
+
+ret = virFileReadValueUint(path, value);
+
+VIR_FREE(path);
+return ret;
+}
diff --git a/src/util/virsysfs.h b/src/util/virsysfs.h
index cd871ff..af1e786 100644
--- a/src/util/virsysfs.h
+++ b/src/util/virsysfs.h
@@ -25,6 +25,7 @@
 # include "virbitmap.h"
 
 const char * virSysfsGetSystemPath(void);
+const char * virSysfsGetResctrlPath(void);
 
 int
 virSysfsGetValueInt(const char *file,
@@ -67,4 +68,19 @@ virSysfsGetNodeValueBitmap(unsigned int cpu,
const char *file,
virBitmapPtr *value);
 
+int
+virSysfsGetResctrlString(const char* file,
+ char **value);
+
+int
+virSysfsGetResctrlUint(const char* file,
+   unsigned int *value);
+
+int
+virSysfsGetResctrlInfoString(const char* file,
+ char **value);
+
+int
+virSysfsGetResctrlInfoUint(const char *file,
+  unsigned int *value);
 #endif /* __VIR_

Re: [libvirt] [PATCH resend V10 01/12] Resctrl: Add some utils functions

2017-03-29 Thread Eli Qiao



On Wednesday, 29 March 2017 at 5:47 PM, Martin Kletzander wrote:

> On Wed, Mar 29, 2017 at 05:31:42PM +0800, Eli Qiao wrote:
> >  
> >  
> > --
> > Best regards
> > Eli
> >  
> > 天涯无处不重逢
> > a leaf duckweed belongs to the sea, where not to meet in life
> >  
> > Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
>  
> OK, please do something with your client. Having the footer here on top
> for every reply is *so* bothersome when you are replying inline
> (that part is fine).
>  
Sorry, I removed footer, better now?
  
>  
> > On Wednesday, 29 March 2017 at 5:19 PM, Martin Kletzander wrote:
> >  
> > > On Wed, Mar 29, 2017 at 04:22:16PM +0800, Eli Qiao wrote:
> > > >  
> > > >  
> > > > --
> > > > Best regards
> > > > Eli
> > > >  
> > > > 天涯无处不重逢
> > > > a leaf duckweed belongs to the sea, where not to meet in life
> > > >  
> > > > Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
> > > >  
> > > >  
> > > > On Wednesday, 29 March 2017 at 3:45 PM, Martin Kletzander wrote:
> > > >  
> > > > > On Tue, Mar 28, 2017 at 03:22:34PM +0800, Eli Qiao wrote:
> > > > > > hi Martin
> > > > > >  
> > > > > > (cc libvir-list)
> > > > > >  
> > > > > > I am a little confused about cat support.
> > > > > >  
> > > > > > I am currently rebasing my code on top of pre-cat branch from your 
> > > > > > private github repo, today when I check it you have removed it and 
> > > > > > create a cat branch and there are some related code pushed[1], can 
> > > > > > I know what ’s your plan for my patch set for CAT support ? should 
> > > > > > I continue my rebasing work? your though?
> > > > >  
> > > > > So we can work together on that. Since the rework of the sysfs
> > > > > functions, some patches are easier to write from scratch then rewrite,
> > > > > but I'm now just trying to setup the test suite, so that we have
> > > > > something to test on, at least some of the code. So where are you in
> > > > > the rebase right now? Do you think anything from the virsysfs.c code
> > > > > could be enhanced?
> > > > >  
> > > >  
> > > >  
> > > >  
> > > >  
> > > >  
> > > > Not so fast, only the first patch [1], I found that nodeinfo.c is 
> > > > removed :(
> > > >  
> > > > I think we need to extend virResCtrlGetInfoStr and 
> > > > virResCtrlGetInfoUint to virsysfs.c
> > > >  
> > > > thought ?
> > >  
> > > Yeah, we should wrap around /sys/fs/resctrl as we do with
> > > /sys/devices/system so that it can be easily tested.
> > >  
> >  
> > Sure, working on it, and done, will push it for review.
> >  
> > Also I will push some fake data for resctrl testing..
> >  
> >  
> > >  
> > > Also I got another idea about keeping the resource info. There is no
> > > need for any global data to be stored as you are re-reading almost all
> > > of it. The only info that stays the same is caches (that is already
> > > saved in capabilities) and what caches are available for resource
> > > control (that will be there as well). So I don't think we need yet
> > > another global data storage.
> > >  
> >  
> >  
> > Do you mean, we re-create all struct (reading them from /sys/fs/resctrl) 
> > when we create/destroy instance?
> > also, for get free cache ?
> >  
>  
>  
> You have to update that for every request anyway, so what's the point of
> keeping the data when they immediately become old?
>  
I was thought that may reduce the time costing, not all of the content be 
refreshed, anyway, I will try to avoid global files in my later version.  

LoL lots of rebasing  :(  

Thanks for your suggestion.
>  
> > This is what I did in my early PoC, that will much easier… but please keep 
> > in mind that only one thread can read/write to /sys/fs/resctrl at one time.
>  
>  
> Yeah, that's what we have locks for.
>  
> > the neck bottle is /sys/fs/resctrl
>  
> Sure you mean bottleneck, right? :)
yes, bottleneck,  

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATCH] test: Add fake cpu id file

2017-03-29 Thread Eli Qiao

Added fake cpu cache id and resctrl file

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 tests/vircaps2xmldata/linux-caches/cpu/cpu0/cache/index0/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu0/cache/index1/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu0/cache/index2/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu0/cache/index3/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu1/cache/index0/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu1/cache/index1/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu1/cache/index2/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu1/cache/index3/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu2/cache/index0/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu2/cache/index1/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu2/cache/index2/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu2/cache/index3/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu3/cache/index0/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu3/cache/index1/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu3/cache/index2/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu3/cache/index3/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu4/cache/index0/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu4/cache/index1/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu4/cache/index2/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu4/cache/index3/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu5/cache/index0/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu5/cache/index1/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu5/cache/index2/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu5/cache/index3/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu6/cache/index0/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu6/cache/index1/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu6/cache/index2/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu6/cache/index3/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu7/cache/index0/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu7/cache/index1/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu7/cache/index2/id | 1 +
 tests/vircaps2xmldata/linux-caches/cpu/cpu7/cache/index3/id | 1 +
 tests/vircaps2xmldata/linux-resctrl/cpus| 1 +
 tests/vircaps2xmldata/linux-resctrl/info/L3/cbm_mask| 1 +
 tests/vircaps2xmldata/linux-resctrl/info/L3/min_cbm_bits| 1 +
 tests/vircaps2xmldata/linux-resctrl/info/L3/num_closids | 1 +
 tests/vircaps2xmldata/linux-resctrl/schemata| 1 +
 tests/vircaps2xmldata/linux-resctrl/tasks   | 2 ++
 38 files changed, 39 insertions(+)
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu0/cache/index0/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu0/cache/index1/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu0/cache/index2/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu0/cache/index3/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu1/cache/index0/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu1/cache/index1/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu1/cache/index2/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu1/cache/index3/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu2/cache/index0/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu2/cache/index1/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu2/cache/index2/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu2/cache/index3/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu3/cache/index0/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu3/cache/index1/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu3/cache/index2/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu3/cache/index3/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu4/cache/index0/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu4/cache/index1/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu4/cache/index2/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu4/cache/index3/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu5/cache/index0/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu5/cache/index1/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu5/cache/index2/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu5/cache/index3/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu6/cache/index0/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu6/cache/index1/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu6/cache/index2/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu6/cache/index3/id
 create mode 100644 tests/vircaps2xmldata/linux-caches/cpu/cpu7/cache/ind

[libvirt] [PATCH] util: Add more virsysfs functions for handling resctrl sysfs

2017-03-29 Thread Eli Qiao

Extended /sys/fs/resctrl sysfs handling.

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 src/libvirt_private.syms |  4 ++
 src/util/virsysfs.c  | 98 +++-
 src/util/virsysfs.h  | 15 
 3 files changed, 116 insertions(+), 1 deletion(-)

diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms
index b551cb8..e07ae79 100644
--- a/src/libvirt_private.syms
+++ b/src/libvirt_private.syms
@@ -2631,6 +2631,10 @@ virSysfsGetValueBitmap;
 virSysfsGetValueInt;
 virSysfsGetValueString;
 virSysfsSetSystemPath;
+virSysfsGetResctrlString;
+virSysfsGetResctrlUint;
+virSysfsGetResctrlInfoString;
+virSysfsGetResCtrInfoUint;
 
 
 # util/virsysinfo.h
diff --git a/src/util/virsysfs.c b/src/util/virsysfs.c
index 7a98b48..d1be289 100644
--- a/src/util/virsysfs.c
+++ b/src/util/virsysfs.c
@@ -36,8 +36,10 @@ VIR_LOG_INIT("util.sysfs");
 
 #define VIR_SYSFS_VALUE_MAXLEN 8192
 #define SYSFS_SYSTEM_PATH "/sys/devices/system"
+#define SYSFS_RESCTRL_PATH "/sys/fs/resctrl"
 
 static const char *sysfs_system_path = SYSFS_SYSTEM_PATH;
+static const char *sysfs_resctrl_path = SYSFS_RESCTRL_PATH;
 
 
 void virSysfsSetSystemPath(const char *path)
@@ -48,13 +50,26 @@ void virSysfsSetSystemPath(const char *path)
 sysfs_system_path = SYSFS_SYSTEM_PATH;
 }
 
-
 const char *
 virSysfsGetSystemPath(void)
 {
 return sysfs_system_path;
 }
 
+void virSysfsSetResctrlPath(const char *path)
+{
+if (path)
+sysfs_resctrl_path  = path;
+else
+sysfs_resctrl_path = SYSFS_RESCTRL_PATH;
+}
+
+const char *
+virSysfsGetResctrlPath(void)
+{
+return sysfs_resctrl_path;
+}
+
 int
 virSysfsGetValueInt(const char *file,
 int *value)
@@ -227,3 +242,84 @@ virSysfsGetNodeValueBitmap(unsigned int node,
 VIR_FREE(path);
 return ret;
 }
+
+int
+virSysfsGetResctrlString(const char* file,
+ char **value)
+{
+chat *path = NULL;
+int ret = -1;
+if (virAsprintf(, "%s/%s", sysfs_resctrl_path, file) < 0)
+if (!virFileExists(path)) {
+ret = -2;
+goto cleanup;
+}
+
+if (virFileReadAll(path, VIR_SYSFS_VALUE_MAXLEN, value) < 0)
+goto cleanup;
+
+ret = 0;
+
+ cleanup:
+VIR_FREE(path);
+return ret;
+}
+
+int
+virSysfsGetResctrlUint(const char* file,
+   unsigned int **value)
+{
+chat *path = NULL;
+int ret = -1;
+if (virAsprintf(, "%s/%s", sysfs_resctrl_path, file) < 0)
+if (!virFileExists(path)) {
+ret = -2;
+goto cleanup;
+}
+
+ret = virFileReadValueUint(path, value);
+
+ cleanup:
+VIR_FREE(path);
+return ret;
+}
+
+int
+virSysfsGetResctrlInfoString(const char* file,
+ char **value)
+{
+chat *path = NULL;
+int ret = -1;
+if (virAsprintf(, "%s/info/%s", sysfs_resctrl_path, file) < 0)
+return -1;
+
+if (!virFileExists(path)) {
+ret = -2;
+goto cleanup;
+}
+
+if (virFileReadAll(path, VIR_SYSFS_VALUE_MAXLEN, value) < 0)
+goto cleanup;
+
+ret = 0;
+
+ cleanup:
+VIR_FREE(path);
+return ret;
+}
+
+int
+virSysfsGetResCtrInfoUint(const char *file,
+  unsigned int *value)
+{
+char *path = NULL;
+int ret = -1;
+
+if (virAsprintf(, "%s/info/%s", sysfs_resctrl_path, file) < 0)
+return -1;
+
+ret = virFileReadValueUint(path, value);
+
+VIR_FREE(path);
+return ret;
+}
diff --git a/src/util/virsysfs.h b/src/util/virsysfs.h
index cd871ff..81a27c2 100644
--- a/src/util/virsysfs.h
+++ b/src/util/virsysfs.h
@@ -67,4 +67,19 @@ virSysfsGetNodeValueBitmap(unsigned int cpu,
const char *file,
virBitmapPtr *value);
 
+int
+virSysfsGetResctrlString(const char* file,
+ char **value);
+
+int
+virSysfsGetResctrlUint(const char* file,
+   unsigned int **value);
+
+int
+virSysfsGetResctrlInfoString(const char* file,
+ char **value);
+
+int
+virSysfsGetResCtrInfoUint(const char *file,
+  unsigned int *value);
 #endif /* __VIR_SYSFS_H__*/
-- 
1.9.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH resend V10 01/12] Resctrl: Add some utils functions

2017-03-29 Thread Eli Qiao



--  
Best regards  
Eli

天涯无处不重逢
a leaf duckweed belongs to the sea, where not to meet in life  

Sent with Sparrow (http://www.sparrowmailapp.com/?sig)


On Wednesday, 29 March 2017 at 5:19 PM, Martin Kletzander wrote:

> On Wed, Mar 29, 2017 at 04:22:16PM +0800, Eli Qiao wrote:
> >  
> >  
> > --
> > Best regards
> > Eli
> >  
> > 天涯无处不重逢
> > a leaf duckweed belongs to the sea, where not to meet in life
> >  
> > Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
> >  
> >  
> > On Wednesday, 29 March 2017 at 3:45 PM, Martin Kletzander wrote:
> >  
> > > On Tue, Mar 28, 2017 at 03:22:34PM +0800, Eli Qiao wrote:
> > > > hi Martin
> > > >  
> > > > (cc libvir-list)
> > > >  
> > > > I am a little confused about cat support.
> > > >  
> > > > I am currently rebasing my code on top of pre-cat branch from your 
> > > > private github repo, today when I check it you have removed it and 
> > > > create a cat branch and there are some related code pushed[1], can I 
> > > > know what ’s your plan for my patch set for CAT support ? should I 
> > > > continue my rebasing work? your though?
> > >  
> > > So we can work together on that. Since the rework of the sysfs
> > > functions, some patches are easier to write from scratch then rewrite,
> > > but I'm now just trying to setup the test suite, so that we have
> > > something to test on, at least some of the code. So where are you in
> > > the rebase right now? Do you think anything from the virsysfs.c code
> > > could be enhanced?
> > >  
> >  
> >  
> >  
> >  
> > Not so fast, only the first patch [1], I found that nodeinfo.c is removed :(
> >  
> > I think we need to extend virResCtrlGetInfoStr and virResCtrlGetInfoUint to 
> > virsysfs.c
> >  
> > thought ?
>  
> Yeah, we should wrap around /sys/fs/resctrl as we do with
> /sys/devices/system so that it can be easily tested.
>  
Sure, working on it, and done, will push it for review.

Also I will push some fake data for resctrl testing..
  
  
>  
> Also I got another idea about keeping the resource info. There is no
> need for any global data to be stored as you are re-reading almost all
> of it. The only info that stays the same is caches (that is already
> saved in capabilities) and what caches are available for resource
> control (that will be there as well). So I don't think we need yet
> another global data storage.
>  
>  

Do you mean, we re-create all struct (reading them from /sys/fs/resctrl) when 
we create/destroy instance?  
also, for get free cache ?

This is what I did in my early PoC, that will much easier… but please keep in 
mind that only one thread can read/write to /sys/fs/resctrl at one time.

the neck bottle is /sys/fs/resctrl
  
>   
>  
>  


--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH resend V10 01/12] Resctrl: Add some utils functions

2017-03-29 Thread Eli Qiao



--  
Best regards  
Eli

天涯无处不重逢
a leaf duckweed belongs to the sea, where not to meet in life  

Sent with Sparrow (http://www.sparrowmailapp.com/?sig)


On Wednesday, 29 March 2017 at 3:45 PM, Martin Kletzander wrote:

> On Tue, Mar 28, 2017 at 03:22:34PM +0800, Eli Qiao wrote:
> > hi Martin
> >  
> > (cc libvir-list)
> >  
> > I am a little confused about cat support.
> >  
> > I am currently rebasing my code on top of pre-cat branch from your private 
> > github repo, today when I check it you have removed it and create a cat 
> > branch and there are some related code pushed[1], can I know what ’s your 
> > plan for my patch set for CAT support ? should I continue my rebasing work? 
> > your though?
>  
> So we can work together on that. Since the rework of the sysfs
> functions, some patches are easier to write from scratch then rewrite,
> but I'm now just trying to setup the test suite, so that we have
> something to test on, at least some of the code. So where are you in
> the rebase right now? Do you think anything from the virsysfs.c code
> could be enhanced?
>  
>  



Not so fast, only the first patch [1], I found that nodeinfo.c is removed :(  

I think we need to extend virResCtrlGetInfoStr and virResCtrlGetInfoUint to 
virsysfs.c

thought ?

[1]https://github.com/taget/libvirt/commits/cdp_v11
  
>  
> > [1] 
> > https://github.com/nertpinx/libvirt/commit/c335de47a4efeca87f23e641a93587b1e036e558
> >  
> > Thanks Eli.
> >  
> >  
> >  
> > --
> > Best regards
> > Eli
> >  
> > 天涯无处不重逢
> > a leaf duckweed belongs to the sea, where not to meet in life
> >  
> > Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
> >  
> >  
> > On Friday, 24 March 2017 at 3:42 PM, Martin Kletzander wrote:
> >  
> > > On Fri, Mar 24, 2017 at 09:35:33AM +0800, Eli Qiao wrote:
> > > >  
> > > >  
> > > > --
> > > > Best regards
> > > > Eli
> > > >  
> > > > 天涯无处不重逢
> > > > a leaf duckweed belongs to the sea, where not to meet in life
> > > >  
> > > > Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
> > > >  
> > > >  
> > > > On Thursday, 16 March 2017 at 3:52 PM, Eli Qiao wrote:
> > > >  
> > > > >  
> > > > >  
> > > > > --
> > > > > Best regards
> > > > > Eli
> > > > >  
> > > > > 天涯无处不重逢
> > > > > a leaf duckweed belongs to the sea, where not to meet in life
> > > > >  
> > > > > Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
> > > > >  
> > > > >  
> > > > > On Wednesday, 15 March 2017 at 7:57 PM, Martin Kletzander wrote:
> > > > >  
> > > > > > On Mon, Mar 06, 2017 at 06:06:30PM +0800, Eli Qiao wrote:
> > > > > > > This patch adds some utils struct and functions to expose resctrl
> > > > > > > information.
> > > > > > >  
> > > > > > > virResCtrlAvailable: if resctrl interface exist on host.
> > > > > > > virResCtrlGet: get specific type resource control information.
> > > > > > > virResCtrlInit: initialize resctrl struct from the host's sys fs.
> > > > > > > resctrlall[]: an array to maintain resource control information.
> > > > > > >  
> > > > > > > Some of host cpu related information methods was added in 
> > > > > > > virhostcpu.c
> > > > > >  
> > > > > > So to be able to test all this we need to make a bit different 
> > > > > > approach.
> > > > > > I'm not in favour of pushing this without proper tests. Some paths 
> > > > > > need
> > > > > > to be configurable, some readings should be unified. Unfortunately 
> > > > > > lot
> > > > > > of the code is just copy-paste mess from the past. Fortunately for 
> > > > > > you,
> > > > > >  
> > > > > >  
> > > > > >  
> > > > > > I'm working on cleaning this up, at least a little bit, so that we 
> > > > > > can
> > > > >  
> > > > > Good news.
> > > > > > add the tests easily. I got almost up to the test (I stumbled upon 
> > > > > > few
> > > > > > rabbit holes on the way a

Re: [libvirt] [PATCH resend V10 01/12] Resctrl: Add some utils functions

2017-03-28 Thread Eli Qiao

hi Martin

(cc libvir-list)
  
I am a little confused about cat support.  

I am currently rebasing my code on top of pre-cat branch from your private 
github repo, today when I check it you have removed it and create a cat branch 
and there are some related code pushed[1], can I know what ’s your plan for my 
patch set for CAT support ? should I continue my rebasing work? your though?

[1] 
https://github.com/nertpinx/libvirt/commit/c335de47a4efeca87f23e641a93587b1e036e558

Thanks Eli.



--  
Best regards  
Eli

天涯无处不重逢
a leaf duckweed belongs to the sea, where not to meet in life  

Sent with Sparrow (http://www.sparrowmailapp.com/?sig)


On Friday, 24 March 2017 at 3:42 PM, Martin Kletzander wrote:

> On Fri, Mar 24, 2017 at 09:35:33AM +0800, Eli Qiao wrote:
> >  
> >  
> > --
> > Best regards
> > Eli
> >  
> > 天涯无处不重逢
> > a leaf duckweed belongs to the sea, where not to meet in life
> >  
> > Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
> >  
> >  
> > On Thursday, 16 March 2017 at 3:52 PM, Eli Qiao wrote:
> >  
> > >  
> > >  
> > > --
> > > Best regards
> > > Eli
> > >  
> > > 天涯无处不重逢
> > > a leaf duckweed belongs to the sea, where not to meet in life
> > >  
> > > Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
> > >  
> > >  
> > > On Wednesday, 15 March 2017 at 7:57 PM, Martin Kletzander wrote:
> > >  
> > > > On Mon, Mar 06, 2017 at 06:06:30PM +0800, Eli Qiao wrote:
> > > > > This patch adds some utils struct and functions to expose resctrl
> > > > > information.
> > > > >  
> > > > > virResCtrlAvailable: if resctrl interface exist on host.
> > > > > virResCtrlGet: get specific type resource control information.
> > > > > virResCtrlInit: initialize resctrl struct from the host's sys fs.
> > > > > resctrlall[]: an array to maintain resource control information.
> > > > >  
> > > > > Some of host cpu related information methods was added in virhostcpu.c
> > > >  
> > > > So to be able to test all this we need to make a bit different approach.
> > > > I'm not in favour of pushing this without proper tests. Some paths need
> > > > to be configurable, some readings should be unified. Unfortunately lot
> > > > of the code is just copy-paste mess from the past. Fortunately for you,
> > > >  
> > > >  
> > > >  
> > > > I'm working on cleaning this up, at least a little bit, so that we can
> > >  
> > > Good news.
> > > > add the tests easily. I got almost up to the test (I stumbled upon few
> > > > rabbit holes on the way and some clean-ups went wrong along the way), so
> > > > it should be pretty easy for you to modify this code to use what I'll be
> > > > proposing to add. It's not ready yet, but you can start rebasing your
> > > > series on top of my branch pre-cat from my github repo [1]. The commits
> > > > are not very well described right now (for some temporary ones I used
> > > > whatthecommit.com (http://whatthecommit.com), sorry), but I'll fix all 
> > > > that. I'll be updating the
> > > > branch, but it will be done with force pushes, so be careful when
> > > > rebasing on top of newer versions.
> > > >  
> > > > I can do that if you don't want, just let me know so we can coordinate.
> > > of cause we can do some coordinate, but I am glad that you can help on 
> > > this to speed up the progress to merge them, as you know this patch is in 
> > > V10 already, and it has 12 patch set, kinds of hard to doing rebase… :(
> > >  
> > >  
> > > > Just a quick heads-up, there will be virsysfs that will be used for the
> > > > reads, some additional helper functions in virhostcpu and virfile, test
> > > > that scans copy of /sys/devices/system (with that path faked thanks to
> > > > using the aforementioned virsysfs) and outputs capabilities so that we
> > > > can check the capability XML and so on.
> > > >  
> > >  
> > >  
> > >  
> > > Ah, that’s a good news..
> > > >  
> > > > Martin
> > > >  
> > > > [1] https://github.com/nertpinx/libvirt
> >  
> > hi Martin
> >  
> > So, if I understand you correctly , you want all my patch set to rebased on 
> > top of pre-cat branch [1] , I checked that the last commit is 15th March, I 
> > wonder if that ’s ready to merged into master?
> > so that I can start doing the rebasing
> >  
>  
>  
> I forgot to do the usual push, it's updated now. The test fails in one
> occasion, but it's the code's fault, the test is fine. That's the last
> thing I'm looking at now, after that I'll send it to the list.
>  
> Look at the changes and see what you can use, it will help simplifying
> your code a lot, I thing. You can start rebasing on top of that, I'll
> do that as well after it's posted and I'll be either using and modifying
> your patches or maybe doing some myself.
>  
> Martin  

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH resend V10 01/12] Resctrl: Add some utils functions

2017-03-23 Thread Eli Qiao



--  
Best regards  
Eli

天涯无处不重逢
a leaf duckweed belongs to the sea, where not to meet in life  

Sent with Sparrow (http://www.sparrowmailapp.com/?sig)


On Thursday, 16 March 2017 at 3:52 PM, Eli Qiao wrote:

>  
>  
> --  
> Best regards  
> Eli
>  
> 天涯无处不重逢
> a leaf duckweed belongs to the sea, where not to meet in life  
>  
> Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
>  
>  
> On Wednesday, 15 March 2017 at 7:57 PM, Martin Kletzander wrote:
>  
> > On Mon, Mar 06, 2017 at 06:06:30PM +0800, Eli Qiao wrote:
> > > This patch adds some utils struct and functions to expose resctrl
> > > information.
> > >  
> > > virResCtrlAvailable: if resctrl interface exist on host.
> > > virResCtrlGet: get specific type resource control information.
> > > virResCtrlInit: initialize resctrl struct from the host's sys fs.
> > > resctrlall[]: an array to maintain resource control information.
> > >  
> > > Some of host cpu related information methods was added in virhostcpu.c
> >  
> > So to be able to test all this we need to make a bit different approach.
> > I'm not in favour of pushing this without proper tests. Some paths need
> > to be configurable, some readings should be unified. Unfortunately lot
> > of the code is just copy-paste mess from the past. Fortunately for you,  
> >  
> >  
> >  
> > I'm working on cleaning this up, at least a little bit, so that we can
> >  
> >  
> >  
>  
> Good news.  
> > add the tests easily. I got almost up to the test (I stumbled upon few
> > rabbit holes on the way and some clean-ups went wrong along the way), so
> > it should be pretty easy for you to modify this code to use what I'll be
> > proposing to add. It's not ready yet, but you can start rebasing your
> > series on top of my branch pre-cat from my github repo [1]. The commits
> > are not very well described right now (for some temporary ones I used
> > whatthecommit.com (http://whatthecommit.com), sorry), but I'll fix all 
> > that. I'll be updating the
> > branch, but it will be done with force pushes, so be careful when
> > rebasing on top of newer versions.
> >  
> > I can do that if you don't want, just let me know so we can coordinate.
> >  
> of cause we can do some coordinate, but I am glad that you can help on this 
> to speed up the progress to merge them, as you know this patch is in V10 
> already, and it has 12 patch set, kinds of hard to doing rebase… :(
>  
>   
> > Just a quick heads-up, there will be virsysfs that will be used for the
> > reads, some additional helper functions in virhostcpu and virfile, test
> > that scans copy of /sys/devices/system (with that path faked thanks to
> > using the aforementioned virsysfs) and outputs capabilities so that we
> > can check the capability XML and so on.
> >  
> >  
> >  
>  
>  
> Ah, that’s a good news..  
> >  
> > Martin
> >  
> > [1] https://github.com/nertpinx/libvirt

hi Martin

So, if I understand you correctly , you want all my patch set to rebased on top 
of pre-cat branch [1] , I checked that the last commit is 15th March, I wonder 
if that ’s ready to merged into master?
so that I can start doing the rebasing

Thx Eli.  
> >  
> > > Signed-off-by: Eli Qiao <liyong.q...@intel.com 
> > > (mailto:liyong.q...@intel.com)>
> > > ---
> > > include/libvirt/virterror.h | 1 +
> > > po/POTFILES.in (http://POTFILES.in) | 1 +
> > > src/Makefile.am (http://Makefile.am) | 1 +
> > > src/libvirt_private.syms | 4 +
> > > src/util/virerror.c | 1 +
> > > src/util/virhostcpu.c | 186 
> > > src/util/virhostcpu.h | 6 ++
> > > src/util/virresctrl.c | 201 
> > > src/util/virresctrl.h | 78 +
> > > 9 files changed, 462 insertions(+), 17 deletions(-)
> > > create mode 100644 src/util/virresctrl.c
> > > create mode 100644 src/util/virresctrl.h
> > >  
> > > diff --git a/include/libvirt/virterror.h b/include/libvirt/virterror.h
> > > index 2efee8f..3dd2d08 100644
> > > --- a/include/libvirt/virterror.h
> > > +++ b/include/libvirt/virterror.h
> > > @@ -132,6 +132,7 @@ typedef enum {
> > >  
> > > VIR_FROM_PERF = 65, /* Error from perf */
> > > VIR_FROM_LIBSSH = 66, /* Error from libssh connection transport */
> > > + VIR_FROM_RESCTRL = 67, /* Error from resource control */
> > >  
> > > # ifdef VIR_ENUM_SENTINELS
> >

Re: [libvirt] [PATCH resend V10 02/12] Resctrl: expose cache information to capabilities

2017-03-16 Thread Eli Qiao



--  
Best regards  
Eli

天涯无处不重逢
a leaf duckweed belongs to the sea, where not to meet in life  

Sent with Sparrow (http://www.sparrowmailapp.com/?sig)


On Wednesday, 15 March 2017 at 8:53 PM, Martin Kletzander wrote:

> On Mon, Mar 06, 2017 at 06:06:31PM +0800, Eli Qiao wrote:
> > This patch expose cache information to host's capabilites xml.
> >  
> > For l3 cache allocation
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> >  
> > For l3 cache allocation supported cdp(seperate data/code):
> > 
> > 
> > 
> > 
> >  
>  
>  
> I know this was discussed before, but why having a vector value in a
> single attribute? Why don't we split it to two scalars? It will also
> be SOO much more readable and close to what other tools expose, e.g.:
>  
> 
> 
>  
> or
>  
> 
>  
> I also skipped the 'l3' because that is already in the  and will
> never be different, right?
>  

 I am okay to ether of them, just follow with previous discussion.
  
>  
> > 
> > 
> > 
> > 
> > 
> > 
> >  
> > RFC on mailing list.
> > https://www.redhat.com/archives/libvir-list/2017-January/msg00644.html
> >  
> > Signed-off-by: Eli Qiao <liyong.q...@intel.com 
> > (mailto:liyong.q...@intel.com)>
> > ---
> > src/conf/capabilities.c | 56 ++
> > src/conf/capabilities.h | 23 
> > src/libvirt_private.syms | 3 +++
> > src/nodeinfo.c | 64 
> > src/nodeinfo.h | 1 +
> > src/qemu/qemu_capabilities.c | 8 ++
> > src/qemu/qemu_driver.c | 4 +++
> > 7 files changed, 159 insertions(+)
> >  
> > diff --git a/src/conf/capabilities.c b/src/conf/capabilities.c
> > index 9ab343b..23e21e4 100644
> > --- a/src/conf/capabilities.c
> > +++ b/src/conf/capabilities.c
> > @@ -198,6 +198,18 @@ virCapabilitiesClearSecModel(virCapsHostSecModelPtr 
> > secmodel)
> > }
> >  
> > static void
> > +virCapabilitiesClearCacheBank(virCapsHostCacheBankPtr cachebank)
> > +{
> > + size_t i;
> > + for (i = 0; i < cachebank->ncontrol; i++)
> > + VIR_FREE(cachebank->control[i].scope);
> > +
> > + VIR_FREE(cachebank->type);
> > + VIR_FREE(cachebank->cpus);
> > +}
> > +
> > +
> > +static void
> > virCapabilitiesDispose(void *object)
> > {
> > virCapsPtr caps = object;
> > @@ -221,6 +233,10 @@ virCapabilitiesDispose(void *object)
> > virCapabilitiesClearSecModel(>host.secModels[i]);
> > VIR_FREE(caps->host.secModels);
> >  
> > + for (i = 0; i < caps->host.ncachebank; i++)
> > + virCapabilitiesClearCacheBank(caps->host.cachebank[i]);
> > + VIR_FREE(caps->host.cachebank);
> > +
> > VIR_FREE(caps->host.netprefix);
> > VIR_FREE(caps->host.pagesSize);
> > virCPUDefFree(caps->host.cpu);
> > @@ -844,6 +860,41 @@ virCapabilitiesFormatNUMATopology(virBufferPtr buf,
> > return 0;
> > }
> >  
> > +static int
> > +virCapabilitiesFormatCache(virBufferPtr buf,
> > + size_t ncachebank,
> > + virCapsHostCacheBankPtr *cachebank)
> > +{
> > + size_t i;
> > + size_t j;
> > +
> > + virBufferAddLit(buf, "\n");
> > + virBufferAdjustIndent(buf, 2);
> > +
> > + for (i = 0; i < ncachebank; i++) {
> > + virBufferAsprintf(buf,
> > + "\n",
> > + cachebank[i]->id,
> > + cachebank[i]->type,
> > + cachebank[i]->size,
> > + cachebank[i]->cpus);
> > +
> > + virBufferAdjustIndent(buf, 2);
> > + for (j = 0; j < cachebank[i]->ncontrol; j++) {
> > + virBufferAsprintf(buf,
> > + "\n",
> > + cachebank[i]->control[j].min,
> > + cachebank[i]->control[j].reserved,
> > + cachebank[i]->control[j].scope);
> > + }
> > + virBufferAdjustIndent(buf, -2);
> > + virBufferAddLit(buf, "\n");
> > + }
> > + virBufferAdjustIndent(buf, -2);
> > + virBufferAddLit(buf, "\n");
> > + return 0;
> > +}
> > +
> > /**
> > * virCapabilitiesFormatXML:
> > * @caps: capabilities to format
> > @@ -931,6 +982,11 @@ virCapabilitiesFormatXML(virCapsPtr caps)
> > virBufferAddLit(, "\n");
> > }
> >  
> > + if (caps->host.ncachebank &&
> > + virCapabilitiesFormatCache(, caps->host.ncachebank,
> > + caps->host.cachebank) < 0)
> > + return NULL;
> > +
> > if (caps->host.netprefix)
> > virBufferAsprintf(,

Re: [libvirt] [PATCH resend V10 01/12] Resctrl: Add some utils functions

2017-03-16 Thread Eli Qiao



--  
Best regards  
Eli

天涯无处不重逢
a leaf duckweed belongs to the sea, where not to meet in life  

Sent with Sparrow (http://www.sparrowmailapp.com/?sig)


On Wednesday, 15 March 2017 at 7:57 PM, Martin Kletzander wrote:

> On Mon, Mar 06, 2017 at 06:06:30PM +0800, Eli Qiao wrote:
> > This patch adds some utils struct and functions to expose resctrl
> > information.
> >  
> > virResCtrlAvailable: if resctrl interface exist on host.
> > virResCtrlGet: get specific type resource control information.
> > virResCtrlInit: initialize resctrl struct from the host's sys fs.
> > resctrlall[]: an array to maintain resource control information.
> >  
> > Some of host cpu related information methods was added in virhostcpu.c
>  
> So to be able to test all this we need to make a bit different approach.
> I'm not in favour of pushing this without proper tests. Some paths need
> to be configurable, some readings should be unified. Unfortunately lot
> of the code is just copy-paste mess from the past. Fortunately for you,  
>  
>  

> I'm working on cleaning this up, at least a little bit, so that we can
>  
>  

Good news.  
> add the tests easily. I got almost up to the test (I stumbled upon few
> rabbit holes on the way and some clean-ups went wrong along the way), so
> it should be pretty easy for you to modify this code to use what I'll be
> proposing to add. It's not ready yet, but you can start rebasing your
> series on top of my branch pre-cat from my github repo [1]. The commits
> are not very well described right now (for some temporary ones I used
> whatthecommit.com (http://whatthecommit.com), sorry), but I'll fix all that. 
> I'll be updating the
> branch, but it will be done with force pushes, so be careful when
> rebasing on top of newer versions.
>  
> I can do that if you don't want, just let me know so we can coordinate.
>  
of cause we can do some coordinate, but I am glad that you can help on this to 
speed up the progress to merge them, as you know this patch is in V10 already, 
and it has 12 patch set, kinds of hard to doing rebase… :(

  
>  
> Just a quick heads-up, there will be virsysfs that will be used for the
> reads, some additional helper functions in virhostcpu and virfile, test
> that scans copy of /sys/devices/system (with that path faked thanks to
> using the aforementioned virsysfs) and outputs capabilities so that we
> can check the capability XML and so on.
>  
>  


Ah, that’s a good news..  
>  
> Martin
>  
> [1] https://github.com/nertpinx/libvirt
>  
> > Signed-off-by: Eli Qiao <liyong.q...@intel.com 
> > (mailto:liyong.q...@intel.com)>
> > ---
> > include/libvirt/virterror.h | 1 +
> > po/POTFILES.in (http://POTFILES.in) | 1 +
> > src/Makefile.am (http://Makefile.am) | 1 +
> > src/libvirt_private.syms | 4 +
> > src/util/virerror.c | 1 +
> > src/util/virhostcpu.c | 186 
> > src/util/virhostcpu.h | 6 ++
> > src/util/virresctrl.c | 201 
> > src/util/virresctrl.h | 78 +
> > 9 files changed, 462 insertions(+), 17 deletions(-)
> > create mode 100644 src/util/virresctrl.c
> > create mode 100644 src/util/virresctrl.h
> >  
> > diff --git a/include/libvirt/virterror.h b/include/libvirt/virterror.h
> > index 2efee8f..3dd2d08 100644
> > --- a/include/libvirt/virterror.h
> > +++ b/include/libvirt/virterror.h
> > @@ -132,6 +132,7 @@ typedef enum {
> >  
> > VIR_FROM_PERF = 65, /* Error from perf */
> > VIR_FROM_LIBSSH = 66, /* Error from libssh connection transport */
> > + VIR_FROM_RESCTRL = 67, /* Error from resource control */
> >  
> > # ifdef VIR_ENUM_SENTINELS
> > VIR_ERR_DOMAIN_LAST
> > diff --git a/po/POTFILES.in (http://POTFILES.in) b/po/POTFILES.in 
> > (http://POTFILES.in)
> > index 7c7f530..4147bc6 100644
> > --- a/po/POTFILES.in (http://POTFILES.in)
> > +++ b/po/POTFILES.in (http://POTFILES.in)
> > @@ -241,6 +241,7 @@ src/util/virportallocator.c
> > src/util/virprocess.c
> > src/util/virqemu.c
> > src/util/virrandom.c
> > +src/util/virresctrl.c
> > src/util/virrotatingfile.c
> > src/util/virscsi.c
> > src/util/virscsihost.c
> > diff --git a/src/Makefile.am (http://Makefile.am) b/src/Makefile.am 
> > (http://Makefile.am)
> > index 7d42eac..edb946a 100644
> > --- a/src/Makefile.am (http://Makefile.am)
> > +++ b/src/Makefile.am (http://Makefile.am)
> > @@ -162,6 +162,7 @@ UTIL_SOURCES = \
> > util/virprocess.c util/virprocess.h \
> > util/virqemu.c util/virqemu.h \
> > util/virrandom.h util/virrandom.c \
> &

Re: [libvirt] [PATCH resend V10 01/12] Resctrl: Add some utils functions

2017-03-16 Thread Eli Qiao



--  
Best regards  
Eli

天涯无处不重逢
a leaf duckweed belongs to the sea, where not to meet in life  

Sent with Sparrow (http://www.sparrowmailapp.com/?sig)


On Wednesday, 15 March 2017 at 8:20 PM, Martin Kletzander wrote:

> On Mon, Mar 06, 2017 at 06:06:30PM +0800, Eli Qiao wrote:
> > This patch adds some utils struct and functions to expose resctrl
> > information.
> >  
> > virResCtrlAvailable: if resctrl interface exist on host.
> > virResCtrlGet: get specific type resource control information.
> > virResCtrlInit: initialize resctrl struct from the host's sys fs.
> > resctrlall[]: an array to maintain resource control information.
> >  
> > Some of host cpu related information methods was added in virhostcpu.c
> >  
> > Signed-off-by: Eli Qiao <liyong.q...@intel.com 
> > (mailto:liyong.q...@intel.com)>
> > ---
> > include/libvirt/virterror.h | 1 +
> > po/POTFILES.in (http://POTFILES.in) | 1 +
> > src/Makefile.am (http://Makefile.am) | 1 +
> > src/libvirt_private.syms | 4 +
> > src/util/virerror.c | 1 +
> > src/util/virhostcpu.c | 186 
> > src/util/virhostcpu.h | 6 ++
> > src/util/virresctrl.c | 201 
> > src/util/virresctrl.h | 78 +
> > 9 files changed, 462 insertions(+), 17 deletions(-)
> > create mode 100644 src/util/virresctrl.c
> > create mode 100644 src/util/virresctrl.h
> >  
> > diff --git a/src/util/virerror.c b/src/util/virerror.c
> > index ef17fb5..0ba15e6 100644
> > --- a/src/util/virerror.c
> > +++ b/src/util/virerror.c
> > @@ -139,6 +139,7 @@ VIR_ENUM_IMPL(virErrorDomain, VIR_ERR_DOMAIN_LAST,
> >  
> > "Perf", /* 65 */
> > "Libssh transport layer",
> > + "Resouce Control",
> >  
>  
>  
> s/resouce/resource/
>  
> > diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c
> > new file mode 100644
> > index 000..44a47cc
> > --- /dev/null
> > +++ b/src/util/virresctrl.c
> > @@ -0,0 +1,201 @@
> >  
>  
>  
> [...]
>  
> > +
> > +static unsigned int host_id;
> > +
> > +static virResCtrl resctrlall[] = {
> > + {
> > + .name = "L3",
> > + .cache_level = "l3",
> > + },
> > + {
> > + .name = "L3DATA",
> > + .cache_level = "l3",
> > + },
> > + {
> > + .name = "L3CODE",
> > + .cache_level = "l3",
> > + },
> > + {
> > + .name = "L2",
> > + .cache_level = "l2",
> > + },
> > +};
> > +
> >  
>  
>  
> You are using global variables, still. But I *still* see no locking.
> What if yet another driver (not just QEMU) will want to use resctrl?
> Bunch of these accesses can happen at the same time and break
> everything. How much of this information do we really need to keep (and
> not reload)?
>  
Yes, we need to maintain a global one as /sys/fs/resctrl is a global one.

most of these information are in-mutble and don’t need to reload.
  
>  
> For example host_id can screw up a lot of things. I might be discussing
> in the latter patches as well.
>  
>  

yep. I see them.  
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATH V10 12/12] Resctrl: Add nodecachestats

2017-03-06 Thread Eli Qiao



--  
Best regards  
Eli

天涯无处不重逢
a leaf duckweed belongs to the sea, where not to meet in life  

Sent with Sparrow (http://www.sparrowmailapp.com/?sig)


On Tuesday, 7 March 2017 at 5:33 AM, Marcelo Tosatti wrote:

> On Mon, Mar 06, 2017 at 05:50:43PM +0800, Eli Qiao wrote:
> > Add new virsh command line `nodecachestats` to expose the cache usage
> > on a node.
> >  
> > Signed-off-by: Eli Qiao <liyong.q...@intel.com 
> > (mailto:liyong.q...@intel.com)>
> > ---
> > src/libvirt_private.syms | 3 ++-
> > src/qemu/qemu_driver.c | 12 ++
> > src/util/virresctrl.c | 62 
> > src/util/virresctrl.h | 8 +++
> > tools/virsh-host.c | 49 ++
> > 5 files changed, 133 insertions(+), 1 deletion(-)
> >  
>  
>  
> It perhaps would be good to also report the largest contiguous region
> available: So that if the space is fragmented, management software
> can detect the situation beforehand.
>  
> Like this perhaps:
>  
>  
> # LD_LIBRARY_PATH=/root/git/libvirt/src/.libs/
> # /root/git/virshnodegetcachestats  
> ret=0 nparams=10
> L3.0: free=11520, max_contiguous=X.
> L3.1: free=11520, max_contiguous=X.
>  
>  


Marclo, thanks for your testing and comments.

Actually, I only expose the contiguous cache which can be allocated to VMs for 
now.

so

free == max_contiguous.

Othere will be one case:

schemata 1 = 1000 
schemata 2 = 0010 
default schemata should be 0001  (schemata should be contiguous, so it 
can’t be 0101

and free_size = 5 * min_cache_unit

virsh nodecachestats will report free=5 * min_cache_unit

next if a new vm requires cache of 1 * min_cache_unit

schemata 1 = 1000 
schemata 2 = 0010 
new VM’s schemata = 0100 
default schemata should be 0001 


so free_size is still 5 * min_cache_unit.

and   
virsh nodecachestats will report free=5 * min_cache_unit



I am not sure if that will meaningful to user we expose (fragmented) cache 
which can not be allocated to VMs.

>  
> Or:
>  
> L3.0.free: 11520
> L3.0.max_contiguous=X.
> L3.1.free: 11520
> L3.1.max_contiguous=X.
>  
> Not sure what is the preferred way to do this in libvirt.
>  
> Otherwise, testing looks good now.  

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATH V10 08/12] Resctrl: Compatible mode for cdp enabled

2017-03-06 Thread Eli Qiao

This patch support l3 cache allocation compatible mode if cdp enabled
on host.

In this case l3code/l3data has same schemata.

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 src/conf/domain_conf.c | 15 +--
 src/util/virresctrl.c  | 10 ++
 2 files changed, 23 insertions(+), 2 deletions(-)

diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
index 86c292d..710c327 100644
--- a/src/conf/domain_conf.c
+++ b/src/conf/domain_conf.c
@@ -15823,9 +15823,20 @@ virDomainCacheTuneDefParseXML(virDomainDefPtr def,
 resctrl = virResCtrlGet(type);
 
 if (resctrl == NULL || !resctrl->enabled) {
-virReportError(VIR_ERR_XML_ERROR,
+/* support cdp compatible */
+if (type == VIR_RDT_RESOURCE_L3) {
+resctrl = virResCtrlGet(type + 1);
+if (resctrl == NULL || !resctrl->enabled) {
+virReportError(VIR_ERR_XML_ERROR,
 _("'host doesn't enabled cache type '%s'"), tmp);
-goto cleanup;
+goto cleanup;
+}
+VIR_WARN("Use cdp compatible mode.");
+} else {
+virReportError(VIR_ERR_XML_ERROR,
+_("'host doesn't enabled cache type '%s'"), tmp);
+goto cleanup;
+}
 }
 
 bool found_host_id = false;
diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c
index ee5f043..9f0d05f 100644
--- a/src/util/virresctrl.c
+++ b/src/util/virresctrl.c
@@ -760,6 +760,7 @@ int virResCtrlSetCacheBanks(virDomainCachetunePtr cachetune,
 char name[VIR_UUID_STRING_BUFLEN];
 virResDomainPtr p;
 int type;
+int pair_type = -1;
 int sid;
 int schemata;
 
@@ -793,6 +794,13 @@ int virResCtrlSetCacheBanks(virDomainCachetunePtr 
cachetune,
  cachetune->cache_banks[i].type);
 continue;
 }
+/* use cdp compatible mode */
+if (!VIR_RESCTRL_ENABLED(type) &&
+(type == VIR_RDT_RESOURCE_L3) &&
+VIR_RESCTRL_ENABLED(VIR_RDT_RESOURCE_L3DATA)) {
+type = VIR_RDT_RESOURCE_L3DATA;
+pair_type = VIR_RDT_RESOURCE_L3CODE;
+}
 
 if ((sid = virResCtrlGetSocketIdByHostID(
 type, cachetune->cache_banks[i].host_id)) < 0) {
@@ -810,6 +818,8 @@ int virResCtrlSetCacheBanks(virDomainCachetunePtr cachetune,
 }
 
 p->schematas[type]->schemata_items[sid].schemata = schemata;
+if (pair_type > 0)
+p->schematas[pair_type]->schemata_items[sid].schemata = 
schemata;
 }
 
 for (i = 0; i < npid; i++)
-- 
1.9.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATCH resend V10 06/12] Resctrl: enable l3code/l3data

2017-03-06 Thread Eli Qiao

Enable l3code/l3data while doing cache tune.

l3code/l3data should use a continus cbm in their seperated schemata and
the cache size are shared between them, so we need to deal them
differently with l3 cache.

This should enable cdp feature while mounting /sys/fs/resctrl, eg:
mount -t resctrl resctrl -o cdp  /sys/fs/resctrl

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 docs/schemas/domaincommon.rng |  2 +-
 src/util/virresctrl.c | 27 +--
 2 files changed, 26 insertions(+), 3 deletions(-)

diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng
index efc84c5..ed8bdb9 100644
--- a/docs/schemas/domaincommon.rng
+++ b/docs/schemas/domaincommon.rng
@@ -5528,7 +5528,7 @@
   
   
 
-  (l3)
+  (l3|l3code|l3data)
 
   
   
diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c
index 43af0f5..e7376a0 100644
--- a/src/util/virresctrl.c
+++ b/src/util/virresctrl.c
@@ -417,6 +417,7 @@ int virResCtrlRefreshSchemata(void)
 unsigned int tmp_schemata;
 unsigned int default_schemata;
 unsigned int min_schemata;
+int pair_type = 0;
 
 virResDomainPtr header, p;
 
@@ -429,6 +430,11 @@ int virResCtrlRefreshSchemata(void)
 if (VIR_RESCTRL_ENABLED(i)) {
 min_schemata = 
VIR_RESCTRL_GET_SCHEMATA(resctrlall[i].min_cbm_bits);
 
+if (i == VIR_RDT_RESOURCE_L3DATA)
+pair_type = VIR_RDT_RESOURCE_L3CODE;
+if (i == VIR_RDT_RESOURCE_L3CODE)
+pair_type = VIR_RDT_RESOURCE_L3DATA;
+
 for (j = 0; j < header->schematas[i]->n_schemata_items; j ++) {
 p = header->next;
 // Reset to default schemata 0xf
@@ -436,8 +442,11 @@ int virResCtrlRefreshSchemata(void)
 tmp_schemata = 0;
 /* NOTEs: if only header domain, the schemata will be set to 
default one*/
 for (k = 1; k < domainall.num_domains; k++) {
-if (p->schematas[i]->schemata_items[j].schemata > 
min_schemata)
+if (p->schematas[i]->schemata_items[j].schemata > 
min_schemata) {
 tmp_schemata |= 
p->schematas[i]->schemata_items[j].schemata;
+if (pair_type > 0)
+tmp_schemata |= 
p->schematas[pair_type]->schemata_items[j].schemata;
+}
 p = p->next;
 }
 
@@ -503,6 +512,7 @@ virResCtrlWrite(const char *name, const char *item, const 
char *content)
 goto cleanup;
 
 rc = 0;
+
  cleanup:
 VIR_FREE(path);
 VIR_FORCE_CLOSE(writefd);
@@ -707,6 +717,7 @@ virResCtrlCalculateSchemata(int type,
 virResDomainPtr p;
 unsigned int tmp_schemata;
 unsigned int schemata_sum = 0;
+int pair_type = 0;
 
 if (resctrlall[type].cache_banks[sid].cache_left < size) {
 VIR_ERROR(_("Not enough cache left on bank %u"), hostid);
@@ -721,8 +732,18 @@ virResCtrlCalculateSchemata(int type,
 
 p = domainall.domains;
 p = p->next;
+
+/* for type is l3code and l3data, we need to deal them specially*/
+if (type == VIR_RDT_RESOURCE_L3DATA)
+pair_type = VIR_RDT_RESOURCE_L3CODE;
+
+if (type == VIR_RDT_RESOURCE_L3CODE)
+pair_type = VIR_RDT_RESOURCE_L3DATA;
+
 for (i = 1; i < domainall.num_domains; i ++) {
 schemata_sum |= p->schematas[type]->schemata_items[sid].schemata;
+if (pair_type > 0)
+schemata_sum |= 
p->schematas[pair_type]->schemata_items[sid].schemata;
 p = p->next;
 }
 
@@ -763,6 +784,9 @@ int virResCtrlSetCacheBanks(virDomainCachetunePtr cachetune,
 }
 
 if (p != NULL) {
+
+virResCtrlAppendDomain(p);
+
 for (i = 0; i < cachetune->n_banks; i++) {
 if ((type = virResCtrlTypeFromString(
 cachetune->cache_banks[i].type)) < 0) {
@@ -797,7 +821,6 @@ int virResCtrlSetCacheBanks(virDomainCachetunePtr cachetune,
 virResCtrlDestroyDomain(p);
 return -1;
 }
-virResCtrlAppendDomain(p);
 } else {
 VIR_ERROR(_("Failed to create a domain in sysfs"));
 return -1;
-- 
1.9.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATCH resend V10 04/12] Resctrl: Add private interfaces to operate cache bank

2017-03-06 Thread Eli Qiao

virResCtrlSetCacheBanks: Set cache banks of a libvirt domain. It will
 create new resource domain under
 `/sys/fs/resctrl` and fill the schemata according
 the cache bank configration.

virResCtrlUpdate: Destroy resctrl domain directory, and update the schemata
  after libvirt domain destroy.

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 src/libvirt_private.syms |   6 +-
 src/util/virresctrl.c| 677 +++
 src/util/virresctrl.h|   5 +-
 3 files changed, 686 insertions(+), 2 deletions(-)

diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms
index b8445ef..9cfffb8 100644
--- a/src/libvirt_private.syms
+++ b/src/libvirt_private.syms
@@ -2325,7 +2325,11 @@ virRandomInt;
 virResCtrlAvailable;
 virResCtrlGet;
 virResCtrlInit;
-
+virResCtrlSetCacheBanks;
+virResCtrlTypeFromString;
+virResCtrlTypeToString;
+virResCtrlUpdate;
+#
 
 # util/virrotatingfile.h
 virRotatingFileReaderConsume;
diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c
index eee6675..43af0f5 100644
--- a/src/util/virresctrl.c
+++ b/src/util/virresctrl.c
@@ -19,6 +19,8 @@
  *  Eli Qiao <liyong.q...@intel.com>
  */
 #include 
+#include 
+#include 
 
 #include "virresctrl.h"
 #include "viralloc.h"
@@ -61,8 +63,58 @@ do { \
 
 #define VIR_RESCTRL_GET_SCHEMATA(count) ((1 << count) - 1)
 
+/**
+ * a virResSchemata represents a schemata object under a resource control
+ * domain.
+ */
+typedef struct _virResSchemataItem virResSchemataItem;
+typedef virResSchemataItem *virResSchemataItemPtr;
+struct _virResSchemataItem {
+unsigned int socket_no;
+unsigned schemata;
+};
+
+typedef struct _virResSchemata virResSchemata;
+typedef virResSchemata *virResSchemataPtr;
+struct _virResSchemata {
+unsigned int n_schemata_items;
+virResSchemataItemPtr schemata_items;
+};
+
+/**
+ * a virResDomain represents a resource control domain. It's a double linked
+ * list.
+ */
+
+typedef struct _virResDomain virResDomain;
+typedef virResDomain *virResDomainPtr;
+
+struct _virResDomain {
+char *name;
+virResSchemataPtr schematas[VIR_RDT_RESOURCE_LAST];
+char **tasks;
+size_t n_tasks;
+size_t n_sockets;
+virResDomainPtr pre;
+virResDomainPtr next;
+};
+
+/* All resource control domains on this host*/
+typedef struct _virResCtrlDomain virResCtrlDomain;
+typedef virResCtrlDomain *virResCtrlDomainPtr;
+
+struct _virResCtrlDomain {
+unsigned int num_domains;
+virResDomainPtr domains;
+};
+
 static unsigned int host_id;
 
+/* Global static struct to be maintained which is a interface */
+static virResCtrlDomain domainall;
+
+/* Global static struct array to be maintained which indicate
+ * resource status on a host */
 static virResCtrl resctrlall[] = {
 {
 .name = "L3",
@@ -82,6 +134,78 @@ static virResCtrl resctrlall[] = {
 },
 };
 
+/*
+ * How many bits is set in schemata
+ * eg:
+ * virResCtrlBitsNum(10110) = 2 */
+static int virResCtrlBitsContinuesNum(unsigned schemata)
+{
+size_t i;
+int ret = 0;
+for (i = 0; i < MAX_CBM_BIT_LEN; i ++) {
+if ((schemata & 0x1) == 0x1)
+ret++;
+else
+if (ret > 0 || schemata == 0) break;
+
+schemata = schemata >> 1;
+}
+return ret;
+}
+
+/* Position of the highest continue 1 bit of in schemata
+ * eg:
+ * virResctrlBitsContinuesPos(10110) = 3 */
+static int virResCtrlBitsContinuesPos(unsigned schemata)
+{
+size_t i;
+int flag = 0;
+for (i = 0; i < MAX_CBM_BIT_LEN; i ++) {
+if ((schemata & 0x1) == 0x0 && flag == 1)
+return i;
+else if ((schemata & 0x1) == 0x1) flag = 1;
+
+schemata = schemata >> 1;
+}
+return 0;
+}
+
+static int virResCtrlGetStr(const char *domain_name, const char *item_name, 
char **ret)
+{
+char *path;
+int rc = 0;
+
+CONSTRUCT_RESCTRL_PATH(domain_name, item_name);
+
+if (virFileReadAll(path, MAX_FILE_LEN, ret) < 0) {
+rc = -1;
+goto cleanup;
+}
+
+ cleanup:
+VIR_FREE(path);
+return rc;
+}
+
+static int virResCtrlGetSchemata(const int type, const char *name, char 
**schemata)
+{
+int rc;
+char *tmp, *end;
+char *buf;
+
+if ((rc = virResCtrlGetStr(name, "schemata", )) < 0)
+return rc;
+
+tmp = strstr(buf, resctrlall[type].name);
+end = strchr(tmp, '\n');
+*end = '\0';
+if (VIR_STRDUP(*schemata, tmp) < 0)
+rc = -1;
+
+VIR_FREE(buf);
+return rc;
+}
+
 static int virResCtrlGetInfoStr(const int type, const char *item, char **str)
 {
 int ret = 0;
@@ -102,6 +226,71 @@ static int virResCtrlGetInfoStr(const int type, const char 
*item, char **str)
 return ret;
 }
 
+/* Return pointer of and ncount of schemata*/
+static virResSchemataPtr virParseSchemata(const char *schema

[libvirt] [PATCH resend V10 08/12] Resctrl: Compatible mode for cdp enabled

2017-03-06 Thread Eli Qiao

This patch support l3 cache allocation compatible mode if cdp enabled
on host.

In this case l3code/l3data has same schemata.

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 src/conf/domain_conf.c | 15 +--
 src/util/virresctrl.c  | 10 ++
 2 files changed, 23 insertions(+), 2 deletions(-)

diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
index 86c292d..710c327 100644
--- a/src/conf/domain_conf.c
+++ b/src/conf/domain_conf.c
@@ -15823,9 +15823,20 @@ virDomainCacheTuneDefParseXML(virDomainDefPtr def,
 resctrl = virResCtrlGet(type);
 
 if (resctrl == NULL || !resctrl->enabled) {
-virReportError(VIR_ERR_XML_ERROR,
+/* support cdp compatible */
+if (type == VIR_RDT_RESOURCE_L3) {
+resctrl = virResCtrlGet(type + 1);
+if (resctrl == NULL || !resctrl->enabled) {
+virReportError(VIR_ERR_XML_ERROR,
 _("'host doesn't enabled cache type '%s'"), tmp);
-goto cleanup;
+goto cleanup;
+}
+VIR_WARN("Use cdp compatible mode.");
+} else {
+virReportError(VIR_ERR_XML_ERROR,
+_("'host doesn't enabled cache type '%s'"), tmp);
+goto cleanup;
+}
 }
 
 bool found_host_id = false;
diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c
index ee5f043..9f0d05f 100644
--- a/src/util/virresctrl.c
+++ b/src/util/virresctrl.c
@@ -760,6 +760,7 @@ int virResCtrlSetCacheBanks(virDomainCachetunePtr cachetune,
 char name[VIR_UUID_STRING_BUFLEN];
 virResDomainPtr p;
 int type;
+int pair_type = -1;
 int sid;
 int schemata;
 
@@ -793,6 +794,13 @@ int virResCtrlSetCacheBanks(virDomainCachetunePtr 
cachetune,
  cachetune->cache_banks[i].type);
 continue;
 }
+/* use cdp compatible mode */
+if (!VIR_RESCTRL_ENABLED(type) &&
+(type == VIR_RDT_RESOURCE_L3) &&
+VIR_RESCTRL_ENABLED(VIR_RDT_RESOURCE_L3DATA)) {
+type = VIR_RDT_RESOURCE_L3DATA;
+pair_type = VIR_RDT_RESOURCE_L3CODE;
+}
 
 if ((sid = virResCtrlGetSocketIdByHostID(
 type, cachetune->cache_banks[i].host_id)) < 0) {
@@ -810,6 +818,8 @@ int virResCtrlSetCacheBanks(virDomainCachetunePtr cachetune,
 }
 
 p->schematas[type]->schemata_items[sid].schemata = schemata;
+if (pair_type > 0)
+p->schematas[pair_type]->schemata_items[sid].schemata = 
schemata;
 }
 
 for (i = 0; i < npid; i++)
-- 
1.9.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATCH resend V10 07/12] Resctrl: Make sure l3data/l3code are pairs

2017-03-06 Thread Eli Qiao

l3data and l3code type of cache banks should be configured pairs.

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 src/conf/domain_conf.c | 19 +++
 src/util/virresctrl.c  |  1 -
 src/util/virresctrl.h  |  2 ++
 3 files changed, 21 insertions(+), 1 deletion(-)

diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
index 652f4ca..86c292d 100644
--- a/src/conf/domain_conf.c
+++ b/src/conf/domain_conf.c
@@ -15760,9 +15760,13 @@ virDomainCacheTuneDefParseXML(virDomainDefPtr def,
 int type = -1;
 virDomainCacheBankPtr bank = NULL;
 virResCtrlPtr resctrl;
+/* An array to make sure l3code and l3data are pairs */
+int* sem = NULL;
 
 if (VIR_ALLOC_N(bank, n) < 0)
 goto cleanup;
+if (VIR_ALLOC_N(sem, MAX_CPU_SOCKET_NUM) < 0)
+goto cleanup;
 
 for (i = 0; i < n; i++) {
 if (!(tmp = virXMLPropString(nodes[i], "id"))) {
@@ -15810,6 +15814,12 @@ virDomainCacheTuneDefParseXML(virDomainDefPtr def,
 goto cleanup;
 }
 
+/* VIR_RDT_RESOURCE_L3DATA and VIR_RDT_RESOURCE_L3CODE should be pair 
*/
+if (type == VIR_RDT_RESOURCE_L3DATA)
+sem[bank[i].host_id] ++;
+else if (type == VIR_RDT_RESOURCE_L3CODE)
+sem[bank[i].host_id] --;
+
 resctrl = virResCtrlGet(type);
 
 if (resctrl == NULL || !resctrl->enabled) {
@@ -15858,6 +15868,14 @@ virDomainCacheTuneDefParseXML(virDomainDefPtr def,
 }
 }
 
+for (i = 0; i < MAX_CPU_SOCKET_NUM; i ++) {
+if (sem[i] != 0) {
+virReportError(VIR_ERR_XML_ERROR,
+_("'l3code and l3data shoud be show up pairs on bank 
%zu'"),
+i);
+goto cleanup;
+}
+}
 def->cachetune.cache_banks = bank;
 def->cachetune.n_banks = n;
 return 0;
@@ -15865,6 +15883,7 @@ virDomainCacheTuneDefParseXML(virDomainDefPtr def,
  cleanup:
 VIR_FREE(bank);
 VIR_FREE(tmp);
+VIR_FREE(sem);
 return -1;
 }
 
diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c
index e7376a0..ee5f043 100644
--- a/src/util/virresctrl.c
+++ b/src/util/virresctrl.c
@@ -34,7 +34,6 @@
 VIR_LOG_INIT("util.resctrl");
 
 #define VIR_FROM_THIS VIR_FROM_RESCTRL
-#define MAX_CPU_SOCKET_NUM 8
 #define MAX_CBM_BIT_LEN 32
 #define MAX_SCHEMATA_LEN 1024
 #define MAX_FILE_LEN (10 * 1024 * 1024)
diff --git a/src/util/virresctrl.h b/src/util/virresctrl.h
index 3a6fb95..d639de1 100644
--- a/src/util/virresctrl.h
+++ b/src/util/virresctrl.h
@@ -28,6 +28,8 @@
 # include "virutil.h"
 # include "conf/domain_conf.h"
 
+#define MAX_CPU_SOCKET_NUM 8
+
 enum {
 VIR_RDT_RESOURCE_L3,
 VIR_RDT_RESOURCE_L3DATA,
-- 
1.9.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATCH resend V10 11/12] Resctrl: Add Public API for nodecachestats

2017-03-06 Thread Eli Qiao

This patch expose a public API virNodeCacheStats to query cache stats
on a host.

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 daemon/remote.c| 67 ++
 include/libvirt/libvirt-host.h | 32 
 src/driver-hypervisor.h|  7 +
 src/libvirt-host.c | 41 ++
 src/libvirt_public.syms|  1 +
 src/remote/remote_driver.c | 52 
 src/remote/remote_protocol.x   | 25 +++-
 src/remote_protocol-structs| 16 ++
 8 files changed, 240 insertions(+), 1 deletion(-)

diff --git a/daemon/remote.c b/daemon/remote.c
index f2b9b9a..af291a7 100644
--- a/daemon/remote.c
+++ b/daemon/remote.c
@@ -7079,3 +7079,70 @@ remoteSerializeDomainDiskErrors(virDomainDiskErrorPtr 
errors,
 }
 return -1;
 }
+
+ static int
+remoteDispatchNodeGetCacheStats(virNetServerPtr server ATTRIBUTE_UNUSED,
+virNetServerClientPtr client ATTRIBUTE_UNUSED,
+virNetMessagePtr msg ATTRIBUTE_UNUSED,
+virNetMessageErrorPtr rerr,
+remote_node_get_cache_stats_args *args,
+remote_node_get_cache_stats_ret *ret)
+{
+virNodeCacheStatsPtr params = NULL;
+size_t i;
+int nparams = 0;
+unsigned int flags;
+int rv = -1;
+struct daemonClientPrivate *priv =
+virNetServerClientGetPrivateData(client);
+
+if (!priv->conn) {
+virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("connection not open"));
+goto cleanup;
+}
+
+flags = args->flags;
+
+if (args->nparams && VIR_ALLOC_N(params, args->nparams) < 0)
+goto cleanup;
+nparams = args->nparams;
+
+if (virNodeGetCacheStats(priv->conn, params, , flags) < 0)
+goto cleanup;
+
+/* In this case, we need to send back the number of stats
+ * supported
+ */
+if (args->nparams == 0) {
+ret->nparams = nparams;
+goto success;
+}
+
+/* Serialise the memory parameters. */
+ret->params.params_len = nparams;
+if (VIR_ALLOC_N(ret->params.params_val, nparams) < 0)
+goto cleanup;
+
+for (i = 0; i < nparams; ++i) {
+/* remoteDispatchClientRequest will free this: */
+if (VIR_STRDUP(ret->params.params_val[i].field, params[i].field) < 0)
+goto cleanup;
+
+ret->params.params_val[i].value = params[i].value;
+}
+
+ success:
+rv = 0;
+
+ cleanup:
+if (rv < 0) {
+virNetMessageSaveError(rerr);
+if (ret->params.params_val) {
+for (i = 0; i < nparams; i++)
+VIR_FREE(ret->params.params_val[i].field);
+VIR_FREE(ret->params.params_val);
+}
+}
+VIR_FREE(params);
+return rv;
+}
diff --git a/include/libvirt/libvirt-host.h b/include/libvirt/libvirt-host.h
index 07b5d15..222f361 100644
--- a/include/libvirt/libvirt-host.h
+++ b/include/libvirt/libvirt-host.h
@@ -140,6 +140,32 @@ struct _virSecurityModel {
  */
 typedef virSecurityModel *virSecurityModelPtr;
 
+/**
+ * VIR_NODE_CACHE_STATS_FIELD_LENGTH:
+ *
+ * Macro providing the field length of virNodeCacheStats
+ */
+
+# define VIR_NODE_CACHE_STATS_FIELD_LENGTH 16
+
+/**
+ *
+ * virNodeCacheStats is all the cache stats on a host.
+ */
+
+typedef struct _virNodeCacheStats virNodeCacheStats;
+
+struct _virNodeCacheStats {
+char field[VIR_NODE_CACHE_STATS_FIELD_LENGTH];
+unsigned long long value;
+};
+
+/**
+ * a virNodeCacheStatsPtr is a pointer to a virNodeCacheStats.
+ */
+
+typedef virNodeCacheStats *virNodeCacheStatsPtr;
+
 
 /* data types related to virNodePtr */
 
@@ -603,6 +629,12 @@ int virNodeSuspendForDuration 
(virConnectPtr conn,
unsigned long long duration,
unsigned int flags);
 
+int virNodeGetCacheStats (virConnectPtr conn,
+  virNodeCacheStatsPtr params,
+  int *nparams,
+  unsigned int flags);
+
+
 /*
  * NUMA support
  */
diff --git a/src/driver-hypervisor.h b/src/driver-hypervisor.h
index b81420a..a6a1350 100644
--- a/src/driver-hypervisor.h
+++ b/src/driver-hypervisor.h
@@ -578,6 +578,12 @@ typedef int
 unsigned int flags);
 
 typedef int
+(*virDrvNodeGetCacheStats)(virConnectPtr conn,
+   virNodeCacheStatsPtr params,
+   int *nparams,
+   unsigned int flags);
+
+typedef int
 (*virDrvNodeGetCellsFreeMemory)(virConnectPtr conn,
 unsigned long long *freeMem

[libvirt] [PATCH resend V10 01/12] Resctrl: Add some utils functions

2017-03-06 Thread Eli Qiao

This patch adds some utils struct and functions to expose resctrl
information.

virResCtrlAvailable: if resctrl interface exist on host.
virResCtrlGet: get specific type resource control information.
virResCtrlInit: initialize resctrl struct from the host's sys fs.
resctrlall[]: an array to maintain resource control information.

Some of host cpu related information methods was added in virhostcpu.c

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 include/libvirt/virterror.h |   1 +
 po/POTFILES.in  |   1 +
 src/Makefile.am |   1 +
 src/libvirt_private.syms|   4 +
 src/util/virerror.c |   1 +
 src/util/virhostcpu.c   | 186 
 src/util/virhostcpu.h   |   6 ++
 src/util/virresctrl.c   | 201 
 src/util/virresctrl.h   |  78 +
 9 files changed, 462 insertions(+), 17 deletions(-)
 create mode 100644 src/util/virresctrl.c
 create mode 100644 src/util/virresctrl.h

diff --git a/include/libvirt/virterror.h b/include/libvirt/virterror.h
index 2efee8f..3dd2d08 100644
--- a/include/libvirt/virterror.h
+++ b/include/libvirt/virterror.h
@@ -132,6 +132,7 @@ typedef enum {
 
 VIR_FROM_PERF = 65, /* Error from perf */
 VIR_FROM_LIBSSH = 66,   /* Error from libssh connection transport */
+VIR_FROM_RESCTRL = 67,  /* Error from resource control */
 
 # ifdef VIR_ENUM_SENTINELS
 VIR_ERR_DOMAIN_LAST
diff --git a/po/POTFILES.in b/po/POTFILES.in
index 7c7f530..4147bc6 100644
--- a/po/POTFILES.in
+++ b/po/POTFILES.in
@@ -241,6 +241,7 @@ src/util/virportallocator.c
 src/util/virprocess.c
 src/util/virqemu.c
 src/util/virrandom.c
+src/util/virresctrl.c
 src/util/virrotatingfile.c
 src/util/virscsi.c
 src/util/virscsihost.c
diff --git a/src/Makefile.am b/src/Makefile.am
index 7d42eac..edb946a 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -162,6 +162,7 @@ UTIL_SOURCES =  
\
util/virprocess.c util/virprocess.h \
util/virqemu.c util/virqemu.h   \
util/virrandom.h util/virrandom.c   \
+   util/virresctrl.h util/virresctrl.c \
util/virrotatingfile.h util/virrotatingfile.c   \
util/virscsi.c util/virscsi.h   \
util/virscsihost.c util/virscsihost.h   \
diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms
index aed1d3d..bb7c3ad 100644
--- a/src/libvirt_private.syms
+++ b/src/libvirt_private.syms
@@ -2320,6 +2320,10 @@ virRandomGenerateWWN;
 virRandomInt;
 
 
+# util/virresctrl.h
+virResCtrlAvailable;
+virResCtrlInit;
+
 # util/virrotatingfile.h
 virRotatingFileReaderConsume;
 virRotatingFileReaderFree;
diff --git a/src/util/virerror.c b/src/util/virerror.c
index ef17fb5..0ba15e6 100644
--- a/src/util/virerror.c
+++ b/src/util/virerror.c
@@ -139,6 +139,7 @@ VIR_ENUM_IMPL(virErrorDomain, VIR_ERR_DOMAIN_LAST,
 
   "Perf", /* 65 */
   "Libssh transport layer",
+  "Resouce Control",
 )
 
 
diff --git a/src/util/virhostcpu.c b/src/util/virhostcpu.c
index f29f312..e6d5102 100644
--- a/src/util/virhostcpu.c
+++ b/src/util/virhostcpu.c
@@ -206,29 +206,21 @@ void virHostCPUSetSysFSSystemPathLinux(const char *path)
 sysfs_system_path = SYSFS_SYSTEM_PATH;
 }
 
-/* Return the positive decimal contents of the given
- * DIR/cpu%u/FILE, or -1 on error.  If DEFAULT_VALUE is non-negative
- * and the file could not be found, return that instead of an error;
- * this is useful for machines that cannot hot-unplug cpu0, or where
- * hot-unplugging is disabled, or where the kernel is too old
- * to support NUMA cells, etc.  */
+/* Get a String value*/
 static int
-virHostCPUGetValue(const char *dir, unsigned int cpu, const char *file,
-   int default_value)
+virHostCPUGetStrValue(const char *dir, unsigned int cpu, const char *file, 
char *value_str)
 {
 char *path;
 FILE *pathfp;
-int value = -1;
-char value_str[INT_BUFSIZE_BOUND(value)];
-char *tmp;
+int ret = -1;
 
 if (virAsprintf(, "%s/cpu%u/%s", dir, cpu, file) < 0)
 return -1;
 
 pathfp = fopen(path, "r");
 if (pathfp == NULL) {
-if (default_value >= 0 && errno == ENOENT)
-value = default_value;
+if (errno == ENOENT)
+return -2;
 else
 virReportSystemError(errno, _("cannot open %s"), path);
 goto cleanup;
@@ -238,17 +230,84 @@ virHostCPUGetValue(const char *dir, unsigned int cpu, 
const char *file,
 virReportSystemError(errno, _("cannot read from %s"), path);
 goto cleanup;
 }
+
+ret = 0;
+
+ cleanup:
+VIR_FORCE_FCLOSE(pathfp);
+VIR_FREE(path);
+return ret;
+}
+

[libvirt] [PATCH resend V10 02/12] Resctrl: expose cache information to capabilities

2017-03-06 Thread Eli Qiao

This patch expose cache information to host's capabilites xml.

For l3 cache allocation

  

  
  

  


For l3 cache allocation supported cdp(seperate data/code):

  


  
  


  


RFC on mailing list.
https://www.redhat.com/archives/libvir-list/2017-January/msg00644.html

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 src/conf/capabilities.c  | 56 ++
 src/conf/capabilities.h  | 23 
 src/libvirt_private.syms |  3 +++
 src/nodeinfo.c   | 64 
 src/nodeinfo.h   |  1 +
 src/qemu/qemu_capabilities.c |  8 ++
 src/qemu/qemu_driver.c   |  4 +++
 7 files changed, 159 insertions(+)

diff --git a/src/conf/capabilities.c b/src/conf/capabilities.c
index 9ab343b..23e21e4 100644
--- a/src/conf/capabilities.c
+++ b/src/conf/capabilities.c
@@ -198,6 +198,18 @@ virCapabilitiesClearSecModel(virCapsHostSecModelPtr 
secmodel)
 }
 
 static void
+virCapabilitiesClearCacheBank(virCapsHostCacheBankPtr cachebank)
+{
+size_t i;
+for (i = 0; i < cachebank->ncontrol; i++)
+VIR_FREE(cachebank->control[i].scope);
+
+VIR_FREE(cachebank->type);
+VIR_FREE(cachebank->cpus);
+}
+
+
+static void
 virCapabilitiesDispose(void *object)
 {
 virCapsPtr caps = object;
@@ -221,6 +233,10 @@ virCapabilitiesDispose(void *object)
 virCapabilitiesClearSecModel(>host.secModels[i]);
 VIR_FREE(caps->host.secModels);
 
+for (i = 0; i < caps->host.ncachebank; i++)
+virCapabilitiesClearCacheBank(caps->host.cachebank[i]);
+VIR_FREE(caps->host.cachebank);
+
 VIR_FREE(caps->host.netprefix);
 VIR_FREE(caps->host.pagesSize);
 virCPUDefFree(caps->host.cpu);
@@ -844,6 +860,41 @@ virCapabilitiesFormatNUMATopology(virBufferPtr buf,
 return 0;
 }
 
+static int
+virCapabilitiesFormatCache(virBufferPtr buf,
+   size_t ncachebank,
+   virCapsHostCacheBankPtr *cachebank)
+{
+size_t i;
+size_t j;
+
+virBufferAddLit(buf, "\n");
+virBufferAdjustIndent(buf, 2);
+
+for (i = 0; i < ncachebank; i++) {
+virBufferAsprintf(buf,
+  "\n",
+  cachebank[i]->id,
+  cachebank[i]->type,
+  cachebank[i]->size,
+  cachebank[i]->cpus);
+
+virBufferAdjustIndent(buf, 2);
+for (j = 0; j < cachebank[i]->ncontrol; j++) {
+virBufferAsprintf(buf,
+  "\n",
+  cachebank[i]->control[j].min,
+  cachebank[i]->control[j].reserved,
+  cachebank[i]->control[j].scope);
+}
+virBufferAdjustIndent(buf, -2);
+virBufferAddLit(buf, "\n");
+}
+virBufferAdjustIndent(buf, -2);
+virBufferAddLit(buf, "\n");
+return 0;
+}
+
 /**
  * virCapabilitiesFormatXML:
  * @caps: capabilities to format
@@ -931,6 +982,11 @@ virCapabilitiesFormatXML(virCapsPtr caps)
 virBufferAddLit(, "\n");
 }
 
+if (caps->host.ncachebank &&
+virCapabilitiesFormatCache(, caps->host.ncachebank,
+   caps->host.cachebank) < 0)
+return NULL;
+
 if (caps->host.netprefix)
 virBufferAsprintf(, "%s\n",
   caps->host.netprefix);
diff --git a/src/conf/capabilities.h b/src/conf/capabilities.h
index cfdc34a..b446de5 100644
--- a/src/conf/capabilities.h
+++ b/src/conf/capabilities.h
@@ -138,6 +138,25 @@ struct _virCapsHostSecModel {
 virCapsHostSecModelLabelPtr labels;
 };
 
+typedef struct _virCapsHostCacheControl virCapsHostCacheControl;
+typedef virCapsHostCacheControl *virCapsHostCacheControlPtr;
+struct _virCapsHostCacheControl {
+unsigned long long min;
+unsigned long long reserved;
+char* scope;
+};
+
+typedef struct _virCapsHostCacheBank virCapsHostCacheBank;
+typedef virCapsHostCacheBank *virCapsHostCacheBankPtr;
+struct _virCapsHostCacheBank {
+unsigned int id;
+char* type;
+char* cpus;
+unsigned long long size;
+size_t ncontrol;
+virCapsHostCacheControlPtr control;
+};
+
 typedef struct _virCapsHost virCapsHost;
 typedef virCapsHost *virCapsHostPtr;
 struct _virCapsHost {
@@ -160,6 +179,10 @@ struct _virCapsHost {
 size_t nsecModels;
 virCapsHostSecModelPtr secModels;
 
+size_t ncachebank;
+size_t ncachebank_max;
+virCapsHostCacheBankPtr *cachebank;
+
 char *netprefix;
 virCPUDefPtr cpu;
 int nPagesSize; /* size of pagesSize array */
diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms
index bb7c3ad..b8445ef 100644
--- a/src/libvirt_private.syms
+++ b/src

[libvirt] [PATCH resend V10 12/12] Resctrl: Add nodecachestats

2017-03-06 Thread Eli Qiao

Add new virsh command line `nodecachestats` to expose the cache usage
on a node.

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 src/libvirt_private.syms |  3 ++-
 src/qemu/qemu_driver.c   | 12 ++
 src/util/virresctrl.c| 62 
 src/util/virresctrl.h|  8 +++
 tools/virsh-host.c   | 49 ++
 5 files changed, 133 insertions(+), 1 deletion(-)

diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms
index 9cfffb8..75a4c98 100644
--- a/src/libvirt_private.syms
+++ b/src/libvirt_private.syms
@@ -2323,13 +2323,14 @@ virRandomInt;
 
 # util/virresctrl.h
 virResCtrlAvailable;
+virResCtrlCacheGetStats;
 virResCtrlGet;
 virResCtrlInit;
 virResCtrlSetCacheBanks;
 virResCtrlTypeFromString;
 virResCtrlTypeToString;
 virResCtrlUpdate;
-#
+
 
 # util/virrotatingfile.h
 virRotatingFileReaderConsume;
diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c
index 0f11ae2..4677406 100644
--- a/src/qemu/qemu_driver.c
+++ b/src/qemu/qemu_driver.c
@@ -18285,6 +18285,17 @@ qemuNodeGetCPUStats(virConnectPtr conn,
 return virHostCPUGetStats(cpuNum, params, nparams, flags);
 }
 
+static int
+qemuNodeGetCacheStats(virConnectPtr conn,
+virNodeCacheStatsPtr params,
+int *nparams,
+unsigned int flags)
+{
+if (virNodeGetCacheStatsEnsureACL(conn) < 0)
+return -1;
+
+return virResCtrlCacheGetStats(params, nparams, flags);
+}
 
 static int
 qemuNodeGetMemoryStats(virConnectPtr conn,
@@ -20391,6 +20402,7 @@ static virHypervisorDriver qemuHypervisorDriver = {
 .domainMemoryPeek = qemuDomainMemoryPeek, /* 0.4.4 */
 .domainGetBlockInfo = qemuDomainGetBlockInfo, /* 0.8.1 */
 .nodeGetCPUStats = qemuNodeGetCPUStats, /* 0.9.3 */
+.nodeGetCacheStats = qemuNodeGetCacheStats, /* 3.1.0 */
 .nodeGetMemoryStats = qemuNodeGetMemoryStats, /* 0.9.3 */
 .nodeGetCellsFreeMemory = qemuNodeGetCellsFreeMemory, /* 0.4.4 */
 .nodeGetFreeMemory = qemuNodeGetFreeMemory, /* 0.4.4 */
diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c
index 16c01a2..97f7e84 100644
--- a/src/util/virresctrl.c
+++ b/src/util/virresctrl.c
@@ -1034,3 +1034,65 @@ virResCtrlGet(int type)
 {
 return [type];
 }
+
+int virResCtrlCacheGetStats(virNodeCacheStatsPtr params,
+int *nparams,
+unsigned int flags)
+{
+virCheckFlags(0, -1);
+size_t i, j, k;
+char *value;
+int rc = -1;
+int lockfd;
+
+if (*nparams == 0) {
+for (i = 0; i < VIR_RDT_RESOURCE_LAST; i++) {
+if (VIR_RESCTRL_ENABLED(i))
+*nparams += resctrlall[i].num_banks;
+}
+}
+if (params == NULL)
+return 0;
+
+if ((lockfd = open(RESCTRL_DIR, O_RDONLY)) < 0)
+goto cleanup;
+
+if (VIR_RESCTRL_LOCK(lockfd, LOCK_SH) < 0) {
+virReportSystemError(errno, _("Unable to lock '%s'"), RESCTRL_DIR);
+goto cleanup;
+}
+if (virResCtrlScan() < 0) {
+VIR_ERROR(_("Failed to scan resctrl domain dir"));
+goto cleanup;
+}
+
+virResCtrlRefreshSchemata();
+
+if ((rc = virResCtrlFlushDomainToSysfs(domainall.domains)) < 0)
+goto cleanup;
+
+k = 0;
+
+for (i = 0; i < VIR_RDT_RESOURCE_LAST; i++) {
+if (VIR_RESCTRL_ENABLED(i)) {
+for (j = 0; j < resctrlall[i].num_banks; j++) {
+
+if (virAsprintf(, "%s.%zu",
+resctrlall[i].name, j) < 0)
+goto cleanup;
+
+if (virStrcpyStatic(([k])->field, value) == NULL)
+goto cleanup;
+
+([k++])->value = 
resctrlall[i].cache_banks[j].cache_left;
+}
+}
+}
+
+rc = 0;
+
+ cleanup:
+VIR_FREE(value);
+VIR_RESCTRL_UNLOCK(lockfd);
+return rc;
+}
diff --git a/src/util/virresctrl.h b/src/util/virresctrl.h
index 968e0dc..eef5370 100644
--- a/src/util/virresctrl.h
+++ b/src/util/virresctrl.h
@@ -80,9 +80,17 @@ struct _virResCtrl {
 };
 
 bool virResCtrlAvailable(void);
+
 int virResCtrlInit(void);
+
 virResCtrlPtr virResCtrlGet(int);
+
 int virResCtrlSetCacheBanks(virDomainCachetunePtr,
 unsigned char *, pid_t *, int);
+
 int virResCtrlUpdate(unsigned char *);
+
+int virResCtrlCacheGetStats(virNodeCacheStatsPtr params,
+int *nparams,
+unsigned int flags);
 #endif
diff --git a/tools/virsh-host.c b/tools/virsh-host.c
index 24ebde2..c90bd2e 100644
--- a/tools/virsh-host.c
+++ b/tools/virsh-host.c
@@ -946,6 +946,49 @@ cmdNodeMemStats(vshControl *ctl, const vshCmd *cmd)
 VIR_FREE(params);
 return ret;
 }
+/* "nodecachestats" command
+ */
+static const vshCmdInfo info_nodecachestats[] = {
+{.name = "help",
+ .data = N_("Prints cache stats

[libvirt] [PATCH resend V10 00/12] Support cache tune in libvirt

2017-03-06 Thread Eli Qiao

Addressed comment from v10 -> v9

Marcelo:
  * Improve default schemata calculate, support allocate from low schemata 
bits for other APP.

Addressed comment from v9 -> v8

Marcelo:
  * New public API to query cache usage
Eli:
  * Fix core dump while multiple tasks are added.

Addressed comment from v8 -> v7

Martin:
   * Patch subject prefix.
   * Move some of cpu related information to virhostcpu.c.
   * Fix some memory leak in src/utils/resctrl.c

Martin & Marcelo:
   * Don't remove directories which are not maintained by libvirt.

Addressed comment from v7 -> v6

Marcelo:
   * Fix flock usage while VM initialization.

Addressed comment from v6 -> v5

Marcelo:
   * Support other APPs to operate /sys/fs/resctrl at same time

 Libvirt will scan /sys/fs/resctrl again before doing cache allocation.
 patch 10 will address this.

Addressed comment from v4 -> v5:

Marcelo:
   * Several typos
   * Use flock instead of virFileLock

Addressed comment from v3 -> v4:

Daniel & Marcelo:
   * Added concurrence support

Addressed comment from v2 -> v3:

Daniel:
  * Fixed coding style, passed `make check` and `make syntax-check`

  * Variables renaming and move from header file to c file.

  * For locking/mutex support, no progress.

  There are some discussion from mailing list, but I can not find a better
  way to add locking support without performance impact.

  I'll explain the process and please help to advice what shoud we do.

  VM create:
  1) Get the cache left value on each bank of the host. This should be
 shared amount all VMs.
  2) Calculate the schemata on the bank based on all created resctrl
 domain's schemata
  3) Calculate the default schemata by scaning all domain's schemata.
  4) Flush default schemata to /sys/fs/resctrl/schemata

  VM destroy:
  1) Remove the resctrl domain of that VM
  2) Recalculate default schemata
  3) Flush default schemata to /sys/fs/resctrl/schemata

  The key point is that all VMs shares /sys/fs/resctrl/schemata, and
  when a VM create a resctrl domain, the schemata of that VM depends on
  the default schemata and all other exsited schematas. So a global
  mutex is reqired.

  Before calculate a schemata or update default schemata, libvirt
  should gain this global mutex.

  I will try to think more about how to support this gracefully in next
  patch set.

Marcelo:
  * Added vcpu support for cachetune, this will allow user to define which
vcpu using which cache allocation bank.



vcpus is a cpumap, the vcpu pids will be added to tasks file

  * Added cdp compatible, user can specify l3 cache even host enable cdp.
See patch 8.
On a cdp enabled host, specify l3code/l3data by



This will create a schemata like:
L3data:0=0xff00;...
L3code:0=0xff00;...

  * Would you please help to test if the functions work.

Martin:
  * Xml test case, I have no time to work on this yet, would you please
show me an example, would like to amend it later.

This series patches are for supportting CAT featues, which also
called cache tune in libvirt.

First to expose cache information which could be tuned in capabilites XML.
Then add new domain xml element support to add cacahe bank which will apply
on this libvirt domain.

This series patches add a util file `resctrl.c/h`, an interface to talk with
linux kernel's system fs.

There are still one TODO left:
2. Expose a new public interface to set cachetune lively.

Some discussion about this feature support can be found from:
https://www.redhat.com/archives/libvir-list/2017-January/msg00644.html


*** BLURB HERE ***

Eli Qiao (12):
  Resctrl: Add some utils functions
  Resctrl: expose cache information to capabilities
  Resctrl: Add new xml element to support cache tune
  Resctrl: Add private interfaces to operate cache bank
  Qemu: Set cache tune while booting a new domain.
  Resctrl: enable l3code/l3data
  Resctrl: Make sure l3data/l3code are pairs
  Resctrl: Compatible mode for cdp enabled
  Resctrl: concurrence support
  Resctrl: Scan resctrl before doing cache allocation
  Resctrl: Add Public API for nodecachestats
  Resctrl: Add nodecachestats

 daemon/remote.c|   67 +++
 docs/schemas/domaincommon.rng  |   46 ++
 include/libvirt/libvirt-host.h |   32 ++
 include/libvirt/virterror.h|1 +
 po/POTFILES.in |1 +
 src/Makefile.am|1 +
 src/conf/capabilities.c|   56 ++
 src/conf/capabilities.h|   23 +
 src/conf/domain_conf.c |  182 +++
 src/conf/domain_conf.h |   19 +
 src/driver-hypervisor.h|7 +
 src/libvirt-host.c |   41 ++
 src/libvirt_private.syms   |   12 +
 src/libvirt_public.syms|1 +
 src/n

[libvirt] [PATCH resend V10 03/12] Resctrl: Add new xml element to support cache tune

2017-03-06 Thread Eli Qiao

This patch adds new xml element to support cache tune as:


  ...
  
  ...


id: any non-minus number
host_id: reference of the host's cache banks id, it's from capabilities
type: cache bank type
size: should be multiples of the min_size of the bank on host.
vcpus: cache allocation on vcpu set, if empty, will apply the allocation
on all vcpus
---
 docs/schemas/domaincommon.rng |  46 +
 src/conf/domain_conf.c| 152 ++
 src/conf/domain_conf.h|  19 ++
 src/util/virresctrl.c |  29 ++--
 src/util/virresctrl.h |   4 +-
 5 files changed, 244 insertions(+), 6 deletions(-)

diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng
index c64544a..efc84c5 100644
--- a/docs/schemas/domaincommon.rng
+++ b/docs/schemas/domaincommon.rng
@@ -825,6 +825,32 @@
 
   
 
+
+  
+
+  
+
+
+  
+
+
+  
+
+
+  
+
+
+  
+
+  
+
+
+  
+
+  
+
+  
+
 
   
 
@@ -5490,6 +5516,26 @@
   -1
 
   
+  
+
+  [0-9]+
+
+  
+  
+
+  [0-9]+
+
+  
+  
+
+  (l3)
+
+  
+  
+
+  KiB
+
+  
   
   
 
diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
index 97d42fe..652f4ca 100644
--- a/src/conf/domain_conf.c
+++ b/src/conf/domain_conf.c
@@ -56,6 +56,7 @@
 #include "virstring.h"
 #include "virnetdev.h"
 #include "virhostdev.h"
+#include "virresctrl.h"
 
 #define VIR_FROM_THIS VIR_FROM_DOMAIN
 
@@ -15745,6 +15746,127 @@ virDomainVcpuPinDefParseXML(virDomainDefPtr def,
 return ret;
 }
 
+/* Parse the XML definition for cachetune
+ * and a cachetune has the form
+ * 
+ */
+static int
+virDomainCacheTuneDefParseXML(virDomainDefPtr def,
+  int n,
+  xmlNodePtr* nodes)
+{
+char* tmp = NULL;
+size_t i, j;
+int type = -1;
+virDomainCacheBankPtr bank = NULL;
+virResCtrlPtr resctrl;
+
+if (VIR_ALLOC_N(bank, n) < 0)
+goto cleanup;
+
+for (i = 0; i < n; i++) {
+if (!(tmp = virXMLPropString(nodes[i], "id"))) {
+virReportError(VIR_ERR_XML_ERROR, "%s", _("missing id in cache 
tune"));
+goto cleanup;
+}
+if (virStrToLong_uip(tmp, NULL, 10, &(bank[i].id)) < 0) {
+virReportError(VIR_ERR_XML_ERROR,
+   _("invalid setting for cache id '%s'"), tmp);
+goto cleanup;
+}
+
+VIR_FREE(tmp);
+if (!(tmp = virXMLPropString(nodes[i], "host_id"))) {
+virReportError(VIR_ERR_XML_ERROR, "%s", _("missing host id in 
cache tune"));
+goto cleanup;
+}
+if (virStrToLong_uip(tmp, NULL, 10, &(bank[i].host_id)) < 0) {
+virReportError(VIR_ERR_XML_ERROR,
+   _("invalid setting for cache host id '%s'"), tmp);
+goto cleanup;
+}
+VIR_FREE(tmp);
+
+if (!(tmp = virXMLPropString(nodes[i], "size"))) {
+virReportError(VIR_ERR_XML_ERROR, "%s", _("missing size in cache 
tune"));
+goto cleanup;
+}
+if (virStrToLong_ull(tmp, NULL, 10, &(bank[i].size)) < 0) {
+virReportError(VIR_ERR_XML_ERROR,
+   _("invalid setting for cache size '%s'"), tmp);
+goto cleanup;
+}
+VIR_FREE(tmp);
+
+if (!(tmp = virXMLPropString(nodes[i], "type"))) {
+virReportError(VIR_ERR_INTERNAL_ERROR, "%s",
+   _("missing cache type"));
+goto cleanup;
+}
+
+if ((type = virResCtrlTypeFromString(tmp)) < 0) {
+virReportError(VIR_ERR_XML_ERROR,
+_("'unsupported cache type '%s'"), tmp);
+goto cleanup;
+}
+
+resctrl = virResCtrlGet(type);
+
+if (resctrl == NULL || !resctrl->enabled) {
+virReportError(VIR_ERR_XML_ERROR,
+_("'host doesn't enabled cache type '%s'"), tmp);
+goto cleanup;
+}
+
+bool found_host_id = false;
+/* Loop for banks to search host_id */
+for (j = 0; j < resctrl->num_banks; j++) {
+if (resctrl->cache_banks[j].host_id == bank[i].host_id) {
+found_host_id = true;
+break;
+}
+}
+
+if (! found_host_id) {
+virReportError(VIR_ERR_XML_ERROR,
+_("'cache bank's host id %u not found on the host"),
+bank[i].host_id);
+goto cleanup;
+}
+
+if (bank[i].size == 0 ||
+bank[i].size % resctrl->cache_banks[j].cache_min

[libvirt] [PATCH resend V10 09/12] Resctrl: concurrence support

2017-03-06 Thread Eli Qiao

The internal struct list domainall is a list which are resctral domain
status shared by all VMs, especiall the default domain, each VM should
access it concomitantly. Ues a mutex to control it.

Each bank's cache_left field is also a global shared resource we need
to be care, add a mutex for each bank.

We need also to add lock to access /sys/fs/resctrl, use flock.

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 src/util/virresctrl.c | 93 ---
 src/util/virresctrl.h |  3 ++
 2 files changed, 83 insertions(+), 13 deletions(-)

diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c
index 9f0d05f..3c54e44 100644
--- a/src/util/virresctrl.c
+++ b/src/util/virresctrl.c
@@ -20,7 +20,9 @@
  */
 #include 
 #include 
+#include 
 #include 
+#include 
 
 #include "virresctrl.h"
 #include "viralloc.h"
@@ -62,6 +64,9 @@ do { \
 
 #define VIR_RESCTRL_GET_SCHEMATA(count) ((1 << count) - 1)
 
+#define VIR_RESCTRL_LOCK(fd, op) flock(fd, op)
+#define VIR_RESCTRL_UNLOCK(fd) flock(fd, LOCK_UN)
+
 /**
  * a virResSchemata represents a schemata object under a resource control
  * domain.
@@ -105,6 +110,8 @@ typedef virResCtrlDomain *virResCtrlDomainPtr;
 struct _virResCtrlDomain {
 unsigned int num_domains;
 virResDomainPtr domains;
+
+virMutex lock;
 };
 
 static unsigned int host_id;
@@ -176,11 +183,16 @@ static int virResCtrlGetStr(const char *domain_name, 
const char *item_name, char
 
 CONSTRUCT_RESCTRL_PATH(domain_name, item_name);
 
+if (!virFileExists(path))
+goto cleanup;
+
 if (virFileReadAll(path, MAX_FILE_LEN, ret) < 0) {
 rc = -1;
 goto cleanup;
 }
 
+rc = 0;
+
  cleanup:
 VIR_FREE(path);
 return rc;
@@ -687,10 +699,15 @@ static int
 virResCtrlAppendDomain(virResDomainPtr dom)
 {
 virResDomainPtr p = domainall.domains;
+
+virMutexLock();
+
 while (p->next != NULL) p = p->next;
 p->next = dom;
 dom->pre = p;
 domainall.num_domains += 1;
+
+virMutexUnlock();
 return 0;
 }
 
@@ -713,18 +730,22 @@ virResCtrlCalculateSchemata(int type,
 {
 size_t i;
 int count;
+int rc = -1;
 virResDomainPtr p;
 unsigned int tmp_schemata;
 unsigned int schemata_sum = 0;
 int pair_type = 0;
 
+virMutexLock([type].cache_banks[sid].lock);
+
 if (resctrlall[type].cache_banks[sid].cache_left < size) {
 VIR_ERROR(_("Not enough cache left on bank %u"), hostid);
-return -1;
+goto cleanup;
 }
+
 if ((count = size / resctrlall[type].cache_banks[sid].cache_min) <= 0) {
 VIR_ERROR(_("Error cache size %llu"), size);
-return -1;
+goto cleanup;
 }
 
 tmp_schemata = VIR_RESCTRL_GET_SCHEMATA(count);
@@ -739,7 +760,7 @@ virResCtrlCalculateSchemata(int type,
 if (type == VIR_RDT_RESOURCE_L3CODE)
 pair_type = VIR_RDT_RESOURCE_L3DATA;
 
-for (i = 1; i < domainall.num_domains; i ++) {
+for (i = 1; i < domainall.num_domains; i++) {
 schemata_sum |= p->schematas[type]->schemata_items[sid].schemata;
 if (pair_type > 0)
 schemata_sum |= 
p->schematas[pair_type]->schemata_items[sid].schemata;
@@ -750,7 +771,16 @@ virResCtrlCalculateSchemata(int type,
 
 while ((tmp_schemata & schemata_sum) != 0)
 tmp_schemata = tmp_schemata >> 1;
-return tmp_schemata;
+
+resctrlall[type].cache_banks[sid].cache_left -= size;
+if (pair_type > 0)
+resctrlall[pair_type].cache_banks[sid].cache_left = 
resctrlall[type].cache_banks[sid].cache_left;
+
+rc = tmp_schemata;
+
+ cleanup:
+virMutexUnlock([type].cache_banks[sid].lock);
+return rc;
 }
 
 int virResCtrlSetCacheBanks(virDomainCachetunePtr cachetune,
@@ -763,8 +793,8 @@ int virResCtrlSetCacheBanks(virDomainCachetunePtr cachetune,
 int pair_type = -1;
 int sid;
 int schemata;
-
-virUUIDFormat(uuid, name);
+int lockfd;
+int rc = -1;
 
 for (i = 0; i < cachetune->n_banks; i++) {
 VIR_DEBUG("cache_banks %u, %u, %llu, %s",
@@ -777,12 +807,21 @@ int virResCtrlSetCacheBanks(virDomainCachetunePtr 
cachetune,
 if (cachetune->n_banks < 1)
 return 0;
 
+virUUIDFormat(uuid, name);
+
+if ((lockfd = open(RESCTRL_DIR, O_RDONLY)) < 0)
+goto cleanup;
+
+if (VIR_RESCTRL_LOCK(lockfd, LOCK_EX) < 0) {
+virReportSystemError(errno, _("Unable to lock '%s'"), RESCTRL_DIR);
+goto cleanup;
+}
+
 p = virResCtrlGetDomain(name);
 if (p == NULL) {
 VIR_DEBUG("no domain name %s found, create new one!", name);
 p = virResCtrlCreateDomain(name);
 }
-
 if (p != NULL) {
 
 virResCtrlAppendDomain(p);
@@ -828,19 +867,25 @@ int virResCtrlSetCacheBanks(virDomainCachetunePtr 
cachetune,
 if (virResCtrlFlushDomainToSysfs(p) < 0) {

[libvirt] [PATCH resend V10 05/12] Qemu: Set cache tune while booting a new domain.

2017-03-06 Thread Eli Qiao

While user can assign some specific vcpus list in , adds the
vcpus' pids to cache bank, else vm->pid will be added to cache bank.

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 src/qemu/qemu_driver.c  |  6 --
 src/qemu/qemu_process.c | 54 +
 2 files changed, 58 insertions(+), 2 deletions(-)

diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c
index 3bfb4ec..0f11ae2 100644
--- a/src/qemu/qemu_driver.c
+++ b/src/qemu/qemu_driver.c
@@ -850,8 +850,10 @@ qemuStateInitialize(bool privileged,
 run_gid = cfg->group;
 }
 
-if (virResCtrlAvailable() && virResCtrlInit() < 0)
-VIR_WARN("Faild to initialize resource control.");
+if (virResCtrlAvailable() && virResCtrlInit() < 0) {
+VIR_ERROR(_("Faild to initialize resource control"));
+goto error;
+}
 
 qemu_driver->qemuCapsCache = virQEMUCapsCacheNew(cfg->libDir,
  cfg->cacheDir,
diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c
index 68378c9..5935502 100644
--- a/src/qemu/qemu_process.c
+++ b/src/qemu/qemu_process.c
@@ -76,6 +76,7 @@
 #include "configmake.h"
 #include "nwfilter_conf.h"
 #include "netdev_bandwidth_conf.h"
+#include "virresctrl.h"
 
 #define VIR_FROM_THIS VIR_FROM_QEMU
 
@@ -5055,6 +5056,50 @@ qemuProcessVcpusSortOrder(const void *a,
 return vcpua->order - vcpub->order;
 }
 
+static int
+qemuProcessSetCacheBanks(virDomainObjPtr vm)
+{
+size_t i, j;
+virDomainCachetunePtr cachetune;
+unsigned int max_vcpus = virDomainDefGetVcpusMax(vm->def);
+pid_t *pids = NULL;
+virDomainVcpuDefPtr vcpu;
+size_t npid = 0;
+size_t count = 0;
+int ret = -1;
+
+cachetune = &(vm->def->cachetune);
+
+for (i = 0; i < cachetune->n_banks; i++) {
+if (cachetune->cache_banks[i].vcpus) {
+for (j = 0; j < max_vcpus; j++) {
+if (virBitmapIsBitSet(cachetune->cache_banks[i].vcpus, j)) {
+
+vcpu = virDomainDefGetVcpu(vm->def, j);
+if (!vcpu->online)
+continue;
+
+if (VIR_RESIZE_N(pids, npid, count, 1) < 0)
+goto cleanup;
+pids[count ++] = qemuDomainGetVcpuPid(vm, j);
+}
+}
+}
+}
+
+/* If not specific vcpus in cachetune, add vm->pid */
+if (pids == NULL) {
+if (VIR_ALLOC_N(pids, 1) < 0)
+goto cleanup;
+pids[0] = vm->pid;
+count = 1;
+}
+ret = virResCtrlSetCacheBanks(cachetune, vm->def->uuid, pids, count);
+
+ cleanup:
+VIR_FREE(pids);
+return ret;
+}
 
 static int
 qemuProcessSetupHotpluggableVcpus(virQEMUDriverPtr driver,
@@ -5749,6 +5794,11 @@ qemuProcessLaunch(virConnectPtr conn,
 qemuProcessAutoDestroyAdd(driver, vm, conn) < 0)
 goto cleanup;
 
+VIR_DEBUG("Cache allocation");
+
+if (virResCtrlAvailable() && qemuProcessSetCacheBanks(vm) < 0)
+goto cleanup;
+
 ret = 0;
 
  cleanup:
@@ -6251,6 +6301,10 @@ void qemuProcessStop(virQEMUDriverPtr driver,
 virPerfFree(priv->perf);
 priv->perf = NULL;
 
+if (virResCtrlAvailable() && virResCtrlUpdate(vm->def->uuid) < 0)
+VIR_WARN("Failed to update resource control for %s",
+ vm->def->name);
+
 qemuProcessRemoveDomainStatus(driver, vm);
 
 /* Remove VNC and Spice ports from port reservation bitmap, but only if
-- 
1.9.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATCH resend V10 10/12] Resctrl: Scan resctrl before doing cache allocation

2017-03-06 Thread Eli Qiao

Other application may touch resctrl while libvirt's running, scan
resctrl again before allocating cache information to the newly created
VM.

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 src/util/virresctrl.c | 40 
 1 file changed, 40 insertions(+)

diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c
index 3c54e44..16c01a2 100644
--- a/src/util/virresctrl.c
+++ b/src/util/virresctrl.c
@@ -711,6 +711,42 @@ virResCtrlAppendDomain(virResDomainPtr dom)
 return 0;
 }
 
+/* scan /sys/fs/resctrl again and refresh default schemata */
+static
+int virResCtrlScan(void)
+{
+struct dirent *ent;
+DIR *dp = NULL;
+int direrr;
+virResDomainPtr p;
+int rc = -1;
+
+if (virDirOpenQuiet(, RESCTRL_DIR) < 0) {
+if (errno == ENOENT)
+return -1;
+VIR_ERROR(_("Unable to open %s (%d)"), RESCTRL_DIR, errno);
+goto cleanup;
+}
+
+while ((direrr = virDirRead(dp, , NULL)) > 0) {
+if ((ent->d_type != DT_DIR) || STREQ(ent->d_name, "info"))
+continue;
+/* test if we'v tracked all domains */
+p = virResCtrlGetDomain(ent->d_name);
+if (p == NULL) {
+p = virResCtrlLoadDomain(ent->d_name);
+if (p == NULL)
+continue;
+virResCtrlAppendDomain(p);
+}
+}
+rc = 0;
+
+ cleanup:
+VIR_DIR_CLOSE(dp);
+return rc;
+}
+
 static int
 virResCtrlGetSocketIdByHostID(int type, unsigned int hostid)
 {
@@ -817,6 +853,10 @@ int virResCtrlSetCacheBanks(virDomainCachetunePtr 
cachetune,
 goto cleanup;
 }
 
+if (virResCtrlScan() < 0) {
+VIR_ERROR(_("Failed to scan resctrl domain dir"));
+goto cleanup;
+}
 p = virResCtrlGetDomain(name);
 if (p == NULL) {
 VIR_DEBUG("no domain name %s found, create new one!", name);
-- 
1.9.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATH V10 05/12] Qemu: Set cache tune while booting a new domain.

2017-03-06 Thread Eli Qiao

While user can assign some specific vcpus list in , adds the
vcpus' pids to cache bank, else vm->pid will be added to cache bank.

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 src/qemu/qemu_driver.c  |  6 --
 src/qemu/qemu_process.c | 54 +
 2 files changed, 58 insertions(+), 2 deletions(-)

diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c
index 3bfb4ec..0f11ae2 100644
--- a/src/qemu/qemu_driver.c
+++ b/src/qemu/qemu_driver.c
@@ -850,8 +850,10 @@ qemuStateInitialize(bool privileged,
 run_gid = cfg->group;
 }
 
-if (virResCtrlAvailable() && virResCtrlInit() < 0)
-VIR_WARN("Faild to initialize resource control.");
+if (virResCtrlAvailable() && virResCtrlInit() < 0) {
+VIR_ERROR(_("Faild to initialize resource control"));
+goto error;
+}
 
 qemu_driver->qemuCapsCache = virQEMUCapsCacheNew(cfg->libDir,
  cfg->cacheDir,
diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c
index 68378c9..5935502 100644
--- a/src/qemu/qemu_process.c
+++ b/src/qemu/qemu_process.c
@@ -76,6 +76,7 @@
 #include "configmake.h"
 #include "nwfilter_conf.h"
 #include "netdev_bandwidth_conf.h"
+#include "virresctrl.h"
 
 #define VIR_FROM_THIS VIR_FROM_QEMU
 
@@ -5055,6 +5056,50 @@ qemuProcessVcpusSortOrder(const void *a,
 return vcpua->order - vcpub->order;
 }
 
+static int
+qemuProcessSetCacheBanks(virDomainObjPtr vm)
+{
+size_t i, j;
+virDomainCachetunePtr cachetune;
+unsigned int max_vcpus = virDomainDefGetVcpusMax(vm->def);
+pid_t *pids = NULL;
+virDomainVcpuDefPtr vcpu;
+size_t npid = 0;
+size_t count = 0;
+int ret = -1;
+
+cachetune = &(vm->def->cachetune);
+
+for (i = 0; i < cachetune->n_banks; i++) {
+if (cachetune->cache_banks[i].vcpus) {
+for (j = 0; j < max_vcpus; j++) {
+if (virBitmapIsBitSet(cachetune->cache_banks[i].vcpus, j)) {
+
+vcpu = virDomainDefGetVcpu(vm->def, j);
+if (!vcpu->online)
+continue;
+
+if (VIR_RESIZE_N(pids, npid, count, 1) < 0)
+goto cleanup;
+pids[count ++] = qemuDomainGetVcpuPid(vm, j);
+}
+}
+}
+}
+
+/* If not specific vcpus in cachetune, add vm->pid */
+if (pids == NULL) {
+if (VIR_ALLOC_N(pids, 1) < 0)
+goto cleanup;
+pids[0] = vm->pid;
+count = 1;
+}
+ret = virResCtrlSetCacheBanks(cachetune, vm->def->uuid, pids, count);
+
+ cleanup:
+VIR_FREE(pids);
+return ret;
+}
 
 static int
 qemuProcessSetupHotpluggableVcpus(virQEMUDriverPtr driver,
@@ -5749,6 +5794,11 @@ qemuProcessLaunch(virConnectPtr conn,
 qemuProcessAutoDestroyAdd(driver, vm, conn) < 0)
 goto cleanup;
 
+VIR_DEBUG("Cache allocation");
+
+if (virResCtrlAvailable() && qemuProcessSetCacheBanks(vm) < 0)
+goto cleanup;
+
 ret = 0;
 
  cleanup:
@@ -6251,6 +6301,10 @@ void qemuProcessStop(virQEMUDriverPtr driver,
 virPerfFree(priv->perf);
 priv->perf = NULL;
 
+if (virResCtrlAvailable() && virResCtrlUpdate(vm->def->uuid) < 0)
+VIR_WARN("Failed to update resource control for %s",
+ vm->def->name);
+
 qemuProcessRemoveDomainStatus(driver, vm);
 
 /* Remove VNC and Spice ports from port reservation bitmap, but only if
-- 
1.9.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATH V10 04/12] Resctrl: Add private interfaces to operate cache bank

2017-03-06 Thread Eli Qiao

virResCtrlSetCacheBanks: Set cache banks of a libvirt domain. It will
 create new resource domain under
 `/sys/fs/resctrl` and fill the schemata according
 the cache bank configration.

virResCtrlUpdate: Destroy resctrl domain directory, and update the schemata
  after libvirt domain destroy.

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 src/libvirt_private.syms |   6 +-
 src/util/virresctrl.c| 677 +++
 src/util/virresctrl.h|   5 +-
 3 files changed, 686 insertions(+), 2 deletions(-)

diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms
index b8445ef..9cfffb8 100644
--- a/src/libvirt_private.syms
+++ b/src/libvirt_private.syms
@@ -2325,7 +2325,11 @@ virRandomInt;
 virResCtrlAvailable;
 virResCtrlGet;
 virResCtrlInit;
-
+virResCtrlSetCacheBanks;
+virResCtrlTypeFromString;
+virResCtrlTypeToString;
+virResCtrlUpdate;
+#
 
 # util/virrotatingfile.h
 virRotatingFileReaderConsume;
diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c
index eee6675..43af0f5 100644
--- a/src/util/virresctrl.c
+++ b/src/util/virresctrl.c
@@ -19,6 +19,8 @@
  *  Eli Qiao <liyong.q...@intel.com>
  */
 #include 
+#include 
+#include 
 
 #include "virresctrl.h"
 #include "viralloc.h"
@@ -61,8 +63,58 @@ do { \
 
 #define VIR_RESCTRL_GET_SCHEMATA(count) ((1 << count) - 1)
 
+/**
+ * a virResSchemata represents a schemata object under a resource control
+ * domain.
+ */
+typedef struct _virResSchemataItem virResSchemataItem;
+typedef virResSchemataItem *virResSchemataItemPtr;
+struct _virResSchemataItem {
+unsigned int socket_no;
+unsigned schemata;
+};
+
+typedef struct _virResSchemata virResSchemata;
+typedef virResSchemata *virResSchemataPtr;
+struct _virResSchemata {
+unsigned int n_schemata_items;
+virResSchemataItemPtr schemata_items;
+};
+
+/**
+ * a virResDomain represents a resource control domain. It's a double linked
+ * list.
+ */
+
+typedef struct _virResDomain virResDomain;
+typedef virResDomain *virResDomainPtr;
+
+struct _virResDomain {
+char *name;
+virResSchemataPtr schematas[VIR_RDT_RESOURCE_LAST];
+char **tasks;
+size_t n_tasks;
+size_t n_sockets;
+virResDomainPtr pre;
+virResDomainPtr next;
+};
+
+/* All resource control domains on this host*/
+typedef struct _virResCtrlDomain virResCtrlDomain;
+typedef virResCtrlDomain *virResCtrlDomainPtr;
+
+struct _virResCtrlDomain {
+unsigned int num_domains;
+virResDomainPtr domains;
+};
+
 static unsigned int host_id;
 
+/* Global static struct to be maintained which is a interface */
+static virResCtrlDomain domainall;
+
+/* Global static struct array to be maintained which indicate
+ * resource status on a host */
 static virResCtrl resctrlall[] = {
 {
 .name = "L3",
@@ -82,6 +134,78 @@ static virResCtrl resctrlall[] = {
 },
 };
 
+/*
+ * How many bits is set in schemata
+ * eg:
+ * virResCtrlBitsNum(10110) = 2 */
+static int virResCtrlBitsContinuesNum(unsigned schemata)
+{
+size_t i;
+int ret = 0;
+for (i = 0; i < MAX_CBM_BIT_LEN; i ++) {
+if ((schemata & 0x1) == 0x1)
+ret++;
+else
+if (ret > 0 || schemata == 0) break;
+
+schemata = schemata >> 1;
+}
+return ret;
+}
+
+/* Position of the highest continue 1 bit of in schemata
+ * eg:
+ * virResctrlBitsContinuesPos(10110) = 3 */
+static int virResCtrlBitsContinuesPos(unsigned schemata)
+{
+size_t i;
+int flag = 0;
+for (i = 0; i < MAX_CBM_BIT_LEN; i ++) {
+if ((schemata & 0x1) == 0x0 && flag == 1)
+return i;
+else if ((schemata & 0x1) == 0x1) flag = 1;
+
+schemata = schemata >> 1;
+}
+return 0;
+}
+
+static int virResCtrlGetStr(const char *domain_name, const char *item_name, 
char **ret)
+{
+char *path;
+int rc = 0;
+
+CONSTRUCT_RESCTRL_PATH(domain_name, item_name);
+
+if (virFileReadAll(path, MAX_FILE_LEN, ret) < 0) {
+rc = -1;
+goto cleanup;
+}
+
+ cleanup:
+VIR_FREE(path);
+return rc;
+}
+
+static int virResCtrlGetSchemata(const int type, const char *name, char 
**schemata)
+{
+int rc;
+char *tmp, *end;
+char *buf;
+
+if ((rc = virResCtrlGetStr(name, "schemata", )) < 0)
+return rc;
+
+tmp = strstr(buf, resctrlall[type].name);
+end = strchr(tmp, '\n');
+*end = '\0';
+if (VIR_STRDUP(*schemata, tmp) < 0)
+rc = -1;
+
+VIR_FREE(buf);
+return rc;
+}
+
 static int virResCtrlGetInfoStr(const int type, const char *item, char **str)
 {
 int ret = 0;
@@ -102,6 +226,71 @@ static int virResCtrlGetInfoStr(const int type, const char 
*item, char **str)
 return ret;
 }
 
+/* Return pointer of and ncount of schemata*/
+static virResSchemataPtr virParseSchemata(const char *schema

Re: [libvirt] [PATH v9 12/12] Resctrl: Add nodecachestats

2017-03-06 Thread Eli Qiao



Best regards  
- Eli

天涯无处不重逢
a leaf duckweed belongs to the sea, where not to meet in life  

2017-03-03 10:24 GMT+08:00 Marcelo Tosatti <mtosa...@redhat.com 
(mailto:mtosa...@redhat.com)>:
> On Mon, Feb 27, 2017 at 03:22:56PM +0800, Eli Qiao wrote:
> > Add new virsh command line `nodecachestats` to expose the cache usage
> > on a node.
>  
> Testing:
>  
> Setup: one guest with 3072 bytes reserved for it.

hi Marcelo

Thanks for the testing.  

I improved default schemata calculating from V10, and V10 is on the top of 
master branch.

>  
> Test-1: start libvirt with free cache.
> resctrlfs default: L3:0=f;1=f
>  
> # LD_LIBRARY_PATH=/root/git/libvirt/src/.libs/ ./virshnodegetcachestats
> ret=0 nparams=10
> L3.0: 13824
> L3.1: 13824
>  
> Result: correct.
>  
> Test-2: start libvirt with a manual reservation in place
> (sharing space with default reservation, which is a
> valid scenario).
>  
> ./schemata: L3:0=f;1=f
> ./manualres/schemata: L3:0=f;1=f
>  
> How many bytes free there are to allocate?
> Mask 0x0 is free. Bytes = 12288.
>  
> After start libvirt daemon:
> Value of resctrlfs files:
> ./schemata: L3:0=f;1=f
> ./manualres/schemata: L3:0=f;1=f
>  
> Run cmd:
> # LD_LIBRARY_PATH=/root/git/libvirt/src/.libs/ 
> /root/git/virshnodegetcachestats
> ret=0 nparams=10
> L3.0: 0
> L3.1: 0
>  
> ./schemata: L3:0=3;1=3
> ./manualres/schemata: L3:0=f;1=f1
>  
> Result: incorrect.
>  
> Test-3: start libvirt with a manual reservation in place
> (not sharing space with default reservation):
>  
> ./schemata: L3:0=0;1=0
> ./manualres/schemata: L3:0=f;1=f
>  
> How many bytes free there are to allocate?
> Mask 0x0 is free. Bytes = 12288.
>  
> ./schemata: L3:0=0;1=0
> ./manualres/schemata: L3:0=f;1=f
>  
> # LD_LIBRARY_PATH=/root/git/libvirt/src/.libs/ 
> /root/git/virshnodegetcachestats
> ret=0 nparams=10
> L3.0: 0
> L3.1: 0
>  
> So the interface is very buggy, please fix it (and execute the tests
> above to make sure it is fixed).
>  


  
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATH V10 11/12] Resctrl: Add Public API for nodecachestats

2017-03-06 Thread Eli Qiao

This patch expose a public API virNodeCacheStats to query cache stats
on a host.

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 daemon/remote.c| 67 ++
 include/libvirt/libvirt-host.h | 32 
 src/driver-hypervisor.h|  7 +
 src/libvirt-host.c | 41 ++
 src/libvirt_public.syms|  1 +
 src/remote/remote_driver.c | 52 
 src/remote/remote_protocol.x   | 25 +++-
 src/remote_protocol-structs| 16 ++
 8 files changed, 240 insertions(+), 1 deletion(-)

diff --git a/daemon/remote.c b/daemon/remote.c
index f2b9b9a..af291a7 100644
--- a/daemon/remote.c
+++ b/daemon/remote.c
@@ -7079,3 +7079,70 @@ remoteSerializeDomainDiskErrors(virDomainDiskErrorPtr 
errors,
 }
 return -1;
 }
+
+ static int
+remoteDispatchNodeGetCacheStats(virNetServerPtr server ATTRIBUTE_UNUSED,
+virNetServerClientPtr client ATTRIBUTE_UNUSED,
+virNetMessagePtr msg ATTRIBUTE_UNUSED,
+virNetMessageErrorPtr rerr,
+remote_node_get_cache_stats_args *args,
+remote_node_get_cache_stats_ret *ret)
+{
+virNodeCacheStatsPtr params = NULL;
+size_t i;
+int nparams = 0;
+unsigned int flags;
+int rv = -1;
+struct daemonClientPrivate *priv =
+virNetServerClientGetPrivateData(client);
+
+if (!priv->conn) {
+virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("connection not open"));
+goto cleanup;
+}
+
+flags = args->flags;
+
+if (args->nparams && VIR_ALLOC_N(params, args->nparams) < 0)
+goto cleanup;
+nparams = args->nparams;
+
+if (virNodeGetCacheStats(priv->conn, params, , flags) < 0)
+goto cleanup;
+
+/* In this case, we need to send back the number of stats
+ * supported
+ */
+if (args->nparams == 0) {
+ret->nparams = nparams;
+goto success;
+}
+
+/* Serialise the memory parameters. */
+ret->params.params_len = nparams;
+if (VIR_ALLOC_N(ret->params.params_val, nparams) < 0)
+goto cleanup;
+
+for (i = 0; i < nparams; ++i) {
+/* remoteDispatchClientRequest will free this: */
+if (VIR_STRDUP(ret->params.params_val[i].field, params[i].field) < 0)
+goto cleanup;
+
+ret->params.params_val[i].value = params[i].value;
+}
+
+ success:
+rv = 0;
+
+ cleanup:
+if (rv < 0) {
+virNetMessageSaveError(rerr);
+if (ret->params.params_val) {
+for (i = 0; i < nparams; i++)
+VIR_FREE(ret->params.params_val[i].field);
+VIR_FREE(ret->params.params_val);
+}
+}
+VIR_FREE(params);
+return rv;
+}
diff --git a/include/libvirt/libvirt-host.h b/include/libvirt/libvirt-host.h
index 07b5d15..222f361 100644
--- a/include/libvirt/libvirt-host.h
+++ b/include/libvirt/libvirt-host.h
@@ -140,6 +140,32 @@ struct _virSecurityModel {
  */
 typedef virSecurityModel *virSecurityModelPtr;
 
+/**
+ * VIR_NODE_CACHE_STATS_FIELD_LENGTH:
+ *
+ * Macro providing the field length of virNodeCacheStats
+ */
+
+# define VIR_NODE_CACHE_STATS_FIELD_LENGTH 16
+
+/**
+ *
+ * virNodeCacheStats is all the cache stats on a host.
+ */
+
+typedef struct _virNodeCacheStats virNodeCacheStats;
+
+struct _virNodeCacheStats {
+char field[VIR_NODE_CACHE_STATS_FIELD_LENGTH];
+unsigned long long value;
+};
+
+/**
+ * a virNodeCacheStatsPtr is a pointer to a virNodeCacheStats.
+ */
+
+typedef virNodeCacheStats *virNodeCacheStatsPtr;
+
 
 /* data types related to virNodePtr */
 
@@ -603,6 +629,12 @@ int virNodeSuspendForDuration 
(virConnectPtr conn,
unsigned long long duration,
unsigned int flags);
 
+int virNodeGetCacheStats (virConnectPtr conn,
+  virNodeCacheStatsPtr params,
+  int *nparams,
+  unsigned int flags);
+
+
 /*
  * NUMA support
  */
diff --git a/src/driver-hypervisor.h b/src/driver-hypervisor.h
index b81420a..a6a1350 100644
--- a/src/driver-hypervisor.h
+++ b/src/driver-hypervisor.h
@@ -578,6 +578,12 @@ typedef int
 unsigned int flags);
 
 typedef int
+(*virDrvNodeGetCacheStats)(virConnectPtr conn,
+   virNodeCacheStatsPtr params,
+   int *nparams,
+   unsigned int flags);
+
+typedef int
 (*virDrvNodeGetCellsFreeMemory)(virConnectPtr conn,
 unsigned long long *freeMem

[libvirt] [PATH V10 07/12] Resctrl: Make sure l3data/l3code are pairs

2017-03-06 Thread Eli Qiao

l3data and l3code type of cache banks should be configured pairs.

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 src/conf/domain_conf.c | 19 +++
 src/util/virresctrl.c  |  1 -
 src/util/virresctrl.h  |  2 ++
 3 files changed, 21 insertions(+), 1 deletion(-)

diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
index 652f4ca..86c292d 100644
--- a/src/conf/domain_conf.c
+++ b/src/conf/domain_conf.c
@@ -15760,9 +15760,13 @@ virDomainCacheTuneDefParseXML(virDomainDefPtr def,
 int type = -1;
 virDomainCacheBankPtr bank = NULL;
 virResCtrlPtr resctrl;
+/* An array to make sure l3code and l3data are pairs */
+int* sem = NULL;
 
 if (VIR_ALLOC_N(bank, n) < 0)
 goto cleanup;
+if (VIR_ALLOC_N(sem, MAX_CPU_SOCKET_NUM) < 0)
+goto cleanup;
 
 for (i = 0; i < n; i++) {
 if (!(tmp = virXMLPropString(nodes[i], "id"))) {
@@ -15810,6 +15814,12 @@ virDomainCacheTuneDefParseXML(virDomainDefPtr def,
 goto cleanup;
 }
 
+/* VIR_RDT_RESOURCE_L3DATA and VIR_RDT_RESOURCE_L3CODE should be pair 
*/
+if (type == VIR_RDT_RESOURCE_L3DATA)
+sem[bank[i].host_id] ++;
+else if (type == VIR_RDT_RESOURCE_L3CODE)
+sem[bank[i].host_id] --;
+
 resctrl = virResCtrlGet(type);
 
 if (resctrl == NULL || !resctrl->enabled) {
@@ -15858,6 +15868,14 @@ virDomainCacheTuneDefParseXML(virDomainDefPtr def,
 }
 }
 
+for (i = 0; i < MAX_CPU_SOCKET_NUM; i ++) {
+if (sem[i] != 0) {
+virReportError(VIR_ERR_XML_ERROR,
+_("'l3code and l3data shoud be show up pairs on bank 
%zu'"),
+i);
+goto cleanup;
+}
+}
 def->cachetune.cache_banks = bank;
 def->cachetune.n_banks = n;
 return 0;
@@ -15865,6 +15883,7 @@ virDomainCacheTuneDefParseXML(virDomainDefPtr def,
  cleanup:
 VIR_FREE(bank);
 VIR_FREE(tmp);
+VIR_FREE(sem);
 return -1;
 }
 
diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c
index e7376a0..ee5f043 100644
--- a/src/util/virresctrl.c
+++ b/src/util/virresctrl.c
@@ -34,7 +34,6 @@
 VIR_LOG_INIT("util.resctrl");
 
 #define VIR_FROM_THIS VIR_FROM_RESCTRL
-#define MAX_CPU_SOCKET_NUM 8
 #define MAX_CBM_BIT_LEN 32
 #define MAX_SCHEMATA_LEN 1024
 #define MAX_FILE_LEN (10 * 1024 * 1024)
diff --git a/src/util/virresctrl.h b/src/util/virresctrl.h
index 3a6fb95..d639de1 100644
--- a/src/util/virresctrl.h
+++ b/src/util/virresctrl.h
@@ -28,6 +28,8 @@
 # include "virutil.h"
 # include "conf/domain_conf.h"
 
+#define MAX_CPU_SOCKET_NUM 8
+
 enum {
 VIR_RDT_RESOURCE_L3,
 VIR_RDT_RESOURCE_L3DATA,
-- 
1.9.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATH V10 06/12] Resctrl: enable l3code/l3data

2017-03-06 Thread Eli Qiao

Enable l3code/l3data while doing cache tune.

l3code/l3data should use a continus cbm in their seperated schemata and
the cache size are shared between them, so we need to deal them
differently with l3 cache.

This should enable cdp feature while mounting /sys/fs/resctrl, eg:
mount -t resctrl resctrl -o cdp  /sys/fs/resctrl

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 docs/schemas/domaincommon.rng |  2 +-
 src/util/virresctrl.c | 27 +--
 2 files changed, 26 insertions(+), 3 deletions(-)

diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng
index efc84c5..ed8bdb9 100644
--- a/docs/schemas/domaincommon.rng
+++ b/docs/schemas/domaincommon.rng
@@ -5528,7 +5528,7 @@
   
   
 
-  (l3)
+  (l3|l3code|l3data)
 
   
   
diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c
index 43af0f5..e7376a0 100644
--- a/src/util/virresctrl.c
+++ b/src/util/virresctrl.c
@@ -417,6 +417,7 @@ int virResCtrlRefreshSchemata(void)
 unsigned int tmp_schemata;
 unsigned int default_schemata;
 unsigned int min_schemata;
+int pair_type = 0;
 
 virResDomainPtr header, p;
 
@@ -429,6 +430,11 @@ int virResCtrlRefreshSchemata(void)
 if (VIR_RESCTRL_ENABLED(i)) {
 min_schemata = 
VIR_RESCTRL_GET_SCHEMATA(resctrlall[i].min_cbm_bits);
 
+if (i == VIR_RDT_RESOURCE_L3DATA)
+pair_type = VIR_RDT_RESOURCE_L3CODE;
+if (i == VIR_RDT_RESOURCE_L3CODE)
+pair_type = VIR_RDT_RESOURCE_L3DATA;
+
 for (j = 0; j < header->schematas[i]->n_schemata_items; j ++) {
 p = header->next;
 // Reset to default schemata 0xf
@@ -436,8 +442,11 @@ int virResCtrlRefreshSchemata(void)
 tmp_schemata = 0;
 /* NOTEs: if only header domain, the schemata will be set to 
default one*/
 for (k = 1; k < domainall.num_domains; k++) {
-if (p->schematas[i]->schemata_items[j].schemata > 
min_schemata)
+if (p->schematas[i]->schemata_items[j].schemata > 
min_schemata) {
 tmp_schemata |= 
p->schematas[i]->schemata_items[j].schemata;
+if (pair_type > 0)
+tmp_schemata |= 
p->schematas[pair_type]->schemata_items[j].schemata;
+}
 p = p->next;
 }
 
@@ -503,6 +512,7 @@ virResCtrlWrite(const char *name, const char *item, const 
char *content)
 goto cleanup;
 
 rc = 0;
+
  cleanup:
 VIR_FREE(path);
 VIR_FORCE_CLOSE(writefd);
@@ -707,6 +717,7 @@ virResCtrlCalculateSchemata(int type,
 virResDomainPtr p;
 unsigned int tmp_schemata;
 unsigned int schemata_sum = 0;
+int pair_type = 0;
 
 if (resctrlall[type].cache_banks[sid].cache_left < size) {
 VIR_ERROR(_("Not enough cache left on bank %u"), hostid);
@@ -721,8 +732,18 @@ virResCtrlCalculateSchemata(int type,
 
 p = domainall.domains;
 p = p->next;
+
+/* for type is l3code and l3data, we need to deal them specially*/
+if (type == VIR_RDT_RESOURCE_L3DATA)
+pair_type = VIR_RDT_RESOURCE_L3CODE;
+
+if (type == VIR_RDT_RESOURCE_L3CODE)
+pair_type = VIR_RDT_RESOURCE_L3DATA;
+
 for (i = 1; i < domainall.num_domains; i ++) {
 schemata_sum |= p->schematas[type]->schemata_items[sid].schemata;
+if (pair_type > 0)
+schemata_sum |= 
p->schematas[pair_type]->schemata_items[sid].schemata;
 p = p->next;
 }
 
@@ -763,6 +784,9 @@ int virResCtrlSetCacheBanks(virDomainCachetunePtr cachetune,
 }
 
 if (p != NULL) {
+
+virResCtrlAppendDomain(p);
+
 for (i = 0; i < cachetune->n_banks; i++) {
 if ((type = virResCtrlTypeFromString(
 cachetune->cache_banks[i].type)) < 0) {
@@ -797,7 +821,6 @@ int virResCtrlSetCacheBanks(virDomainCachetunePtr cachetune,
 virResCtrlDestroyDomain(p);
 return -1;
 }
-virResCtrlAppendDomain(p);
 } else {
 VIR_ERROR(_("Failed to create a domain in sysfs"));
 return -1;
-- 
1.9.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATH V10 12/12] Resctrl: Add nodecachestats

2017-03-06 Thread Eli Qiao

Add new virsh command line `nodecachestats` to expose the cache usage
on a node.

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 src/libvirt_private.syms |  3 ++-
 src/qemu/qemu_driver.c   | 12 ++
 src/util/virresctrl.c| 62 
 src/util/virresctrl.h|  8 +++
 tools/virsh-host.c   | 49 ++
 5 files changed, 133 insertions(+), 1 deletion(-)

diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms
index 9cfffb8..75a4c98 100644
--- a/src/libvirt_private.syms
+++ b/src/libvirt_private.syms
@@ -2323,13 +2323,14 @@ virRandomInt;
 
 # util/virresctrl.h
 virResCtrlAvailable;
+virResCtrlCacheGetStats;
 virResCtrlGet;
 virResCtrlInit;
 virResCtrlSetCacheBanks;
 virResCtrlTypeFromString;
 virResCtrlTypeToString;
 virResCtrlUpdate;
-#
+
 
 # util/virrotatingfile.h
 virRotatingFileReaderConsume;
diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c
index 0f11ae2..4677406 100644
--- a/src/qemu/qemu_driver.c
+++ b/src/qemu/qemu_driver.c
@@ -18285,6 +18285,17 @@ qemuNodeGetCPUStats(virConnectPtr conn,
 return virHostCPUGetStats(cpuNum, params, nparams, flags);
 }
 
+static int
+qemuNodeGetCacheStats(virConnectPtr conn,
+virNodeCacheStatsPtr params,
+int *nparams,
+unsigned int flags)
+{
+if (virNodeGetCacheStatsEnsureACL(conn) < 0)
+return -1;
+
+return virResCtrlCacheGetStats(params, nparams, flags);
+}
 
 static int
 qemuNodeGetMemoryStats(virConnectPtr conn,
@@ -20391,6 +20402,7 @@ static virHypervisorDriver qemuHypervisorDriver = {
 .domainMemoryPeek = qemuDomainMemoryPeek, /* 0.4.4 */
 .domainGetBlockInfo = qemuDomainGetBlockInfo, /* 0.8.1 */
 .nodeGetCPUStats = qemuNodeGetCPUStats, /* 0.9.3 */
+.nodeGetCacheStats = qemuNodeGetCacheStats, /* 3.1.0 */
 .nodeGetMemoryStats = qemuNodeGetMemoryStats, /* 0.9.3 */
 .nodeGetCellsFreeMemory = qemuNodeGetCellsFreeMemory, /* 0.4.4 */
 .nodeGetFreeMemory = qemuNodeGetFreeMemory, /* 0.4.4 */
diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c
index 16c01a2..97f7e84 100644
--- a/src/util/virresctrl.c
+++ b/src/util/virresctrl.c
@@ -1034,3 +1034,65 @@ virResCtrlGet(int type)
 {
 return [type];
 }
+
+int virResCtrlCacheGetStats(virNodeCacheStatsPtr params,
+int *nparams,
+unsigned int flags)
+{
+virCheckFlags(0, -1);
+size_t i, j, k;
+char *value;
+int rc = -1;
+int lockfd;
+
+if (*nparams == 0) {
+for (i = 0; i < VIR_RDT_RESOURCE_LAST; i++) {
+if (VIR_RESCTRL_ENABLED(i))
+*nparams += resctrlall[i].num_banks;
+}
+}
+if (params == NULL)
+return 0;
+
+if ((lockfd = open(RESCTRL_DIR, O_RDONLY)) < 0)
+goto cleanup;
+
+if (VIR_RESCTRL_LOCK(lockfd, LOCK_SH) < 0) {
+virReportSystemError(errno, _("Unable to lock '%s'"), RESCTRL_DIR);
+goto cleanup;
+}
+if (virResCtrlScan() < 0) {
+VIR_ERROR(_("Failed to scan resctrl domain dir"));
+goto cleanup;
+}
+
+virResCtrlRefreshSchemata();
+
+if ((rc = virResCtrlFlushDomainToSysfs(domainall.domains)) < 0)
+goto cleanup;
+
+k = 0;
+
+for (i = 0; i < VIR_RDT_RESOURCE_LAST; i++) {
+if (VIR_RESCTRL_ENABLED(i)) {
+for (j = 0; j < resctrlall[i].num_banks; j++) {
+
+if (virAsprintf(, "%s.%zu",
+resctrlall[i].name, j) < 0)
+goto cleanup;
+
+if (virStrcpyStatic(([k])->field, value) == NULL)
+goto cleanup;
+
+([k++])->value = 
resctrlall[i].cache_banks[j].cache_left;
+}
+}
+}
+
+rc = 0;
+
+ cleanup:
+VIR_FREE(value);
+VIR_RESCTRL_UNLOCK(lockfd);
+return rc;
+}
diff --git a/src/util/virresctrl.h b/src/util/virresctrl.h
index 968e0dc..eef5370 100644
--- a/src/util/virresctrl.h
+++ b/src/util/virresctrl.h
@@ -80,9 +80,17 @@ struct _virResCtrl {
 };
 
 bool virResCtrlAvailable(void);
+
 int virResCtrlInit(void);
+
 virResCtrlPtr virResCtrlGet(int);
+
 int virResCtrlSetCacheBanks(virDomainCachetunePtr,
 unsigned char *, pid_t *, int);
+
 int virResCtrlUpdate(unsigned char *);
+
+int virResCtrlCacheGetStats(virNodeCacheStatsPtr params,
+int *nparams,
+unsigned int flags);
 #endif
diff --git a/tools/virsh-host.c b/tools/virsh-host.c
index 24ebde2..c90bd2e 100644
--- a/tools/virsh-host.c
+++ b/tools/virsh-host.c
@@ -946,6 +946,49 @@ cmdNodeMemStats(vshControl *ctl, const vshCmd *cmd)
 VIR_FREE(params);
 return ret;
 }
+/* "nodecachestats" command
+ */
+static const vshCmdInfo info_nodecachestats[] = {
+{.name = "help",
+ .data = N_("Prints cache stats

[libvirt] [PATH V10 01/12] Resctrl: Add some utils functions

2017-03-06 Thread Eli Qiao

This patch adds some utils struct and functions to expose resctrl
information.

virResCtrlAvailable: if resctrl interface exist on host.
virResCtrlGet: get specific type resource control information.
virResCtrlInit: initialize resctrl struct from the host's sys fs.
resctrlall[]: an array to maintain resource control information.

Some of host cpu related information methods was added in virhostcpu.c

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 include/libvirt/virterror.h |   1 +
 po/POTFILES.in  |   1 +
 src/Makefile.am |   1 +
 src/libvirt_private.syms|   4 +
 src/util/virerror.c |   1 +
 src/util/virhostcpu.c   | 186 
 src/util/virhostcpu.h   |   6 ++
 src/util/virresctrl.c   | 201 
 src/util/virresctrl.h   |  78 +
 9 files changed, 462 insertions(+), 17 deletions(-)
 create mode 100644 src/util/virresctrl.c
 create mode 100644 src/util/virresctrl.h

diff --git a/include/libvirt/virterror.h b/include/libvirt/virterror.h
index 2efee8f..3dd2d08 100644
--- a/include/libvirt/virterror.h
+++ b/include/libvirt/virterror.h
@@ -132,6 +132,7 @@ typedef enum {
 
 VIR_FROM_PERF = 65, /* Error from perf */
 VIR_FROM_LIBSSH = 66,   /* Error from libssh connection transport */
+VIR_FROM_RESCTRL = 67,  /* Error from resource control */
 
 # ifdef VIR_ENUM_SENTINELS
 VIR_ERR_DOMAIN_LAST
diff --git a/po/POTFILES.in b/po/POTFILES.in
index 7c7f530..4147bc6 100644
--- a/po/POTFILES.in
+++ b/po/POTFILES.in
@@ -241,6 +241,7 @@ src/util/virportallocator.c
 src/util/virprocess.c
 src/util/virqemu.c
 src/util/virrandom.c
+src/util/virresctrl.c
 src/util/virrotatingfile.c
 src/util/virscsi.c
 src/util/virscsihost.c
diff --git a/src/Makefile.am b/src/Makefile.am
index 7d42eac..edb946a 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -162,6 +162,7 @@ UTIL_SOURCES =  
\
util/virprocess.c util/virprocess.h \
util/virqemu.c util/virqemu.h   \
util/virrandom.h util/virrandom.c   \
+   util/virresctrl.h util/virresctrl.c \
util/virrotatingfile.h util/virrotatingfile.c   \
util/virscsi.c util/virscsi.h   \
util/virscsihost.c util/virscsihost.h   \
diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms
index aed1d3d..bb7c3ad 100644
--- a/src/libvirt_private.syms
+++ b/src/libvirt_private.syms
@@ -2320,6 +2320,10 @@ virRandomGenerateWWN;
 virRandomInt;
 
 
+# util/virresctrl.h
+virResCtrlAvailable;
+virResCtrlInit;
+
 # util/virrotatingfile.h
 virRotatingFileReaderConsume;
 virRotatingFileReaderFree;
diff --git a/src/util/virerror.c b/src/util/virerror.c
index ef17fb5..0ba15e6 100644
--- a/src/util/virerror.c
+++ b/src/util/virerror.c
@@ -139,6 +139,7 @@ VIR_ENUM_IMPL(virErrorDomain, VIR_ERR_DOMAIN_LAST,
 
   "Perf", /* 65 */
   "Libssh transport layer",
+  "Resouce Control",
 )
 
 
diff --git a/src/util/virhostcpu.c b/src/util/virhostcpu.c
index f29f312..e6d5102 100644
--- a/src/util/virhostcpu.c
+++ b/src/util/virhostcpu.c
@@ -206,29 +206,21 @@ void virHostCPUSetSysFSSystemPathLinux(const char *path)
 sysfs_system_path = SYSFS_SYSTEM_PATH;
 }
 
-/* Return the positive decimal contents of the given
- * DIR/cpu%u/FILE, or -1 on error.  If DEFAULT_VALUE is non-negative
- * and the file could not be found, return that instead of an error;
- * this is useful for machines that cannot hot-unplug cpu0, or where
- * hot-unplugging is disabled, or where the kernel is too old
- * to support NUMA cells, etc.  */
+/* Get a String value*/
 static int
-virHostCPUGetValue(const char *dir, unsigned int cpu, const char *file,
-   int default_value)
+virHostCPUGetStrValue(const char *dir, unsigned int cpu, const char *file, 
char *value_str)
 {
 char *path;
 FILE *pathfp;
-int value = -1;
-char value_str[INT_BUFSIZE_BOUND(value)];
-char *tmp;
+int ret = -1;
 
 if (virAsprintf(, "%s/cpu%u/%s", dir, cpu, file) < 0)
 return -1;
 
 pathfp = fopen(path, "r");
 if (pathfp == NULL) {
-if (default_value >= 0 && errno == ENOENT)
-value = default_value;
+if (errno == ENOENT)
+return -2;
 else
 virReportSystemError(errno, _("cannot open %s"), path);
 goto cleanup;
@@ -238,17 +230,84 @@ virHostCPUGetValue(const char *dir, unsigned int cpu, 
const char *file,
 virReportSystemError(errno, _("cannot read from %s"), path);
 goto cleanup;
 }
+
+ret = 0;
+
+ cleanup:
+VIR_FORCE_FCLOSE(pathfp);
+VIR_FREE(path);
+return ret;
+}
+

[libvirt] [PATH V10 09/12] Resctrl: concurrence support

2017-03-06 Thread Eli Qiao

The internal struct list domainall is a list which are resctral domain
status shared by all VMs, especiall the default domain, each VM should
access it concomitantly. Ues a mutex to control it.

Each bank's cache_left field is also a global shared resource we need
to be care, add a mutex for each bank.

We need also to add lock to access /sys/fs/resctrl, use flock.

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 src/util/virresctrl.c | 93 ---
 src/util/virresctrl.h |  3 ++
 2 files changed, 83 insertions(+), 13 deletions(-)

diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c
index 9f0d05f..3c54e44 100644
--- a/src/util/virresctrl.c
+++ b/src/util/virresctrl.c
@@ -20,7 +20,9 @@
  */
 #include 
 #include 
+#include 
 #include 
+#include 
 
 #include "virresctrl.h"
 #include "viralloc.h"
@@ -62,6 +64,9 @@ do { \
 
 #define VIR_RESCTRL_GET_SCHEMATA(count) ((1 << count) - 1)
 
+#define VIR_RESCTRL_LOCK(fd, op) flock(fd, op)
+#define VIR_RESCTRL_UNLOCK(fd) flock(fd, LOCK_UN)
+
 /**
  * a virResSchemata represents a schemata object under a resource control
  * domain.
@@ -105,6 +110,8 @@ typedef virResCtrlDomain *virResCtrlDomainPtr;
 struct _virResCtrlDomain {
 unsigned int num_domains;
 virResDomainPtr domains;
+
+virMutex lock;
 };
 
 static unsigned int host_id;
@@ -176,11 +183,16 @@ static int virResCtrlGetStr(const char *domain_name, 
const char *item_name, char
 
 CONSTRUCT_RESCTRL_PATH(domain_name, item_name);
 
+if (!virFileExists(path))
+goto cleanup;
+
 if (virFileReadAll(path, MAX_FILE_LEN, ret) < 0) {
 rc = -1;
 goto cleanup;
 }
 
+rc = 0;
+
  cleanup:
 VIR_FREE(path);
 return rc;
@@ -687,10 +699,15 @@ static int
 virResCtrlAppendDomain(virResDomainPtr dom)
 {
 virResDomainPtr p = domainall.domains;
+
+virMutexLock();
+
 while (p->next != NULL) p = p->next;
 p->next = dom;
 dom->pre = p;
 domainall.num_domains += 1;
+
+virMutexUnlock();
 return 0;
 }
 
@@ -713,18 +730,22 @@ virResCtrlCalculateSchemata(int type,
 {
 size_t i;
 int count;
+int rc = -1;
 virResDomainPtr p;
 unsigned int tmp_schemata;
 unsigned int schemata_sum = 0;
 int pair_type = 0;
 
+virMutexLock([type].cache_banks[sid].lock);
+
 if (resctrlall[type].cache_banks[sid].cache_left < size) {
 VIR_ERROR(_("Not enough cache left on bank %u"), hostid);
-return -1;
+goto cleanup;
 }
+
 if ((count = size / resctrlall[type].cache_banks[sid].cache_min) <= 0) {
 VIR_ERROR(_("Error cache size %llu"), size);
-return -1;
+goto cleanup;
 }
 
 tmp_schemata = VIR_RESCTRL_GET_SCHEMATA(count);
@@ -739,7 +760,7 @@ virResCtrlCalculateSchemata(int type,
 if (type == VIR_RDT_RESOURCE_L3CODE)
 pair_type = VIR_RDT_RESOURCE_L3DATA;
 
-for (i = 1; i < domainall.num_domains; i ++) {
+for (i = 1; i < domainall.num_domains; i++) {
 schemata_sum |= p->schematas[type]->schemata_items[sid].schemata;
 if (pair_type > 0)
 schemata_sum |= 
p->schematas[pair_type]->schemata_items[sid].schemata;
@@ -750,7 +771,16 @@ virResCtrlCalculateSchemata(int type,
 
 while ((tmp_schemata & schemata_sum) != 0)
 tmp_schemata = tmp_schemata >> 1;
-return tmp_schemata;
+
+resctrlall[type].cache_banks[sid].cache_left -= size;
+if (pair_type > 0)
+resctrlall[pair_type].cache_banks[sid].cache_left = 
resctrlall[type].cache_banks[sid].cache_left;
+
+rc = tmp_schemata;
+
+ cleanup:
+virMutexUnlock([type].cache_banks[sid].lock);
+return rc;
 }
 
 int virResCtrlSetCacheBanks(virDomainCachetunePtr cachetune,
@@ -763,8 +793,8 @@ int virResCtrlSetCacheBanks(virDomainCachetunePtr cachetune,
 int pair_type = -1;
 int sid;
 int schemata;
-
-virUUIDFormat(uuid, name);
+int lockfd;
+int rc = -1;
 
 for (i = 0; i < cachetune->n_banks; i++) {
 VIR_DEBUG("cache_banks %u, %u, %llu, %s",
@@ -777,12 +807,21 @@ int virResCtrlSetCacheBanks(virDomainCachetunePtr 
cachetune,
 if (cachetune->n_banks < 1)
 return 0;
 
+virUUIDFormat(uuid, name);
+
+if ((lockfd = open(RESCTRL_DIR, O_RDONLY)) < 0)
+goto cleanup;
+
+if (VIR_RESCTRL_LOCK(lockfd, LOCK_EX) < 0) {
+virReportSystemError(errno, _("Unable to lock '%s'"), RESCTRL_DIR);
+goto cleanup;
+}
+
 p = virResCtrlGetDomain(name);
 if (p == NULL) {
 VIR_DEBUG("no domain name %s found, create new one!", name);
 p = virResCtrlCreateDomain(name);
 }
-
 if (p != NULL) {
 
 virResCtrlAppendDomain(p);
@@ -828,19 +867,25 @@ int virResCtrlSetCacheBanks(virDomainCachetunePtr 
cachetune,
 if (virResCtrlFlushDomainToSysfs(p) < 0) {

[libvirt] [PATH V10 10/12] Resctrl: Scan resctrl before doing cache allocation

2017-03-06 Thread Eli Qiao

Other application may touch resctrl while libvirt's running, scan
resctrl again before allocating cache information to the newly created
VM.

Signed-off-by: Eli Qiao <liyong.q...@intel.com>
---
 src/util/virresctrl.c | 40 
 1 file changed, 40 insertions(+)

diff --git a/src/util/virresctrl.c b/src/util/virresctrl.c
index 3c54e44..16c01a2 100644
--- a/src/util/virresctrl.c
+++ b/src/util/virresctrl.c
@@ -711,6 +711,42 @@ virResCtrlAppendDomain(virResDomainPtr dom)
 return 0;
 }
 
+/* scan /sys/fs/resctrl again and refresh default schemata */
+static
+int virResCtrlScan(void)
+{
+struct dirent *ent;
+DIR *dp = NULL;
+int direrr;
+virResDomainPtr p;
+int rc = -1;
+
+if (virDirOpenQuiet(, RESCTRL_DIR) < 0) {
+if (errno == ENOENT)
+return -1;
+VIR_ERROR(_("Unable to open %s (%d)"), RESCTRL_DIR, errno);
+goto cleanup;
+}
+
+while ((direrr = virDirRead(dp, , NULL)) > 0) {
+if ((ent->d_type != DT_DIR) || STREQ(ent->d_name, "info"))
+continue;
+/* test if we'v tracked all domains */
+p = virResCtrlGetDomain(ent->d_name);
+if (p == NULL) {
+p = virResCtrlLoadDomain(ent->d_name);
+if (p == NULL)
+continue;
+virResCtrlAppendDomain(p);
+}
+}
+rc = 0;
+
+ cleanup:
+VIR_DIR_CLOSE(dp);
+return rc;
+}
+
 static int
 virResCtrlGetSocketIdByHostID(int type, unsigned int hostid)
 {
@@ -817,6 +853,10 @@ int virResCtrlSetCacheBanks(virDomainCachetunePtr 
cachetune,
 goto cleanup;
 }
 
+if (virResCtrlScan() < 0) {
+VIR_ERROR(_("Failed to scan resctrl domain dir"));
+goto cleanup;
+}
 p = virResCtrlGetDomain(name);
 if (p == NULL) {
 VIR_DEBUG("no domain name %s found, create new one!", name);
-- 
1.9.1

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

[libvirt] [PATH V10 00/12] Support cache tune in libvirt

2017-03-06 Thread Eli Qiao

Addressed comment from v9 -> v8

Marcelo:
  * Improve default schemata calculate, support allocate from low schemata 
bits for other APP.

Addressed comment from v9 -> v8

Marcelo:
  * New public API to query cache usage
Eli:
  * Fix core dump while multiple tasks are added.

Addressed comment from v8 -> v7

Martin:
   * Patch subject prefix.
   * Move some of cpu related information to virhostcpu.c.
   * Fix some memory leak in src/utils/resctrl.c

Martin & Marcelo:
   * Don't remove directories which are not maintained by libvirt.

Addressed comment from v7 -> v6

Marcelo:
   * Fix flock usage while VM initialization.

Addressed comment from v6 -> v5

Marcelo:
   * Support other APPs to operate /sys/fs/resctrl at same time

 Libvirt will scan /sys/fs/resctrl again before doing cache allocation.
 patch 10 will address this.

Addressed comment from v4 -> v5:

Marcelo:
   * Several typos
   * Use flock instead of virFileLock

Addressed comment from v3 -> v4:

Daniel & Marcelo:
   * Added concurrence support

Addressed comment from v2 -> v3:

Daniel:
  * Fixed coding style, passed `make check` and `make syntax-check`

  * Variables renaming and move from header file to c file.

  * For locking/mutex support, no progress.

  There are some discussion from mailing list, but I can not find a better
  way to add locking support without performance impact.

  I'll explain the process and please help to advice what shoud we do.

  VM create:
  1) Get the cache left value on each bank of the host. This should be
 shared amount all VMs.
  2) Calculate the schemata on the bank based on all created resctrl
 domain's schemata
  3) Calculate the default schemata by scaning all domain's schemata.
  4) Flush default schemata to /sys/fs/resctrl/schemata

  VM destroy:
  1) Remove the resctrl domain of that VM
  2) Recalculate default schemata
  3) Flush default schemata to /sys/fs/resctrl/schemata

  The key point is that all VMs shares /sys/fs/resctrl/schemata, and
  when a VM create a resctrl domain, the schemata of that VM depends on
  the default schemata and all other exsited schematas. So a global
  mutex is reqired.

  Before calculate a schemata or update default schemata, libvirt
  should gain this global mutex.

  I will try to think more about how to support this gracefully in next
  patch set.

Marcelo:
  * Added vcpu support for cachetune, this will allow user to define which
vcpu using which cache allocation bank.



vcpus is a cpumap, the vcpu pids will be added to tasks file

  * Added cdp compatible, user can specify l3 cache even host enable cdp.
See patch 8.
On a cdp enabled host, specify l3code/l3data by



This will create a schemata like:
L3data:0=0xff00;...
L3code:0=0xff00;...

  * Would you please help to test if the functions work.

Martin:
  * Xml test case, I have no time to work on this yet, would you please
show me an example, would like to amend it later.

This series patches are for supportting CAT featues, which also
called cache tune in libvirt.

First to expose cache information which could be tuned in capabilites XML.
Then add new domain xml element support to add cacahe bank which will apply
on this libvirt domain.

This series patches add a util file `resctrl.c/h`, an interface to talk with
linux kernel's system fs.

There are still one TODO left:
1. Expose a new public interface to get free cache information.
2. Expose a new public interface to set cachetune lively.

Some discussion about this feature support can be found from:
https://www.redhat.com/archives/libvir-list/2017-January/msg00644.html


Eli Qiao (12):
  Resctrl: Add some utils functions
  Resctrl: expose cache information to capabilities
  Resctrl: Add new xml element to support cache tune
  Resctrl: Add private interfaces to operate cache bank
  Qemu: Set cache tune while booting a new domain.
  Resctrl: enable l3code/l3data
  Resctrl: Make sure l3data/l3code are pairs
  Resctrl: Compatible mode for cdp enabled
  Resctrl: concurrence support
  Resctrl: Scan resctrl before doing cache allocation
  Resctrl: Add Public API for nodecachestats
  Resctrl: Add nodecachestats

 daemon/remote.c|   67 +++
 docs/schemas/domaincommon.rng  |   46 ++
 include/libvirt/libvirt-host.h |   32 ++
 include/libvirt/virterror.h|1 +
 po/POTFILES.in |1 +
 src/Makefile.am|1 +
 src/conf/capabilities.c|   56 ++
 src/conf/capabilities.h|   23 +
 src/conf/domain_conf.c |  182 +++
 src/conf/domain_conf.h |   19 +
 src/driver-hypervisor.h|7 +
 src/libvirt-host.c |   41 ++
 src/libvirt_private.syms   |   12 +
 src/l

1 2 3 >

1 - 100 of 233 matches

Mail list logo