On Wed, Oct 12, 2016 at 03:04:53PM +1100, Sam Bobroff wrote:
At the moment, guests that are backed by hugepages in the host are
only able to use policy to control the placement of those hugepages
on a per-(guest-)CPU basis. Policy applied globally is ignored.

Such guests would use <memoryBacking><hugepages/></memoryBacking> and
a <numatune> block with <memory mode=... nodeset=.../> but no <memnode
.../> elements.

This patch corrects this by, in this specific case, changing the QEMU
command line from "-mem-prealloc -mem-path=..." (which cannot
specify NUMA policy) to "-object memory-backend-file ..." (which can).

Note: This is not visible to the guest and does not appear to create
a migration incompatibility.

It could make sense, I haven't tried yet, though.  However, I still
don't see the point in using memory-backend-file.  Is it that when you
don't have cpuset cgroup the allocation doesn't work well?  Because it
certainly does work for me.

Signed-off-by: Sam Bobroff <sam.bobr...@au1.ibm.com>
There was some discussion leading up to this patch, here:


During that discussion it seemed that fixing this issue would cause
migration incompatibility but after some careful testing, it appears
that it only does when a memory-backend object is attached to a guest
NUMA node and that is not the case here. If only one is created, and
used globally (not attached via mem=<id>), the migration data does not
seem to be changed and so it seems reasonable to try something like
this patch.

This patch does work for my test cases but I don't claim a deep
understanding of the libvirt code so this is at least partly a RFC.
Comments welcome :-)


src/qemu/qemu_command.c | 55 ++++++++++++++++++++++++++++++++++++++++---------
src/qemu/qemu_command.h |  2 +-
2 files changed, 46 insertions(+), 11 deletions(-)

diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c
index 0804133..c28c8f2 100644
--- a/src/qemu/qemu_command.c
+++ b/src/qemu/qemu_command.c
@@ -3143,7 +3143,7 @@ qemuBuildMemoryBackendStr(unsigned long long size,
                          int guestNode,
                          virBitmapPtr userNodeset,
                          virBitmapPtr autoNodeset,
-                          virDomainDefPtr def,
+                          const virDomainDefPtr def,
                          virQEMUCapsPtr qemuCaps,
                          virQEMUDriverConfigPtr cfg,
                          const char **backendType,
@@ -7129,12 +7129,18 @@ qemuBuildSmpCommandLine(virCommandPtr cmd,

static int
qemuBuildMemPathStr(virQEMUDriverConfigPtr cfg,
-                    const virDomainDef *def,
+                    const virDomainDefPtr def,

These two hunks have nothing to do with the change.  If you want to do
them for some reason, explain the reasoning and put it into separate
patch, please.

                    virQEMUCapsPtr qemuCaps,
-                    virCommandPtr cmd)
+                    virCommandPtr cmd,
+                    virBitmapPtr auto_nodeset)
    const long system_page_size = virGetSystemPageSizeKB();
    char *mem_path = NULL;
+    virBitmapPtr nodemask = NULL;
+    const char *backendType = NULL;
+    char *backendStr = NULL;
+    virJSONValuePtr props;
+    int rv = -1;

     *  No-op if hugepages were not requested.
@@ -7159,18 +7165,47 @@ qemuBuildMemPathStr(virQEMUDriverConfigPtr cfg,
    if (qemuGetHupageMemPath(cfg, def->mem.hugepages[0].size, &mem_path) < 0)
        return -1;

-    virCommandAddArgList(cmd, "-mem-prealloc", "-mem-path", mem_path, NULL);
+    if (virDomainNumatuneMaybeGetNodeset(def->numa, auto_nodeset,
+                                         &nodemask, -1) < 0)
+        return -1;
+    if (nodemask && virQEMUCapsGet(qemuCaps, QEMU_CAPS_OBJECT_MEMORY_FILE)) {
+        props = virJSONValueNewObject();
+        if (qemuBuildMemoryBackendStr(virDomainDefGetMemoryInitial(def),
+                                      0, -1, NULL, auto_nodeset,
+                                      def, qemuCaps, cfg, &backendType,
+                                      &props, false) < 0)
+            goto cleanup;
+        if (!(backendStr = virQEMUBuildObjectCommandlineFromJSON(backendType,
+                                                                 "mem",
+                                                                 props)))
+            goto cleanup;
+        virCommandAddArgList(cmd, "-object", backendStr, NULL);

So now a function that's called BuildMemPathStr does more than that.  It
is also called when the memory is added in a different way in which case
this function would just add even more memory which is clearly wrong.

+        rv = 0;
+        VIR_FREE(backendStr);
+        VIR_FREE(props);
+    }
+    else {
+        if (nodemask || 1)

" || 1" ?  Leftover from debugging?

+            virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
+                           _("Memory file backend objects are "
+                             "unsupported by QEMU binary. Global NUMA "
+                             "hugepage policy will be ignored."));

Reporting error without erroring out is wrong.  VIR_WARN is probably
what you wanted to do here.

As I said, I don't feel the need for the change.  If there is a need for
it, it should clearly be done differently, so NACK to this particular


Attachment: signature.asc
Description: Digital signature

libvir-list mailing list

Reply via email to