Re: [libvirt] [PATCHv10 0/4] Introduce x86 Cache Monitoring Technology (CMT)
On 11/27/18 8:28 AM, Wang, Huaqiang wrote:
> Hi John,
>
> The issue mentioned is generated by a -1 returned by
> qemuDomainGetIOThreadsMon.
>
> Further, it is caused by a -1 returned by virQEMUCapsGet:
> 6174 if (!virQEMUCapsGet(priv->qemuCaps, QEMU_CAPS_OBJECT_IOTHREAD))
> {
> 6175 virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
> 6176_("IOThreads not supported with this
> binary"));
> 6177 goto endjob;
> 6178 }
>
> By checking qemuCaps->flags, it seems the bit of
> QEMU_CAPS_OBJECT_IOTHREAD(176) is not set.
> (gdb) call virBitmapFormat(qemuCaps->flags)
> $4 = 0x7f08a4004cb0
> "13,33,46,50,54-55,58,61-64,66-70,72-73,77-78,80,87-88,92-93,95-97,99,102-110,112,114,119-121,128-131,141-142,146,148-149,151-156,158-161,165,167,174-175,180,182,188,193-198,210,214,221-222,225,250,255"...
>
Hmm... Thanks for the details...
This is an "interesting" failure. There's a couple of bugs I suppose.
One simple to fix - the qemuConnectGetAllDomainStats should have saved
the error, but the call to virDomainStatsRecordListFree wipes out the
error when calling virDomainFree (via virResetLastError) when there's
more than one domain's worth of stats being collect and "some" domains
had already succeeded. So saving that error across the failure is
possible via a couple of means.
The other part of this would be a change to qemuDomainGetStatsIOThread
to not fail when the cap isn't available, but yet still allow a failure
if the cap was there, but the mon call failed.
As an aside, your QEMU must be fairly "old" to fail on the lack of the
capability QEMU_CAPS_OBJECT_IOTHREAD... Perhaps older than 2.0 - which
doesn't quite make sense given everything else.
I'll post a couple of patches shortly...
Tks,
John
> Breakpoint 1 at 0x7f088ef00880: file qemu/qemu_driver.c, line 5492.
> (gdb) c
> Continuing.
> [Switching to Thread 0x7f08bcdcb700 (LWP 24711)]
>
> Breakpoint 1, qemuDomainGetIOThreadsMon (driver=0x7f0874049b80,
> vm=0x7f08740cd840, iothreads=0x7f08bcdca8b8) at qemu/qemu_driver.c:5492
> 5492{
> Unprocessed breakpoint [1]
> (gdb) finish
> Run till exit from #0 qemuDomainGetIOThreadsMon (driver=0x7f0874049b80,
> vm=0x7f08740cd840, iothreads=0x7f08bcdca8b8) at qemu/qemu_driver.c:5492
> 0x7f088ef0098f in qemuDomainGetStatsIOThread (driver=,
> dom=, record=0x7f08ac000b60, maxparams=0x7f08bcdca9e4,
> privflags=) at qemu/qemu_driver.c:20885
> 20885 if ((niothreads = qemuDomainGetIOThreadsMon(driver, dom,
> )) < 0)
> Value returned is $1 = -1
> (gdb) bt
> #0 0x7f088ef0098f in qemuDomainGetStatsIOThread (driver=,
> dom=, record=0x7f08ac000b60, maxparams=0x7f08bcdca9e4,
> privflags=) at qemu/qemu_driver.c:20885
> #1 0x7f088eee70a9 in qemuDomainGetStats (flags=1, record= pointer>, stats=255, dom=0x7f08740cd840, conn=0x7f08ac000c50) at
> qemu/qemu_driver.c:21062
> #2 qemuConnectGetAllDomainStats (conn=0x7f08ac000c50, doms=,
> ndoms=, stats=255, retStats=0x7f08bcdcab10, flags= out>) at qemu/qemu_driver.c:21162
> #3 0x7f08cd147646 in virConnectGetAllDomainStats (conn=0x7f08ac000c50,
> stats=0, retStats=retStats@entry=0x7f08bcdcab10, flags=0) at
> libvirt-domain.c:11685
> #4 0x562e2404d2c0 in remoteDispatchConnectGetAllDomainStats
> (server=0x562e24ee74a0, msg=, ret=0x7f08ac001ec0,
> args=0x7f08ac001f30, rerr=0x7f08bcdcac50, client=0x562e24ef5200) at
> remote/remote_daemon_dispatch.c:6665
> #5 remoteDispatchConnectGetAllDomainStatsHelper (server=0x562e24ee74a0,
> client=0x562e24ef5200, msg=, rerr=0x7f08bcdcac50,
> args=0x7f08ac001f30, ret=0x7f08ac001ec0) at
> remote/remote_daemon_dispatch_stubs.h:743
> #6 0x7f08cd05dad5 in virNetServerProgramDispatchCall
> (msg=0x562e24ef5910, client=0x562e24ef5200, server=0x562e24ee74a0,
> prog=0x562e24ee6650) at rpc/virnetserverprogram.c:437
> #7 virNetServerProgramDispatch (prog=0x562e24ee6650,
> server=server@entry=0x562e24ee74a0, client=0x562e24ef5200,
> msg=0x562e24ef5910) at rpc/virnetserverprogram.c:304
> #8 0x7f08cd063f7d in virNetServerProcessMsg (msg=,
> prog=, client=, srv=0x562e24ee74a0) at
> rpc/virnetserver.c:144
> #9 virNetServerHandleJob (jobOpaque=, opaque=0x562e24ee74a0)
> at rpc/virnetserver.c:165
> #10 0x7f08ccf95171 in virThreadPoolWorker
> (opaque=opaque@entry=0x562e24ee69a0) at util/virthreadpool.c:167
> #11 0x7f08ccf944f8 in virThreadHelper (data=) at
> util/virthread.c:206
> #12 0x7f08cb213dc5 in start_thread (arg=0x7f08bcdcb700) at
> pthread_create.c:308
> #13 0x7f08cab3b76d in clone () at
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:113
> (gdb)
>
> My domain configuration is：
> [root@dl-c200 libvirt]# virsh list
> v^H Id Name State
> --
> 1ubuntu16.04-base running
>
> [root@dl-c200 libvirt]# virsh dumpxml 1
>
> ubuntu16.04-base
> 19d01bd1-ad54-4176-84bd-e510de08fe00
> 10485760
> 10485760
> 8
>
>
>
>
>
>
>
Re: [libvirt] [PATCHv10 0/4] Introduce x86 Cache Monitoring Technology (CMT)
Hi John,
The issue mentioned is generated by a -1 returned by qemuDomainGetIOThreadsMon.
Further, it is caused by a -1 returned by virQEMUCapsGet:
6174 if (!virQEMUCapsGet(priv->qemuCaps, QEMU_CAPS_OBJECT_IOTHREAD)) {
6175 virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
6176_("IOThreads not supported with this binary"));
6177 goto endjob;
6178 }
By checking qemuCaps->flags, it seems the bit of QEMU_CAPS_OBJECT_IOTHREAD(176)
is not set.
(gdb) call virBitmapFormat(qemuCaps->flags)
$4 = 0x7f08a4004cb0
"13,33,46,50,54-55,58,61-64,66-70,72-73,77-78,80,87-88,92-93,95-97,99,102-110,112,114,119-121,128-131,141-142,146,148-149,151-156,158-161,165,167,174-175,180,182,188,193-198,210,214,221-222,225,250,255"...
Breakpoint 1 at 0x7f088ef00880: file qemu/qemu_driver.c, line 5492.
(gdb) c
Continuing.
[Switching to Thread 0x7f08bcdcb700 (LWP 24711)]
Breakpoint 1, qemuDomainGetIOThreadsMon (driver=0x7f0874049b80,
vm=0x7f08740cd840, iothreads=0x7f08bcdca8b8) at qemu/qemu_driver.c:5492
5492{
Unprocessed breakpoint [1]
(gdb) finish
Run till exit from #0 qemuDomainGetIOThreadsMon (driver=0x7f0874049b80,
vm=0x7f08740cd840, iothreads=0x7f08bcdca8b8) at qemu/qemu_driver.c:5492
0x7f088ef0098f in qemuDomainGetStatsIOThread (driver=,
dom=, record=0x7f08ac000b60, maxparams=0x7f08bcdca9e4,
privflags=) at qemu/qemu_driver.c:20885
20885 if ((niothreads = qemuDomainGetIOThreadsMon(driver, dom,
)) < 0)
Value returned is $1 = -1
(gdb) bt
#0 0x7f088ef0098f in qemuDomainGetStatsIOThread (driver=,
dom=, record=0x7f08ac000b60, maxparams=0x7f08bcdca9e4,
privflags=) at qemu/qemu_driver.c:20885
#1 0x7f088eee70a9 in qemuDomainGetStats (flags=1, record=, stats=255, dom=0x7f08740cd840, conn=0x7f08ac000c50) at
qemu/qemu_driver.c:21062
#2 qemuConnectGetAllDomainStats (conn=0x7f08ac000c50, doms=,
ndoms=, stats=255, retStats=0x7f08bcdcab10, flags=) at qemu/qemu_driver.c:21162
#3 0x7f08cd147646 in virConnectGetAllDomainStats (conn=0x7f08ac000c50,
stats=0, retStats=retStats@entry=0x7f08bcdcab10, flags=0) at
libvirt-domain.c:11685
#4 0x562e2404d2c0 in remoteDispatchConnectGetAllDomainStats
(server=0x562e24ee74a0, msg=, ret=0x7f08ac001ec0,
args=0x7f08ac001f30, rerr=0x7f08bcdcac50, client=0x562e24ef5200) at
remote/remote_daemon_dispatch.c:6665
#5 remoteDispatchConnectGetAllDomainStatsHelper (server=0x562e24ee74a0,
client=0x562e24ef5200, msg=, rerr=0x7f08bcdcac50,
args=0x7f08ac001f30, ret=0x7f08ac001ec0) at
remote/remote_daemon_dispatch_stubs.h:743
#6 0x7f08cd05dad5 in virNetServerProgramDispatchCall (msg=0x562e24ef5910,
client=0x562e24ef5200, server=0x562e24ee74a0, prog=0x562e24ee6650) at
rpc/virnetserverprogram.c:437
#7 virNetServerProgramDispatch (prog=0x562e24ee6650,
server=server@entry=0x562e24ee74a0, client=0x562e24ef5200, msg=0x562e24ef5910)
at rpc/virnetserverprogram.c:304
#8 0x7f08cd063f7d in virNetServerProcessMsg (msg=,
prog=, client=, srv=0x562e24ee74a0) at
rpc/virnetserver.c:144
#9 virNetServerHandleJob (jobOpaque=, opaque=0x562e24ee74a0) at
rpc/virnetserver.c:165
#10 0x7f08ccf95171 in virThreadPoolWorker
(opaque=opaque@entry=0x562e24ee69a0) at util/virthreadpool.c:167
#11 0x7f08ccf944f8 in virThreadHelper (data=) at
util/virthread.c:206
#12 0x7f08cb213dc5 in start_thread (arg=0x7f08bcdcb700) at
pthread_create.c:308
#13 0x7f08cab3b76d in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:113
(gdb)
My domain configuration is：
[root@dl-c200 libvirt]# virsh list
v^H Id Name State
--
1ubuntu16.04-base running
[root@dl-c200 libvirt]# virsh dumpxml 1
ubuntu16.04-base
19d01bd1-ad54-4176-84bd-e510de08fe00
10485760
10485760
8
/machine
hvm
Broadwell
destroy
restart
destroy
/usr/libexec/qemu-kvm
system_u:system_r:svirt_t:s0:c157,c912
system_u:object_r:svirt_image_t:s0:c157,c912
+0:+0
+0:+0
I wonder if these message is enough for you to locate the root cause, I don’t
have too much time in investigating this today, I'll try to create a fix
tomorrow if you need.
Huaqiang
> -Original Message-
> From: Wang, Huaqiang
> Sent: Tuesday, November 27, 2018 11:21 AM
> To: 'John Ferlan' ; libvir-list@redhat.com
> Subject: RE: [PATCHv10 0/4] Introduce x86 Cache
Re: [libvirt] [PATCHv10 0/4] Introduce x86 Cache Monitoring Technology (CMT)
> -Original Message- > From: John Ferlan [mailto:jfer...@redhat.com] > Sent: Tuesday, November 27, 2018 10:59 AM > To: Wang, Huaqiang ; libvir-list@redhat.com > Subject: Re: [PATCHv10 0/4] Introduce x86 Cache Monitoring Technology > (CMT) > > > > On 11/26/18 9:39 PM, Wang, Huaqiang wrote: > > Hi John, > > > > Really appreciate your hard work for the CMT series. Next I'll working on > the MBM. > > > > In testing the newly pushed code, I find a problem: > > > > > > [david@dl-c200 ~]$ sudo virsh domstats > > error: An error occurred, but the cause is unknown > > > > I couldn't reproduced in a quick test here. Can you get a thread trace of the > failure? > > What I usually do, build libvirt, then in a terminal session at the top of the > git tree "./run gdb src/libvirtd" (dbg> r)... THen in another terminal > session > run the virsh command and when the libvirtd session stops do a "t a a bt" > (thread apply all backtrace)... > > John I'll trace the error. Thanks. Huaqiang > > (done for the night) > > > seems it is caused by qemuDomainGetStatsIOThread not by the new > series. > > What I test is return 0 immediately at top of > > qemuDomainGetStatsIOThread, the command 'virsh domstats' reports > the cache statistics normally. > > > > BR > > Huaqiang > > > >> -Original Message- > >> From: John Ferlan [mailto:jfer...@redhat.com] > >> Sent: Tuesday, November 27, 2018 9:49 AM > >> To: Wang, Huaqiang ; libvir- > l...@redhat.com > >> Subject: Re: [PATCHv10 0/4] Introduce x86 Cache Monitoring > Technology > >> (CMT) > >> > >> > >> > >> On 11/26/18 12:56 PM, Wang Huaqiang wrote: > >>> These patches are the remaining part for the CMT enabling series, > >>> and most of the series have been merged. > >>> > >>> This series is addressing John's review comments and suggestions, > >>> which are > >>> https://www.redhat.com/archives/libvir-list/2018- > >> November/msg00907.htm > >>> l > >>> https://www.redhat.com/archives/libvir-list/2018- > >> November/msg00510.htm > >>> l > >>> https://www.redhat.com/archives/libvir-list/2018- > >> November/msg00561.htm > >>> l > >>> > >>> Change lists: > >>> Changes in v10: > >>> -. Add tag (virResctrlMonitorType) in qemuDomainGetResctrlMonData, > >> thus > >>>qemuDomainGetResctrlMonData could be reused for MBM. > >>> -. Using VIR_APPEND_ELEMENT to append virQEMUResctrlMonDataPtr > >> list. > >>> -. Add qemuDomainFreeResctrlMonData. > >>> -. Add virResctrlMonitorFreeStats. > >>> -. Return a list of virResctrlMonitorStatsPtr instead of > >>>a virResctrlMonitorStats array in virResctrlMonitorGetStats. > >>> > >>> Changes in V9: > >>> -. Addressing code review comments form John. > >>> -. Refined the names for new data structure and new functions. > >>> -. Merged qemuDomainGetStatsCpuResMonitorPerTag and > >> qemuDomainGetStatsCpuResMonitor, > >>>and refined new function name based on the fact that we only > >>> support > >> cache monitor now. > >>> Wang Huaqiang (4): > >>> util: Return a list of pointer in virResctrlMonitorGetStats > >>> util: Add function to free monitor statistical data > >>> qemu: Report cache occupancy (CMT) with domstats > >>> docs: Updated news.xml for CMT > >>> > >>> docs/news.xml| 12 > >>> src/libvirt-domain.c | 12 > >>> src/libvirt_private.syms | 1 + > >>> src/qemu/qemu_driver.c | 183 > >> ++- > >>> src/util/virresctrl.c| 26 +-- > >>> src/util/virresctrl.h| 8 ++- > >>> tools/virsh.pod | 14 > >>> 7 files changed, 248 insertions(+), 8 deletions(-) > >>> > >> > >> Reviewed-by: John Ferlan > >> (series) > >> > >> and pushed, > >> > >> John -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCHv10 0/4] Introduce x86 Cache Monitoring Technology (CMT)
On 11/26/18 9:39 PM, Wang, Huaqiang wrote: > Hi John, > > Really appreciate your hard work for the CMT series. Next I'll working on the > MBM. > > In testing the newly pushed code, I find a problem: > > > [david@dl-c200 ~]$ sudo virsh domstats > error: An error occurred, but the cause is unknown > > I couldn't reproduced in a quick test here. Can you get a thread trace of the failure? What I usually do, build libvirt, then in a terminal session at the top of the git tree "./run gdb src/libvirtd" (dbg> r)... THen in another terminal session run the virsh command and when the libvirtd session stops do a "t a a bt" (thread apply all backtrace)... John (done for the night) > seems it is caused by qemuDomainGetStatsIOThread not by the new series. > What I test is return 0 immediately at top of qemuDomainGetStatsIOThread, the > command 'virsh domstats' reports the cache statistics normally. > > BR > Huaqiang > >> -Original Message- >> From: John Ferlan [mailto:jfer...@redhat.com] >> Sent: Tuesday, November 27, 2018 9:49 AM >> To: Wang, Huaqiang ; libvir-list@redhat.com >> Subject: Re: [PATCHv10 0/4] Introduce x86 Cache Monitoring Technology >> (CMT) >> >> >> >> On 11/26/18 12:56 PM, Wang Huaqiang wrote: >>> These patches are the remaining part for the CMT enabling series, and >>> most of the series have been merged. >>> >>> This series is addressing John's review comments and suggestions, >>> which are >>> https://www.redhat.com/archives/libvir-list/2018- >> November/msg00907.htm >>> l >>> https://www.redhat.com/archives/libvir-list/2018- >> November/msg00510.htm >>> l >>> https://www.redhat.com/archives/libvir-list/2018- >> November/msg00561.htm >>> l >>> >>> Change lists: >>> Changes in v10: >>> -. Add tag (virResctrlMonitorType) in qemuDomainGetResctrlMonData, >> thus >>>qemuDomainGetResctrlMonData could be reused for MBM. >>> -. Using VIR_APPEND_ELEMENT to append virQEMUResctrlMonDataPtr >> list. >>> -. Add qemuDomainFreeResctrlMonData. >>> -. Add virResctrlMonitorFreeStats. >>> -. Return a list of virResctrlMonitorStatsPtr instead of >>>a virResctrlMonitorStats array in virResctrlMonitorGetStats. >>> >>> Changes in V9: >>> -. Addressing code review comments form John. >>> -. Refined the names for new data structure and new functions. >>> -. Merged qemuDomainGetStatsCpuResMonitorPerTag and >> qemuDomainGetStatsCpuResMonitor, >>>and refined new function name based on the fact that we only support >> cache monitor now. >>> Wang Huaqiang (4): >>> util: Return a list of pointer in virResctrlMonitorGetStats >>> util: Add function to free monitor statistical data >>> qemu: Report cache occupancy (CMT) with domstats >>> docs: Updated news.xml for CMT >>> >>> docs/news.xml| 12 >>> src/libvirt-domain.c | 12 >>> src/libvirt_private.syms | 1 + >>> src/qemu/qemu_driver.c | 183 >> ++- >>> src/util/virresctrl.c| 26 +-- >>> src/util/virresctrl.h| 8 ++- >>> tools/virsh.pod | 14 >>> 7 files changed, 248 insertions(+), 8 deletions(-) >>> >> >> Reviewed-by: John Ferlan >> (series) >> >> and pushed, >> >> John -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCHv10 0/4] Introduce x86 Cache Monitoring Technology (CMT)
Hi John, Really appreciate your hard work for the CMT series. Next I'll working on the MBM. In testing the newly pushed code, I find a problem: [david@dl-c200 ~]$ sudo virsh domstats error: An error occurred, but the cause is unknown seems it is caused by qemuDomainGetStatsIOThread not by the new series. What I test is return 0 immediately at top of qemuDomainGetStatsIOThread, the command 'virsh domstats' reports the cache statistics normally. BR Huaqiang > -Original Message- > From: John Ferlan [mailto:jfer...@redhat.com] > Sent: Tuesday, November 27, 2018 9:49 AM > To: Wang, Huaqiang ; libvir-list@redhat.com > Subject: Re: [PATCHv10 0/4] Introduce x86 Cache Monitoring Technology > (CMT) > > > > On 11/26/18 12:56 PM, Wang Huaqiang wrote: > > These patches are the remaining part for the CMT enabling series, and > > most of the series have been merged. > > > > This series is addressing John's review comments and suggestions, > > which are > > https://www.redhat.com/archives/libvir-list/2018- > November/msg00907.htm > > l > > https://www.redhat.com/archives/libvir-list/2018- > November/msg00510.htm > > l > > https://www.redhat.com/archives/libvir-list/2018- > November/msg00561.htm > > l > > > > Change lists: > > Changes in v10: > > -. Add tag (virResctrlMonitorType) in qemuDomainGetResctrlMonData, > thus > >qemuDomainGetResctrlMonData could be reused for MBM. > > -. Using VIR_APPEND_ELEMENT to append virQEMUResctrlMonDataPtr > list. > > -. Add qemuDomainFreeResctrlMonData. > > -. Add virResctrlMonitorFreeStats. > > -. Return a list of virResctrlMonitorStatsPtr instead of > >a virResctrlMonitorStats array in virResctrlMonitorGetStats. > > > > Changes in V9: > > -. Addressing code review comments form John. > > -. Refined the names for new data structure and new functions. > > -. Merged qemuDomainGetStatsCpuResMonitorPerTag and > qemuDomainGetStatsCpuResMonitor, > >and refined new function name based on the fact that we only support > cache monitor now. > > Wang Huaqiang (4): > > util: Return a list of pointer in virResctrlMonitorGetStats > > util: Add function to free monitor statistical data > > qemu: Report cache occupancy (CMT) with domstats > > docs: Updated news.xml for CMT > > > > docs/news.xml| 12 > > src/libvirt-domain.c | 12 > > src/libvirt_private.syms | 1 + > > src/qemu/qemu_driver.c | 183 > ++- > > src/util/virresctrl.c| 26 +-- > > src/util/virresctrl.h| 8 ++- > > tools/virsh.pod | 14 > > 7 files changed, 248 insertions(+), 8 deletions(-) > > > > Reviewed-by: John Ferlan > (series) > > and pushed, > > John -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCHv10 0/4] Introduce x86 Cache Monitoring Technology (CMT)
On 11/26/18 12:56 PM, Wang Huaqiang wrote: > These patches are the remaining part for the CMT enabling series, > and most of the series have been merged. > > This series is addressing John's review comments and suggestions, > which are > https://www.redhat.com/archives/libvir-list/2018-November/msg00907.html > https://www.redhat.com/archives/libvir-list/2018-November/msg00510.html > https://www.redhat.com/archives/libvir-list/2018-November/msg00561.html > > Change lists: > Changes in v10: > -. Add tag (virResctrlMonitorType) in qemuDomainGetResctrlMonData, thus >qemuDomainGetResctrlMonData could be reused for MBM. > -. Using VIR_APPEND_ELEMENT to append virQEMUResctrlMonDataPtr list. > -. Add qemuDomainFreeResctrlMonData. > -. Add virResctrlMonitorFreeStats. > -. Return a list of virResctrlMonitorStatsPtr instead of >a virResctrlMonitorStats array in virResctrlMonitorGetStats. > > Changes in V9: > -. Addressing code review comments form John. > -. Refined the names for new data structure and new functions. > -. Merged qemuDomainGetStatsCpuResMonitorPerTag and > qemuDomainGetStatsCpuResMonitor, >and refined new function name based on the fact that we only support cache > monitor now. > Wang Huaqiang (4): > util: Return a list of pointer in virResctrlMonitorGetStats > util: Add function to free monitor statistical data > qemu: Report cache occupancy (CMT) with domstats > docs: Updated news.xml for CMT > > docs/news.xml| 12 > src/libvirt-domain.c | 12 > src/libvirt_private.syms | 1 + > src/qemu/qemu_driver.c | 183 > ++- > src/util/virresctrl.c| 26 +-- > src/util/virresctrl.h| 8 ++- > tools/virsh.pod | 14 > 7 files changed, 248 insertions(+), 8 deletions(-) > Reviewed-by: John Ferlan (series) and pushed, John -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
