Re: [Qemu-devel] [QAPI+QGA 3/3] QEMU Guest Agent (virtagent) v6
On Thu, 14 Jul 2011 10:53:51 +0800
Zhi Yong Wu zwu.ker...@gmail.com wrote:
HI, Michael,
On Tue, Jul 5, 2011 at 9:21 PM, Michael Roth mdr...@linux.vnet.ibm.com
wrote:
This is Set 3/3 of the QAPI+QGA patchsets.
These patches apply on top of qapi-backport-set2-v5, and can also be
obtained from:
git://repo.or.cz/qemu/mdroth.git qapi-backport-set3-v6
(Set1+2 are a backport of some of the QAPI-related work from Anthony's
glib tree. The main goal is to get the basic code generation infrastructure
in
place so that it can be used by the guest agent to implement a QMP-like
guest
interface, and so that future work regarding the QMP conversion to QAPI can
be
decoupled from the infrastructure bits. Set3 is the Qemu Guest Agent
(virtagent), rebased on the new code QAPI code generation infrastructure.
This
is the first user of QAPI, QMP will follow.)
___
CHANGES SINCE V5:
- switched to using qemu malloc/list functions where possible
- removed unused proxy_path field in struct GAState
- pid file now opened write-only, removed lockf() in favor of O_EXCL,
added SIGINT/SIGTERM signal handlers to handle cleanup
- cleaned up error-handling, switched to asserts where appropriate,
removed unecessary gotos and NULL checks for qemu_free()/qobject_decref()
- refactored send_payload() using helper functions
- fixed improper handling of pidfile fd==0
- changed guest-shutdown's shutdown_mode param to mode
- switched to using kernel-generated FDs for guest-file-open rather than
an autoincrement value
- add maximum chunk size of guest-file-read/guest-file-write
- added checks to avoid guest-file-write from writing data beyond the
provided data buffer
- made logging best-effort, removed handling of failures to log as errors
- guest-shutdown exec errors now logged to guest syslog, clarified
shutdown's asynchronous, no gauruntee nature in schema.
CHANGES SINCE V4:
- Removed timeout mechanism via worker thread/pthread_cancel due to
potential memory leak. Will re-introduce guest-side timeout support in
future version.
- Fixed up fsfreeze code to use enums specified within the guest agent's
qapi schema.
- Fixed memory leak due to a log statement, and added missing cleanup
functions for heap-allocated g_error objects.
- Made mode param to guest-file-open optional, defaults to r
(read-only)
CHANGES SINCE V3:
- Fixed error-handling issues in fsfreeze commands leading to certain
mounted directories causing freeze/thaw operations to fail
- Added cleanup hook to thaw filesystems on graceful guest agent exit
- Removed unused enum values and added additional details to schema
documentation
- Fixed build issue that was missed due to deprecated files in source
tree, removed unused includes
CHANGES SINCE V2:
- Rebased on new QAPI code generation framework
- Dropped ability for QMP to act as a proxy for the guest agent, will be
added when new QMP server is backported from Anthony's glib tree
- Replaced negotiation/control events with a simple 2-way handshake
implemented by a standard RPC (guest-sync)
- Removed enforcement of pristine sessions, state is now
global/persistant across multiple clients/connections
- Fixed segfault in logging code
- Added Jes' filesystem freeze patches
- General cleanups
CHANGES SINCE V1:
- Added guest agent worker thread to execute RPCs in the guest. With this
in place we have a reliable timeout mechanism for hung commands, currently
set at 30 seconds.
- Add framework for registering init/cleanup routines for stateful RPCs to
clean up after themselves after a timeout.
- Added the following RPCs: guest-file-{open,close,read,write,seek},
guest-shutdown, guest-info, and removed stubs for guest-view-file (now
deprecated)
- Added GUEST_AGENT_UP/GUEST_AGENT_DOWN QMP events
- Switched to a TCP-style host-initiated 3-way handshake for channel
negotiation, this simplifies client negotiation/interaction over the wire
- Added configurable log level/log file/pid file options for guest agent
- Various fixes for bugs/memory leaks and checkpatch.pl fixups
ISSUES/TODOS:
- Add unit tests for guest agent wire protocol
OVERVIEW
For a better overview of what these patches are meant to accomplish, please
reference the RFC for virtagent:
http://comments.gmane.org/gmane.comp.emulators.qemu/96096
These patches integrate the previous virtagent guest agent work directly in
QAPI/QMP to leverage it's auto-generated marshalling code. This has
numerous benefits:
- addresses previous concerns over relying on external libraries to handle
data encapsulation
- reduces the need for manual unmarshalling of requests/responses, which
makes adding new RPCs much safer/less error-prone, as well as cutting down
on redundant code
- QAPI documentation aligns completely with guest-side
Re: [Qemu-devel] [QAPI+QGA 3/3] QEMU Guest Agent (virtagent) v6
On 07/14/2011 08:53 AM, Zhi Yong Wu wrote:
On Thu, Jul 14, 2011 at 8:55 PM, Luiz Capitulinolcapitul...@redhat.com wrote:
On Thu, 14 Jul 2011 10:53:51 +0800
Zhi Yong Wuzwu.ker...@gmail.com wrote:
HI, Michael,
On Tue, Jul 5, 2011 at 9:21 PM, Michael Rothmdr...@linux.vnet.ibm.com wrote:
This is Set 3/3 of the QAPI+QGA patchsets.
These patches apply on top of qapi-backport-set2-v5, and can also be obtained
from:
git://repo.or.cz/qemu/mdroth.git qapi-backport-set3-v6
(Set1+2 are a backport of some of the QAPI-related work from Anthony's
glib tree. The main goal is to get the basic code generation infrastructure in
place so that it can be used by the guest agent to implement a QMP-like guest
interface, and so that future work regarding the QMP conversion to QAPI can be
decoupled from the infrastructure bits. Set3 is the Qemu Guest Agent
(virtagent), rebased on the new code QAPI code generation infrastructure. This
is the first user of QAPI, QMP will follow.)
___
CHANGES SINCE V5:
- switched to using qemu malloc/list functions where possible
- removed unused proxy_path field in struct GAState
- pid file now opened write-only, removed lockf() in favor of O_EXCL, added
SIGINT/SIGTERM signal handlers to handle cleanup
- cleaned up error-handling, switched to asserts where appropriate, removed
unecessary gotos and NULL checks for qemu_free()/qobject_decref()
- refactored send_payload() using helper functions
- fixed improper handling of pidfile fd==0
- changed guest-shutdown's shutdown_mode param to mode
- switched to using kernel-generated FDs for guest-file-open rather than an
autoincrement value
- add maximum chunk size of guest-file-read/guest-file-write
- added checks to avoid guest-file-write from writing data beyond the
provided data buffer
- made logging best-effort, removed handling of failures to log as errors
- guest-shutdown exec errors now logged to guest syslog, clarified shutdown's
asynchronous, no gauruntee nature in schema.
CHANGES SINCE V4:
- Removed timeout mechanism via worker thread/pthread_cancel due to potential
memory leak. Will re-introduce guest-side timeout support in future version.
- Fixed up fsfreeze code to use enums specified within the guest agent's qapi
schema.
- Fixed memory leak due to a log statement, and added missing cleanup
functions for heap-allocated g_error objects.
- Made mode param to guest-file-open optional, defaults to r (read-only)
CHANGES SINCE V3:
- Fixed error-handling issues in fsfreeze commands leading to certain mounted
directories causing freeze/thaw operations to fail
- Added cleanup hook to thaw filesystems on graceful guest agent exit
- Removed unused enum values and added additional details to schema
documentation
- Fixed build issue that was missed due to deprecated files in source tree,
removed unused includes
CHANGES SINCE V2:
- Rebased on new QAPI code generation framework
- Dropped ability for QMP to act as a proxy for the guest agent, will be
added when new QMP server is backported from Anthony's glib tree
- Replaced negotiation/control events with a simple 2-way handshake
implemented by a standard RPC (guest-sync)
- Removed enforcement of pristine sessions, state is now global/persistant
across multiple clients/connections
- Fixed segfault in logging code
- Added Jes' filesystem freeze patches
- General cleanups
CHANGES SINCE V1:
- Added guest agent worker thread to execute RPCs in the guest. With this in
place we have a reliable timeout mechanism for hung commands, currently set at
30 seconds.
- Add framework for registering init/cleanup routines for stateful RPCs to
clean up after themselves after a timeout.
- Added the following RPCs: guest-file-{open,close,read,write,seek},
guest-shutdown, guest-info, and removed stubs for guest-view-file (now
deprecated)
- Added GUEST_AGENT_UP/GUEST_AGENT_DOWN QMP events
- Switched to a TCP-style host-initiated 3-way handshake for channel
negotiation, this simplifies client negotiation/interaction over the wire
- Added configurable log level/log file/pid file options for guest agent
- Various fixes for bugs/memory leaks and checkpatch.pl fixups
ISSUES/TODOS:
- Add unit tests for guest agent wire protocol
OVERVIEW
For a better overview of what these patches are meant to accomplish, please
reference the RFC for virtagent:
http://comments.gmane.org/gmane.comp.emulators.qemu/96096
These patches integrate the previous virtagent guest agent work directly in
QAPI/QMP to leverage it's auto-generated marshalling code. This has numerous
benefits:
- addresses previous concerns over relying on external libraries to handle
data encapsulation
- reduces the need for manual unmarshalling of requests/responses, which
makes adding new RPCs much safer/less error-prone, as well as cutting down on
redundant code
- QAPI documentation aligns completely with guest-side RPC implementation
- is Just Better (TM)
Re: [Qemu-devel] [QAPI+QGA 3/3] QEMU Guest Agent (virtagent) v6
On Thu, Jul 14, 2011 at 8:55 PM, Luiz Capitulino lcapitul...@redhat.com wrote:
On Thu, 14 Jul 2011 10:53:51 +0800
Zhi Yong Wu zwu.ker...@gmail.com wrote:
HI, Michael,
On Tue, Jul 5, 2011 at 9:21 PM, Michael Roth mdr...@linux.vnet.ibm.com
wrote:
This is Set 3/3 of the QAPI+QGA patchsets.
These patches apply on top of qapi-backport-set2-v5, and can also be
obtained from:
git://repo.or.cz/qemu/mdroth.git qapi-backport-set3-v6
(Set1+2 are a backport of some of the QAPI-related work from Anthony's
glib tree. The main goal is to get the basic code generation
infrastructure in
place so that it can be used by the guest agent to implement a QMP-like
guest
interface, and so that future work regarding the QMP conversion to QAPI
can be
decoupled from the infrastructure bits. Set3 is the Qemu Guest Agent
(virtagent), rebased on the new code QAPI code generation infrastructure.
This
is the first user of QAPI, QMP will follow.)
___
CHANGES SINCE V5:
- switched to using qemu malloc/list functions where possible
- removed unused proxy_path field in struct GAState
- pid file now opened write-only, removed lockf() in favor of O_EXCL,
added SIGINT/SIGTERM signal handlers to handle cleanup
- cleaned up error-handling, switched to asserts where appropriate,
removed unecessary gotos and NULL checks for qemu_free()/qobject_decref()
- refactored send_payload() using helper functions
- fixed improper handling of pidfile fd==0
- changed guest-shutdown's shutdown_mode param to mode
- switched to using kernel-generated FDs for guest-file-open rather than
an autoincrement value
- add maximum chunk size of guest-file-read/guest-file-write
- added checks to avoid guest-file-write from writing data beyond the
provided data buffer
- made logging best-effort, removed handling of failures to log as errors
- guest-shutdown exec errors now logged to guest syslog, clarified
shutdown's asynchronous, no gauruntee nature in schema.
CHANGES SINCE V4:
- Removed timeout mechanism via worker thread/pthread_cancel due to
potential memory leak. Will re-introduce guest-side timeout support in
future version.
- Fixed up fsfreeze code to use enums specified within the guest agent's
qapi schema.
- Fixed memory leak due to a log statement, and added missing cleanup
functions for heap-allocated g_error objects.
- Made mode param to guest-file-open optional, defaults to r
(read-only)
CHANGES SINCE V3:
- Fixed error-handling issues in fsfreeze commands leading to certain
mounted directories causing freeze/thaw operations to fail
- Added cleanup hook to thaw filesystems on graceful guest agent exit
- Removed unused enum values and added additional details to schema
documentation
- Fixed build issue that was missed due to deprecated files in source
tree, removed unused includes
CHANGES SINCE V2:
- Rebased on new QAPI code generation framework
- Dropped ability for QMP to act as a proxy for the guest agent, will be
added when new QMP server is backported from Anthony's glib tree
- Replaced negotiation/control events with a simple 2-way handshake
implemented by a standard RPC (guest-sync)
- Removed enforcement of pristine sessions, state is now
global/persistant across multiple clients/connections
- Fixed segfault in logging code
- Added Jes' filesystem freeze patches
- General cleanups
CHANGES SINCE V1:
- Added guest agent worker thread to execute RPCs in the guest. With this
in place we have a reliable timeout mechanism for hung commands, currently
set at 30 seconds.
- Add framework for registering init/cleanup routines for stateful RPCs
to clean up after themselves after a timeout.
- Added the following RPCs: guest-file-{open,close,read,write,seek},
guest-shutdown, guest-info, and removed stubs for guest-view-file (now
deprecated)
- Added GUEST_AGENT_UP/GUEST_AGENT_DOWN QMP events
- Switched to a TCP-style host-initiated 3-way handshake for channel
negotiation, this simplifies client negotiation/interaction over the wire
- Added configurable log level/log file/pid file options for guest agent
- Various fixes for bugs/memory leaks and checkpatch.pl fixups
ISSUES/TODOS:
- Add unit tests for guest agent wire protocol
OVERVIEW
For a better overview of what these patches are meant to accomplish,
please reference the RFC for virtagent:
http://comments.gmane.org/gmane.comp.emulators.qemu/96096
These patches integrate the previous virtagent guest agent work directly
in QAPI/QMP to leverage it's auto-generated marshalling code. This has
numerous benefits:
- addresses previous concerns over relying on external libraries to
handle data encapsulation
- reduces the need for manual unmarshalling of requests/responses, which
makes adding new RPCs much safer/less error-prone, as well as cutting down
Re: [Qemu-devel] [QAPI+QGA 3/3] QEMU Guest Agent (virtagent) v6
On Tue, Jul 05, 2011 at 08:21:36AM -0500, Michael Roth wrote: BUILD/USAGE build: ./configure --target-list=x86_64-softmmu make make qemu-ga #should be built on|for target guest start guest: qemu \ -drive file=/home/mdroth/vm/rhel6_64_base.raw,snapshot=off,if=virtio \ -net nic,model=virtio,macaddr=52:54:00:12:34:00 \ -net tap,script=/etc/qemu-ifup \ -vnc :1 -m 1024 --enable-kvm \ -chardev socket,path=/tmp/qga.sock,server,nowait,id=qga \ -device virtio-serial \ -device virtserialport,chardev=qga,name=qga use guest agent: ./qemu-ga -h ./qemu-ga -c virtio-serial -p /dev/virtio-ports/qga Have we documented any naming convention for virtio serial ports yet ? For both Matahari, and libguestfs we've followed the reverse domain name style naming adopted by things like DBus and AMQP, or language package namespaces. eg org.libguestfs.channel.0 org.apache.qpid.matahari.0 The '.0' is just in case we find we need to add further channels for each agent later with different usage. I think this would be a good general naming convention to recommend to app developers in order to avoid naming clashes, and thus think that the QEMU guest agent should use a channel name prefixed with 'org.qemu.' eg perhaps ... -device virtserialport,chardev=qga,name=org.qemu.guestagent.0 ... ./qemu-ga -c virtio-serial -p /dev/virtio-ports/org.qemu.guestagent.0 Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
Re: [Qemu-devel] [QAPI+QGA 3/3] QEMU Guest Agent (virtagent) v6
On 07/13/2011 08:14 AM, Daniel P. Berrange wrote: On Tue, Jul 05, 2011 at 08:21:36AM -0500, Michael Roth wrote: BUILD/USAGE build: ./configure --target-list=x86_64-softmmu make make qemu-ga #should be built on|for target guest start guest: qemu \ -drive file=/home/mdroth/vm/rhel6_64_base.raw,snapshot=off,if=virtio \ -net nic,model=virtio,macaddr=52:54:00:12:34:00 \ -net tap,script=/etc/qemu-ifup \ -vnc :1 -m 1024 --enable-kvm \ -chardev socket,path=/tmp/qga.sock,server,nowait,id=qga \ -device virtio-serial \ -device virtserialport,chardev=qga,name=qga use guest agent: ./qemu-ga -h ./qemu-ga -c virtio-serial -p /dev/virtio-ports/qga Have we documented any naming convention for virtio serial ports yet ? For both Matahari, and libguestfs we've followed the reverse domain name style naming adopted by things like DBus and AMQP, or language package namespaces. eg org.libguestfs.channel.0 org.apache.qpid.matahari.0 The '.0' is just in case we find we need to add further channels for each agent later with different usage. I think this would be a good general naming convention to recommend to app developers in order to avoid naming clashes, and thus think that the QEMU guest agent should use a channel name prefixed with 'org.qemu.' eg perhaps ... -device virtserialport,chardev=qga,name=org.qemu.guestagent.0 ... ./qemu-ga -c virtio-serial -p /dev/virtio-ports/org.qemu.guestagent.0 Regards, Daniel Hi Daniel, If you don't specify the -p option explicitly, qemu-ga will look for /dev/virtio-ports/org.qemu.guest_agent Adding .0 would be useful though, so I'll make /dev/virtio-ports/org.qemu.guest_agent.0 the default, and update any surrounding documentation to encourage that convention.
Re: [Qemu-devel] [QAPI+QGA 3/3] QEMU Guest Agent (virtagent) v6
HI, Michael,
On Tue, Jul 5, 2011 at 9:21 PM, Michael Roth mdr...@linux.vnet.ibm.com wrote:
This is Set 3/3 of the QAPI+QGA patchsets.
These patches apply on top of qapi-backport-set2-v5, and can also be obtained
from:
git://repo.or.cz/qemu/mdroth.git qapi-backport-set3-v6
(Set1+2 are a backport of some of the QAPI-related work from Anthony's
glib tree. The main goal is to get the basic code generation infrastructure in
place so that it can be used by the guest agent to implement a QMP-like guest
interface, and so that future work regarding the QMP conversion to QAPI can be
decoupled from the infrastructure bits. Set3 is the Qemu Guest Agent
(virtagent), rebased on the new code QAPI code generation infrastructure. This
is the first user of QAPI, QMP will follow.)
___
CHANGES SINCE V5:
- switched to using qemu malloc/list functions where possible
- removed unused proxy_path field in struct GAState
- pid file now opened write-only, removed lockf() in favor of O_EXCL, added
SIGINT/SIGTERM signal handlers to handle cleanup
- cleaned up error-handling, switched to asserts where appropriate, removed
unecessary gotos and NULL checks for qemu_free()/qobject_decref()
- refactored send_payload() using helper functions
- fixed improper handling of pidfile fd==0
- changed guest-shutdown's shutdown_mode param to mode
- switched to using kernel-generated FDs for guest-file-open rather than an
autoincrement value
- add maximum chunk size of guest-file-read/guest-file-write
- added checks to avoid guest-file-write from writing data beyond the
provided data buffer
- made logging best-effort, removed handling of failures to log as errors
- guest-shutdown exec errors now logged to guest syslog, clarified
shutdown's asynchronous, no gauruntee nature in schema.
CHANGES SINCE V4:
- Removed timeout mechanism via worker thread/pthread_cancel due to
potential memory leak. Will re-introduce guest-side timeout support in future
version.
- Fixed up fsfreeze code to use enums specified within the guest agent's
qapi schema.
- Fixed memory leak due to a log statement, and added missing cleanup
functions for heap-allocated g_error objects.
- Made mode param to guest-file-open optional, defaults to r (read-only)
CHANGES SINCE V3:
- Fixed error-handling issues in fsfreeze commands leading to certain
mounted directories causing freeze/thaw operations to fail
- Added cleanup hook to thaw filesystems on graceful guest agent exit
- Removed unused enum values and added additional details to schema
documentation
- Fixed build issue that was missed due to deprecated files in source tree,
removed unused includes
CHANGES SINCE V2:
- Rebased on new QAPI code generation framework
- Dropped ability for QMP to act as a proxy for the guest agent, will be
added when new QMP server is backported from Anthony's glib tree
- Replaced negotiation/control events with a simple 2-way handshake
implemented by a standard RPC (guest-sync)
- Removed enforcement of pristine sessions, state is now global/persistant
across multiple clients/connections
- Fixed segfault in logging code
- Added Jes' filesystem freeze patches
- General cleanups
CHANGES SINCE V1:
- Added guest agent worker thread to execute RPCs in the guest. With this in
place we have a reliable timeout mechanism for hung commands, currently set
at 30 seconds.
- Add framework for registering init/cleanup routines for stateful RPCs to
clean up after themselves after a timeout.
- Added the following RPCs: guest-file-{open,close,read,write,seek},
guest-shutdown, guest-info, and removed stubs for guest-view-file (now
deprecated)
- Added GUEST_AGENT_UP/GUEST_AGENT_DOWN QMP events
- Switched to a TCP-style host-initiated 3-way handshake for channel
negotiation, this simplifies client negotiation/interaction over the wire
- Added configurable log level/log file/pid file options for guest agent
- Various fixes for bugs/memory leaks and checkpatch.pl fixups
ISSUES/TODOS:
- Add unit tests for guest agent wire protocol
OVERVIEW
For a better overview of what these patches are meant to accomplish, please
reference the RFC for virtagent:
http://comments.gmane.org/gmane.comp.emulators.qemu/96096
These patches integrate the previous virtagent guest agent work directly in
QAPI/QMP to leverage it's auto-generated marshalling code. This has numerous
benefits:
- addresses previous concerns over relying on external libraries to handle
data encapsulation
- reduces the need for manual unmarshalling of requests/responses, which
makes adding new RPCs much safer/less error-prone, as well as cutting down on
redundant code
- QAPI documentation aligns completely with guest-side RPC implementation
- is Just Better (TM)
BUILD/USAGE
build:
./configure --target-list=x86_64-softmmu
make
make qemu-ga #should be built on|for target guest
start guest:
qemu \
