I realizes my reply wasn't sent to the forum. Thanks Greg for the feedback. I'll keep this in mind for the next revision.
Jim ________________________________________ From: Greg Kurz <gr...@kaod.org> Sent: Wednesday, September 07, 2016 2:33 PM To: Nutaro, James J. Cc: qemu-devel@nongnu.org Subject: Re: [Qemu-devel] [PATCH v3] qqq: module for synchronizing with a simulation On Wed, 7 Sep 2016 13:53:18 -0400 "James J. Nutaro" <nutar...@ornl.gov> wrote: > This patch adds an interface for pacing the execution of > QEMU to match its rtc with an external simulation clock. > Its aim is to permit QEMU to be used as a module within a > larger simulation system. This revision (v3) corrects > formatting errors detected by Patchew. > The changelog should only mention what the patch actually does. If you have to re-post newer versions before the patch is finally accepted, changes should be mentioned... > Signed-off-by: James J. Nutaro <nutar...@ornl.gov> > --- ... here, below the ---, so they don't appear in the commit changelog. Cheers. -- Greg > Makefile.target | 3 ++ > docs/simulation-sync.txt | 60 ++++++++++++++++++++++++++ > qemu-options.hx | 16 +++++++ > qqq.c | 109 > +++++++++++++++++++++++++++++++++++++++++++++++ > qqq.h | 35 +++++++++++++++ > vl.c | 32 ++++++++++++++ > 6 files changed, 255 insertions(+) > create mode 100644 docs/simulation-sync.txt > create mode 100644 qqq.c > create mode 100644 qqq.h > > diff --git a/Makefile.target b/Makefile.target > index 8c7a072..b7f6c4c 100644 > --- a/Makefile.target > +++ b/Makefile.target > @@ -103,6 +103,9 @@ obj-$(CONFIG_LIBDECNUMBER) += libdecnumber/dpd/decimal32.o > obj-$(CONFIG_LIBDECNUMBER) += libdecnumber/dpd/decimal64.o > obj-$(CONFIG_LIBDECNUMBER) += libdecnumber/dpd/decimal128.o > > +# qqq > +obj-y += qqq.o > + > ######################################################### > # Linux user emulator target > > diff --git a/docs/simulation-sync.txt b/docs/simulation-sync.txt > new file mode 100644 > index 0000000..2133093 > --- /dev/null > +++ b/docs/simulation-sync.txt > @@ -0,0 +1,60 @@ > += Synchronizing the virtual clock with an external source = > + > +QEMU has a protocol for synchronizing its virtual clock > +with the clock of a simulator in which QEMU is embedded > +as a component. This options is enabled with the -qqq > +argument, and it should generally be accompanied by the > +following additional command line arguments: > + > +-icount 1,sleep=off -rtc clock=vm > + > +The -qqq argument is used to supply file descriptors > +for two Unix pipes. The read pipe is used by QEMU to > +receive synchronization data from the external simulator. > +The write pipe is used by QEMU to supply synchronization > +data to the external emulator. The typical procedure for > +launching QEMU in is synchronization mode has three steps: > + > +(1) Create two pairs of pipes with the Linux pipe function. > + The code segment that does this might look like > + > + int pipefd1[2]; > + int pipefd2[2]; > + pipe(pipefd1); > + pipe(pipefd2); > + > +(2) Fork QEMU with the appropriate command line arguments. > + The -qqq part of the argument will look something like > + > + -qqq write=pipefd1[1],read=pipefd2[0] > + > +(3) After forking QEMU, close pipefd1[1] and pipefd2[0]. > + Retain the other pair of pipes for communicating with QEMU. > + > +The synchronization protocol is very simple. To start, the > +external simulator writes an integer to its write pipe with > +the amount of time in microseconds that QEMU is allowed to > +advance. The code segment that does this might look like: > + > + int ta = 1000; // Advance by 1 millisecond > + write(pipefd2[1],&ta,sizeof(int)); > + > +The external simulator can then advance its clock by this > +same amount. During this time, QEMU and the external simulator > +will be executing in parallel. When the external simulator > +completes its time advance, it waits for QEMU by reading from > +its read pipe. The value read will be the actual number of > +virtual microseconds by which QEMU has advanced its virtual clock. > +This will be greater than or equal to the requested advance. > +The code that does this might look like: > + > + read(pipefd1[0],&ta,sizeof(int)); > + > +These steps are repeated until either (1) the external simulator > +closes its pipes thereby causing QEMU to terminate or (2) QEMU > +stops executing (e.g., if the emulated computer is shutdown) and > +causes SIGPIPE to be generated by the closing of its pipes. > + > +You can find an example of a simulator using this protocol in > +the adevs simulation package at http://sourceforge.net/projects/adevs/ > + > diff --git a/qemu-options.hx b/qemu-options.hx > index a71aaf8..d52cc9c 100644 > --- a/qemu-options.hx > +++ b/qemu-options.hx > @@ -3359,6 +3359,22 @@ many timer interrupts were not processed by the > Windows guest and will > re-inject them. > ETEXI > > +DEF("qqq", HAS_ARG, QEMU_OPTION_qqq, \ > + "-qqq read=fd,write=fd\n" \ > + " enable synchronization of the virtual clock \n" \ > + " with an external simulation clock\n", QEMU_ARCH_ALL) > +STEXI > +@item -qqq read=@var{fd0},write=@var{fd1} > +@findex -qqq > +Qemu will use the supplied pipes to synchronize its virtual clock with > +an external simulation clock. Qemu will wait until a time slice size in > +microseconds is supplied on the read pipe. Then it will execute for at > +least that number of virtual microseconds before writing the actual > +virtual time that has elapsed in microseconds to the write pipe. This > +cycle will repeat until a zero is elaspsed time is requested, which > +will cause qemu to exit. > +ETEXI > + > DEF("icount", HAS_ARG, QEMU_OPTION_icount, \ > "-icount > [shift=N|auto][,align=on|off][,sleep=on|off,rr=record|replay,rrfile=<filename>]\n" > \ > " enable virtual instruction counter with 2^N clock ticks > per\n" \ > diff --git a/qqq.c b/qqq.c > new file mode 100644 > index 0000000..f94711f > --- /dev/null > +++ b/qqq.c > @@ -0,0 +1,109 @@ > + > +#include "qemu/osdep.h" > +#include "qemu/timer.h" > +#include "qemu/main-loop.h" > +#include "sysemu/cpus.h" > +#include "qqq.h" > +#include <time.h> > + > +/* This is a Linux only feature */ > + > +#ifndef _WIN32 > + > +#include <unistd.h> > +#include <assert.h> > + > +static int elapsed; > +static int time_advance = -1; > +static int read_fd = -1, write_fd = -1; > +static int64_t t; > +static QEMUTimer *sync_timer; > + > +static void cleanup_and_exit(void) > +{ > + close(read_fd); > + close(write_fd); > + exit(0); > +} > + > +static void write_mem_value(int val) > +{ > + if (write(write_fd, &val, sizeof(int)) != sizeof(int)) { > + /* If the pipe is no good, then assume this is an > + * indication that we should exit. > + */ > + cleanup_and_exit(); > + } > +} > + > +static int read_mem_value(void) > +{ > + int tmp; > + if (read(read_fd, &tmp, sizeof(int)) != sizeof(int)) { > + /* If the pipe is no good, then assume this is an > + * indication that we should exit. > + */ > + cleanup_and_exit(); > + } > + return tmp; > +} > + > +static void schedule_next_event(void) > +{ > + /* If we got the time advance in fd_read, then don't do it > + * again here. */ > + if (time_advance < 0) { > + /* Otherwise read the value from the pipe */ > + time_advance = read_mem_value(); > + } > + /* Schedule the next synchronization point */ > + timer_mod(sync_timer, t + time_advance); > + /* Note that we need to read the time advance again on the next pass */ > + time_advance = -1; > +} > + > +static void sync_func(void *data) > +{ > + /* Report the actual elapsed time. */ > + int64_t tnow = qemu_clock_get_us(QEMU_CLOCK_VIRTUAL); > + elapsed = tnow - t; > + write_mem_value(elapsed); > + /* Update our time of last event */ > + t = tnow; > + /* Schedule the next event */ > + schedule_next_event(); > +} > + > +static void fd_read(void *opaque) > +{ > + /* Read the time advance if its becomes available > + * before our timer expires */ > + time_advance = read_mem_value(); > +} > + > +void setup_qqq(QemuOpts *opts) > +{ > + /* Initialize the simulation clock */ > + t = 0; > + /* Get the communication pipes */ > + read_fd = qemu_opt_get_number(opts, "read", 0); > + write_fd = qemu_opt_get_number(opts, "write", 0); > + /* Start the timer to ensure time warps advance the clock */ > + sync_timer = timer_new_us(QEMU_CLOCK_VIRTUAL, sync_func, NULL); > + /* Get the time advance that is requested by the simulation */ > + schedule_next_event(); > + /* Register the file descriptor with qemu. This should ensure > + * the emulator doesn't pause for lack of I/O and thereby > + * cause the attached simulator to pause with it. */ > + qemu_set_fd_handler(read_fd, fd_read, NULL, NULL); > +} > + > +#else > + > +void setup_qqq(QemuOpts *opts) > +{ > + fprintf(stderr, "-qqq is not supported on Windows, exiting\n"); > + exit(0); > +} > + > +#endif > diff --git a/qqq.h b/qqq.h > new file mode 100644 > index 0000000..40a24e3 > --- /dev/null > +++ b/qqq.h > @@ -0,0 +1,35 @@ > +/* > + * This work is licensed under the terms of the GNU GPL > + * version 2. Seethe COPYING file in the top-level directory. > + * > + * A module for pacing the rate of advance of the computer > + * clock in reference to an external simulation clock. The > + * basic approach used here is adapted from QBox from Green > + * Socs. The mode of operation is as follows: > + * > + * The simulator uses pipes to exchange time advance data. > + * The external simulator starts the exchange by forking a > + * QEMU process and passing is descriptors for a read and > + * write pipe. Then the external simulator writes an integer > + * (native endian) to the pipe to indicate the number of > + * microseconds that QEMU should advance. QEMU advances its > + * virtual clock by this amount and writes to its write pipe > + * the actual number of microseconds that have advanced. > + * This process continues until a pipe on either side is > + * closed, which will either cause QEMU to exit (if the > + * external simulator closes a pipe) or raise SIGPIPE in the > + * external simulator (if QEMU closes a pipe). > + * > + * Authors: > + * James Nutaro <nut...@gmail.com> > + * > + */ > +#ifndef QQQ_H > +#define QQQ_H > + > +#include "qemu/osdep.h" > +#include "qemu-options.h" > + > +void setup_qqq(QemuOpts *opts); > + > +#endif > diff --git a/vl.c b/vl.c > index ee557a1..fe72990 100644 > --- a/vl.c > +++ b/vl.c > @@ -122,6 +122,8 @@ int main(int argc, char **argv) > #include "sysemu/replay.h" > #include "qapi/qmp/qerror.h" > > +#include "qqq.h" > + > #define MAX_VIRTIO_CONSOLES 1 > #define MAX_SCLP_CONSOLES 1 > > @@ -231,6 +233,23 @@ static struct { > { .driver = "virtio-vga", .flag = &default_vga }, > }; > > +static QemuOptsList qemu_qqq_opts = { > + .name = "qqq", > + .implied_opt_name = "", > + .merge_lists = true, > + .head = QTAILQ_HEAD_INITIALIZER(qemu_qqq_opts.head), > + .desc = { > + { > + .name = "read", > + .type = QEMU_OPT_NUMBER, > + }, { > + .name = "write", > + .type = QEMU_OPT_NUMBER, > + }, > + { /* end of list */ } > + }, > +}; > + > static QemuOptsList qemu_rtc_opts = { > .name = "rtc", > .head = QTAILQ_HEAD_INITIALIZER(qemu_rtc_opts.head), > @@ -2952,6 +2971,7 @@ int main(int argc, char **argv, char **envp) > DisplayState *ds; > int cyls, heads, secs, translation; > QemuOpts *hda_opts = NULL, *opts, *machine_opts, *icount_opts = NULL; > + QemuOpts *qqq_opts = NULL; > QemuOptsList *olist; > int optind; > const char *optarg; > @@ -2987,6 +3007,7 @@ int main(int argc, char **argv, char **envp) > > module_call_init(MODULE_INIT_QOM); > > + qemu_add_opts(&qemu_qqq_opts); > qemu_add_opts(&qemu_drive_opts); > qemu_add_drive_opts(&qemu_legacy_drive_opts); > qemu_add_drive_opts(&qemu_common_drive_opts); > @@ -3847,6 +3868,13 @@ int main(int argc, char **argv, char **envp) > exit(1); > } > break; > + case QEMU_OPTION_qqq: > + qqq_opts = qemu_opts_parse_noisily(qemu_find_opts("qqq"), > + optarg, true); > + if (!qqq_opts) { > + exit(1); > + } > + break; > case QEMU_OPTION_incoming: > if (!incoming) { > runstate_set(RUN_STATE_INMIGRATE); > @@ -4350,6 +4378,10 @@ int main(int argc, char **argv, char **envp) > /* spice needs the timers to be initialized by this point */ > qemu_spice_init(); > > + if (qqq_opts) { > + setup_qqq(qqq_opts); > + } > + > cpu_ticks_init(); > if (icount_opts) { > if (kvm_enabled() || xen_enabled()) {
