Re: [Qemu-devel] [RFC PATCH v8 11/21] replay: recording and replaying clock ticks
From: Paolo Bonzini [mailto:pbonz...@redhat.com] On 03/02/2015 11:51, Pavel Dovgaluk wrote: From: Paolo Bonzini [mailto:pbonz...@redhat.com] On 22/01/2015 09:52, Pavel Dovgalyuk wrote: Clock ticks are considered as the sources of non-deterministic data for virtual machine. This patch implements saving the clock values when they are acquired (virtual, host clock, rdtsc, and some other timers). When replaying the execution corresponding values are read from log and transfered to the module, which wants to read the values. Such a design required the clock polling to be synchronized. Sometimes it is not true - e.g. when timeouts for timer lists are checked. In this case we use a cached value of the clock, passing it to the client code. Signed-off-by: Pavel Dovgalyuk pavel.dovga...@ispras.ru --- cpus.c |3 +- include/qemu/timer.h | 10 + qemu-timer.c |7 ++-- replay/Makefile.objs |1 + replay/replay-internal.h | 13 +++ replay/replay-time.c | 84 ++ replay/replay.h | 25 ++ stubs/replay.c |9 + 8 files changed, 147 insertions(+), 5 deletions(-) create mode 100755 replay/replay-time.c diff --git a/cpus.c b/cpus.c index 8787277..01d89aa 100644 --- a/cpus.c +++ b/cpus.c @@ -353,7 +353,8 @@ static void icount_warp_rt(void *opaque) seqlock_write_lock(timers_state.vm_clock_seqlock); if (runstate_is_running()) { -int64_t clock = cpu_get_clock_locked(); +int64_t clock = REPLAY_CLOCK(REPLAY_CLOCK_VIRTUAL_RT, + cpu_get_clock_locked()); int64_t warp_delta; warp_delta = clock - vm_clock_warp_start; diff --git a/include/qemu/timer.h b/include/qemu/timer.h index 0666920..0c2472c 100644 --- a/include/qemu/timer.h +++ b/include/qemu/timer.h @@ -4,6 +4,7 @@ #include qemu/typedefs.h #include qemu-common.h #include qemu/notify.h +#include replay/replay.h /* timers */ @@ -760,6 +761,8 @@ int64_t cpu_icount_to_ns(int64_t icount); /***/ /* host CPU ticks (if available) */ +#define cpu_get_real_ticks cpu_get_real_ticks_impl + #if defined(_ARCH_PPC) static inline int64_t cpu_get_real_ticks(void) @@ -913,6 +916,13 @@ static inline int64_t cpu_get_real_ticks (void) } #endif +#undef cpu_get_real_ticks + +static inline int64_t cpu_get_real_ticks(void) cpu_get_real_ticks should never be used. Please instead wrap cpu_get_ticks() with REPLAY_CLOCK. I don't quite understand this comment. Do you mean that I should move REPLAY_CLOCK to the cpu_get_real_ticks usages instead of it's implementation? Only to the cpu_get_ticks usage. The others are okay. cpu_get_ticks cannot call cpu_get_real_ticks in icount mode. And other functions can. Then we should put REPLAY_CLOCK into those functions? +/*! Reads next clock value from the file. +If clock kind read from the file is different from the parameter, +the value is not used. +If the parameter is -1, the clock value is read to the cache anyway. */ In what case could the clock kind not match? It was used in full version which had to skip clock from the log while loading the VM state. So can it be removed for now? I think it can. Pavel Dovgalyuk
Re: [Qemu-devel] [RFC PATCH v8 11/21] replay: recording and replaying clock ticks
On 03/02/2015 12:23, Pavel Dovgaluk wrote: Only to the cpu_get_ticks usage. The others are okay. cpu_get_ticks cannot call cpu_get_real_ticks in icount mode. You're right, but... And other functions can. ... which functions? cpu_enable_ticks and cpu_disable_ticks call it, but the result is only used by cpu_get_ticks. As you said, this cannot be called in icount mode. A couple of PPC device models call it, but they probably should call cpu_get_ticks instead. So it looks like handling REAL_TICKS is altogether unnecessary. Paolo
Re: [Qemu-devel] [RFC PATCH v8 11/21] replay: recording and replaying clock ticks
From: Paolo Bonzini [mailto:pbonz...@redhat.com] On 22/01/2015 09:52, Pavel Dovgalyuk wrote: Clock ticks are considered as the sources of non-deterministic data for virtual machine. This patch implements saving the clock values when they are acquired (virtual, host clock, rdtsc, and some other timers). When replaying the execution corresponding values are read from log and transfered to the module, which wants to read the values. Such a design required the clock polling to be synchronized. Sometimes it is not true - e.g. when timeouts for timer lists are checked. In this case we use a cached value of the clock, passing it to the client code. Signed-off-by: Pavel Dovgalyuk pavel.dovga...@ispras.ru --- cpus.c |3 +- include/qemu/timer.h | 10 + qemu-timer.c |7 ++-- replay/Makefile.objs |1 + replay/replay-internal.h | 13 +++ replay/replay-time.c | 84 ++ replay/replay.h | 25 ++ stubs/replay.c |9 + 8 files changed, 147 insertions(+), 5 deletions(-) create mode 100755 replay/replay-time.c diff --git a/cpus.c b/cpus.c index 8787277..01d89aa 100644 --- a/cpus.c +++ b/cpus.c @@ -353,7 +353,8 @@ static void icount_warp_rt(void *opaque) seqlock_write_lock(timers_state.vm_clock_seqlock); if (runstate_is_running()) { -int64_t clock = cpu_get_clock_locked(); +int64_t clock = REPLAY_CLOCK(REPLAY_CLOCK_VIRTUAL_RT, + cpu_get_clock_locked()); int64_t warp_delta; warp_delta = clock - vm_clock_warp_start; diff --git a/include/qemu/timer.h b/include/qemu/timer.h index 0666920..0c2472c 100644 --- a/include/qemu/timer.h +++ b/include/qemu/timer.h @@ -4,6 +4,7 @@ #include qemu/typedefs.h #include qemu-common.h #include qemu/notify.h +#include replay/replay.h /* timers */ @@ -760,6 +761,8 @@ int64_t cpu_icount_to_ns(int64_t icount); /***/ /* host CPU ticks (if available) */ +#define cpu_get_real_ticks cpu_get_real_ticks_impl + #if defined(_ARCH_PPC) static inline int64_t cpu_get_real_ticks(void) @@ -913,6 +916,13 @@ static inline int64_t cpu_get_real_ticks (void) } #endif +#undef cpu_get_real_ticks + +static inline int64_t cpu_get_real_ticks(void) cpu_get_real_ticks should never be used. Please instead wrap cpu_get_ticks() with REPLAY_CLOCK. I don't quite understand this comment. Do you mean that I should move REPLAY_CLOCK to the cpu_get_real_ticks usages instead of it's implementation? +{ +return REPLAY_CLOCK(REPLAY_CLOCK_REAL_TICKS, cpu_get_real_ticks_impl()); +} + #ifdef CONFIG_PROFILER static inline int64_t profile_getclock(void) { diff --git a/qemu-timer.c b/qemu-timer.c index 98d9d1b..bc981a2 100644 --- a/qemu-timer.c +++ b/qemu-timer.c @@ -25,6 +25,7 @@ #include sysemu/sysemu.h #include monitor/monitor.h #include ui/console.h +#include replay/replay.h #include hw/hw.h @@ -566,15 +567,15 @@ int64_t qemu_clock_get_ns(QEMUClockType type) return cpu_get_clock(); } case QEMU_CLOCK_HOST: -now = get_clock_realtime(); +now = REPLAY_CLOCK(REPLAY_CLOCK_HOST, get_clock_realtime()); last = clock-last; clock-last = now; -if (now last) { +if (now last replay_mode == REPLAY_MODE_NONE) { notifier_list_notify(clock-reset_notifiers, now); } return now; case QEMU_CLOCK_VIRTUAL_RT: -return cpu_get_clock(); +return REPLAY_CLOCK(REPLAY_CLOCK_VIRTUAL_RT, cpu_get_clock()); } } diff --git a/replay/Makefile.objs b/replay/Makefile.objs index 56da09c..257c320 100755 --- a/replay/Makefile.objs +++ b/replay/Makefile.objs @@ -1,3 +1,4 @@ obj-y += replay.o obj-y += replay-internal.o obj-y += replay-events.o +obj-y += replay-time.o diff --git a/replay/replay-internal.h b/replay/replay-internal.h index 1666d6e..e906ec3 100755 --- a/replay/replay-internal.h +++ b/replay/replay-internal.h @@ -22,7 +22,10 @@ enum ReplayEvents { /* for emulated exceptions */ EVENT_EXCEPTION, /* for async events */ -EVENT_ASYNC +EVENT_ASYNC, +/* for clock read/writes */ +/* some of grteater codes are reserved for clocks */ +EVENT_CLOCK }; /* Asynchronous events IDs */ @@ -34,6 +37,8 @@ enum ReplayAsyncEventKind { typedef enum ReplayAsyncEventKind ReplayAsyncEventKind; typedef struct ReplayState { +/*! Cached clock values. */ +int64_t cached_clock[REPLAY_CLOCK_COUNT]; /*! Current step - number of processed instructions and timer events. */ uint64_t current_step; /*! Number of instructions to
Re: [Qemu-devel] [RFC PATCH v8 11/21] replay: recording and replaying clock ticks
On 03/02/2015 11:51, Pavel Dovgaluk wrote: From: Paolo Bonzini [mailto:pbonz...@redhat.com] On 22/01/2015 09:52, Pavel Dovgalyuk wrote: Clock ticks are considered as the sources of non-deterministic data for virtual machine. This patch implements saving the clock values when they are acquired (virtual, host clock, rdtsc, and some other timers). When replaying the execution corresponding values are read from log and transfered to the module, which wants to read the values. Such a design required the clock polling to be synchronized. Sometimes it is not true - e.g. when timeouts for timer lists are checked. In this case we use a cached value of the clock, passing it to the client code. Signed-off-by: Pavel Dovgalyuk pavel.dovga...@ispras.ru --- cpus.c |3 +- include/qemu/timer.h | 10 + qemu-timer.c |7 ++-- replay/Makefile.objs |1 + replay/replay-internal.h | 13 +++ replay/replay-time.c | 84 ++ replay/replay.h | 25 ++ stubs/replay.c |9 + 8 files changed, 147 insertions(+), 5 deletions(-) create mode 100755 replay/replay-time.c diff --git a/cpus.c b/cpus.c index 8787277..01d89aa 100644 --- a/cpus.c +++ b/cpus.c @@ -353,7 +353,8 @@ static void icount_warp_rt(void *opaque) seqlock_write_lock(timers_state.vm_clock_seqlock); if (runstate_is_running()) { -int64_t clock = cpu_get_clock_locked(); +int64_t clock = REPLAY_CLOCK(REPLAY_CLOCK_VIRTUAL_RT, + cpu_get_clock_locked()); int64_t warp_delta; warp_delta = clock - vm_clock_warp_start; diff --git a/include/qemu/timer.h b/include/qemu/timer.h index 0666920..0c2472c 100644 --- a/include/qemu/timer.h +++ b/include/qemu/timer.h @@ -4,6 +4,7 @@ #include qemu/typedefs.h #include qemu-common.h #include qemu/notify.h +#include replay/replay.h /* timers */ @@ -760,6 +761,8 @@ int64_t cpu_icount_to_ns(int64_t icount); /***/ /* host CPU ticks (if available) */ +#define cpu_get_real_ticks cpu_get_real_ticks_impl + #if defined(_ARCH_PPC) static inline int64_t cpu_get_real_ticks(void) @@ -913,6 +916,13 @@ static inline int64_t cpu_get_real_ticks (void) } #endif +#undef cpu_get_real_ticks + +static inline int64_t cpu_get_real_ticks(void) cpu_get_real_ticks should never be used. Please instead wrap cpu_get_ticks() with REPLAY_CLOCK. I don't quite understand this comment. Do you mean that I should move REPLAY_CLOCK to the cpu_get_real_ticks usages instead of it's implementation? Only to the cpu_get_ticks usage. The others are okay. +/*! Reads next clock value from the file. +If clock kind read from the file is different from the parameter, +the value is not used. +If the parameter is -1, the clock value is read to the cache anyway. */ In what case could the clock kind not match? It was used in full version which had to skip clock from the log while loading the VM state. So can it be removed for now? Paolo +void replay_read_next_clock(unsigned int kind); + /* Asynchronous events queue */ Pavel Dovgalyuk
Re: [Qemu-devel] [RFC PATCH v8 11/21] replay: recording and replaying clock ticks
On 22/01/2015 09:52, Pavel Dovgalyuk wrote: Clock ticks are considered as the sources of non-deterministic data for virtual machine. This patch implements saving the clock values when they are acquired (virtual, host clock, rdtsc, and some other timers). When replaying the execution corresponding values are read from log and transfered to the module, which wants to read the values. Such a design required the clock polling to be synchronized. Sometimes it is not true - e.g. when timeouts for timer lists are checked. In this case we use a cached value of the clock, passing it to the client code. Signed-off-by: Pavel Dovgalyuk pavel.dovga...@ispras.ru --- cpus.c |3 +- include/qemu/timer.h | 10 + qemu-timer.c |7 ++-- replay/Makefile.objs |1 + replay/replay-internal.h | 13 +++ replay/replay-time.c | 84 ++ replay/replay.h | 25 ++ stubs/replay.c |9 + 8 files changed, 147 insertions(+), 5 deletions(-) create mode 100755 replay/replay-time.c diff --git a/cpus.c b/cpus.c index 8787277..01d89aa 100644 --- a/cpus.c +++ b/cpus.c @@ -353,7 +353,8 @@ static void icount_warp_rt(void *opaque) seqlock_write_lock(timers_state.vm_clock_seqlock); if (runstate_is_running()) { -int64_t clock = cpu_get_clock_locked(); +int64_t clock = REPLAY_CLOCK(REPLAY_CLOCK_VIRTUAL_RT, + cpu_get_clock_locked()); int64_t warp_delta; warp_delta = clock - vm_clock_warp_start; diff --git a/include/qemu/timer.h b/include/qemu/timer.h index 0666920..0c2472c 100644 --- a/include/qemu/timer.h +++ b/include/qemu/timer.h @@ -4,6 +4,7 @@ #include qemu/typedefs.h #include qemu-common.h #include qemu/notify.h +#include replay/replay.h /* timers */ @@ -760,6 +761,8 @@ int64_t cpu_icount_to_ns(int64_t icount); /***/ /* host CPU ticks (if available) */ +#define cpu_get_real_ticks cpu_get_real_ticks_impl + #if defined(_ARCH_PPC) static inline int64_t cpu_get_real_ticks(void) @@ -913,6 +916,13 @@ static inline int64_t cpu_get_real_ticks (void) } #endif +#undef cpu_get_real_ticks + +static inline int64_t cpu_get_real_ticks(void) cpu_get_real_ticks should never be used. Please instead wrap cpu_get_ticks() with REPLAY_CLOCK. +{ +return REPLAY_CLOCK(REPLAY_CLOCK_REAL_TICKS, cpu_get_real_ticks_impl()); +} + #ifdef CONFIG_PROFILER static inline int64_t profile_getclock(void) { diff --git a/qemu-timer.c b/qemu-timer.c index 98d9d1b..bc981a2 100644 --- a/qemu-timer.c +++ b/qemu-timer.c @@ -25,6 +25,7 @@ #include sysemu/sysemu.h #include monitor/monitor.h #include ui/console.h +#include replay/replay.h #include hw/hw.h @@ -566,15 +567,15 @@ int64_t qemu_clock_get_ns(QEMUClockType type) return cpu_get_clock(); } case QEMU_CLOCK_HOST: -now = get_clock_realtime(); +now = REPLAY_CLOCK(REPLAY_CLOCK_HOST, get_clock_realtime()); last = clock-last; clock-last = now; -if (now last) { +if (now last replay_mode == REPLAY_MODE_NONE) { notifier_list_notify(clock-reset_notifiers, now); } return now; case QEMU_CLOCK_VIRTUAL_RT: -return cpu_get_clock(); +return REPLAY_CLOCK(REPLAY_CLOCK_VIRTUAL_RT, cpu_get_clock()); } } diff --git a/replay/Makefile.objs b/replay/Makefile.objs index 56da09c..257c320 100755 --- a/replay/Makefile.objs +++ b/replay/Makefile.objs @@ -1,3 +1,4 @@ obj-y += replay.o obj-y += replay-internal.o obj-y += replay-events.o +obj-y += replay-time.o diff --git a/replay/replay-internal.h b/replay/replay-internal.h index 1666d6e..e906ec3 100755 --- a/replay/replay-internal.h +++ b/replay/replay-internal.h @@ -22,7 +22,10 @@ enum ReplayEvents { /* for emulated exceptions */ EVENT_EXCEPTION, /* for async events */ -EVENT_ASYNC +EVENT_ASYNC, +/* for clock read/writes */ +/* some of grteater codes are reserved for clocks */ +EVENT_CLOCK }; /* Asynchronous events IDs */ @@ -34,6 +37,8 @@ enum ReplayAsyncEventKind { typedef enum ReplayAsyncEventKind ReplayAsyncEventKind; typedef struct ReplayState { +/*! Cached clock values. */ +int64_t cached_clock[REPLAY_CLOCK_COUNT]; /*! Current step - number of processed instructions and timer events. */ uint64_t current_step; /*! Number of instructions to be executed before other events happen. */ @@ -88,6 +93,12 @@ bool skip_async_events(int stop_event); reports an error and stops the execution. */ void skip_async_events_until(unsigned int kind); +/*! Reads next clock value from the file. +If clock kind read from the file is different from the parameter,
[Qemu-devel] [RFC PATCH v8 11/21] replay: recording and replaying clock ticks
Clock ticks are considered as the sources of non-deterministic data for virtual machine. This patch implements saving the clock values when they are acquired (virtual, host clock, rdtsc, and some other timers). When replaying the execution corresponding values are read from log and transfered to the module, which wants to read the values. Such a design required the clock polling to be synchronized. Sometimes it is not true - e.g. when timeouts for timer lists are checked. In this case we use a cached value of the clock, passing it to the client code. Signed-off-by: Pavel Dovgalyuk pavel.dovga...@ispras.ru --- cpus.c |3 +- include/qemu/timer.h | 10 + qemu-timer.c |7 ++-- replay/Makefile.objs |1 + replay/replay-internal.h | 13 +++ replay/replay-time.c | 84 ++ replay/replay.h | 25 ++ stubs/replay.c |9 + 8 files changed, 147 insertions(+), 5 deletions(-) create mode 100755 replay/replay-time.c diff --git a/cpus.c b/cpus.c index 8787277..01d89aa 100644 --- a/cpus.c +++ b/cpus.c @@ -353,7 +353,8 @@ static void icount_warp_rt(void *opaque) seqlock_write_lock(timers_state.vm_clock_seqlock); if (runstate_is_running()) { -int64_t clock = cpu_get_clock_locked(); +int64_t clock = REPLAY_CLOCK(REPLAY_CLOCK_VIRTUAL_RT, + cpu_get_clock_locked()); int64_t warp_delta; warp_delta = clock - vm_clock_warp_start; diff --git a/include/qemu/timer.h b/include/qemu/timer.h index 0666920..0c2472c 100644 --- a/include/qemu/timer.h +++ b/include/qemu/timer.h @@ -4,6 +4,7 @@ #include qemu/typedefs.h #include qemu-common.h #include qemu/notify.h +#include replay/replay.h /* timers */ @@ -760,6 +761,8 @@ int64_t cpu_icount_to_ns(int64_t icount); /***/ /* host CPU ticks (if available) */ +#define cpu_get_real_ticks cpu_get_real_ticks_impl + #if defined(_ARCH_PPC) static inline int64_t cpu_get_real_ticks(void) @@ -913,6 +916,13 @@ static inline int64_t cpu_get_real_ticks (void) } #endif +#undef cpu_get_real_ticks + +static inline int64_t cpu_get_real_ticks(void) +{ +return REPLAY_CLOCK(REPLAY_CLOCK_REAL_TICKS, cpu_get_real_ticks_impl()); +} + #ifdef CONFIG_PROFILER static inline int64_t profile_getclock(void) { diff --git a/qemu-timer.c b/qemu-timer.c index 98d9d1b..bc981a2 100644 --- a/qemu-timer.c +++ b/qemu-timer.c @@ -25,6 +25,7 @@ #include sysemu/sysemu.h #include monitor/monitor.h #include ui/console.h +#include replay/replay.h #include hw/hw.h @@ -566,15 +567,15 @@ int64_t qemu_clock_get_ns(QEMUClockType type) return cpu_get_clock(); } case QEMU_CLOCK_HOST: -now = get_clock_realtime(); +now = REPLAY_CLOCK(REPLAY_CLOCK_HOST, get_clock_realtime()); last = clock-last; clock-last = now; -if (now last) { +if (now last replay_mode == REPLAY_MODE_NONE) { notifier_list_notify(clock-reset_notifiers, now); } return now; case QEMU_CLOCK_VIRTUAL_RT: -return cpu_get_clock(); +return REPLAY_CLOCK(REPLAY_CLOCK_VIRTUAL_RT, cpu_get_clock()); } } diff --git a/replay/Makefile.objs b/replay/Makefile.objs index 56da09c..257c320 100755 --- a/replay/Makefile.objs +++ b/replay/Makefile.objs @@ -1,3 +1,4 @@ obj-y += replay.o obj-y += replay-internal.o obj-y += replay-events.o +obj-y += replay-time.o diff --git a/replay/replay-internal.h b/replay/replay-internal.h index 1666d6e..e906ec3 100755 --- a/replay/replay-internal.h +++ b/replay/replay-internal.h @@ -22,7 +22,10 @@ enum ReplayEvents { /* for emulated exceptions */ EVENT_EXCEPTION, /* for async events */ -EVENT_ASYNC +EVENT_ASYNC, +/* for clock read/writes */ +/* some of grteater codes are reserved for clocks */ +EVENT_CLOCK }; /* Asynchronous events IDs */ @@ -34,6 +37,8 @@ enum ReplayAsyncEventKind { typedef enum ReplayAsyncEventKind ReplayAsyncEventKind; typedef struct ReplayState { +/*! Cached clock values. */ +int64_t cached_clock[REPLAY_CLOCK_COUNT]; /*! Current step - number of processed instructions and timer events. */ uint64_t current_step; /*! Number of instructions to be executed before other events happen. */ @@ -88,6 +93,12 @@ bool skip_async_events(int stop_event); reports an error and stops the execution. */ void skip_async_events_until(unsigned int kind); +/*! Reads next clock value from the file. +If clock kind read from the file is different from the parameter, +the value is not used. +If the parameter is -1, the clock value is read to the cache anyway. */ +void replay_read_next_clock(unsigned int kind); + /* Asynchronous events queue */ /*! Initializes events' processing internals */ diff --git a/replay/replay-time.c