Re: QEMU Clock record and replay

[Arnabjyoti Kalita]

The host clock refers to one of the clock types used by QEMU -

```
@QEMU_CLOCK_HOST: host clock
*
* The host clock should be used for device models that emulate accurate
* real time sources. It will continue to run when the virtual machine
* is suspended, and it will reflect system time changes the host may
* undergo (e.g. due to NTP).
```

Thanks.

Best Regards,
Arnab

On Fri, Feb 26, 2021 at 3:25 PM Alex Bennée  wrote:

>
> Pavel Dovgalyuk  writes:
>
> > On 25.02.2021 18:43, Philippe Mathieu-Daudé wrote:
> >> Cc'ing Pavel/Alex.
> >>
> >> On 2/25/21 4:09 PM, Arnabjyoti Kalita wrote:
> >>> Hello all,
> >>>
> >>> I am trying to understand how the clock values are recorded and
> replayed
> >>> in QEMU (when it runs in TCG mode). I have been specifically following
> >>> the document that has been provided here -
> >>> https://github.com/qemu/qemu/blob/master/docs/replay.txt
> >>> 
> >>>
> >>> I have the following 4 questions to ask-
> >>>
> >>> - Why are clock values being replayed off a cache and not directly off
> >>> the file as seen here -
> >>> https://github.com/qemu/qemu/blob/stable-5.0/replay/replay-time.c#L45?
> >>>  ?>?
> >
> > The cache was needed when replay does not follow the record behavior.
> > E.g., included additional deadline calculations or timer checks.
> > I'm not sure, that it is still needed, and I'll check it during future
> > refactoring session.
> >
> >>> - I have a requirement to record and replay host clock values. Can it
> so
> >>> happen that if incorrect values of the host clock are replayed, then
> the
> >>> timers in the guest start expiring incorrectly, during replay?
> >
> > This is a bug, when the host clock is replayed incorrectly.
>
> When you say host clock what do you mean? Surely all the cycle counters
> are VIRTUAL? Or do you mean things like the RTC?
>
> --
> Alex Bennée
>

Re: QEMU Clock record and replay

[Arnabjyoti Kalita]

Thank you Pavel.

You answered all of my questions. The clock record-replay functionality is
much clearer now.

Best Regards,
Arnab

On Fri, Feb 26, 2021 at 11:54 AM Pavel Dovgalyuk 
wrote:

> On 25.02.2021 18:43, Philippe Mathieu-Daudé wrote:
> > Cc'ing Pavel/Alex.
> >
> > On 2/25/21 4:09 PM, Arnabjyoti Kalita wrote:
> >> Hello all,
> >>
> >> I am trying to understand how the clock values are recorded and replayed
> >> in QEMU (when it runs in TCG mode). I have been specifically following
> >> the document that has been provided here -
> >> https://github.com/qemu/qemu/blob/master/docs/replay.txt
> >> 
> >>
> >> I have the following 4 questions to ask-
> >>
> >> - Why are clock values being replayed off a cache and not directly off
> >> the file as seen here -
> >> https://github.com/qemu/qemu/blob/stable-5.0/replay/replay-time.c#L45?
> >>  ?>?
>
> The cache was needed when replay does not follow the record behavior.
> E.g., included additional deadline calculations or timer checks.
> I'm not sure, that it is still needed, and I'll check it during future
> refactoring session.
>
> >> - I have a requirement to record and replay host clock values. Can it so
> >> happen that if incorrect values of the host clock are replayed, then the
> >> timers in the guest start expiring incorrectly, during replay?
>
> This is a bug, when the host clock is replayed incorrectly.
>
> >> - Let's say we record the clock values when the guest starts in KVM
> >> mode, and we replay the clock values when the guest starts in TCG mode.
> >> Does the record-replay functionality change a lot?
>
> Only if you want to replay just the clock.
> But replaying whole execution includes instruction counting and thread
> synchronization.
>
> >> - Also, I couldn't understand the specific purpose of replaying the
> >> 'virtual real time clock' ? Do I still need to record and replay the
> >> 'virtual real time clock' if we record the clock values in KVM mode, and
> >> replay the clock values in TCG mode?
>
> This clock is used for icount-related timers. For now icount is required
> for record replay. If you want to replay only host clock, you don't need
> this one.
>
> Pavel Dovgalyuk
>

Re: QEMU Clock record and replay

[Pavel Dovgalyuk]

On 25.02.2021 18:43, Philippe Mathieu-Daudé wrote:

Cc'ing Pavel/Alex.

On 2/25/21 4:09 PM, Arnabjyoti Kalita wrote:

Hello all,

I am trying to understand how the clock values are recorded and replayed
in QEMU (when it runs in TCG mode). I have been specifically following
the document that has been provided here -
https://github.com/qemu/qemu/blob/master/docs/replay.txt


I have the following 4 questions to ask-

- Why are clock values being replayed off a cache and not directly off
the file as seen here -
https://github.com/qemu/qemu/blob/stable-5.0/replay/replay-time.c#L45?
?


The cache was needed when replay does not follow the record behavior.
E.g., included additional deadline calculations or timer checks.
I'm not sure, that it is still needed, and I'll check it during future
refactoring session.


- I have a requirement to record and replay host clock values. Can it so
happen that if incorrect values of the host clock are replayed, then the
timers in the guest start expiring incorrectly, during replay?


This is a bug, when the host clock is replayed incorrectly.


- Let's say we record the clock values when the guest starts in KVM
mode, and we replay the clock values when the guest starts in TCG mode.
Does the record-replay functionality change a lot?


Only if you want to replay just the clock.
But replaying whole execution includes instruction counting and thread 
synchronization.



- Also, I couldn't understand the specific purpose of replaying the
'virtual real time clock' ? Do I still need to record and replay the
'virtual real time clock' if we record the clock values in KVM mode, and
replay the clock values in TCG mode?


This clock is used for icount-related timers. For now icount is required
for record replay. If you want to replay only host clock, you don't need 
this one.


Pavel Dovgalyuk

Re: QEMU Clock record and replay

[Alex Bennée]

Pavel Dovgalyuk  writes:

> On 25.02.2021 18:43, Philippe Mathieu-Daudé wrote:
>> Cc'ing Pavel/Alex.
>> 
>> On 2/25/21 4:09 PM, Arnabjyoti Kalita wrote:
>>> Hello all,
>>>
>>> I am trying to understand how the clock values are recorded and replayed
>>> in QEMU (when it runs in TCG mode). I have been specifically following
>>> the document that has been provided here -
>>> https://github.com/qemu/qemu/blob/master/docs/replay.txt
>>> 
>>>
>>> I have the following 4 questions to ask-
>>>
>>> - Why are clock values being replayed off a cache and not directly off
>>> the file as seen here -
>>> https://github.com/qemu/qemu/blob/stable-5.0/replay/replay-time.c#L45?
>>> ?
>
> The cache was needed when replay does not follow the record behavior.
> E.g., included additional deadline calculations or timer checks.
> I'm not sure, that it is still needed, and I'll check it during future
> refactoring session.
>
>>> - I have a requirement to record and replay host clock values. Can it so
>>> happen that if incorrect values of the host clock are replayed, then the
>>> timers in the guest start expiring incorrectly, during replay?
>
> This is a bug, when the host clock is replayed incorrectly.

When you say host clock what do you mean? Surely all the cycle counters
are VIRTUAL? Or do you mean things like the RTC?

-- 
Alex Bennée

Re: QEMU Clock record and replay

[Arnabjyoti Kalita]

Thank you Alex for such a detailed answer. It explains a lot of aspects
about the QEMU timer and icount features.

Let's say we start a guest in KVM mode, save its state and then allow the
guest to continue. Now, let's say we load the 'saved' guest in TCG mode. I
am looking for a way to ensure that the timer activity is synchronized with
the KVM mode, when I load the guest in TCG mode. Will this be possible to
achieve ?

Thank you very much again.

Best Regards,
Arnab

On Thu, Feb 25, 2021 at 10:30 PM Alex Bennée  wrote:

>
> Arnabjyoti Kalita  writes:
>
> > Hello all,
> >
> > I am trying to understand how the clock values are recorded and replayed
> in
> > QEMU (when it runs in TCG mode). I have been specifically following the
> > document that has been provided here -
> > https://github.com/qemu/qemu/blob/master/docs/replay.txt
> >
> > I have the following 4 questions to ask-
> >
> > - Why are clock values being replayed off a cache and not directly off
> the
> > file as seen here -
> > https://github.com/qemu/qemu/blob/stable-5.0/replay/replay-time.c#L45?
> >
> > - I have a requirement to record and replay host clock values. Can it so
> > happen that if incorrect values of the host clock are replayed, then the
> > timers in the guest start expiring incorrectly, during replay?
>
> Host clock values would be meaningless in the context of record/replay.
> "time" is simply a function of the number of instruction executed. As
> all asynchronous events (e.g. HW timers expiring) are a function of this
> executed instruction count you get determinism as no external events
> influence the behaviour of the run. This is then paired with recording
> other hardware events like blocks and packets against this time to get
> your full record/replay experience.
>
> The underlying icount is explained in a bit more detail at:
>
>   https://qemu.readthedocs.io/en/latest/devel/tcg-icount.html
>
> > - Let's say we record the clock values when the guest starts in KVM mode,
> > and we replay the clock values when the guest starts in TCG mode. Does
> the
> > record-replay functionality change a lot?
> >
> > - Also, I couldn't understand the specific purpose of replaying the
> > 'virtual real time clock' ? Do I still need to record and replay the
> > 'virtual real time clock' if we record the clock values in KVM mode, and
> > replay the clock values in TCG mode?
>
> It won't work. You can only ever record when running in TCG icount mode
> because having that stable and deterministic behaviour is a
> pre-requisite.
>
> --
> Alex Bennée
>

Re: QEMU Clock record and replay

[Arnabjyoti Kalita]

Thank you Lukas for your answer.

I was only looking to record and replay host clock values (not any other
events), i.e. record in KVM mode and replay in TCG mode.

Best Regards,
Arnab

On Thu, Feb 25, 2021 at 10:00 PM Lukas Straub  wrote:

> On Thu, 25 Feb 2021 20:39:20 +0530
> Arnabjyoti Kalita  wrote:
>
> > Hello all,
> >
> > [...]
> >
> > - Let's say we record the clock values when the guest starts in KVM mode,
> > and we replay the clock values when the guest starts in TCG mode. Does
> the
> > record-replay functionality change a lot?
>
> Record-replay only works in TCG mode, as one needs to intercept rdtsc/wrtsc
> instructions (among others) and KVM doesn't support that.
>
> Regards,
> Lukas Straub
>
> > - Also, I couldn't understand the specific purpose of replaying the
> > 'virtual real time clock' ? Do I still need to record and replay the
> > 'virtual real time clock' if we record the clock values in KVM mode, and
> > replay the clock values in TCG mode?
> >
> > Thank you very much.
> >
> > Best Regards,
> > Arnab
>
>
>
> --
>
>

Re: QEMU Clock record and replay

[Alex Bennée]

Arnabjyoti Kalita  writes:

> Hello all,
>
> I am trying to understand how the clock values are recorded and replayed in
> QEMU (when it runs in TCG mode). I have been specifically following the
> document that has been provided here -
> https://github.com/qemu/qemu/blob/master/docs/replay.txt
>
> I have the following 4 questions to ask-
>
> - Why are clock values being replayed off a cache and not directly off the
> file as seen here -
> https://github.com/qemu/qemu/blob/stable-5.0/replay/replay-time.c#L45?
>
> - I have a requirement to record and replay host clock values. Can it so
> happen that if incorrect values of the host clock are replayed, then the
> timers in the guest start expiring incorrectly, during replay?

Host clock values would be meaningless in the context of record/replay.
"time" is simply a function of the number of instruction executed. As
all asynchronous events (e.g. HW timers expiring) are a function of this
executed instruction count you get determinism as no external events
influence the behaviour of the run. This is then paired with recording
other hardware events like blocks and packets against this time to get
your full record/replay experience.

The underlying icount is explained in a bit more detail at:

  https://qemu.readthedocs.io/en/latest/devel/tcg-icount.html

> - Let's say we record the clock values when the guest starts in KVM mode,
> and we replay the clock values when the guest starts in TCG mode. Does the
> record-replay functionality change a lot?
>
> - Also, I couldn't understand the specific purpose of replaying the
> 'virtual real time clock' ? Do I still need to record and replay the
> 'virtual real time clock' if we record the clock values in KVM mode, and
> replay the clock values in TCG mode?

It won't work. You can only ever record when running in TCG icount mode
because having that stable and deterministic behaviour is a
pre-requisite.

-- 
Alex Bennée

Re: QEMU Clock record and replay

[Lukas Straub]

On Thu, 25 Feb 2021 20:39:20 +0530
Arnabjyoti Kalita  wrote:

> Hello all,
>
> [...]
> 
> - Let's say we record the clock values when the guest starts in KVM mode,
> and we replay the clock values when the guest starts in TCG mode. Does the
> record-replay functionality change a lot?

Record-replay only works in TCG mode, as one needs to intercept rdtsc/wrtsc
instructions (among others) and KVM doesn't support that.

Regards,
Lukas Straub

> - Also, I couldn't understand the specific purpose of replaying the
> 'virtual real time clock' ? Do I still need to record and replay the
> 'virtual real time clock' if we record the clock values in KVM mode, and
> replay the clock values in TCG mode?
> 
> Thank you very much.
> 
> Best Regards,
> Arnab



-- 



pgpRctXrsPYJt.pgp
Description: OpenPGP digital signature

Re: QEMU Clock record and replay

[Philippe Mathieu-Daudé]

Cc'ing Pavel/Alex.

On 2/25/21 4:09 PM, Arnabjyoti Kalita wrote:
> Hello all,
> 
> I am trying to understand how the clock values are recorded and replayed
> in QEMU (when it runs in TCG mode). I have been specifically following
> the document that has been provided here -
> https://github.com/qemu/qemu/blob/master/docs/replay.txt
> 
> 
> I have the following 4 questions to ask-
> 
> - Why are clock values being replayed off a cache and not directly off
> the file as seen here -
> https://github.com/qemu/qemu/blob/stable-5.0/replay/replay-time.c#L45?
> ?
> 
> - I have a requirement to record and replay host clock values. Can it so
> happen that if incorrect values of the host clock are replayed, then the
> timers in the guest start expiring incorrectly, during replay?
> 
> - Let's say we record the clock values when the guest starts in KVM
> mode, and we replay the clock values when the guest starts in TCG mode.
> Does the record-replay functionality change a lot?
> 
> - Also, I couldn't understand the specific purpose of replaying the
> 'virtual real time clock' ? Do I still need to record and replay the
> 'virtual real time clock' if we record the clock values in KVM mode, and
> replay the clock values in TCG mode?
> 
> Thank you very much.
> 
> Best Regards,
> Arnab
> 
>