Re: How does QEMU in TCG mode handle interrupts ?
Thank you for the detailed explanation, Peter. It makes a lot of things
clear.
Arnab
On Tue, Aug 25, 2020 at 4:00 PM Peter Maydell
wrote:
> On Tue, 25 Aug 2020 at 06:53, Arnabjyoti Kalita
> wrote:
> > This makes sense. In this scenario, when QEMU takes an interrupt at the
> end of a TB, I understand that the TB execution will not happen.
>
> When QEMU takes an interrupt at the *end* of a TB, that TB
> has already executed. (Really we take interrupts between
> TBs, so at the end of one and the start of the next.)
>
> To be a bit more precise, the way this works is that there's
> a loop in cpu_exec():
>
>while (!cpu_handle_exception(...)) {
>while (!cpu_handle_interrupt(...)) {
>tb = tb_find(...);
>cpu_loop_exec_tb(...);
>}
>}
>
> where cpu_loop_exec_tb() will start the TB, which might then
> chain forward to the next TB without returning to this C loop.
> So the first bit of generated code in each TB checks a flag
> to see if there's a pending interrupt that means it should
> just exit back to the C code loop without executing the TB
> (we'll then call cpu_handle_interrupt() to process it).
>
> > The interrupt will be taken and then the same TB will be
> > re-translated again and later executed, right ?
>
> We won't re-translate a TB just because it returned without
> executing. We will likely re-execute it later after the guest
> returns from the interrupt, but that depends on guest behaviour.
> (There is no special casing for this: we just implement "return
> from interrupt means we set the guest PC and other registers
> as the guest CPU architecture specifies"; then when we find
> the first TB after that, it will turn out to be the same TB
> we abandoned execution of, if the guest made the return-from-interrupt
> go back to the same place it left off. But if the guest did
> something clever, eg OS context switch after interrupt handler
> runs, then the guest PC will be something else and we'll execute
> from there.)
>
> > If so, does this methodology apply for all kinds of interrupts,
> > hardware/software/faults/traps/vmexits etc. ?
>
> You can divide these things into three categories:
>
> * things which affect the emulated CPU and which are
>asynchronously triggered by other parts of the system --
>most notably, device interrupts. These are handled as I
>describe above, with flags that tell the guest CPU to stop
>executing at the next convenient point so it can deal with
>the interrupt. QEMU calls all these things "interrupts"
>in its generic code.
>
> * "expected" exceptions: things like software traps where
>a syscall instruction, for instance, will always cause an
>exception. These we handle by having the generated code
>for the instruction just be "raise an exception". (This is
>generally 'write the current PC etc back to the CPU state
>struct, call a C helper which sets some state about what
>kind of exception this is and then calls cpu_loop_exit(),
>which does a siglongjmp() out to cpu_exec() so it can do
>the "cpu_handle_exception()" call immediately'.) We also
>handle cases like "undefined instruction" and "tried to
>exec an FPU instruction but the FPU is disabled" this way.
>You can also deal with "usually happens" exceptions
>like this too -- if a helper you call turns out not to
>raise an exception, that's fine, it just means you updated
>the PC unnecessarily, which is slightly inefficient but not
>a big deal in most cases.
>
> * "unexpected" exceptions: the primary example is a fault on
>a data load/store insn. This is similar to the "expected"
>case, except that we want to avoid the overhead of updating
>the CPU state struct with the current PC because almost always
>loads and stores succeed and don't take exceptions, and
>loads and stores are really common so we care about avoiding
>the extra insns that update the PC. So instead, when QEMU
>detects that a load/store failed it will call cpu_restore_state().
>This uses the *host* PC where the fault was detected (which
>will be somewhere inside the generated code for the TB),
>plus metadata that was generated and stored alongside the TB
>when we first translated it, to identify "for this host PC,
>what is the guest PC (plus any other per-target info we need)",
>and then we can fix up the CPU state using that info. Once
>the CPU state has been fixed up we call cpu_loop_exit() and
>the rest is like the "expected" exception case.
>
> In all this I have been ignoring the 'icount' feature, which
> when enabled does cause us sometimes to stop 'halfway' through
> a TB and re-translate just the first part of it, in order
> to cope with I/O. If you care about the details of that you can
> look at docs/devel/tcg-icount.rst.
>
> thanks
> -- PMM
>
Re: How does QEMU in TCG mode handle interrupts ?
On Tue, 25 Aug 2020 at 06:53, Arnabjyoti Kalita
wrote:
> This makes sense. In this scenario, when QEMU takes an interrupt at the end
> of a TB, I understand that the TB execution will not happen.
When QEMU takes an interrupt at the *end* of a TB, that TB
has already executed. (Really we take interrupts between
TBs, so at the end of one and the start of the next.)
To be a bit more precise, the way this works is that there's
a loop in cpu_exec():
while (!cpu_handle_exception(...)) {
while (!cpu_handle_interrupt(...)) {
tb = tb_find(...);
cpu_loop_exec_tb(...);
}
}
where cpu_loop_exec_tb() will start the TB, which might then
chain forward to the next TB without returning to this C loop.
So the first bit of generated code in each TB checks a flag
to see if there's a pending interrupt that means it should
just exit back to the C code loop without executing the TB
(we'll then call cpu_handle_interrupt() to process it).
> The interrupt will be taken and then the same TB will be
> re-translated again and later executed, right ?
We won't re-translate a TB just because it returned without
executing. We will likely re-execute it later after the guest
returns from the interrupt, but that depends on guest behaviour.
(There is no special casing for this: we just implement "return
from interrupt means we set the guest PC and other registers
as the guest CPU architecture specifies"; then when we find
the first TB after that, it will turn out to be the same TB
we abandoned execution of, if the guest made the return-from-interrupt
go back to the same place it left off. But if the guest did
something clever, eg OS context switch after interrupt handler
runs, then the guest PC will be something else and we'll execute
from there.)
> If so, does this methodology apply for all kinds of interrupts,
> hardware/software/faults/traps/vmexits etc. ?
You can divide these things into three categories:
* things which affect the emulated CPU and which are
asynchronously triggered by other parts of the system --
most notably, device interrupts. These are handled as I
describe above, with flags that tell the guest CPU to stop
executing at the next convenient point so it can deal with
the interrupt. QEMU calls all these things "interrupts"
in its generic code.
* "expected" exceptions: things like software traps where
a syscall instruction, for instance, will always cause an
exception. These we handle by having the generated code
for the instruction just be "raise an exception". (This is
generally 'write the current PC etc back to the CPU state
struct, call a C helper which sets some state about what
kind of exception this is and then calls cpu_loop_exit(),
which does a siglongjmp() out to cpu_exec() so it can do
the "cpu_handle_exception()" call immediately'.) We also
handle cases like "undefined instruction" and "tried to
exec an FPU instruction but the FPU is disabled" this way.
You can also deal with "usually happens" exceptions
like this too -- if a helper you call turns out not to
raise an exception, that's fine, it just means you updated
the PC unnecessarily, which is slightly inefficient but not
a big deal in most cases.
* "unexpected" exceptions: the primary example is a fault on
a data load/store insn. This is similar to the "expected"
case, except that we want to avoid the overhead of updating
the CPU state struct with the current PC because almost always
loads and stores succeed and don't take exceptions, and
loads and stores are really common so we care about avoiding
the extra insns that update the PC. So instead, when QEMU
detects that a load/store failed it will call cpu_restore_state().
This uses the *host* PC where the fault was detected (which
will be somewhere inside the generated code for the TB),
plus metadata that was generated and stored alongside the TB
when we first translated it, to identify "for this host PC,
what is the guest PC (plus any other per-target info we need)",
and then we can fix up the CPU state using that info. Once
the CPU state has been fixed up we call cpu_loop_exit() and
the rest is like the "expected" exception case.
In all this I have been ignoring the 'icount' feature, which
when enabled does cause us sometimes to stop 'halfway' through
a TB and re-translate just the first part of it, in order
to cope with I/O. If you care about the details of that you can
look at docs/devel/tcg-icount.rst.
thanks
-- PMM
Re: How does QEMU in TCG mode handle interrupts ?
Thank you Peter. This makes sense. In this scenario, when QEMU takes an interrupt at the end of a TB, I understand that the TB execution will not happen. The interrupt will be taken and then the same TB will be re-translated again and later executed, right ? If so, does this methodology apply for all kinds of interrupts, hardware/software/faults/traps/vmexits etc. ? Best Regards, Arnab On Sat, Aug 22, 2020 at 7:01 PM Peter Maydell wrote: > On Sat, 22 Aug 2020 at 09:42, Arnabjyoti Kalita > wrote: > > I am running QEMU-3.0.0 in TCG mode, and my QEMU as well as TCG target > is x86_64 architecture. > > > > What I am trying to do is inject an I/O interrupt in the middle of a > translation block. > > You can't. QEMU will only ever check for and take interrupts > at the end of a TB. The best you can do is that you can > use the '-singlestep' command line option to force QEMU > to put exactly 1 instruction in each TB. This will massively > slow down execution but it also will allow an interrupt > to occur anywhere. This is useful sometimes for debugging, > and also occasionally to run test suite type code that makes > dubious assumptions about exact CPU behaviour. "Real" guest > code should not require it. > > thanks > -- PMM >
Re: How does QEMU in TCG mode handle interrupts ?
On Sat, 22 Aug 2020 at 09:42, Arnabjyoti Kalita wrote: > I am running QEMU-3.0.0 in TCG mode, and my QEMU as well as TCG target is > x86_64 architecture. > > What I am trying to do is inject an I/O interrupt in the middle of a > translation block. You can't. QEMU will only ever check for and take interrupts at the end of a TB. The best you can do is that you can use the '-singlestep' command line option to force QEMU to put exactly 1 instruction in each TB. This will massively slow down execution but it also will allow an interrupt to occur anywhere. This is useful sometimes for debugging, and also occasionally to run test suite type code that makes dubious assumptions about exact CPU behaviour. "Real" guest code should not require it. thanks -- PMM
How does QEMU in TCG mode handle interrupts ?
Hello all, I am running QEMU-3.0.0 in TCG mode, and my QEMU as well as TCG target is x86_64 architecture. What I am trying to do is inject an I/O interrupt in the middle of a translation block. I have started TCG mode using the following debug flags *-d in_asm,cpu,exec,nochain* Let's say, I have this translation block -- IN: 0x810630a0: 0f 1f 44 00 00 nopl (%rax, %rax) 0x810630a5: 55 pushq%rbp 0x810630a6: 48 8b 05 6b 90 0a 01 movq 0x10a906b(%rip), %rax 0x810630ad: 48 89 e5 movq %rsp, %rbp 0x810630b0: 8b 80 f0 00 00 00 movl 0xf0(%rax), %eax 0x810630b6: 89 c0 movl %eax, %eax --- interrupt here 0x810630b8: 5d popq %rbp 0x810630b9: c3 retq And say, I have to make QEMU take an interrupt at the start of the instruction ending with "810630b6". How do I go about doing this? I have read that the user needs to asynchronously run a function to do this. But should we run the function during the translation phase ? Which function is it ? How can I "safely" inject an interrupt in the middle of a TB, without disturbing the execution of the block ? Regards, Arnab
