Hi Julien,

On Wed, Apr 11, 2018 at 6:02 PM, Julien Grall <julien.gr...@arm.com> wrote:

> Hi,
> On 11/04/18 16:58, Mirela Simonovic wrote:
>> On 04/11/2018 05:07 PM, Julien Grall wrote:
>>> On 11/04/18 14:19, Mirela Simonovic wrote:
>> Migrating interrupts when turning off a CPU already works. However, when
>> a CPU is turned back on there is no interrupt migration back to the
>> hotplugged CPU - all interrupts will remain routed to the CPU#0.
>> Patch 7/7 fixes this
> What do you mean by all interrupts? Interrupts routed to guest will always
> follow the vCPU. So are you sure they are going to be migrated when that
> vCPU is paused/off?
Just to make sure we're on the same page - this is about hotplugging
physical CPUs. Hotplugging vCPUs using virtual PSCI CPU_OFF interface is
already implemented and unrelated to this series.

Assuming that system has 2 pCPUs by 'all interrupts' I mean interrupts that
were targeted to the pCPU#0 and pCPU#1 prior to doing any hotplug.

For example, if a guest is pinned to pCPU#1 an interrupt of a device it
owns will be targeted to pCPU#1.
When pCPU#1 is turned off that interrupt will be migrated to pCPU#0. pCPU#0
finalizes the suspend and receives wake-up interrupts. However, when CPU#1
is turned back on that interrupt will remain targeted to the CPU#0, which I
assumed is wrong.
The scenario described here is also how I tested this.

> Can you give the path in Xen doing that?
Sure, here is a backtrace (dumped on the CPU being turned off):

    0  0x2603dc arch_move_irqs(): vgic.c, line 309
    1  0x22ee58 sched_move_irqs()+20: schedule.c, line 303
    2  0x2318e8 cpu_disable_scheduler()+1000: schedule.c, line 586
    3  0x2318e8 cpu_disable_scheduler()+1000: schedule.c, line 586
    4  0x25aff8 __cpu_disable()+96: smpboot.c, line 386
    5  0x201608 take_cpu_down()+52: cpu.c, line 75
    6  0x23426c stopmachine_action()+188: stop_machine.c, line 159
    7  0x235858 do_tasklet_work()+176: tasklet.c, line 94
    8  0x235c80 do_tasklet()+104: tasklet.c, line 126
    9  0x24daec idle_loop()+144: domain.c, line 72
   10  0x25b1f8 start_secondary()+404: smpboot.c, line 368

>>>> Calls to enable/disable_nonboot_cpus() functions currently don't exist
>>>> in
>>>> Xen ARM code. This will be added with the suspend to RAM support for
>>>> ARM.
>>> What would be the way to test that series?
>> I tested/developed the series on Xilinx Zynq UltraScale+ MPSoC (ZCU102
>> board), but how would others test that - I don't know. We will have to ask
>> Xilinx for detailed answer to this question.
> I am not sure to understand your answer. What I asked, is how did you did
> turned off the CPU? Did you hack Xen or do you rely on another series?
I rely on the suspend to RAM series that will be submitted after this one
is done. Not complete suspend to RAM support is needed in order to do the
Essentially, I have triggered suspend to RAM from Dom0 and the minimum in
Xen is to have the suspend support for secondary CPUs (calls to

> If the former, it would be nice to get the code you used. If the latter,
> then having a hack patch to test that code would be nice. Ideally, you want
> to plug that in the SYSCTL interface for out-of-box testing.
Ok, I have never used that but I'll try to figure it out. I may come up
with additional questions.


> Cheers,
> --
> Julien Grall
Xen-devel mailing list

Reply via email to