Re: [Xen-devel] [PATCH v4 00/10] xen/arm64: Suspend preconditions and CPU hotplug fixes
On 28/05/18 11:15, Mirela Simonovic wrote: Hi, Could you please provide feedback for the remaining patches? The series is in my queue of patches to review but I was focus on getting XSA-263 (Speculation Store Bypass Disabled) ready for Arm. I will have a look at it now. Cheers, -- Julien Grall ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v4 00/10] xen/arm64: Suspend preconditions and CPU hotplug fixes
Hi, Could you please provide feedback for the remaining patches? Thanks, Mirela On Tue, May 15, 2018 at 1:44 PM, Mirela Simonovic < mirela.simono...@aggios.com> wrote: > This patch set contains fixes that are required as precondition for > suspend to > RAM support, including the CPU hotplug which is required to suspend > non-boot > CPUs. > The first two patches in this series: > 1) xen/arm64: Added handling of the trapped access to OSLSR register > 2) xen/arm: Ignore write to GICD_ISACTIVERn registers (vgic-v2) > are required to avoid Dom0 crashes when Dom0 performs its own suspend. This > patch set does not include the implementation of virtual PSCI system > suspend > call that would allow guests to finalize their suspend procedures. This > will > be submitted in the following series. > > Remaining of the patches are related to enabling CPU hotplug for non-boot > CPUs is resume scenario. CPU hotplug of non-boot CPUs will be used for > suspend > to RAM support for ARM. In suspend procedure, the hot-unplug of non-boot > CPUs > will be triggered with disable_nonboot_cpus(), while the hotplug is > triggered > with enable_nonboot_cpus(). Using these calls, the physical non-boot CPUs > could > be powered down/up on suspend/resume, respectively, if the underlying > firmware > allows so. Calls to enable/disable_nonboot_cpus() functions currently do > not > exist in Xen ARM code. This will be added with the suspend to RAM support > for > ARM. > > When non-boot pCPUs are hot-unplugged their interrupts are migrated to the > boot > pCPU. This series also includes a fix that would restore the interrupts > affinity > once non-boot pCPUs are hotplugged. Here only SPIs used by guests are > covered. > Migration of Xen internal SPIs is not covered. According to my > understanding > Xen internal SPIs are routed to the boot CPU which initializes the > respective > devices. Therefore, there is no need to migrate Xen internal SPIs. > > The code is tested on Xilinx Zynq UltraScale+ MPSoC/ZCU102 board (includes > physical power down/up of non-boot CPUs). The testing requires additional > patches for issuing system suspend. These patches and instructions for > testing > will be submitted later, when we get closer to the final version of the > series. > > --- > Changes in v2: > -Rename cover-letter title and emphasize that 2 patches from this series > are not > specific to CPU hotplug (my initial fault, splitting it now could be > confusing) > -Fix cover-letter explanations > -Address all the issues and comments as discussed on mailing list for v1 > -Add 3 patches to ensure that suspend/resume does not cause any memory > leaks. > All the memory allocated when a CPU was hotplugged is now freed when the > CPU is > hot-unplugged. > -Remove from the v1 series the patch which incorrectly dealt with an issue: > [PATCH 4/7] xen/arm: When CPU dies, free percpu area immediatelly > One solution to the issue addressed by the patch above is to add > rcu_barrier() > prior to calling enable_nonboot_cpus() during the suspend. This is how it > is > done in x86 suspend implementation. Until the discussion here > https://lists.xenproject.org/archives/html/xen-devel/2018-04/msg01199.html > doesn't conclude differently, I need to assume that adding rcu_barrier() > prior > to calling enable_nonboot_cpus() as it is done for x86 is the right way to > go. > Therefore, the fix to the issue will be part of the suspend to RAM series. > > Changes in v3: > -Add acked-by where needed > -Fix CPU_OFF PSCI implementation (physical interface) > -Use notifiers to implement freeing memory and releasing interrupts on CPU > hotplug > -Use notifier to trigger setup of virtual paging for non-boot CPUs on CPU > hotplug > -Add enabling errata workarounds on CPU hotplug, also based on a notifier > -Remove patch: > [PATCH v2 10/10] xen/arm: Call check_local_cpu_errata for secondary CPU > only on boot > > Changes in v4: > -Add acked-by/reviewed-by where needed > -Cleanup: use smp_processor_id() instead of get_processor_id(), fixed > indentation, add includes alphabetically, add newline before return, etc. > -Disable timers prior to releasing timer interrupts > -Initialize cpu_smpboot notifier at presmp_init rather than init phase > -In the last patch of the series errata notifier now returns an error > > --- > CC: Stefano Stabellini> CC: Julien Grall > CC: George Dunlap > CC: Dario Faggioli > --- > > Mirela Simonovic (10): > xen/arm64: Added handling of the trapped access to OSLSR register > xen/arm: Ignore write to GICD_ISACTIVERn registers (vgic-v2) > xen/arm: Implement CPU_OFF PSCI call (physical interface) > xen/arm: Remove __initdata and __init to enable CPU hotplug > xen/arm: Setup virtual paging for non-boot CPUs on hotplug/resume > xen/common: Restore IRQ affinity when hotplugging a pCPU > xen/arm: Release maintenance interrupt when CPU is
[Xen-devel] [PATCH v4 00/10] xen/arm64: Suspend preconditions and CPU hotplug fixes
This patch set contains fixes that are required as precondition for suspend to RAM support, including the CPU hotplug which is required to suspend non-boot CPUs. The first two patches in this series: 1) xen/arm64: Added handling of the trapped access to OSLSR register 2) xen/arm: Ignore write to GICD_ISACTIVERn registers (vgic-v2) are required to avoid Dom0 crashes when Dom0 performs its own suspend. This patch set does not include the implementation of virtual PSCI system suspend call that would allow guests to finalize their suspend procedures. This will be submitted in the following series. Remaining of the patches are related to enabling CPU hotplug for non-boot CPUs is resume scenario. CPU hotplug of non-boot CPUs will be used for suspend to RAM support for ARM. In suspend procedure, the hot-unplug of non-boot CPUs will be triggered with disable_nonboot_cpus(), while the hotplug is triggered with enable_nonboot_cpus(). Using these calls, the physical non-boot CPUs could be powered down/up on suspend/resume, respectively, if the underlying firmware allows so. Calls to enable/disable_nonboot_cpus() functions currently do not exist in Xen ARM code. This will be added with the suspend to RAM support for ARM. When non-boot pCPUs are hot-unplugged their interrupts are migrated to the boot pCPU. This series also includes a fix that would restore the interrupts affinity once non-boot pCPUs are hotplugged. Here only SPIs used by guests are covered. Migration of Xen internal SPIs is not covered. According to my understanding Xen internal SPIs are routed to the boot CPU which initializes the respective devices. Therefore, there is no need to migrate Xen internal SPIs. The code is tested on Xilinx Zynq UltraScale+ MPSoC/ZCU102 board (includes physical power down/up of non-boot CPUs). The testing requires additional patches for issuing system suspend. These patches and instructions for testing will be submitted later, when we get closer to the final version of the series. --- Changes in v2: -Rename cover-letter title and emphasize that 2 patches from this series are not specific to CPU hotplug (my initial fault, splitting it now could be confusing) -Fix cover-letter explanations -Address all the issues and comments as discussed on mailing list for v1 -Add 3 patches to ensure that suspend/resume does not cause any memory leaks. All the memory allocated when a CPU was hotplugged is now freed when the CPU is hot-unplugged. -Remove from the v1 series the patch which incorrectly dealt with an issue: [PATCH 4/7] xen/arm: When CPU dies, free percpu area immediatelly One solution to the issue addressed by the patch above is to add rcu_barrier() prior to calling enable_nonboot_cpus() during the suspend. This is how it is done in x86 suspend implementation. Until the discussion here https://lists.xenproject.org/archives/html/xen-devel/2018-04/msg01199.html doesn't conclude differently, I need to assume that adding rcu_barrier() prior to calling enable_nonboot_cpus() as it is done for x86 is the right way to go. Therefore, the fix to the issue will be part of the suspend to RAM series. Changes in v3: -Add acked-by where needed -Fix CPU_OFF PSCI implementation (physical interface) -Use notifiers to implement freeing memory and releasing interrupts on CPU hotplug -Use notifier to trigger setup of virtual paging for non-boot CPUs on CPU hotplug -Add enabling errata workarounds on CPU hotplug, also based on a notifier -Remove patch: [PATCH v2 10/10] xen/arm: Call check_local_cpu_errata for secondary CPU only on boot Changes in v4: -Add acked-by/reviewed-by where needed -Cleanup: use smp_processor_id() instead of get_processor_id(), fixed indentation, add includes alphabetically, add newline before return, etc. -Disable timers prior to releasing timer interrupts -Initialize cpu_smpboot notifier at presmp_init rather than init phase -In the last patch of the series errata notifier now returns an error --- CC: Stefano StabelliniCC: Julien Grall CC: George Dunlap CC: Dario Faggioli --- Mirela Simonovic (10): xen/arm64: Added handling of the trapped access to OSLSR register xen/arm: Ignore write to GICD_ISACTIVERn registers (vgic-v2) xen/arm: Implement CPU_OFF PSCI call (physical interface) xen/arm: Remove __initdata and __init to enable CPU hotplug xen/arm: Setup virtual paging for non-boot CPUs on hotplug/resume xen/common: Restore IRQ affinity when hotplugging a pCPU xen/arm: Release maintenance interrupt when CPU is hot-unplugged xen/arm: Disable timers and release their interrupts on CPU hot-unplug xen/arm: Free memory allocated for sibling/core maps on CPU hot-unplug xen/arm: Enable errata for secondary CPU on hotplug after the boot xen/arch/arm/arm64/smpboot.c | 2 +- xen/arch/arm/arm64/vsysreg.c | 3 ++- xen/arch/arm/cpuerrata.c | 49 +