On 07/17/2017 06:16 AM, Nikunj A Dadhania wrote:
> Rebooting a SMP TCG guest is broken for both single/multi threaded TCG.
>
> When reset happens, all the CPUs are in halted state. First CPU is brought out
> of reset and secondary CPUs would be initialized by the guest kernel using a
> rtas call start-cpu.
>
> However, in case of TCG, decrementer interrupts keep on coming and waking the
> secondary CPUs up.
>
> These secondary CPUs would see the decrementer interrupt pending, which makes
> cpu::has_work() to bring them out of wait loop and start executing
> tcg_exec_cpu().
>
> The problem with this is all the CPUs wake up and start booting SLOF image,
> causing the following exception(4 CPUs TCG VM):
>
> [ 81.440850] reboot: Restarting system
>
> SLOF
> S
> SLOF
> SLOFLOF[0[0m
> **********************************************************************
> QEMU Starting
> Build Date = Mar 3 2017 13:29:19
> FW Version = git-66d250ef0fd06bb8
> [0m **********************************************************************
> QEMU Starting
> Build Date = Mar 3 2017 13:29:19
> FW Version = git-66d250ef0fd06bb8
> [0m *************************************m**********[?25l
> **********************************************************************
> QEMU Starting
> Build Date = Mar 3 2017 13:29:19
> FW Version = git-66d250ef0fd06bb8
> ***********************
> QEMU Starting
> Build Date = Mar 3 2017 13:29:19
> FW Version = git-66d250ef0fd06bb8
> ERROR: Flatten device tree not available!
>
> exception 300
> SRR0 = 00000000000060e4 SRR1 = 800000008000000000000000
> SPRG2 = 0000000000400000 SPRG3 = 0000000000004bd8
> ERROR: Flatten device tree not available!
>
> exception 300
> SRR0 = 00000000000060e4 SRR1 = 800000008000000000000000
> SPRG2 = 0000000000400000 SPRG3 = 0000000000004bd8
>
> During reset, disable decrementer interrupt for secondary CPUs and enable them
> when the secondary CPUs are brought online by rtas start-cpu call.
>
> Reported-by: Bharata B Rao <bhar...@linux.vnet.ibm.com>
> Signed-off-by: Nikunj A Dadhania <nik...@linux.vnet.ibm.com>
> ---
> hw/ppc/spapr_cpu_core.c | 9 +++++++++
> hw/ppc/spapr_rtas.c | 8 ++++++++
> 2 files changed, 17 insertions(+)
>
> diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> index ea278ce..bbfe8c2 100644
> --- a/hw/ppc/spapr_cpu_core.c
> +++ b/hw/ppc/spapr_cpu_core.c
> @@ -87,6 +87,15 @@ static void spapr_cpu_reset(void *opaque)
>
> env->spr[SPR_HIOR] = 0;
>
> + /* Disable DECR for secondary cpus */
> + if (cs != first_cpu) {
> + if (env->mmu_model == POWERPC_MMU_3_00) {
> + env->spr[SPR_LPCR] &= ~LPCR_DEE;
> + } else {
> + /* P7 and P8 both have same bit for DECR */
> + env->spr[SPR_LPCR] &= ~LPCR_P8_PECE3;
> + }
> + }
> /*
> * This is a hack for the benefit of KVM PR - it abuses the SDR1
> * slot in kvm_sregs to communicate the userspace address of the
> diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
> index 94a2799..4623d1d 100644
> --- a/hw/ppc/spapr_rtas.c
> +++ b/hw/ppc/spapr_rtas.c
> @@ -174,6 +174,14 @@ static void rtas_start_cpu(PowerPCCPU *cpu_,
> sPAPRMachineState *spapr,
> kvm_cpu_synchronize_state(cs);
>
> env->msr = (1ULL << MSR_SF) | (1ULL << MSR_ME);
> +
> + /* Enable DECR interrupt */
> + if (env->mmu_model == POWERPC_MMU_3_00) {
> + env->spr[SPR_LPCR] |= LPCR_DEE;
> + } else {
> + /* P7 and P8 both have same bit for DECR */
> + env->spr[SPR_LPCR] |= LPCR_P8_PECE3;
> + }
> env->nip = start;
> env->gpr[3] = r3;
> cs->halted = 0;
you should also disable the decrementer in the stop-cpu call
when a cpu is hot unplugged.
One thing I don't explain is why the decrementer interrupt
does not wake up the cpu after being hot unplugged. I am not
sure the code doing :
env->msr = 0;
is responsible of this behavior.
With the xive patchset, I am having issues in that area.
the decrementer wakes up the cpu just after hot unplugged
and I need to set the LPCR to disable it.
I wonder why XICS is immune to that.
C.
