Re: [PATCH v6 15/24] arm64: Switch to PMR masking when starting CPUs
On 04/12/18 17:51, Catalin Marinas wrote: > On Mon, Nov 12, 2018 at 11:57:06AM +, Julien Thierry wrote: >> diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c >> index 8dc9dde..e495360 100644 >> --- a/arch/arm64/kernel/smp.c >> +++ b/arch/arm64/kernel/smp.c >> @@ -35,6 +35,7 @@ >> #include >> #include >> #include >> +#include >> #include >> #include >> #include >> @@ -175,6 +176,25 @@ int __cpu_up(unsigned int cpu, struct task_struct *idle) >> return ret; >> } >> >> +static void init_gic_priority_masking(void) >> +{ >> +u32 gic_sre = gic_read_sre(); >> +u32 cpuflags; >> + >> +if (WARN_ON(!(gic_sre & ICC_SRE_EL1_SRE))) >> +return; >> + >> +WARN_ON(!irqs_disabled()); >> + >> +gic_write_pmr(GIC_PRIO_IRQOFF); >> + >> +cpuflags = read_sysreg(daif); >> + >> +/* We can only unmask PSR.I if we can take aborts */ >> +if (!(cpuflags & PSR_A_BIT)) >> +write_sysreg(cpuflags & ~PSR_I_BIT, daif); > > I don't understand this. If you don't switch off PSR_I_BIT here, where > does it happen? In which scenario do we actually have the A bit still > set? At a quick look, smp_prepare_boot_cpu() would have the A bit > cleared previously by setup_arch(). We have secondary_start_kernel() > where you call init_gic_priority_masking() before local_daif_restore(). > So this is for secondary CPUs where PSR.A can be still set. The thing is that the daifflags.h establishes the order for disabling types of exceptions: Debug > Abort > IRQ The idea is that when introducing pseudo-NMIs this becomes: Debug > Abort > pseudo-NMI > IRQ Whenever aborts are disabled (maybe because we just took an abort) we don't want to take an NMI. > So what happens if you always turn off PSR_I_BIT here? > So semantically it would be saying "we can take a pseudo-NMI here". Realistically, I think it depends on the state of the GIC redistributor for this CPU: - If the re-distributor was initialized, nothing bad could happen as no NMI could have been configured for this CPU yet. - If the re-distributor initialization is done between the call to init_gic_priority_mask() and the local_daif_restore() then probably bad things could happen I can try to figure out if it is safe to just clear PSR.I always, but I also find it easier to always play by the rule "if PSR.A is set, PSR.I is set". Thanks, -- Julien Thierry
Re: [PATCH v6 15/24] arm64: Switch to PMR masking when starting CPUs
On 04/12/18 17:51, Catalin Marinas wrote: > On Mon, Nov 12, 2018 at 11:57:06AM +, Julien Thierry wrote: >> diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c >> index 8dc9dde..e495360 100644 >> --- a/arch/arm64/kernel/smp.c >> +++ b/arch/arm64/kernel/smp.c >> @@ -35,6 +35,7 @@ >> #include >> #include >> #include >> +#include >> #include >> #include >> #include >> @@ -175,6 +176,25 @@ int __cpu_up(unsigned int cpu, struct task_struct *idle) >> return ret; >> } >> >> +static void init_gic_priority_masking(void) >> +{ >> +u32 gic_sre = gic_read_sre(); >> +u32 cpuflags; >> + >> +if (WARN_ON(!(gic_sre & ICC_SRE_EL1_SRE))) >> +return; >> + >> +WARN_ON(!irqs_disabled()); >> + >> +gic_write_pmr(GIC_PRIO_IRQOFF); >> + >> +cpuflags = read_sysreg(daif); >> + >> +/* We can only unmask PSR.I if we can take aborts */ >> +if (!(cpuflags & PSR_A_BIT)) >> +write_sysreg(cpuflags & ~PSR_I_BIT, daif); > > I don't understand this. If you don't switch off PSR_I_BIT here, where > does it happen? In which scenario do we actually have the A bit still > set? At a quick look, smp_prepare_boot_cpu() would have the A bit > cleared previously by setup_arch(). We have secondary_start_kernel() > where you call init_gic_priority_masking() before local_daif_restore(). > So this is for secondary CPUs where PSR.A can be still set. The thing is that the daifflags.h establishes the order for disabling types of exceptions: Debug > Abort > IRQ The idea is that when introducing pseudo-NMIs this becomes: Debug > Abort > pseudo-NMI > IRQ Whenever aborts are disabled (maybe because we just took an abort) we don't want to take an NMI. > So what happens if you always turn off PSR_I_BIT here? > So semantically it would be saying "we can take a pseudo-NMI here". Realistically, I think it depends on the state of the GIC redistributor for this CPU: - If the re-distributor was initialized, nothing bad could happen as no NMI could have been configured for this CPU yet. - If the re-distributor initialization is done between the call to init_gic_priority_mask() and the local_daif_restore() then probably bad things could happen I can try to figure out if it is safe to just clear PSR.I always, but I also find it easier to always play by the rule "if PSR.A is set, PSR.I is set". Thanks, -- Julien Thierry
Re: [PATCH v6 15/24] arm64: Switch to PMR masking when starting CPUs
On Mon, Nov 12, 2018 at 11:57:06AM +, Julien Thierry wrote: > diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c > index 8dc9dde..e495360 100644 > --- a/arch/arm64/kernel/smp.c > +++ b/arch/arm64/kernel/smp.c > @@ -35,6 +35,7 @@ > #include > #include > #include > +#include > #include > #include > #include > @@ -175,6 +176,25 @@ int __cpu_up(unsigned int cpu, struct task_struct *idle) > return ret; > } > > +static void init_gic_priority_masking(void) > +{ > + u32 gic_sre = gic_read_sre(); > + u32 cpuflags; > + > + if (WARN_ON(!(gic_sre & ICC_SRE_EL1_SRE))) > + return; > + > + WARN_ON(!irqs_disabled()); > + > + gic_write_pmr(GIC_PRIO_IRQOFF); > + > + cpuflags = read_sysreg(daif); > + > + /* We can only unmask PSR.I if we can take aborts */ > + if (!(cpuflags & PSR_A_BIT)) > + write_sysreg(cpuflags & ~PSR_I_BIT, daif); I don't understand this. If you don't switch off PSR_I_BIT here, where does it happen? In which scenario do we actually have the A bit still set? At a quick look, smp_prepare_boot_cpu() would have the A bit cleared previously by setup_arch(). We have secondary_start_kernel() where you call init_gic_priority_masking() before local_daif_restore(). So what happens if you always turn off PSR_I_BIT here? -- Catalin
Re: [PATCH v6 15/24] arm64: Switch to PMR masking when starting CPUs
On Mon, Nov 12, 2018 at 11:57:06AM +, Julien Thierry wrote: > diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c > index 8dc9dde..e495360 100644 > --- a/arch/arm64/kernel/smp.c > +++ b/arch/arm64/kernel/smp.c > @@ -35,6 +35,7 @@ > #include > #include > #include > +#include > #include > #include > #include > @@ -175,6 +176,25 @@ int __cpu_up(unsigned int cpu, struct task_struct *idle) > return ret; > } > > +static void init_gic_priority_masking(void) > +{ > + u32 gic_sre = gic_read_sre(); > + u32 cpuflags; > + > + if (WARN_ON(!(gic_sre & ICC_SRE_EL1_SRE))) > + return; > + > + WARN_ON(!irqs_disabled()); > + > + gic_write_pmr(GIC_PRIO_IRQOFF); > + > + cpuflags = read_sysreg(daif); > + > + /* We can only unmask PSR.I if we can take aborts */ > + if (!(cpuflags & PSR_A_BIT)) > + write_sysreg(cpuflags & ~PSR_I_BIT, daif); I don't understand this. If you don't switch off PSR_I_BIT here, where does it happen? In which scenario do we actually have the A bit still set? At a quick look, smp_prepare_boot_cpu() would have the A bit cleared previously by setup_arch(). We have secondary_start_kernel() where you call init_gic_priority_masking() before local_daif_restore(). So what happens if you always turn off PSR_I_BIT here? -- Catalin