Re: [PATCH] powernv: Restore SPRs correctly upon wake up from hypervisor state loss

[Shreyas B. Prabhu]

Hi Gautham,

Thanks for fixing this.

On Wed, Sep 7, 2016 at 1:16 AM, Gautham R. Shenoy
<e...@linux.vnet.ibm.com> wrote:
> From: "Gautham R. Shenoy" <e...@linux.vnet.ibm.com>
>
> pnv_wakeup_tb_loss function currently expects the cr4 to be "eq" if
> the CPU is waking up from a complete hypervisor state loss. Hence, it
> currently restores the SPR contents only if cr4 is "eq".
>
> However, after the commit bcef83a00dc4 ("powerpc/powernv: Add platform
> support for stop instruction"), on ISA_V300 CPUs, the function
> pnv_restore_hyp_resource sets cr4 to contain the result of the
> comparison between state the CPU has woken up and the first deepest
> stop state before calling pnv_wakeup_tb_loss.
>
> Thus if the CPU woke up from a state that is deeper than the first
> deepest stop state, cr4 have "gt" set and hence, pnv_wakeup_tb_loss
> will fail to restore the SPRs on waking up from such a state.
>
> Fix the code in pnv_wakeup_tb_loss to restore the SPR states when cr4 is
> "eq" or "gt".
>
> Fixes: Commit bcef83a00dc4 ("powerpc/powernv: Add platform support for
> stop instruction")
>
> Cc: Vaidyanathan Srinivasan <sva...@linux.vnet.ibm.com>
> Cc: Michael Neuling <michael.neul...@au1.ibm.com>
> Cc: Michael Ellerman <m...@ellerman.id.au>
> Cc: Shreyas B. Prabhu <shreya...@gmail.com>
> Signed-off-by: Gautham R. Shenoy <e...@linux.vnet.ibm.com>
> ---

Reviewed-by: Shreyas B. Prabhu <shreya...@gmail.com>


Thanks,
Shreyas

Re: [PATCH] powernv: Restore SPRs correctly upon wake up from hypervisor state loss

[Shreyas B. Prabhu]

Hi Gautham,

Thanks for fixing this.

On Wed, Sep 7, 2016 at 1:16 AM, Gautham R. Shenoy
 wrote:
> From: "Gautham R. Shenoy" 
>
> pnv_wakeup_tb_loss function currently expects the cr4 to be "eq" if
> the CPU is waking up from a complete hypervisor state loss. Hence, it
> currently restores the SPR contents only if cr4 is "eq".
>
> However, after the commit bcef83a00dc4 ("powerpc/powernv: Add platform
> support for stop instruction"), on ISA_V300 CPUs, the function
> pnv_restore_hyp_resource sets cr4 to contain the result of the
> comparison between state the CPU has woken up and the first deepest
> stop state before calling pnv_wakeup_tb_loss.
>
> Thus if the CPU woke up from a state that is deeper than the first
> deepest stop state, cr4 have "gt" set and hence, pnv_wakeup_tb_loss
> will fail to restore the SPRs on waking up from such a state.
>
> Fix the code in pnv_wakeup_tb_loss to restore the SPR states when cr4 is
> "eq" or "gt".
>
> Fixes: Commit bcef83a00dc4 ("powerpc/powernv: Add platform support for
> stop instruction")
>
> Cc: Vaidyanathan Srinivasan 
> Cc: Michael Neuling 
> Cc: Michael Ellerman 
> Cc: Shreyas B. Prabhu 
> Signed-off-by: Gautham R. Shenoy 
> ---

Reviewed-by: Shreyas B. Prabhu 


Thanks,
Shreyas

[PATCH v8 09/11] cpuidle/powernv: cleanup cpuidle-powernv.c

[Shreyas B. Prabhu]

 - Use stack instead of kzalloc'ed memory for variables while probing
   device tree for idle states.
 - Set cap for number of idle states that can be added to
   cpuidle_state_table
 - Minor change in way we check of_property_read_u32_array for error
   for sake of consistency
 - Drop unnecessary "&" while assigning function pointer

Cc: Rafael J. Wysocki <rafael.j.wyso...@intel.com>
Cc: Daniel Lezcano <daniel.lezc...@linaro.org>
Cc: linux...@vger.kernel.org
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
Changes in v8
=
 - _loop -> snooze_loop

Changes in v7
=
 - New in v7. This was mainly to make the existing code
   consistent with the review comments for new code

 drivers/cpuidle/cpuidle-powernv.c | 38 --
 1 file changed, 20 insertions(+), 18 deletions(-)

diff --git a/drivers/cpuidle/cpuidle-powernv.c 
b/drivers/cpuidle/cpuidle-powernv.c
index 3a763a8..600bbe1 100644
--- a/drivers/cpuidle/cpuidle-powernv.c
+++ b/drivers/cpuidle/cpuidle-powernv.c
@@ -100,7 +100,7 @@ static struct cpuidle_state 
powernv_states[CPUIDLE_STATE_MAX] = {
.desc = "snooze",
.exit_latency = 0,
.target_residency = 0,
-   .enter = _loop },
+   .enter = snooze_loop },
 };
 
 static int powernv_cpuidle_add_cpu_notifier(struct notifier_block *n,
@@ -166,7 +166,9 @@ static int powernv_add_idle_states(void)
struct device_node *power_mgt;
int nr_idle_states = 1; /* Snooze */
int dt_idle_states;
-   u32 *latency_ns, *residency_ns, *flags;
+   u32 latency_ns[CPUIDLE_STATE_MAX];
+   u32 residency_ns[CPUIDLE_STATE_MAX];
+   u32 flags[CPUIDLE_STATE_MAX];
int i, rc;
 
/* Currently we have snooze statically defined */
@@ -184,22 +186,28 @@ static int powernv_add_idle_states(void)
goto out;
}
 
-   flags = kzalloc(sizeof(*flags) * dt_idle_states, GFP_KERNEL);
+   /*
+* Since snooze is used as first idle state, max idle states allowed is
+* CPUIDLE_STATE_MAX -1
+*/
+   if (dt_idle_states > CPUIDLE_STATE_MAX - 1) {
+   pr_warn("cpuidle-powernv: discovered idle states more than 
allowed");
+   dt_idle_states = CPUIDLE_STATE_MAX - 1;
+   }
+
if (of_property_read_u32_array(power_mgt,
"ibm,cpu-idle-state-flags", flags, dt_idle_states)) {
pr_warn("cpuidle-powernv : missing ibm,cpu-idle-state-flags in 
DT\n");
-   goto out_free_flags;
+   goto out;
}
 
-   latency_ns = kzalloc(sizeof(*latency_ns) * dt_idle_states, GFP_KERNEL);
-   rc = of_property_read_u32_array(power_mgt,
-   "ibm,cpu-idle-state-latencies-ns", latency_ns, dt_idle_states);
-   if (rc) {
+   if (of_property_read_u32_array(power_mgt,
+   "ibm,cpu-idle-state-latencies-ns", latency_ns,
+   dt_idle_states)) {
pr_warn("cpuidle-powernv: missing 
ibm,cpu-idle-state-latencies-ns in DT\n");
-   goto out_free_latency;
+   goto out;
}
 
-   residency_ns = kzalloc(sizeof(*residency_ns) * dt_idle_states, 
GFP_KERNEL);
rc = of_property_read_u32_array(power_mgt,
"ibm,cpu-idle-state-residency-ns", residency_ns, 
dt_idle_states);
 
@@ -215,7 +223,7 @@ static int powernv_add_idle_states(void)
strcpy(powernv_states[nr_idle_states].desc, "Nap");
powernv_states[nr_idle_states].flags = 0;
powernv_states[nr_idle_states].target_residency = 100;
-   powernv_states[nr_idle_states].enter = _loop;
+   powernv_states[nr_idle_states].enter = nap_loop;
}
 
/*
@@ -230,7 +238,7 @@ static int powernv_add_idle_states(void)
strcpy(powernv_states[nr_idle_states].desc, 
"FastSleep");
powernv_states[nr_idle_states].flags = 
CPUIDLE_FLAG_TIMER_STOP;
powernv_states[nr_idle_states].target_residency = 
30;
-   powernv_states[nr_idle_states].enter = _loop;
+   powernv_states[nr_idle_states].enter = fastsleep_loop;
}
 #endif
powernv_states[nr_idle_states].exit_latency =
@@ -243,12 +251,6 @@ static int powernv_add_idle_states(void)
 
nr_idle_states++;
}
-
-   kfree(residency_ns);
-out_free_latency:
-   kfree(latency_ns);
-out_free_flags:
-   kfree(flags);
 out:
return nr_idle_states;
 }
-- 
2.4.11

[PATCH v8 07/11] powerpc/powernv: Add platform support for stop instruction

[Shreyas B. Prabhu]

POWER ISA v3 defines a new idle processor core mechanism. In summary,
 a) new instruction named stop is added. This instruction replaces
instructions like nap, sleep, rvwinkle.
 b) new per thread SPR named Processor Stop Status and Control Register
(PSSCR) is added which controls the behavior of stop instruction.

PSSCR layout:
--
| PLS | /// | SD | ESL | EC | PSLL | /// | TR | MTL | RL |
--
0  4 41   4243   44 4854   5660

PSSCR key fields:
Bits 0:3  - Power-Saving Level Status. This field indicates the lowest
power-saving state the thread entered since stop instruction was last
executed.

Bit 42 - Enable State Loss
0 - No state is lost irrespective of other fields
1 - Allows state loss

Bits 44:47 - Power-Saving Level Limit
This limits the power-saving level that can be entered into.

Bits 60:63 - Requested Level
Used to specify which power-saving level must be entered on executing
stop instruction

This patch adds support for stop instruction and PSSCR handling.

Reviewed-by: Gautham R. Shenoy <e...@linux.vnet.ibm.com>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
Changes in v8
=
 - Initializing pnv_first_deep_stop_state
 - Changing MMU_FTR_SECTION condition to reduce FTR section length

Changes in v7
=
 - LMRR, LMSER and ADSR not restored since its not necessary
 - power_stop0, power_stop renamed to power9_idle and power_idle_stop
 - PSSCR template is now a macro instead of storing in paca
 - power9_idle in C file instead of assembly
 - Fixed TOC related bug
 - Handling subcore within FTR section
 - Functions in idle.c reordered and broken into multiple functions
 - calling __restore_cpu_power8/9 via cur_cpu_spec->cpu_restore 
 - Restoring RPR once per core in P9

Changes in v6
=
 - Save/restore new P9 SPRs when using deep idle states

Changes in v4:
==
 - Added PSSCR layout to commit message
 - Improved / Fixed comments
 - Fixed whitespace error in paca.h
 - Using MAX_POSSIBLE_STOP_STATE macro instead of hardcoding 0xF as 
   max possible stop state

Changes in v3:
==
 - Instead of introducing new file idle_power_stop.S, P9 idle support
   is added to idle_power_common.S using CPU_FTR sections.
 - Fixed r4 reg clobbering in power_stop0
 - Improved comments

Changes in v2:
==
 - Using CPU_FTR_ARCH_300 bit instead of CPU_FTR_STOP_INST

 arch/powerpc/include/asm/cpuidle.h|   2 +
 arch/powerpc/include/asm/kvm_book3s_asm.h |   2 +-
 arch/powerpc/include/asm/opal-api.h   |  11 +-
 arch/powerpc/include/asm/ppc-opcode.h |   4 +
 arch/powerpc/include/asm/processor.h  |   2 +
 arch/powerpc/include/asm/reg.h|  10 ++
 arch/powerpc/kernel/idle_book3s.S | 193 --
 arch/powerpc/platforms/powernv/idle.c | 174 ++-
 8 files changed, 332 insertions(+), 66 deletions(-)

diff --git a/arch/powerpc/include/asm/cpuidle.h 
b/arch/powerpc/include/asm/cpuidle.h
index d2f99ca..3d7fc06 100644
--- a/arch/powerpc/include/asm/cpuidle.h
+++ b/arch/powerpc/include/asm/cpuidle.h
@@ -13,6 +13,8 @@
 #ifndef __ASSEMBLY__
 extern u32 pnv_fastsleep_workaround_at_entry[];
 extern u32 pnv_fastsleep_workaround_at_exit[];
+
+extern u64 pnv_first_deep_stop_state;
 #endif
 
 #endif
diff --git a/arch/powerpc/include/asm/kvm_book3s_asm.h 
b/arch/powerpc/include/asm/kvm_book3s_asm.h
index 72b6225..d318d43 100644
--- a/arch/powerpc/include/asm/kvm_book3s_asm.h
+++ b/arch/powerpc/include/asm/kvm_book3s_asm.h
@@ -162,7 +162,7 @@ struct kvmppc_book3s_shadow_vcpu {
 
 /* Values for kvm_state */
 #define KVM_HWTHREAD_IN_KERNEL 0
-#define KVM_HWTHREAD_IN_NAP1
+#define KVM_HWTHREAD_IN_IDLE   1
 #define KVM_HWTHREAD_IN_KVM2
 
 #endif /* __ASM_KVM_BOOK3S_ASM_H__ */
diff --git a/arch/powerpc/include/asm/opal-api.h 
b/arch/powerpc/include/asm/opal-api.h
index 72b5f27..6de1e4e 100644
--- a/arch/powerpc/include/asm/opal-api.h
+++ b/arch/powerpc/include/asm/opal-api.h
@@ -166,13 +166,20 @@
 
 /* Device tree flags */
 
-/* Flags set in power-mgmt nodes in device tree if
- * respective idle states are supported in the platform.
+/*
+ * Flags set in power-mgmt nodes in device tree describing
+ * idle states that are supported in the platform.
  */
+
+#define OPAL_PM_TIMEBASE_STOP  0x0002
+#define OPAL_PM_LOSE_HYP_CONTEXT   0x2000
+#define OPAL_PM_LOSE_FULL_CONTEXT  0x4000
 #define OPAL_PM_NAP_ENABLED0x0001
 #define OPAL_PM_SLEEP_ENABLED  0x0002
 #define OPAL_PM_WINKLE_ENABLED 0x0004
 #define OPAL_PM_SLEEP_ENABLED_ER1  0x0008 /* with workaround */
+#define OPAL_PM_STOP_INST_FAST 0x0010
+#define OPAL_PM_STOP_INST_DEEP

[PATCH v8 09/11] cpuidle/powernv: cleanup cpuidle-powernv.c

[Shreyas B. Prabhu]

 - Use stack instead of kzalloc'ed memory for variables while probing
   device tree for idle states.
 - Set cap for number of idle states that can be added to
   cpuidle_state_table
 - Minor change in way we check of_property_read_u32_array for error
   for sake of consistency
 - Drop unnecessary "&" while assigning function pointer

Cc: Rafael J. Wysocki 
Cc: Daniel Lezcano 
Cc: linux...@vger.kernel.org
Signed-off-by: Shreyas B. Prabhu 
---
Changes in v8
=
 - _loop -> snooze_loop

Changes in v7
=
 - New in v7. This was mainly to make the existing code
   consistent with the review comments for new code

 drivers/cpuidle/cpuidle-powernv.c | 38 --
 1 file changed, 20 insertions(+), 18 deletions(-)

diff --git a/drivers/cpuidle/cpuidle-powernv.c 
b/drivers/cpuidle/cpuidle-powernv.c
index 3a763a8..600bbe1 100644
--- a/drivers/cpuidle/cpuidle-powernv.c
+++ b/drivers/cpuidle/cpuidle-powernv.c
@@ -100,7 +100,7 @@ static struct cpuidle_state 
powernv_states[CPUIDLE_STATE_MAX] = {
.desc = "snooze",
.exit_latency = 0,
.target_residency = 0,
-   .enter = _loop },
+   .enter = snooze_loop },
 };
 
 static int powernv_cpuidle_add_cpu_notifier(struct notifier_block *n,
@@ -166,7 +166,9 @@ static int powernv_add_idle_states(void)
struct device_node *power_mgt;
int nr_idle_states = 1; /* Snooze */
int dt_idle_states;
-   u32 *latency_ns, *residency_ns, *flags;
+   u32 latency_ns[CPUIDLE_STATE_MAX];
+   u32 residency_ns[CPUIDLE_STATE_MAX];
+   u32 flags[CPUIDLE_STATE_MAX];
int i, rc;
 
/* Currently we have snooze statically defined */
@@ -184,22 +186,28 @@ static int powernv_add_idle_states(void)
goto out;
}
 
-   flags = kzalloc(sizeof(*flags) * dt_idle_states, GFP_KERNEL);
+   /*
+* Since snooze is used as first idle state, max idle states allowed is
+* CPUIDLE_STATE_MAX -1
+*/
+   if (dt_idle_states > CPUIDLE_STATE_MAX - 1) {
+   pr_warn("cpuidle-powernv: discovered idle states more than 
allowed");
+   dt_idle_states = CPUIDLE_STATE_MAX - 1;
+   }
+
if (of_property_read_u32_array(power_mgt,
"ibm,cpu-idle-state-flags", flags, dt_idle_states)) {
pr_warn("cpuidle-powernv : missing ibm,cpu-idle-state-flags in 
DT\n");
-   goto out_free_flags;
+   goto out;
}
 
-   latency_ns = kzalloc(sizeof(*latency_ns) * dt_idle_states, GFP_KERNEL);
-   rc = of_property_read_u32_array(power_mgt,
-   "ibm,cpu-idle-state-latencies-ns", latency_ns, dt_idle_states);
-   if (rc) {
+   if (of_property_read_u32_array(power_mgt,
+   "ibm,cpu-idle-state-latencies-ns", latency_ns,
+   dt_idle_states)) {
pr_warn("cpuidle-powernv: missing 
ibm,cpu-idle-state-latencies-ns in DT\n");
-   goto out_free_latency;
+   goto out;
}
 
-   residency_ns = kzalloc(sizeof(*residency_ns) * dt_idle_states, 
GFP_KERNEL);
rc = of_property_read_u32_array(power_mgt,
"ibm,cpu-idle-state-residency-ns", residency_ns, 
dt_idle_states);
 
@@ -215,7 +223,7 @@ static int powernv_add_idle_states(void)
strcpy(powernv_states[nr_idle_states].desc, "Nap");
powernv_states[nr_idle_states].flags = 0;
powernv_states[nr_idle_states].target_residency = 100;
-   powernv_states[nr_idle_states].enter = _loop;
+   powernv_states[nr_idle_states].enter = nap_loop;
}
 
/*
@@ -230,7 +238,7 @@ static int powernv_add_idle_states(void)
strcpy(powernv_states[nr_idle_states].desc, 
"FastSleep");
powernv_states[nr_idle_states].flags = 
CPUIDLE_FLAG_TIMER_STOP;
powernv_states[nr_idle_states].target_residency = 
30;
-   powernv_states[nr_idle_states].enter = _loop;
+   powernv_states[nr_idle_states].enter = fastsleep_loop;
}
 #endif
powernv_states[nr_idle_states].exit_latency =
@@ -243,12 +251,6 @@ static int powernv_add_idle_states(void)
 
nr_idle_states++;
}
-
-   kfree(residency_ns);
-out_free_latency:
-   kfree(latency_ns);
-out_free_flags:
-   kfree(flags);
 out:
return nr_idle_states;
 }
-- 
2.4.11

[PATCH v8 07/11] powerpc/powernv: Add platform support for stop instruction

[Shreyas B. Prabhu]

POWER ISA v3 defines a new idle processor core mechanism. In summary,
 a) new instruction named stop is added. This instruction replaces
instructions like nap, sleep, rvwinkle.
 b) new per thread SPR named Processor Stop Status and Control Register
(PSSCR) is added which controls the behavior of stop instruction.

PSSCR layout:
--
| PLS | /// | SD | ESL | EC | PSLL | /// | TR | MTL | RL |
--
0  4 41   4243   44 4854   5660

PSSCR key fields:
Bits 0:3  - Power-Saving Level Status. This field indicates the lowest
power-saving state the thread entered since stop instruction was last
executed.

Bit 42 - Enable State Loss
0 - No state is lost irrespective of other fields
1 - Allows state loss

Bits 44:47 - Power-Saving Level Limit
This limits the power-saving level that can be entered into.

Bits 60:63 - Requested Level
Used to specify which power-saving level must be entered on executing
stop instruction

This patch adds support for stop instruction and PSSCR handling.

Reviewed-by: Gautham R. Shenoy 
Signed-off-by: Shreyas B. Prabhu 
---
Changes in v8
=
 - Initializing pnv_first_deep_stop_state
 - Changing MMU_FTR_SECTION condition to reduce FTR section length

Changes in v7
=
 - LMRR, LMSER and ADSR not restored since its not necessary
 - power_stop0, power_stop renamed to power9_idle and power_idle_stop
 - PSSCR template is now a macro instead of storing in paca
 - power9_idle in C file instead of assembly
 - Fixed TOC related bug
 - Handling subcore within FTR section
 - Functions in idle.c reordered and broken into multiple functions
 - calling __restore_cpu_power8/9 via cur_cpu_spec->cpu_restore 
 - Restoring RPR once per core in P9

Changes in v6
=
 - Save/restore new P9 SPRs when using deep idle states

Changes in v4:
==
 - Added PSSCR layout to commit message
 - Improved / Fixed comments
 - Fixed whitespace error in paca.h
 - Using MAX_POSSIBLE_STOP_STATE macro instead of hardcoding 0xF as 
   max possible stop state

Changes in v3:
==
 - Instead of introducing new file idle_power_stop.S, P9 idle support
   is added to idle_power_common.S using CPU_FTR sections.
 - Fixed r4 reg clobbering in power_stop0
 - Improved comments

Changes in v2:
==
 - Using CPU_FTR_ARCH_300 bit instead of CPU_FTR_STOP_INST

 arch/powerpc/include/asm/cpuidle.h|   2 +
 arch/powerpc/include/asm/kvm_book3s_asm.h |   2 +-
 arch/powerpc/include/asm/opal-api.h   |  11 +-
 arch/powerpc/include/asm/ppc-opcode.h |   4 +
 arch/powerpc/include/asm/processor.h  |   2 +
 arch/powerpc/include/asm/reg.h|  10 ++
 arch/powerpc/kernel/idle_book3s.S | 193 --
 arch/powerpc/platforms/powernv/idle.c | 174 ++-
 8 files changed, 332 insertions(+), 66 deletions(-)

diff --git a/arch/powerpc/include/asm/cpuidle.h 
b/arch/powerpc/include/asm/cpuidle.h
index d2f99ca..3d7fc06 100644
--- a/arch/powerpc/include/asm/cpuidle.h
+++ b/arch/powerpc/include/asm/cpuidle.h
@@ -13,6 +13,8 @@
 #ifndef __ASSEMBLY__
 extern u32 pnv_fastsleep_workaround_at_entry[];
 extern u32 pnv_fastsleep_workaround_at_exit[];
+
+extern u64 pnv_first_deep_stop_state;
 #endif
 
 #endif
diff --git a/arch/powerpc/include/asm/kvm_book3s_asm.h 
b/arch/powerpc/include/asm/kvm_book3s_asm.h
index 72b6225..d318d43 100644
--- a/arch/powerpc/include/asm/kvm_book3s_asm.h
+++ b/arch/powerpc/include/asm/kvm_book3s_asm.h
@@ -162,7 +162,7 @@ struct kvmppc_book3s_shadow_vcpu {
 
 /* Values for kvm_state */
 #define KVM_HWTHREAD_IN_KERNEL 0
-#define KVM_HWTHREAD_IN_NAP1
+#define KVM_HWTHREAD_IN_IDLE   1
 #define KVM_HWTHREAD_IN_KVM2
 
 #endif /* __ASM_KVM_BOOK3S_ASM_H__ */
diff --git a/arch/powerpc/include/asm/opal-api.h 
b/arch/powerpc/include/asm/opal-api.h
index 72b5f27..6de1e4e 100644
--- a/arch/powerpc/include/asm/opal-api.h
+++ b/arch/powerpc/include/asm/opal-api.h
@@ -166,13 +166,20 @@
 
 /* Device tree flags */
 
-/* Flags set in power-mgmt nodes in device tree if
- * respective idle states are supported in the platform.
+/*
+ * Flags set in power-mgmt nodes in device tree describing
+ * idle states that are supported in the platform.
  */
+
+#define OPAL_PM_TIMEBASE_STOP  0x0002
+#define OPAL_PM_LOSE_HYP_CONTEXT   0x2000
+#define OPAL_PM_LOSE_FULL_CONTEXT  0x4000
 #define OPAL_PM_NAP_ENABLED0x0001
 #define OPAL_PM_SLEEP_ENABLED  0x0002
 #define OPAL_PM_WINKLE_ENABLED 0x0004
 #define OPAL_PM_SLEEP_ENABLED_ER1  0x0008 /* with workaround */
+#define OPAL_PM_STOP_INST_FAST 0x0010
+#define OPAL_PM_STOP_INST_DEEP 0x0020
 
 /*
  * OPAL_CONFIG_CPU_IDLE_STATE parameters
diff --git a/a

[PATCH v8 03/11] powerpc/powernv: Rename idle_power7.S to idle_book3s.S

[Shreyas B. Prabhu]

idle_power7.S handles idle entry/exit for POWER7, POWER8 and in next
patch for POWER9. Rename the file to a non-hardware specific
name.

Reviewed-by: Gautham R. Shenoy <e...@linux.vnet.ibm.com>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---

Changes in v7:
=
 - File renamed to idle_book3s.S instead of idle_power_common.S

Changes in v3:
==
 - Instead of moving few common functions from idle_power7.S to
   idle_power_common.S, renaming idle_power7.S to idle_power_common.S

 arch/powerpc/kernel/Makefile  |   2 +-
 arch/powerpc/kernel/idle_book3s.S | 527 ++
 arch/powerpc/kernel/idle_power7.S | 527 --
 3 files changed, 528 insertions(+), 528 deletions(-)
 create mode 100644 arch/powerpc/kernel/idle_book3s.S
 delete mode 100644 arch/powerpc/kernel/idle_power7.S

diff --git a/arch/powerpc/kernel/Makefile b/arch/powerpc/kernel/Makefile
index 2da380f..9e7bfc32 100644
--- a/arch/powerpc/kernel/Makefile
+++ b/arch/powerpc/kernel/Makefile
@@ -47,7 +47,7 @@ obj-$(CONFIG_PPC_BOOK3E_64)   += exceptions-64e.o 
idle_book3e.o
 obj-$(CONFIG_PPC64)+= vdso64/
 obj-$(CONFIG_ALTIVEC)  += vecemu.o
 obj-$(CONFIG_PPC_970_NAP)  += idle_power4.o
-obj-$(CONFIG_PPC_P7_NAP)   += idle_power7.o
+obj-$(CONFIG_PPC_P7_NAP)   += idle_book3s.o
 procfs-y   := proc_powerpc.o
 obj-$(CONFIG_PROC_FS)  += $(procfs-y)
 rtaspci-$(CONFIG_PPC64)-$(CONFIG_PCI)  := rtas_pci.o
diff --git a/arch/powerpc/kernel/idle_book3s.S 
b/arch/powerpc/kernel/idle_book3s.S
new file mode 100644
index 000..d5def06
--- /dev/null
+++ b/arch/powerpc/kernel/idle_book3s.S
@@ -0,0 +1,527 @@
+/*
+ *  This file contains the power_save function for Power7 CPUs.
+ *
+ *  This program is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU General Public License
+ *  as published by the Free Software Foundation; either version
+ *  2 of the License, or (at your option) any later version.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#undef DEBUG
+
+/*
+ * Use unused space in the interrupt stack to save and restore
+ * registers for winkle support.
+ */
+#define _SDR1  GPR3
+#define _RPR   GPR4
+#define _SPURR GPR5
+#define _PURR  GPR6
+#define _TSCR  GPR7
+#define _DSCR  GPR8
+#define _AMOR  GPR9
+#define _WORT  GPR10
+#define _WORC  GPR11
+
+/* Idle state entry routines */
+
+#defineIDLE_STATE_ENTER_SEQ(IDLE_INST) \
+   /* Magic NAP/SLEEP/WINKLE mode enter sequence */\
+   std r0,0(r1);   \
+   ptesync;\
+   ld  r0,0(r1);   \
+1: cmp cr0,r0,r0;  \
+   bne 1b; \
+   IDLE_INST;  \
+   b   .
+
+   .text
+
+/*
+ * Used by threads when the lock bit of core_idle_state is set.
+ * Threads will spin in HMT_LOW until the lock bit is cleared.
+ * r14 - pointer to core_idle_state
+ * r15 - used to load contents of core_idle_state
+ */
+
+core_idle_lock_held:
+   HMT_LOW
+3: lwz r15,0(r14)
+   andi.   r15,r15,PNV_CORE_IDLE_LOCK_BIT
+   bne 3b
+   HMT_MEDIUM
+   lwarx   r15,0,r14
+   blr
+
+/*
+ * Pass requested state in r3:
+ * r3 - PNV_THREAD_NAP/SLEEP/WINKLE
+ *
+ * To check IRQ_HAPPENED in r4
+ * 0 - don't check
+ * 1 - check
+ */
+_GLOBAL(power7_powersave_common)
+   /* Use r3 to pass state nap/sleep/winkle */
+   /* NAP is a state loss, we create a regs frame on the
+* stack, fill it up with the state we care about and
+* stick a pointer to it in PACAR1. We really only
+* need to save PC, some CR bits and the NV GPRs,
+* but for now an interrupt frame will do.
+*/
+   mflrr0
+   std r0,16(r1)
+   stdur1,-INT_FRAME_SIZE(r1)
+   std r0,_LINK(r1)
+   std r0,_NIP(r1)
+
+   /* Hard disable interrupts */
+   mfmsr   r9
+   rldicl  r9,r9,48,1
+   rotldi  r9,r9,16
+   mtmsrd  r9,1/* hard-disable interrupts */
+
+   /* Check if something happened while soft-disabled */
+   lbz r0,PACAIRQHAPPENED(r13)
+   andi.   r0,r0,~PACA_IRQ_HARD_DIS@l
+   beq 1f
+   cmpwi   cr0,r4,0
+   beq 1f
+   addir1,r1,INT_FRAME_SIZE
+   ld  r0,16(r1)
+   li  r3,0/* Return 0 (no nap) */
+   mtlrr0
+   blr
+
+1: /* We mark irqs hard disabled as this is the state we'll
+* be in when returning and we need to tell arch_local_irq_restore()
+* about it
+*/
+ 

[PATCH v8 04/11] powerpc/powernv: Rename reusable idle functions to hardware agnostic names

[Shreyas B. Prabhu]

Functions like power7_wakeup_loss, power7_wakeup_noloss,
power7_wakeup_tb_loss are used by POWER7 and POWER8 hardware. They can
also be used by POWER9. Hence rename these functions hardware agnostic
names.

Suggested-by: Gautham R. Shenoy <e...@linux.vnet.ibm.com>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
 - No changes since v4

Changes in v4:
==
 - renaming power7_powersave_common to pnv_powersave_common
 - renaming power7_enter_nap_mode to pnv_enter_arch207_idle_mode

 arch/powerpc/kernel/exceptions-64s.S|  8 
 arch/powerpc/kernel/idle_book3s.S   | 33 +
 arch/powerpc/kvm/book3s_hv_rmhandlers.S |  4 ++--
 3 files changed, 23 insertions(+), 22 deletions(-)

diff --git a/arch/powerpc/kernel/exceptions-64s.S 
b/arch/powerpc/kernel/exceptions-64s.S
index 4a74d6a..2a123cd 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -108,7 +108,7 @@ BEGIN_FTR_SECTION
 
cmpwi   cr3,r13,2
GET_PACA(r13)
-   bl  power7_restore_hyp_resource
+   bl  pnv_restore_hyp_resource
 
li  r0,PNV_THREAD_RUNNING
stb r0,PACA_THREAD_IDLE_STATE(r13)  /* Clear thread state */
@@ -128,8 +128,8 @@ BEGIN_FTR_SECTION
/* Return SRR1 from power7_nap() */
mfspr   r3,SPRN_SRR1
blt cr3,2f
-   b   power7_wakeup_loss
-2: b   power7_wakeup_noloss
+   b   pnv_wakeup_loss
+2: b   pnv_wakeup_noloss
 
 9:
 END_FTR_SECTION_IFSET(CPU_FTR_HVMODE | CPU_FTR_ARCH_206)
@@ -1269,7 +1269,7 @@ machine_check_handle_early:
GET_PACA(r13)
ld  r1,PACAR1(r13)
li  r3,PNV_THREAD_NAP
-   b   power7_enter_nap_mode
+   b   pnv_enter_arch207_idle_mode
 4:
 #endif
/*
diff --git a/arch/powerpc/kernel/idle_book3s.S 
b/arch/powerpc/kernel/idle_book3s.S
index d5def06..34dbfc9 100644
--- a/arch/powerpc/kernel/idle_book3s.S
+++ b/arch/powerpc/kernel/idle_book3s.S
@@ -1,5 +1,6 @@
 /*
- *  This file contains the power_save function for Power7 CPUs.
+ *  This file contains idle entry/exit functions for POWER7 and
+ *  POWER8 CPUs.
  *
  *  This program is free software; you can redistribute it and/or
  *  modify it under the terms of the GNU General Public License
@@ -75,7 +76,7 @@ core_idle_lock_held:
  * 0 - don't check
  * 1 - check
  */
-_GLOBAL(power7_powersave_common)
+_GLOBAL(pnv_powersave_common)
/* Use r3 to pass state nap/sleep/winkle */
/* NAP is a state loss, we create a regs frame on the
 * stack, fill it up with the state we care about and
@@ -135,14 +136,14 @@ _GLOBAL(power7_powersave_common)
LOAD_REG_IMMEDIATE(r5, MSR_IDLE)
li  r6, MSR_RI
andcr6, r9, r6
-   LOAD_REG_ADDR(r7, power7_enter_nap_mode)
+   LOAD_REG_ADDR(r7, pnv_enter_arch207_idle_mode)
mtmsrd  r6, 1   /* clear RI before setting SRR0/1 */
mtspr   SPRN_SRR0, r7
mtspr   SPRN_SRR1, r5
rfid
 
-   .globl  power7_enter_nap_mode
-power7_enter_nap_mode:
+   .globl pnv_enter_arch207_idle_mode
+pnv_enter_arch207_idle_mode:
 #ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
/* Tell KVM we're napping */
li  r4,KVM_HWTHREAD_IN_NAP
@@ -242,19 +243,19 @@ _GLOBAL(power7_idle)
 _GLOBAL(power7_nap)
mr  r4,r3
li  r3,PNV_THREAD_NAP
-   b   power7_powersave_common
+   b   pnv_powersave_common
/* No return */
 
 _GLOBAL(power7_sleep)
li  r3,PNV_THREAD_SLEEP
li  r4,1
-   b   power7_powersave_common
+   b   pnv_powersave_common
/* No return */
 
 _GLOBAL(power7_winkle)
li  r3,PNV_THREAD_WINKLE
li  r4,1
-   b   power7_powersave_common
+   b   pnv_powersave_common
/* No return */
 
 #define CHECK_HMI_INTERRUPT\
@@ -284,7 +285,7 @@ ALT_FTR_SECTION_END_NESTED_IFSET(CPU_FTR_ARCH_207S, 66);
\
  * r13 - Contents of HSPRG0
  * cr3 - set to gt if waking up with partial/complete hypervisor state loss
  */
-_GLOBAL(power7_restore_hyp_resource)
+_GLOBAL(pnv_restore_hyp_resource)
/*
 * Check if last bit of HSPGR0 is set. This indicates whether we are
 * waking up from winkle.
@@ -296,7 +297,7 @@ _GLOBAL(power7_restore_hyp_resource)
 
lbz r0,PACA_THREAD_IDLE_STATE(r13)
cmpwi   cr2,r0,PNV_THREAD_NAP
-   bgt cr2,power7_wakeup_tb_loss   /* Either sleep or Winkle */
+   bgt cr2,pnv_wakeup_tb_loss  /* Either sleep or Winkle */
 
/*
 * We fall through here if PACA_THREAD_IDLE_STATE shows we are waking
@@ -306,10 +307,10 @@ _GLOBAL(power7_restore_hyp_resource)
bgt cr3,.
 
blr /* Return back to System Reset vector from where
-  power7_restore_hyp_resource was invoked */
+  pnv_restore_

[PATCH v8 03/11] powerpc/powernv: Rename idle_power7.S to idle_book3s.S

[Shreyas B. Prabhu]

idle_power7.S handles idle entry/exit for POWER7, POWER8 and in next
patch for POWER9. Rename the file to a non-hardware specific
name.

Reviewed-by: Gautham R. Shenoy 
Signed-off-by: Shreyas B. Prabhu 
---

Changes in v7:
=
 - File renamed to idle_book3s.S instead of idle_power_common.S

Changes in v3:
==
 - Instead of moving few common functions from idle_power7.S to
   idle_power_common.S, renaming idle_power7.S to idle_power_common.S

 arch/powerpc/kernel/Makefile  |   2 +-
 arch/powerpc/kernel/idle_book3s.S | 527 ++
 arch/powerpc/kernel/idle_power7.S | 527 --
 3 files changed, 528 insertions(+), 528 deletions(-)
 create mode 100644 arch/powerpc/kernel/idle_book3s.S
 delete mode 100644 arch/powerpc/kernel/idle_power7.S

diff --git a/arch/powerpc/kernel/Makefile b/arch/powerpc/kernel/Makefile
index 2da380f..9e7bfc32 100644
--- a/arch/powerpc/kernel/Makefile
+++ b/arch/powerpc/kernel/Makefile
@@ -47,7 +47,7 @@ obj-$(CONFIG_PPC_BOOK3E_64)   += exceptions-64e.o 
idle_book3e.o
 obj-$(CONFIG_PPC64)+= vdso64/
 obj-$(CONFIG_ALTIVEC)  += vecemu.o
 obj-$(CONFIG_PPC_970_NAP)  += idle_power4.o
-obj-$(CONFIG_PPC_P7_NAP)   += idle_power7.o
+obj-$(CONFIG_PPC_P7_NAP)   += idle_book3s.o
 procfs-y   := proc_powerpc.o
 obj-$(CONFIG_PROC_FS)  += $(procfs-y)
 rtaspci-$(CONFIG_PPC64)-$(CONFIG_PCI)  := rtas_pci.o
diff --git a/arch/powerpc/kernel/idle_book3s.S 
b/arch/powerpc/kernel/idle_book3s.S
new file mode 100644
index 000..d5def06
--- /dev/null
+++ b/arch/powerpc/kernel/idle_book3s.S
@@ -0,0 +1,527 @@
+/*
+ *  This file contains the power_save function for Power7 CPUs.
+ *
+ *  This program is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU General Public License
+ *  as published by the Free Software Foundation; either version
+ *  2 of the License, or (at your option) any later version.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#undef DEBUG
+
+/*
+ * Use unused space in the interrupt stack to save and restore
+ * registers for winkle support.
+ */
+#define _SDR1  GPR3
+#define _RPR   GPR4
+#define _SPURR GPR5
+#define _PURR  GPR6
+#define _TSCR  GPR7
+#define _DSCR  GPR8
+#define _AMOR  GPR9
+#define _WORT  GPR10
+#define _WORC  GPR11
+
+/* Idle state entry routines */
+
+#defineIDLE_STATE_ENTER_SEQ(IDLE_INST) \
+   /* Magic NAP/SLEEP/WINKLE mode enter sequence */\
+   std r0,0(r1);   \
+   ptesync;\
+   ld  r0,0(r1);   \
+1: cmp cr0,r0,r0;  \
+   bne 1b; \
+   IDLE_INST;  \
+   b   .
+
+   .text
+
+/*
+ * Used by threads when the lock bit of core_idle_state is set.
+ * Threads will spin in HMT_LOW until the lock bit is cleared.
+ * r14 - pointer to core_idle_state
+ * r15 - used to load contents of core_idle_state
+ */
+
+core_idle_lock_held:
+   HMT_LOW
+3: lwz r15,0(r14)
+   andi.   r15,r15,PNV_CORE_IDLE_LOCK_BIT
+   bne 3b
+   HMT_MEDIUM
+   lwarx   r15,0,r14
+   blr
+
+/*
+ * Pass requested state in r3:
+ * r3 - PNV_THREAD_NAP/SLEEP/WINKLE
+ *
+ * To check IRQ_HAPPENED in r4
+ * 0 - don't check
+ * 1 - check
+ */
+_GLOBAL(power7_powersave_common)
+   /* Use r3 to pass state nap/sleep/winkle */
+   /* NAP is a state loss, we create a regs frame on the
+* stack, fill it up with the state we care about and
+* stick a pointer to it in PACAR1. We really only
+* need to save PC, some CR bits and the NV GPRs,
+* but for now an interrupt frame will do.
+*/
+   mflrr0
+   std r0,16(r1)
+   stdur1,-INT_FRAME_SIZE(r1)
+   std r0,_LINK(r1)
+   std r0,_NIP(r1)
+
+   /* Hard disable interrupts */
+   mfmsr   r9
+   rldicl  r9,r9,48,1
+   rotldi  r9,r9,16
+   mtmsrd  r9,1/* hard-disable interrupts */
+
+   /* Check if something happened while soft-disabled */
+   lbz r0,PACAIRQHAPPENED(r13)
+   andi.   r0,r0,~PACA_IRQ_HARD_DIS@l
+   beq 1f
+   cmpwi   cr0,r4,0
+   beq 1f
+   addir1,r1,INT_FRAME_SIZE
+   ld  r0,16(r1)
+   li  r3,0/* Return 0 (no nap) */
+   mtlrr0
+   blr
+
+1: /* We mark irqs hard disabled as this is the state we'll
+* be in when returning and we need to tell arch_local_irq_restore()
+* about it
+*/
+   li  r0,PACA_IRQ_HARD_DIS
+   stb r0,PACAIRQHAPPENED(r13

[PATCH v8 04/11] powerpc/powernv: Rename reusable idle functions to hardware agnostic names

[Shreyas B. Prabhu]

Functions like power7_wakeup_loss, power7_wakeup_noloss,
power7_wakeup_tb_loss are used by POWER7 and POWER8 hardware. They can
also be used by POWER9. Hence rename these functions hardware agnostic
names.

Suggested-by: Gautham R. Shenoy 
Signed-off-by: Shreyas B. Prabhu 
---
 - No changes since v4

Changes in v4:
==
 - renaming power7_powersave_common to pnv_powersave_common
 - renaming power7_enter_nap_mode to pnv_enter_arch207_idle_mode

 arch/powerpc/kernel/exceptions-64s.S|  8 
 arch/powerpc/kernel/idle_book3s.S   | 33 +
 arch/powerpc/kvm/book3s_hv_rmhandlers.S |  4 ++--
 3 files changed, 23 insertions(+), 22 deletions(-)

diff --git a/arch/powerpc/kernel/exceptions-64s.S 
b/arch/powerpc/kernel/exceptions-64s.S
index 4a74d6a..2a123cd 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -108,7 +108,7 @@ BEGIN_FTR_SECTION
 
cmpwi   cr3,r13,2
GET_PACA(r13)
-   bl  power7_restore_hyp_resource
+   bl  pnv_restore_hyp_resource
 
li  r0,PNV_THREAD_RUNNING
stb r0,PACA_THREAD_IDLE_STATE(r13)  /* Clear thread state */
@@ -128,8 +128,8 @@ BEGIN_FTR_SECTION
/* Return SRR1 from power7_nap() */
mfspr   r3,SPRN_SRR1
blt cr3,2f
-   b   power7_wakeup_loss
-2: b   power7_wakeup_noloss
+   b   pnv_wakeup_loss
+2: b   pnv_wakeup_noloss
 
 9:
 END_FTR_SECTION_IFSET(CPU_FTR_HVMODE | CPU_FTR_ARCH_206)
@@ -1269,7 +1269,7 @@ machine_check_handle_early:
GET_PACA(r13)
ld  r1,PACAR1(r13)
li  r3,PNV_THREAD_NAP
-   b   power7_enter_nap_mode
+   b   pnv_enter_arch207_idle_mode
 4:
 #endif
/*
diff --git a/arch/powerpc/kernel/idle_book3s.S 
b/arch/powerpc/kernel/idle_book3s.S
index d5def06..34dbfc9 100644
--- a/arch/powerpc/kernel/idle_book3s.S
+++ b/arch/powerpc/kernel/idle_book3s.S
@@ -1,5 +1,6 @@
 /*
- *  This file contains the power_save function for Power7 CPUs.
+ *  This file contains idle entry/exit functions for POWER7 and
+ *  POWER8 CPUs.
  *
  *  This program is free software; you can redistribute it and/or
  *  modify it under the terms of the GNU General Public License
@@ -75,7 +76,7 @@ core_idle_lock_held:
  * 0 - don't check
  * 1 - check
  */
-_GLOBAL(power7_powersave_common)
+_GLOBAL(pnv_powersave_common)
/* Use r3 to pass state nap/sleep/winkle */
/* NAP is a state loss, we create a regs frame on the
 * stack, fill it up with the state we care about and
@@ -135,14 +136,14 @@ _GLOBAL(power7_powersave_common)
LOAD_REG_IMMEDIATE(r5, MSR_IDLE)
li  r6, MSR_RI
andcr6, r9, r6
-   LOAD_REG_ADDR(r7, power7_enter_nap_mode)
+   LOAD_REG_ADDR(r7, pnv_enter_arch207_idle_mode)
mtmsrd  r6, 1   /* clear RI before setting SRR0/1 */
mtspr   SPRN_SRR0, r7
mtspr   SPRN_SRR1, r5
rfid
 
-   .globl  power7_enter_nap_mode
-power7_enter_nap_mode:
+   .globl pnv_enter_arch207_idle_mode
+pnv_enter_arch207_idle_mode:
 #ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
/* Tell KVM we're napping */
li  r4,KVM_HWTHREAD_IN_NAP
@@ -242,19 +243,19 @@ _GLOBAL(power7_idle)
 _GLOBAL(power7_nap)
mr  r4,r3
li  r3,PNV_THREAD_NAP
-   b   power7_powersave_common
+   b   pnv_powersave_common
/* No return */
 
 _GLOBAL(power7_sleep)
li  r3,PNV_THREAD_SLEEP
li  r4,1
-   b   power7_powersave_common
+   b   pnv_powersave_common
/* No return */
 
 _GLOBAL(power7_winkle)
li  r3,PNV_THREAD_WINKLE
li  r4,1
-   b   power7_powersave_common
+   b   pnv_powersave_common
/* No return */
 
 #define CHECK_HMI_INTERRUPT\
@@ -284,7 +285,7 @@ ALT_FTR_SECTION_END_NESTED_IFSET(CPU_FTR_ARCH_207S, 66);
\
  * r13 - Contents of HSPRG0
  * cr3 - set to gt if waking up with partial/complete hypervisor state loss
  */
-_GLOBAL(power7_restore_hyp_resource)
+_GLOBAL(pnv_restore_hyp_resource)
/*
 * Check if last bit of HSPGR0 is set. This indicates whether we are
 * waking up from winkle.
@@ -296,7 +297,7 @@ _GLOBAL(power7_restore_hyp_resource)
 
lbz r0,PACA_THREAD_IDLE_STATE(r13)
cmpwi   cr2,r0,PNV_THREAD_NAP
-   bgt cr2,power7_wakeup_tb_loss   /* Either sleep or Winkle */
+   bgt cr2,pnv_wakeup_tb_loss  /* Either sleep or Winkle */
 
/*
 * We fall through here if PACA_THREAD_IDLE_STATE shows we are waking
@@ -306,10 +307,10 @@ _GLOBAL(power7_restore_hyp_resource)
bgt cr3,.
 
blr /* Return back to System Reset vector from where
-  power7_restore_hyp_resource was invoked */
+  pnv_restore_hyp_resource was invoked */
 
 
-_GLOBAL(power7_wakeup_tb_loss)
+_GLOBAL

[PATCH v8 02/11] powerpc/kvm: make hypervisor state restore a function

[Shreyas B. Prabhu]

In the current code, when the thread wakes up in reset vector, some
of the state restore code and check for whether a thread needs to
branch to kvm is duplicated. Reorder the code such that this
duplication is avoided.

At a higher level this is what the change looks like-

Before this patch -
power7_wakeup_tb_loss:
restore hypervisor state
if (thread needed by kvm)
goto kvm_start_guest
restore nvgprs, cr, pc
rfid to process context

power7_wakeup_loss:
restore nvgprs, cr, pc
rfid to process context

reset vector:
if (waking from deep idle states)
goto power7_wakeup_tb_loss
else
if (thread needed by kvm)
goto kvm_start_guest
goto power7_wakeup_loss

After this patch -
power7_wakeup_tb_loss:
restore hypervisor state
return

power7_restore_hyp_resource():
if (waking from deep idle states)
goto power7_wakeup_tb_loss
return

power7_wakeup_loss:
restore nvgprs, cr, pc
rfid to process context

reset vector:
power7_restore_hyp_resource()
if (thread needed by kvm)
goto kvm_start_guest
goto power7_wakeup_loss

Reviewed-by: Paul Mackerras <pau...@samba.org>
Reviewed-by: Gautham R. Shenoy <e...@linux.vnet.ibm.com>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
- No changes since v3

Changes in v3:
=
- Retaining GET_PACA(r13) in System Reset vector instead of moving it
  to power7_restore_hyp_resource
- Added comments indicating entry conditions for power7_restore_hyp_resource
- Improved comments around return statements

 arch/powerpc/kernel/exceptions-64s.S | 28 ++
 arch/powerpc/kernel/idle_power7.S| 72 +---
 2 files changed, 46 insertions(+), 54 deletions(-)

diff --git a/arch/powerpc/kernel/exceptions-64s.S 
b/arch/powerpc/kernel/exceptions-64s.S
index 4c94406..4a74d6a 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -107,25 +107,9 @@ BEGIN_FTR_SECTION
beq 9f
 
cmpwi   cr3,r13,2
-
-   /*
-* Check if last bit of HSPGR0 is set. This indicates whether we are
-* waking up from winkle.
-*/
GET_PACA(r13)
-   clrldi  r5,r13,63
-   clrrdi  r13,r13,1
-   cmpwi   cr4,r5,1
-   mtspr   SPRN_HSPRG0,r13
+   bl  power7_restore_hyp_resource
 
-   lbz r0,PACA_THREAD_IDLE_STATE(r13)
-   cmpwi   cr2,r0,PNV_THREAD_NAP
-   bgt cr2,8f  /* Either sleep or Winkle */
-
-   /* Waking up from nap should not cause hypervisor state loss */
-   bgt cr3,.
-
-   /* Waking up from nap */
li  r0,PNV_THREAD_RUNNING
stb r0,PACA_THREAD_IDLE_STATE(r13)  /* Clear thread state */
 
@@ -143,13 +127,9 @@ BEGIN_FTR_SECTION
 
/* Return SRR1 from power7_nap() */
mfspr   r3,SPRN_SRR1
-   beq cr3,2f
-   b   power7_wakeup_noloss
-2: b   power7_wakeup_loss
-
-   /* Fast Sleep wakeup on PowerNV */
-8: GET_PACA(r13)
-   b   power7_wakeup_tb_loss
+   blt cr3,2f
+   b   power7_wakeup_loss
+2: b   power7_wakeup_noloss
 
 9:
 END_FTR_SECTION_IFSET(CPU_FTR_HVMODE | CPU_FTR_ARCH_206)
diff --git a/arch/powerpc/kernel/idle_power7.S 
b/arch/powerpc/kernel/idle_power7.S
index 705c867..d5def06 100644
--- a/arch/powerpc/kernel/idle_power7.S
+++ b/arch/powerpc/kernel/idle_power7.S
@@ -276,6 +276,39 @@ ALT_FTR_SECTION_END_NESTED_IFSET(CPU_FTR_ARCH_207S, 66);   
\
 20:nop;
 
 
+/*
+ * Called from reset vector. Check whether we have woken up with
+ * hypervisor state loss. If yes, restore hypervisor state and return
+ * back to reset vector.
+ *
+ * r13 - Contents of HSPRG0
+ * cr3 - set to gt if waking up with partial/complete hypervisor state loss
+ */
+_GLOBAL(power7_restore_hyp_resource)
+   /*
+* Check if last bit of HSPGR0 is set. This indicates whether we are
+* waking up from winkle.
+*/
+   clrldi  r5,r13,63
+   clrrdi  r13,r13,1
+   cmpwi   cr4,r5,1
+   mtspr   SPRN_HSPRG0,r13
+
+   lbz r0,PACA_THREAD_IDLE_STATE(r13)
+   cmpwi   cr2,r0,PNV_THREAD_NAP
+   bgt cr2,power7_wakeup_tb_loss   /* Either sleep or Winkle */
+
+   /*
+* We fall through here if PACA_THREAD_IDLE_STATE shows we are waking
+* up from nap. At this stage CR3 shouldn't contains 'gt' since that
+* indicates we are waking with hypervisor state loss from nap.
+*/
+   bgt cr3,.
+
+   blr /* Return back to System Reset vector from where
+  power7_restore_hyp_resource was invoked */
+
+
 _GLOBAL(power7_wakeup_tb_loss)
ld  r2,PACATOC(r13);
ld  r1,PACAR1(r13)
@@ -284,11 +317,13 @@ _GLOBAL(pow

[PATCH v8 11/11] powerpc/powernv: Use deepest stop state when cpu is offlined

[Shreyas B. Prabhu]

If hardware supports stop state, use the deepest stop state when
the cpu is offlined.

Reviewed-by: Gautham R. Shenoy <e...@linux.vnet.ibm.com>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
- No changes since v1

 arch/powerpc/platforms/powernv/idle.c| 15 +--
 arch/powerpc/platforms/powernv/powernv.h |  1 +
 arch/powerpc/platforms/powernv/smp.c |  4 +++-
 3 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 8219e22..479c256 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -253,6 +253,11 @@ static void power9_idle(void)
 u64 pnv_first_deep_stop_state = MAX_STOP_STATE;
 
 /*
+ * Deepest stop idle state. Used when a cpu is offlined
+ */
+u64 pnv_deepest_stop_state;
+
+/*
  * Power ISA 3.0 idle initialization.
  *
  * POWER ISA 3.0 defines a new SPR Processor stop Status and Control
@@ -314,8 +319,11 @@ static int __init pnv_arch300_idle_init(struct device_node 
*np, u32 *flags,
}
 
/*
-* Set pnv_first_deep_stop_state to the first stop level
-* to cause hypervisor state loss
+* Set pnv_first_deep_stop_state and pnv_deepest_stop_state.
+* pnv_first_deep_stop_state should be set to the first stop
+* level to cause hypervisor state loss.
+* pnv_deepest_stop_state should be set to the deepest stop
+* stop state.
 */
pnv_first_deep_stop_state = MAX_STOP_STATE;
for (i = 0; i < dt_idle_states; i++) {
@@ -324,6 +332,9 @@ static int __init pnv_arch300_idle_init(struct device_node 
*np, u32 *flags,
if ((flags[i] & OPAL_PM_LOSE_FULL_CONTEXT) &&
 (pnv_first_deep_stop_state > psscr_rl))
pnv_first_deep_stop_state = psscr_rl;
+
+   if (pnv_deepest_stop_state < psscr_rl)
+   pnv_deepest_stop_state = psscr_rl;
}
 
 out:
diff --git a/arch/powerpc/platforms/powernv/powernv.h 
b/arch/powerpc/platforms/powernv/powernv.h
index 6dbc0a1..da7c843 100644
--- a/arch/powerpc/platforms/powernv/powernv.h
+++ b/arch/powerpc/platforms/powernv/powernv.h
@@ -18,6 +18,7 @@ static inline void pnv_pci_shutdown(void) { }
 #endif
 
 extern u32 pnv_get_supported_cpuidle_states(void);
+extern u64 pnv_deepest_stop_state;
 
 extern void pnv_lpc_init(void);
 
diff --git a/arch/powerpc/platforms/powernv/smp.c 
b/arch/powerpc/platforms/powernv/smp.c
index ad7b1a3..c789258 100644
--- a/arch/powerpc/platforms/powernv/smp.c
+++ b/arch/powerpc/platforms/powernv/smp.c
@@ -182,7 +182,9 @@ static void pnv_smp_cpu_kill_self(void)
 
ppc64_runlatch_off();
 
-   if (idle_states & OPAL_PM_WINKLE_ENABLED)
+   if (cpu_has_feature(CPU_FTR_ARCH_300))
+   srr1 = power9_idle_stop(pnv_deepest_stop_state);
+   else if (idle_states & OPAL_PM_WINKLE_ENABLED)
srr1 = power7_winkle();
else if ((idle_states & OPAL_PM_SLEEP_ENABLED) ||
(idle_states & OPAL_PM_SLEEP_ENABLED_ER1))
-- 
2.4.11

[PATCH v8 05/11] powerpc/powernv: Make pnv_powersave_common more generic

[Shreyas B. Prabhu]

pnv_powersave_common does common steps needed before entering idle
state and eventually changes MSR to MSR_IDLE and does rfid to
pnv_enter_arch207_idle_mode.

Move the updation of HSTATE_HWTHREAD_STATE to pnv_powersave_common
from pnv_enter_arch207_idle_mode and make it more generic by passing the
rfid address as a function parameter.

Reviewed-by: Gautham R. Shenoy <e...@linux.vnet.ibm.com>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
 - No changes since v4

Changes in v4:
==
 - Moved renaming of power7_powersave_common to earlier patch

Changes in v3:
==
 - Moved HSTATE_HWTHREAD_STATE updation to power_powersave_common

 arch/powerpc/kernel/idle_book3s.S | 23 ++-
 1 file changed, 14 insertions(+), 9 deletions(-)

diff --git a/arch/powerpc/kernel/idle_book3s.S 
b/arch/powerpc/kernel/idle_book3s.S
index 34dbfc9..a8397e3 100644
--- a/arch/powerpc/kernel/idle_book3s.S
+++ b/arch/powerpc/kernel/idle_book3s.S
@@ -75,6 +75,8 @@ core_idle_lock_held:
  * To check IRQ_HAPPENED in r4
  * 0 - don't check
  * 1 - check
+ *
+ * Address to 'rfid' to in r5
  */
 _GLOBAL(pnv_powersave_common)
/* Use r3 to pass state nap/sleep/winkle */
@@ -127,28 +129,28 @@ _GLOBAL(pnv_powersave_common)
std r9,_MSR(r1)
std r1,PACAR1(r13)
 
+#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
+   /* Tell KVM we're entering idle */
+   li  r4,KVM_HWTHREAD_IN_NAP
+   stb r4,HSTATE_HWTHREAD_STATE(r13)
+#endif
+
/*
 * Go to real mode to do the nap, as required by the architecture.
 * Also, we need to be in real mode before setting hwthread_state,
 * because as soon as we do that, another thread can switch
 * the MMU context to the guest.
 */
-   LOAD_REG_IMMEDIATE(r5, MSR_IDLE)
+   LOAD_REG_IMMEDIATE(r7, MSR_IDLE)
li  r6, MSR_RI
andcr6, r9, r6
-   LOAD_REG_ADDR(r7, pnv_enter_arch207_idle_mode)
mtmsrd  r6, 1   /* clear RI before setting SRR0/1 */
-   mtspr   SPRN_SRR0, r7
-   mtspr   SPRN_SRR1, r5
+   mtspr   SPRN_SRR0, r5
+   mtspr   SPRN_SRR1, r7
rfid
 
.globl pnv_enter_arch207_idle_mode
 pnv_enter_arch207_idle_mode:
-#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
-   /* Tell KVM we're napping */
-   li  r4,KVM_HWTHREAD_IN_NAP
-   stb r4,HSTATE_HWTHREAD_STATE(r13)
-#endif
stb r3,PACA_THREAD_IDLE_STATE(r13)
cmpwi   cr3,r3,PNV_THREAD_SLEEP
bge cr3,2f
@@ -243,18 +245,21 @@ _GLOBAL(power7_idle)
 _GLOBAL(power7_nap)
mr  r4,r3
li  r3,PNV_THREAD_NAP
+   LOAD_REG_ADDR(r5, pnv_enter_arch207_idle_mode)
b   pnv_powersave_common
/* No return */
 
 _GLOBAL(power7_sleep)
li  r3,PNV_THREAD_SLEEP
li  r4,1
+   LOAD_REG_ADDR(r5, pnv_enter_arch207_idle_mode)
b   pnv_powersave_common
/* No return */
 
 _GLOBAL(power7_winkle)
li  r3,PNV_THREAD_WINKLE
li  r4,1
+   LOAD_REG_ADDR(r5, pnv_enter_arch207_idle_mode)
b   pnv_powersave_common
/* No return */
 
-- 
2.4.11

[PATCH v8 06/11] powerpc/powernv: abstraction for saving SPRs before entering deep idle states

[Shreyas B. Prabhu]

Create a function for saving SPRs before entering deep idle states.
This function can be reused for POWER9 deep idle states.

Reviewed-by: Gautham R. Shenoy <e...@linux.vnet.ibm.com>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
 - No changes since v3

Changes in v3:
=
 - Newly added in v3

 arch/powerpc/kernel/idle_book3s.S | 54 +++
 1 file changed, 32 insertions(+), 22 deletions(-)

diff --git a/arch/powerpc/kernel/idle_book3s.S 
b/arch/powerpc/kernel/idle_book3s.S
index a8397e3..2f909a1 100644
--- a/arch/powerpc/kernel/idle_book3s.S
+++ b/arch/powerpc/kernel/idle_book3s.S
@@ -53,6 +53,36 @@
.text
 
 /*
+ * Used by threads before entering deep idle states. Saves SPRs
+ * in interrupt stack frame
+ */
+save_sprs_to_stack:
+   /*
+* Note all register i.e per-core, per-subcore or per-thread is saved
+* here since any thread in the core might wake up first
+*/
+   mfspr   r3,SPRN_SDR1
+   std r3,_SDR1(r1)
+   mfspr   r3,SPRN_RPR
+   std r3,_RPR(r1)
+   mfspr   r3,SPRN_SPURR
+   std r3,_SPURR(r1)
+   mfspr   r3,SPRN_PURR
+   std r3,_PURR(r1)
+   mfspr   r3,SPRN_TSCR
+   std r3,_TSCR(r1)
+   mfspr   r3,SPRN_DSCR
+   std r3,_DSCR(r1)
+   mfspr   r3,SPRN_AMOR
+   std r3,_AMOR(r1)
+   mfspr   r3,SPRN_WORT
+   std r3,_WORT(r1)
+   mfspr   r3,SPRN_WORC
+   std r3,_WORC(r1)
+
+   blr
+
+/*
  * Used by threads when the lock bit of core_idle_state is set.
  * Threads will spin in HMT_LOW until the lock bit is cleared.
  * r14 - pointer to core_idle_state
@@ -209,28 +239,8 @@ fastsleep_workaround_at_entry:
b   common_enter
 
 enter_winkle:
-   /*
-* Note all register i.e per-core, per-subcore or per-thread is saved
-* here since any thread in the core might wake up first
-*/
-   mfspr   r3,SPRN_SDR1
-   std r3,_SDR1(r1)
-   mfspr   r3,SPRN_RPR
-   std r3,_RPR(r1)
-   mfspr   r3,SPRN_SPURR
-   std r3,_SPURR(r1)
-   mfspr   r3,SPRN_PURR
-   std r3,_PURR(r1)
-   mfspr   r3,SPRN_TSCR
-   std r3,_TSCR(r1)
-   mfspr   r3,SPRN_DSCR
-   std r3,_DSCR(r1)
-   mfspr   r3,SPRN_AMOR
-   std r3,_AMOR(r1)
-   mfspr   r3,SPRN_WORT
-   std r3,_WORT(r1)
-   mfspr   r3,SPRN_WORC
-   std r3,_WORC(r1)
+   bl  save_sprs_to_stack
+
IDLE_STATE_ENTER_SEQ(PPC_WINKLE)
 
 _GLOBAL(power7_idle)
-- 
2.4.11

[PATCH v8 08/11] cpuidle/powernv: Use CPUIDLE_STATE_MAX instead of MAX_POWERNV_IDLE_STATES

[Shreyas B. Prabhu]

Use cpuidle's CPUIDLE_STATE_MAX macro instead of powernv specific
MAX_POWERNV_IDLE_STATES.

Cc: Rafael J. Wysocki <rafael.j.wyso...@intel.com>
Cc: Daniel Lezcano <daniel.lezc...@linaro.org>
Cc: linux...@vger.kernel.org
Acked-by: Daniel Lezcano <daniel.lezc...@linaro.org>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
 - No changes after v5

Changes in v5
=
 - New in v5

 drivers/cpuidle/cpuidle-powernv.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/drivers/cpuidle/cpuidle-powernv.c 
b/drivers/cpuidle/cpuidle-powernv.c
index e12dc30..3a763a8 100644
--- a/drivers/cpuidle/cpuidle-powernv.c
+++ b/drivers/cpuidle/cpuidle-powernv.c
@@ -20,8 +20,6 @@
 #include 
 #include 
 
-#define MAX_POWERNV_IDLE_STATES8
-
 struct cpuidle_driver powernv_idle_driver = {
.name = "powernv_idle",
.owner= THIS_MODULE,
@@ -96,7 +94,7 @@ static int fastsleep_loop(struct cpuidle_device *dev,
 /*
  * States for dedicated partition case.
  */
-static struct cpuidle_state powernv_states[MAX_POWERNV_IDLE_STATES] = {
+static struct cpuidle_state powernv_states[CPUIDLE_STATE_MAX] = {
{ /* Snooze */
.name = "snooze",
.desc = "snooze",
-- 
2.4.11

[PATCH v8 06/11] powerpc/powernv: abstraction for saving SPRs before entering deep idle states

[Shreyas B. Prabhu]

Create a function for saving SPRs before entering deep idle states.
This function can be reused for POWER9 deep idle states.

Reviewed-by: Gautham R. Shenoy 
Signed-off-by: Shreyas B. Prabhu 
---
 - No changes since v3

Changes in v3:
=
 - Newly added in v3

 arch/powerpc/kernel/idle_book3s.S | 54 +++
 1 file changed, 32 insertions(+), 22 deletions(-)

diff --git a/arch/powerpc/kernel/idle_book3s.S 
b/arch/powerpc/kernel/idle_book3s.S
index a8397e3..2f909a1 100644
--- a/arch/powerpc/kernel/idle_book3s.S
+++ b/arch/powerpc/kernel/idle_book3s.S
@@ -53,6 +53,36 @@
.text
 
 /*
+ * Used by threads before entering deep idle states. Saves SPRs
+ * in interrupt stack frame
+ */
+save_sprs_to_stack:
+   /*
+* Note all register i.e per-core, per-subcore or per-thread is saved
+* here since any thread in the core might wake up first
+*/
+   mfspr   r3,SPRN_SDR1
+   std r3,_SDR1(r1)
+   mfspr   r3,SPRN_RPR
+   std r3,_RPR(r1)
+   mfspr   r3,SPRN_SPURR
+   std r3,_SPURR(r1)
+   mfspr   r3,SPRN_PURR
+   std r3,_PURR(r1)
+   mfspr   r3,SPRN_TSCR
+   std r3,_TSCR(r1)
+   mfspr   r3,SPRN_DSCR
+   std r3,_DSCR(r1)
+   mfspr   r3,SPRN_AMOR
+   std r3,_AMOR(r1)
+   mfspr   r3,SPRN_WORT
+   std r3,_WORT(r1)
+   mfspr   r3,SPRN_WORC
+   std r3,_WORC(r1)
+
+   blr
+
+/*
  * Used by threads when the lock bit of core_idle_state is set.
  * Threads will spin in HMT_LOW until the lock bit is cleared.
  * r14 - pointer to core_idle_state
@@ -209,28 +239,8 @@ fastsleep_workaround_at_entry:
b   common_enter
 
 enter_winkle:
-   /*
-* Note all register i.e per-core, per-subcore or per-thread is saved
-* here since any thread in the core might wake up first
-*/
-   mfspr   r3,SPRN_SDR1
-   std r3,_SDR1(r1)
-   mfspr   r3,SPRN_RPR
-   std r3,_RPR(r1)
-   mfspr   r3,SPRN_SPURR
-   std r3,_SPURR(r1)
-   mfspr   r3,SPRN_PURR
-   std r3,_PURR(r1)
-   mfspr   r3,SPRN_TSCR
-   std r3,_TSCR(r1)
-   mfspr   r3,SPRN_DSCR
-   std r3,_DSCR(r1)
-   mfspr   r3,SPRN_AMOR
-   std r3,_AMOR(r1)
-   mfspr   r3,SPRN_WORT
-   std r3,_WORT(r1)
-   mfspr   r3,SPRN_WORC
-   std r3,_WORC(r1)
+   bl  save_sprs_to_stack
+
IDLE_STATE_ENTER_SEQ(PPC_WINKLE)
 
 _GLOBAL(power7_idle)
-- 
2.4.11

[PATCH v8 08/11] cpuidle/powernv: Use CPUIDLE_STATE_MAX instead of MAX_POWERNV_IDLE_STATES

[Shreyas B. Prabhu]

Use cpuidle's CPUIDLE_STATE_MAX macro instead of powernv specific
MAX_POWERNV_IDLE_STATES.

Cc: Rafael J. Wysocki 
Cc: Daniel Lezcano 
Cc: linux...@vger.kernel.org
Acked-by: Daniel Lezcano 
Signed-off-by: Shreyas B. Prabhu 
---
 - No changes after v5

Changes in v5
=
 - New in v5

 drivers/cpuidle/cpuidle-powernv.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/drivers/cpuidle/cpuidle-powernv.c 
b/drivers/cpuidle/cpuidle-powernv.c
index e12dc30..3a763a8 100644
--- a/drivers/cpuidle/cpuidle-powernv.c
+++ b/drivers/cpuidle/cpuidle-powernv.c
@@ -20,8 +20,6 @@
 #include 
 #include 
 
-#define MAX_POWERNV_IDLE_STATES8
-
 struct cpuidle_driver powernv_idle_driver = {
.name = "powernv_idle",
.owner= THIS_MODULE,
@@ -96,7 +94,7 @@ static int fastsleep_loop(struct cpuidle_device *dev,
 /*
  * States for dedicated partition case.
  */
-static struct cpuidle_state powernv_states[MAX_POWERNV_IDLE_STATES] = {
+static struct cpuidle_state powernv_states[CPUIDLE_STATE_MAX] = {
{ /* Snooze */
.name = "snooze",
.desc = "snooze",
-- 
2.4.11

[PATCH v8 02/11] powerpc/kvm: make hypervisor state restore a function

[Shreyas B. Prabhu]

In the current code, when the thread wakes up in reset vector, some
of the state restore code and check for whether a thread needs to
branch to kvm is duplicated. Reorder the code such that this
duplication is avoided.

At a higher level this is what the change looks like-

Before this patch -
power7_wakeup_tb_loss:
restore hypervisor state
if (thread needed by kvm)
goto kvm_start_guest
restore nvgprs, cr, pc
rfid to process context

power7_wakeup_loss:
restore nvgprs, cr, pc
rfid to process context

reset vector:
if (waking from deep idle states)
goto power7_wakeup_tb_loss
else
if (thread needed by kvm)
goto kvm_start_guest
goto power7_wakeup_loss

After this patch -
power7_wakeup_tb_loss:
restore hypervisor state
return

power7_restore_hyp_resource():
if (waking from deep idle states)
goto power7_wakeup_tb_loss
return

power7_wakeup_loss:
restore nvgprs, cr, pc
rfid to process context

reset vector:
power7_restore_hyp_resource()
if (thread needed by kvm)
goto kvm_start_guest
goto power7_wakeup_loss

Reviewed-by: Paul Mackerras 
Reviewed-by: Gautham R. Shenoy 
Signed-off-by: Shreyas B. Prabhu 
---
- No changes since v3

Changes in v3:
=
- Retaining GET_PACA(r13) in System Reset vector instead of moving it
  to power7_restore_hyp_resource
- Added comments indicating entry conditions for power7_restore_hyp_resource
- Improved comments around return statements

 arch/powerpc/kernel/exceptions-64s.S | 28 ++
 arch/powerpc/kernel/idle_power7.S| 72 +---
 2 files changed, 46 insertions(+), 54 deletions(-)

diff --git a/arch/powerpc/kernel/exceptions-64s.S 
b/arch/powerpc/kernel/exceptions-64s.S
index 4c94406..4a74d6a 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -107,25 +107,9 @@ BEGIN_FTR_SECTION
beq 9f
 
cmpwi   cr3,r13,2
-
-   /*
-* Check if last bit of HSPGR0 is set. This indicates whether we are
-* waking up from winkle.
-*/
GET_PACA(r13)
-   clrldi  r5,r13,63
-   clrrdi  r13,r13,1
-   cmpwi   cr4,r5,1
-   mtspr   SPRN_HSPRG0,r13
+   bl  power7_restore_hyp_resource
 
-   lbz r0,PACA_THREAD_IDLE_STATE(r13)
-   cmpwi   cr2,r0,PNV_THREAD_NAP
-   bgt cr2,8f  /* Either sleep or Winkle */
-
-   /* Waking up from nap should not cause hypervisor state loss */
-   bgt cr3,.
-
-   /* Waking up from nap */
li  r0,PNV_THREAD_RUNNING
stb r0,PACA_THREAD_IDLE_STATE(r13)  /* Clear thread state */
 
@@ -143,13 +127,9 @@ BEGIN_FTR_SECTION
 
/* Return SRR1 from power7_nap() */
mfspr   r3,SPRN_SRR1
-   beq cr3,2f
-   b   power7_wakeup_noloss
-2: b   power7_wakeup_loss
-
-   /* Fast Sleep wakeup on PowerNV */
-8: GET_PACA(r13)
-   b   power7_wakeup_tb_loss
+   blt cr3,2f
+   b   power7_wakeup_loss
+2: b   power7_wakeup_noloss
 
 9:
 END_FTR_SECTION_IFSET(CPU_FTR_HVMODE | CPU_FTR_ARCH_206)
diff --git a/arch/powerpc/kernel/idle_power7.S 
b/arch/powerpc/kernel/idle_power7.S
index 705c867..d5def06 100644
--- a/arch/powerpc/kernel/idle_power7.S
+++ b/arch/powerpc/kernel/idle_power7.S
@@ -276,6 +276,39 @@ ALT_FTR_SECTION_END_NESTED_IFSET(CPU_FTR_ARCH_207S, 66);   
\
 20:nop;
 
 
+/*
+ * Called from reset vector. Check whether we have woken up with
+ * hypervisor state loss. If yes, restore hypervisor state and return
+ * back to reset vector.
+ *
+ * r13 - Contents of HSPRG0
+ * cr3 - set to gt if waking up with partial/complete hypervisor state loss
+ */
+_GLOBAL(power7_restore_hyp_resource)
+   /*
+* Check if last bit of HSPGR0 is set. This indicates whether we are
+* waking up from winkle.
+*/
+   clrldi  r5,r13,63
+   clrrdi  r13,r13,1
+   cmpwi   cr4,r5,1
+   mtspr   SPRN_HSPRG0,r13
+
+   lbz r0,PACA_THREAD_IDLE_STATE(r13)
+   cmpwi   cr2,r0,PNV_THREAD_NAP
+   bgt cr2,power7_wakeup_tb_loss   /* Either sleep or Winkle */
+
+   /*
+* We fall through here if PACA_THREAD_IDLE_STATE shows we are waking
+* up from nap. At this stage CR3 shouldn't contains 'gt' since that
+* indicates we are waking with hypervisor state loss from nap.
+*/
+   bgt cr3,.
+
+   blr /* Return back to System Reset vector from where
+  power7_restore_hyp_resource was invoked */
+
+
 _GLOBAL(power7_wakeup_tb_loss)
ld  r2,PACATOC(r13);
ld  r1,PACAR1(r13)
@@ -284,11 +317,13 @@ _GLOBAL(power7_wakeup_tb_loss)
 * and they are restored before switching to the process context. Hence

[PATCH v8 11/11] powerpc/powernv: Use deepest stop state when cpu is offlined

[Shreyas B. Prabhu]

If hardware supports stop state, use the deepest stop state when
the cpu is offlined.

Reviewed-by: Gautham R. Shenoy 
Signed-off-by: Shreyas B. Prabhu 
---
- No changes since v1

 arch/powerpc/platforms/powernv/idle.c| 15 +--
 arch/powerpc/platforms/powernv/powernv.h |  1 +
 arch/powerpc/platforms/powernv/smp.c |  4 +++-
 3 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 8219e22..479c256 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -253,6 +253,11 @@ static void power9_idle(void)
 u64 pnv_first_deep_stop_state = MAX_STOP_STATE;
 
 /*
+ * Deepest stop idle state. Used when a cpu is offlined
+ */
+u64 pnv_deepest_stop_state;
+
+/*
  * Power ISA 3.0 idle initialization.
  *
  * POWER ISA 3.0 defines a new SPR Processor stop Status and Control
@@ -314,8 +319,11 @@ static int __init pnv_arch300_idle_init(struct device_node 
*np, u32 *flags,
}
 
/*
-* Set pnv_first_deep_stop_state to the first stop level
-* to cause hypervisor state loss
+* Set pnv_first_deep_stop_state and pnv_deepest_stop_state.
+* pnv_first_deep_stop_state should be set to the first stop
+* level to cause hypervisor state loss.
+* pnv_deepest_stop_state should be set to the deepest stop
+* stop state.
 */
pnv_first_deep_stop_state = MAX_STOP_STATE;
for (i = 0; i < dt_idle_states; i++) {
@@ -324,6 +332,9 @@ static int __init pnv_arch300_idle_init(struct device_node 
*np, u32 *flags,
if ((flags[i] & OPAL_PM_LOSE_FULL_CONTEXT) &&
 (pnv_first_deep_stop_state > psscr_rl))
pnv_first_deep_stop_state = psscr_rl;
+
+   if (pnv_deepest_stop_state < psscr_rl)
+   pnv_deepest_stop_state = psscr_rl;
}
 
 out:
diff --git a/arch/powerpc/platforms/powernv/powernv.h 
b/arch/powerpc/platforms/powernv/powernv.h
index 6dbc0a1..da7c843 100644
--- a/arch/powerpc/platforms/powernv/powernv.h
+++ b/arch/powerpc/platforms/powernv/powernv.h
@@ -18,6 +18,7 @@ static inline void pnv_pci_shutdown(void) { }
 #endif
 
 extern u32 pnv_get_supported_cpuidle_states(void);
+extern u64 pnv_deepest_stop_state;
 
 extern void pnv_lpc_init(void);
 
diff --git a/arch/powerpc/platforms/powernv/smp.c 
b/arch/powerpc/platforms/powernv/smp.c
index ad7b1a3..c789258 100644
--- a/arch/powerpc/platforms/powernv/smp.c
+++ b/arch/powerpc/platforms/powernv/smp.c
@@ -182,7 +182,9 @@ static void pnv_smp_cpu_kill_self(void)
 
ppc64_runlatch_off();
 
-   if (idle_states & OPAL_PM_WINKLE_ENABLED)
+   if (cpu_has_feature(CPU_FTR_ARCH_300))
+   srr1 = power9_idle_stop(pnv_deepest_stop_state);
+   else if (idle_states & OPAL_PM_WINKLE_ENABLED)
srr1 = power7_winkle();
else if ((idle_states & OPAL_PM_SLEEP_ENABLED) ||
(idle_states & OPAL_PM_SLEEP_ENABLED_ER1))
-- 
2.4.11

[PATCH v8 05/11] powerpc/powernv: Make pnv_powersave_common more generic

[Shreyas B. Prabhu]

pnv_powersave_common does common steps needed before entering idle
state and eventually changes MSR to MSR_IDLE and does rfid to
pnv_enter_arch207_idle_mode.

Move the updation of HSTATE_HWTHREAD_STATE to pnv_powersave_common
from pnv_enter_arch207_idle_mode and make it more generic by passing the
rfid address as a function parameter.

Reviewed-by: Gautham R. Shenoy 
Signed-off-by: Shreyas B. Prabhu 
---
 - No changes since v4

Changes in v4:
==
 - Moved renaming of power7_powersave_common to earlier patch

Changes in v3:
==
 - Moved HSTATE_HWTHREAD_STATE updation to power_powersave_common

 arch/powerpc/kernel/idle_book3s.S | 23 ++-
 1 file changed, 14 insertions(+), 9 deletions(-)

diff --git a/arch/powerpc/kernel/idle_book3s.S 
b/arch/powerpc/kernel/idle_book3s.S
index 34dbfc9..a8397e3 100644
--- a/arch/powerpc/kernel/idle_book3s.S
+++ b/arch/powerpc/kernel/idle_book3s.S
@@ -75,6 +75,8 @@ core_idle_lock_held:
  * To check IRQ_HAPPENED in r4
  * 0 - don't check
  * 1 - check
+ *
+ * Address to 'rfid' to in r5
  */
 _GLOBAL(pnv_powersave_common)
/* Use r3 to pass state nap/sleep/winkle */
@@ -127,28 +129,28 @@ _GLOBAL(pnv_powersave_common)
std r9,_MSR(r1)
std r1,PACAR1(r13)
 
+#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
+   /* Tell KVM we're entering idle */
+   li  r4,KVM_HWTHREAD_IN_NAP
+   stb r4,HSTATE_HWTHREAD_STATE(r13)
+#endif
+
/*
 * Go to real mode to do the nap, as required by the architecture.
 * Also, we need to be in real mode before setting hwthread_state,
 * because as soon as we do that, another thread can switch
 * the MMU context to the guest.
 */
-   LOAD_REG_IMMEDIATE(r5, MSR_IDLE)
+   LOAD_REG_IMMEDIATE(r7, MSR_IDLE)
li  r6, MSR_RI
andcr6, r9, r6
-   LOAD_REG_ADDR(r7, pnv_enter_arch207_idle_mode)
mtmsrd  r6, 1   /* clear RI before setting SRR0/1 */
-   mtspr   SPRN_SRR0, r7
-   mtspr   SPRN_SRR1, r5
+   mtspr   SPRN_SRR0, r5
+   mtspr   SPRN_SRR1, r7
rfid
 
.globl pnv_enter_arch207_idle_mode
 pnv_enter_arch207_idle_mode:
-#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
-   /* Tell KVM we're napping */
-   li  r4,KVM_HWTHREAD_IN_NAP
-   stb r4,HSTATE_HWTHREAD_STATE(r13)
-#endif
stb r3,PACA_THREAD_IDLE_STATE(r13)
cmpwi   cr3,r3,PNV_THREAD_SLEEP
bge cr3,2f
@@ -243,18 +245,21 @@ _GLOBAL(power7_idle)
 _GLOBAL(power7_nap)
mr  r4,r3
li  r3,PNV_THREAD_NAP
+   LOAD_REG_ADDR(r5, pnv_enter_arch207_idle_mode)
b   pnv_powersave_common
/* No return */
 
 _GLOBAL(power7_sleep)
li  r3,PNV_THREAD_SLEEP
li  r4,1
+   LOAD_REG_ADDR(r5, pnv_enter_arch207_idle_mode)
b   pnv_powersave_common
/* No return */
 
 _GLOBAL(power7_winkle)
li  r3,PNV_THREAD_WINKLE
li  r4,1
+   LOAD_REG_ADDR(r5, pnv_enter_arch207_idle_mode)
b   pnv_powersave_common
/* No return */
 
-- 
2.4.11

[PATCH v8 10/11] cpuidle/powernv: Add support for POWER ISA v3 idle states

[Shreyas B. Prabhu]

POWER ISA v3 defines a new idle processor core mechanism. In summary,
 a) new instruction named stop is added.
 b) new per thread SPR named PSSCR is added which controls the behavior
of stop instruction.

Supported idle states and value to be written to PSSCR register to enter
any idle state is exposed via ibm,cpu-idle-state-names and
ibm,cpu-idle-state-psscr respectively. To enter an idle state,
platform provided power_stop() needs to be invoked with the appropriate
PSSCR value.

This patch adds support for this new mechanism in cpuidle powernv driver.

Cc: Rafael J. Wysocki <rafael.j.wyso...@intel.com>
Cc: Daniel Lezcano <daniel.lezc...@linaro.org>
Cc: Rob Herring <robh...@kernel.org>
Cc: Lorenzo Pieralisi <lorenzo.pieral...@arm.com>
Cc: linux...@vger.kernel.org
Cc: Michael Ellerman <m...@ellerman.id.au>
Cc: Paul Mackerras <pau...@ozlabs.org>
Cc: linuxppc-...@lists.ozlabs.org
Reviewed-by: Gautham R. Shenoy <e...@linux.vnet.ibm.com>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
Note: Documentation for the device tree bindings is posted here-
http://patchwork.ozlabs.org/patch/629125/

Changes in v8
=
 - Fix a copy paste mistake while reading ibm,cpu-idle-state-names 

Changes in v7
=
 - Using stack instead kzalloc/kcalloc 

Changes in v5
=
 - Use generic cpuidle constant CPUIDLE_NAME_LEN
 - Fix return code handling for of_property_read_string_array
 - Use DT flags to determine if are using stop instruction, instead of
   cpu_has_feature
 - Removed uncessary cast with names
 - _loop -> stop_loop
 - Added POWERNV_THRESHOLD_LATENCY_NS to filter out idle states with high 
latency

 drivers/cpuidle/cpuidle-powernv.c | 61 +++
 1 file changed, 61 insertions(+)

diff --git a/drivers/cpuidle/cpuidle-powernv.c 
b/drivers/cpuidle/cpuidle-powernv.c
index 600bbe1..f7ca891 100644
--- a/drivers/cpuidle/cpuidle-powernv.c
+++ b/drivers/cpuidle/cpuidle-powernv.c
@@ -20,6 +20,8 @@
 #include 
 #include 
 
+#define POWERNV_THRESHOLD_LATENCY_NS 20
+
 struct cpuidle_driver powernv_idle_driver = {
.name = "powernv_idle",
.owner= THIS_MODULE,
@@ -27,6 +29,9 @@ struct cpuidle_driver powernv_idle_driver = {
 
 static int max_idle_state;
 static struct cpuidle_state *cpuidle_state_table;
+
+static u64 stop_psscr_table[CPUIDLE_STATE_MAX];
+
 static u64 snooze_timeout;
 static bool snooze_timeout_en;
 
@@ -91,6 +96,17 @@ static int fastsleep_loop(struct cpuidle_device *dev,
return index;
 }
 #endif
+
+static int stop_loop(struct cpuidle_device *dev,
+struct cpuidle_driver *drv,
+int index)
+{
+   ppc64_runlatch_off();
+   power9_idle_stop(stop_psscr_table[index]);
+   ppc64_runlatch_on();
+   return index;
+}
+
 /*
  * States for dedicated partition case.
  */
@@ -169,6 +185,8 @@ static int powernv_add_idle_states(void)
u32 latency_ns[CPUIDLE_STATE_MAX];
u32 residency_ns[CPUIDLE_STATE_MAX];
u32 flags[CPUIDLE_STATE_MAX];
+   u64 psscr_val[CPUIDLE_STATE_MAX];
+   const char *names[CPUIDLE_STATE_MAX];
int i, rc;
 
/* Currently we have snooze statically defined */
@@ -207,11 +225,34 @@ static int powernv_add_idle_states(void)
pr_warn("cpuidle-powernv: missing 
ibm,cpu-idle-state-latencies-ns in DT\n");
goto out;
}
+   if (of_property_read_string_array(power_mgt,
+   "ibm,cpu-idle-state-names", names, dt_idle_states) < 0) {
+   pr_warn("cpuidle-powernv: missing ibm,cpu-idle-state-names in 
DT\n");
+   goto out;
+   }
+
+   /*
+* If the idle states use stop instruction, probe for psscr values
+* which are necessary to specify required stop level.
+*/
+   if (flags[0] & (OPAL_PM_STOP_INST_FAST | OPAL_PM_STOP_INST_DEEP))
+   if (of_property_read_u64_array(power_mgt,
+   "ibm,cpu-idle-state-psscr", psscr_val, dt_idle_states)) {
+   pr_warn("cpuidle-powernv: missing 
ibm,cpu-idle-states-psscr in DT\n");
+   goto out;
+   }
 
rc = of_property_read_u32_array(power_mgt,
"ibm,cpu-idle-state-residency-ns", residency_ns, 
dt_idle_states);
 
for (i = 0; i < dt_idle_states; i++) {
+   /*
+* If an idle state has exit latency beyond
+* POWERNV_THRESHOLD_LATENCY_NS then don't use it
+* in cpu-idle.
+*/
+   if (latency_ns[i] > POWERNV_THRESHOLD_LATENCY_NS)
+   continue;
 
/*
 * Cpuidle accepts exit_latency and target_residency in us.
@@ -224,6 +265,16 @@ static int powernv_add_idle_states(void)
 

[PATCH v8 10/11] cpuidle/powernv: Add support for POWER ISA v3 idle states

[Shreyas B. Prabhu]

POWER ISA v3 defines a new idle processor core mechanism. In summary,
 a) new instruction named stop is added.
 b) new per thread SPR named PSSCR is added which controls the behavior
of stop instruction.

Supported idle states and value to be written to PSSCR register to enter
any idle state is exposed via ibm,cpu-idle-state-names and
ibm,cpu-idle-state-psscr respectively. To enter an idle state,
platform provided power_stop() needs to be invoked with the appropriate
PSSCR value.

This patch adds support for this new mechanism in cpuidle powernv driver.

Cc: Rafael J. Wysocki 
Cc: Daniel Lezcano 
Cc: Rob Herring 
Cc: Lorenzo Pieralisi 
Cc: linux...@vger.kernel.org
Cc: Michael Ellerman 
Cc: Paul Mackerras 
Cc: linuxppc-...@lists.ozlabs.org
Reviewed-by: Gautham R. Shenoy 
Signed-off-by: Shreyas B. Prabhu 
---
Note: Documentation for the device tree bindings is posted here-
http://patchwork.ozlabs.org/patch/629125/

Changes in v8
=
 - Fix a copy paste mistake while reading ibm,cpu-idle-state-names 

Changes in v7
=
 - Using stack instead kzalloc/kcalloc 

Changes in v5
=
 - Use generic cpuidle constant CPUIDLE_NAME_LEN
 - Fix return code handling for of_property_read_string_array
 - Use DT flags to determine if are using stop instruction, instead of
   cpu_has_feature
 - Removed uncessary cast with names
 - _loop -> stop_loop
 - Added POWERNV_THRESHOLD_LATENCY_NS to filter out idle states with high 
latency

 drivers/cpuidle/cpuidle-powernv.c | 61 +++
 1 file changed, 61 insertions(+)

diff --git a/drivers/cpuidle/cpuidle-powernv.c 
b/drivers/cpuidle/cpuidle-powernv.c
index 600bbe1..f7ca891 100644
--- a/drivers/cpuidle/cpuidle-powernv.c
+++ b/drivers/cpuidle/cpuidle-powernv.c
@@ -20,6 +20,8 @@
 #include 
 #include 
 
+#define POWERNV_THRESHOLD_LATENCY_NS 20
+
 struct cpuidle_driver powernv_idle_driver = {
.name = "powernv_idle",
.owner= THIS_MODULE,
@@ -27,6 +29,9 @@ struct cpuidle_driver powernv_idle_driver = {
 
 static int max_idle_state;
 static struct cpuidle_state *cpuidle_state_table;
+
+static u64 stop_psscr_table[CPUIDLE_STATE_MAX];
+
 static u64 snooze_timeout;
 static bool snooze_timeout_en;
 
@@ -91,6 +96,17 @@ static int fastsleep_loop(struct cpuidle_device *dev,
return index;
 }
 #endif
+
+static int stop_loop(struct cpuidle_device *dev,
+struct cpuidle_driver *drv,
+int index)
+{
+   ppc64_runlatch_off();
+   power9_idle_stop(stop_psscr_table[index]);
+   ppc64_runlatch_on();
+   return index;
+}
+
 /*
  * States for dedicated partition case.
  */
@@ -169,6 +185,8 @@ static int powernv_add_idle_states(void)
u32 latency_ns[CPUIDLE_STATE_MAX];
u32 residency_ns[CPUIDLE_STATE_MAX];
u32 flags[CPUIDLE_STATE_MAX];
+   u64 psscr_val[CPUIDLE_STATE_MAX];
+   const char *names[CPUIDLE_STATE_MAX];
int i, rc;
 
/* Currently we have snooze statically defined */
@@ -207,11 +225,34 @@ static int powernv_add_idle_states(void)
pr_warn("cpuidle-powernv: missing 
ibm,cpu-idle-state-latencies-ns in DT\n");
goto out;
}
+   if (of_property_read_string_array(power_mgt,
+   "ibm,cpu-idle-state-names", names, dt_idle_states) < 0) {
+   pr_warn("cpuidle-powernv: missing ibm,cpu-idle-state-names in 
DT\n");
+   goto out;
+   }
+
+   /*
+* If the idle states use stop instruction, probe for psscr values
+* which are necessary to specify required stop level.
+*/
+   if (flags[0] & (OPAL_PM_STOP_INST_FAST | OPAL_PM_STOP_INST_DEEP))
+   if (of_property_read_u64_array(power_mgt,
+   "ibm,cpu-idle-state-psscr", psscr_val, dt_idle_states)) {
+   pr_warn("cpuidle-powernv: missing 
ibm,cpu-idle-states-psscr in DT\n");
+   goto out;
+   }
 
rc = of_property_read_u32_array(power_mgt,
"ibm,cpu-idle-state-residency-ns", residency_ns, 
dt_idle_states);
 
for (i = 0; i < dt_idle_states; i++) {
+   /*
+* If an idle state has exit latency beyond
+* POWERNV_THRESHOLD_LATENCY_NS then don't use it
+* in cpu-idle.
+*/
+   if (latency_ns[i] > POWERNV_THRESHOLD_LATENCY_NS)
+   continue;
 
/*
 * Cpuidle accepts exit_latency and target_residency in us.
@@ -224,6 +265,16 @@ static int powernv_add_idle_states(void)
powernv_states[nr_idle_states].flags = 0;
powernv_states[nr_idle_states].target_residency = 100;
powernv_states[nr_idle_states].enter = nap_loop;
+   } els

[PATCH v8 00/11] powerpc/powernv/cpuidle: Add support for POWER ISA v3 idle states

[Shreyas B. Prabhu]

POWER ISA v3 defines a new idle processor core mechanism. In summary,
 a) new instruction named stop is added. This instruction replaces
instructions like nap, sleep, rvwinkle.
 b) new per thread SPR named PSSCR is added which controls the behavior
of stop instruction. 

PSSCR has following key fields
Bits 0:3  - Power-Saving Level Status. This field indicates the
lowest power-saving state the thread entered since stop
instruction was last executed.

Bit 42 - Enable State Loss  
0 - No state is lost irrespective of other fields  
1 - Allows state loss

Bits 44:47 - Power-Saving Level Limit  
This limits the power-saving level that can be entered into.

Bits 60:63 - Requested Level  
Used to specify which power-saving level must be entered on
executing stop instruction

Stop idle states and their properties like name, latency, target
residency, psscr value are exposed via device tree.

This patch series adds support for this new mechanism.

Patches 1-6 are cleanups and code movement.
Patch 7 adds platform specific support for stop and psscr handling.
Patch 8 and 9 are minor cleanup in cpuidle driver.
Patch 10 adds cpuidle driver support.
Patch 11 makes offlined cpu use deepest stop state.

Note: Documentation for the device tree bindings is posted here-
http://patchwork.ozlabs.org/patch/629125/

Changes in v8
=
 - Fixed a copy-paste mistake in PATCH 10
 - Initializing pnv_first_deep_stop_state
 - Changing MMU_FTR_SECTION condition to reduce FTR section length
 - _loop -> snooze_loop

Changes in v7
=
 - File renamed to idle_book3s.S instead of idle_power_common.S
 - Comment changes
 - power_stop0, power_stop renamed to power9_idle and power_idle_stop
 - PSSCR template is now a macro instead of storing in paca
 - power9_idle in C file instead of assembly
 - Fixed TOC related bug
 - Handling subcore within FTR section
 - Functions in idle.c reordered and broken into multiple functions
 - calling __restore_cpu_power8/9 via cur_cpu_spec->cpu_restore 
 - Added a minor patch with minor cleanups in cpuidle-powernv.c . This
   was mainly to make the existing code consistent with the review
   comments for new code
 - Using stack for variables while probing for idle states instead of
   kzalloc/kcalloc
 - Restoring RPR once per core in P9

Changes in v6
=
 - Restore new POWER ISA v3 SPRS when waking up from deep idle

Changes in v5
=
 - Use generic cpuidle constant CPUIDLE_NAME_LEN
 - Fix return code handling for of_property_read_string_array
 - Use DT flags to determine if are using stop instruction, instead of
   cpu_has_feature
 - Removed uncessary cast with names
 - _loop -> stop_loop
 - Added POWERNV_THRESHOLD_LATENCY_NS to filter out idle states with high 
latency

Changes in v4
=
 - Added a patch to use PNV_THREAD_WINKLE macro while requesting for winkle
 - Moved power7_powersave_common rename to more appropriate patch
 - renaming power7_enter_nap_mode to pnv_enter_arch207_idle_mode
 - Added PSSCR layout to Patch 7's commit message
 - Improved / Fixed comments
 - Fixed whitespace error in paca.h
 - Using MAX_POSSIBLE_STOP_STATE macro instead of hardcoding 0xF has
   max possible stop state

Changes in v3
=
 - Rebased on powerpc-next
 - Dropping patch 1 since we are not adding a new file for P9 idle support
 - Improved comments in multiple places
 - Moved GET_PACA from power7_restore_hyp_resource to System Reset
 - Instead of moving few functions from idle_power7 to idle_power_common,
   renaming idle_power7.S to idle_power_common.S
 - Moved HSTATE_HWTHREAD_STATE updation to power_powersave_common
 - Dropped earlier patch 5 which moved few macros from idle_power_common to
   asm/cpuidle.h. 
 - Added a patch to rename reusable power7_* idle functions to pnv_*
 - Added new patch that creates abstraction for saving SPRs before
   entering deep idle states
 - Instead of introducing new file idle_power_stop.S, P9 idle support
   is added to idle_power_common.S using CPU_FTR sections.
 - Fixed r4 reg clobbering in power_stop0

Changes in v2
=
 - Rebased on v4.6-rc6
 - Using CPU_FTR_ARCH_300 bit instead of CPU_FTR_STOP_INST

Cc: Rafael J. Wysocki <rafael.j.wyso...@intel.com>
Cc: Daniel Lezcano <daniel.lezc...@linaro.org>
Cc: Rob Herring <robh...@kernel.org>
Cc: Lorenzo Pieralisi <lorenzo.pieral...@arm.com>
Cc: linux...@vger.kernel.org
Cc: Benjamin Herrenschmidt <b...@au1.ibm.com>
Cc: Michael Ellerman <m...@ellerman.id.au>
Cc: Paul Mackerras <pau...@ozlabs.org>
Cc: Michael Neuling <mi...@neuling.org>
Cc: linuxppc-...@lists.ozlabs.org

Shreyas B. Prabhu (11):
  powerpc/powernv: Use PNV_THREAD_WINKLE macro while requesting for
winkle
  powerpc/kvm: make hype

[PATCH v8 01/11] powerpc/powernv: Use PNV_THREAD_WINKLE macro while requesting for winkle

[Shreyas B. Prabhu]

Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
-No changes since v4

Changes in v4
=
- New in v4

 arch/powerpc/kernel/idle_power7.S | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/powerpc/kernel/idle_power7.S 
b/arch/powerpc/kernel/idle_power7.S
index 470ceeb..705c867 100644
--- a/arch/powerpc/kernel/idle_power7.S
+++ b/arch/powerpc/kernel/idle_power7.S
@@ -252,7 +252,7 @@ _GLOBAL(power7_sleep)
/* No return */
 
 _GLOBAL(power7_winkle)
-   li  r3,3
+   li  r3,PNV_THREAD_WINKLE
li  r4,1
b   power7_powersave_common
/* No return */
-- 
2.4.11

[PATCH v8 00/11] powerpc/powernv/cpuidle: Add support for POWER ISA v3 idle states

[Shreyas B. Prabhu]

POWER ISA v3 defines a new idle processor core mechanism. In summary,
 a) new instruction named stop is added. This instruction replaces
instructions like nap, sleep, rvwinkle.
 b) new per thread SPR named PSSCR is added which controls the behavior
of stop instruction. 

PSSCR has following key fields
Bits 0:3  - Power-Saving Level Status. This field indicates the
lowest power-saving state the thread entered since stop
instruction was last executed.

Bit 42 - Enable State Loss  
0 - No state is lost irrespective of other fields  
1 - Allows state loss

Bits 44:47 - Power-Saving Level Limit  
This limits the power-saving level that can be entered into.

Bits 60:63 - Requested Level  
Used to specify which power-saving level must be entered on
executing stop instruction

Stop idle states and their properties like name, latency, target
residency, psscr value are exposed via device tree.

This patch series adds support for this new mechanism.

Patches 1-6 are cleanups and code movement.
Patch 7 adds platform specific support for stop and psscr handling.
Patch 8 and 9 are minor cleanup in cpuidle driver.
Patch 10 adds cpuidle driver support.
Patch 11 makes offlined cpu use deepest stop state.

Note: Documentation for the device tree bindings is posted here-
http://patchwork.ozlabs.org/patch/629125/

Changes in v8
=
 - Fixed a copy-paste mistake in PATCH 10
 - Initializing pnv_first_deep_stop_state
 - Changing MMU_FTR_SECTION condition to reduce FTR section length
 - _loop -> snooze_loop

Changes in v7
=
 - File renamed to idle_book3s.S instead of idle_power_common.S
 - Comment changes
 - power_stop0, power_stop renamed to power9_idle and power_idle_stop
 - PSSCR template is now a macro instead of storing in paca
 - power9_idle in C file instead of assembly
 - Fixed TOC related bug
 - Handling subcore within FTR section
 - Functions in idle.c reordered and broken into multiple functions
 - calling __restore_cpu_power8/9 via cur_cpu_spec->cpu_restore 
 - Added a minor patch with minor cleanups in cpuidle-powernv.c . This
   was mainly to make the existing code consistent with the review
   comments for new code
 - Using stack for variables while probing for idle states instead of
   kzalloc/kcalloc
 - Restoring RPR once per core in P9

Changes in v6
=
 - Restore new POWER ISA v3 SPRS when waking up from deep idle

Changes in v5
=
 - Use generic cpuidle constant CPUIDLE_NAME_LEN
 - Fix return code handling for of_property_read_string_array
 - Use DT flags to determine if are using stop instruction, instead of
   cpu_has_feature
 - Removed uncessary cast with names
 - _loop -> stop_loop
 - Added POWERNV_THRESHOLD_LATENCY_NS to filter out idle states with high 
latency

Changes in v4
=
 - Added a patch to use PNV_THREAD_WINKLE macro while requesting for winkle
 - Moved power7_powersave_common rename to more appropriate patch
 - renaming power7_enter_nap_mode to pnv_enter_arch207_idle_mode
 - Added PSSCR layout to Patch 7's commit message
 - Improved / Fixed comments
 - Fixed whitespace error in paca.h
 - Using MAX_POSSIBLE_STOP_STATE macro instead of hardcoding 0xF has
   max possible stop state

Changes in v3
=
 - Rebased on powerpc-next
 - Dropping patch 1 since we are not adding a new file for P9 idle support
 - Improved comments in multiple places
 - Moved GET_PACA from power7_restore_hyp_resource to System Reset
 - Instead of moving few functions from idle_power7 to idle_power_common,
   renaming idle_power7.S to idle_power_common.S
 - Moved HSTATE_HWTHREAD_STATE updation to power_powersave_common
 - Dropped earlier patch 5 which moved few macros from idle_power_common to
   asm/cpuidle.h. 
 - Added a patch to rename reusable power7_* idle functions to pnv_*
 - Added new patch that creates abstraction for saving SPRs before
   entering deep idle states
 - Instead of introducing new file idle_power_stop.S, P9 idle support
   is added to idle_power_common.S using CPU_FTR sections.
 - Fixed r4 reg clobbering in power_stop0

Changes in v2
=
 - Rebased on v4.6-rc6
 - Using CPU_FTR_ARCH_300 bit instead of CPU_FTR_STOP_INST

Cc: Rafael J. Wysocki 
Cc: Daniel Lezcano 
Cc: Rob Herring 
Cc: Lorenzo Pieralisi 
Cc: linux...@vger.kernel.org
Cc: Benjamin Herrenschmidt 
Cc: Michael Ellerman 
Cc: Paul Mackerras 
Cc: Michael Neuling 
Cc: linuxppc-...@lists.ozlabs.org

Shreyas B. Prabhu (11):
  powerpc/powernv: Use PNV_THREAD_WINKLE macro while requesting for
winkle
  powerpc/kvm: make hypervisor state restore a function
  powerpc/powernv: Rename idle_power7.S to idle_book3s.S
  powerpc/powernv: Rename reusable idle functions to hardware agnostic
names
  powerpc/powernv: Make pnv_powersave_common mor

[PATCH v8 01/11] powerpc/powernv: Use PNV_THREAD_WINKLE macro while requesting for winkle

[Shreyas B. Prabhu]

Signed-off-by: Shreyas B. Prabhu 
---
-No changes since v4

Changes in v4
=
- New in v4

 arch/powerpc/kernel/idle_power7.S | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/powerpc/kernel/idle_power7.S 
b/arch/powerpc/kernel/idle_power7.S
index 470ceeb..705c867 100644
--- a/arch/powerpc/kernel/idle_power7.S
+++ b/arch/powerpc/kernel/idle_power7.S
@@ -252,7 +252,7 @@ _GLOBAL(power7_sleep)
/* No return */
 
 _GLOBAL(power7_winkle)
-   li  r3,3
+   li  r3,PNV_THREAD_WINKLE
li  r4,1
b   power7_powersave_common
/* No return */
-- 
2.4.11

Re: [PATCH v7 07/11] powerpc/powernv: Add platform support for stop instruction

[Shreyas B Prabhu]

On 07/08/2016 07:50 AM, Michael Neuling wrote:
> 
>> diff --git a/arch/powerpc/include/asm/cpuidle.h 
>> b/arch/powerpc/include/asm/cpuidle.h
>> index d2f99ca..3d7fc06 100644
>> --- a/arch/powerpc/include/asm/cpuidle.h
>> +++ b/arch/powerpc/include/asm/cpuidle.h
>> @@ -13,6 +13,8 @@
>>  #ifndef __ASSEMBLY__
>>  extern u32 pnv_fastsleep_workaround_at_entry[];
>>  extern u32 pnv_fastsleep_workaround_at_exit[];
>> +
>> +extern u64 pnv_first_deep_stop_state;
> 
> mpe asked a question about this which you neither answered or addressed.
> "Should this have some safe initial value?"
> 
> I'm thinking we could do this which is what you have in the init call.
>u64 pnv_first_deep_stop_state = MAX_STOP_STATE;
> 

I missed the comment. I'll make the change.
> 
>> @@ -439,7 +540,18 @@ timebase_resync:
>>   */
>>  bne cr4,clear_lock
>>  
>> -/* Restore per core state */
>> +/*
>> + * First thread in the core to wake up and its waking up with
>> + * complete hypervisor state loss. Restore per core hypervisor
>> + * state.
>> + */
>> +BEGIN_FTR_SECTION
>> +ld  r4,_PTCR(r1)
>> +mtspr   SPRN_PTCR,r4
>> +ld  r4,_RPR(r1)
>> +mtspr   SPRN_RPR,r4
> 
> RPR looks wrong here.  This should be on POWER8 too.
> 
> This has changed since v6 and not noted in the v7 comments.  Why are you
> changing this now?
> 
RPR is a per-core resource in P9. So with this patch, RPR will continue
to be restored per-subcore in P8 and will restored once per core in P9.

>> +END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300)
>> +
>>  ld  r4,_TSCR(r1)
>>  mtspr   SPRN_TSCR,r4
>>  ld  r4,_WORC(r1)
>> @@ -461,9 +573,7 @@ common_exit:
>>  
>>  /* Waking up from winkle */
>>  
>> -/* Restore per thread state */
>> -bl  __restore_cpu_power8
>> -
>> +BEGIN_MMU_FTR_SECTION
>>  /* Restore SLB  from PACA */
>>  ld  r8,PACA_SLBSHADOWPTR(r13)
>>  
>> @@ -477,6 +587,9 @@ common_exit:
>>  slbmte  r6,r5
>>  1:  addir8,r8,16
>>  .endr
>> +END_MMU_FTR_SECTION_IFCLR(MMU_FTR_RADIX)
>> +
>> +/* Restore per thread state */
> 
> This FTR section is too big  It ends up at 25 instructions with the loop.
> Probably better like this:
> 
> BEGIN_MMU_FTR_SECTION
>   b   no_segments
> END_MMU_FTR_SECTION_IFSET(MMU_FTR_RADIX)
>   /* Restore SLB  from PACA */
>   ld  r8,PACA_SLBSHADOWPTR(r13)
> 
>   .rept   SLB_NUM_BOLTED
>   li  r3, SLBSHADOW_SAVEAREA
>   LDX_BE  r5, r8, r3
>   addir3, r3, 8
>   LDX_BE  r6, r8, r3
>   andis.  r7,r5,SLB_ESID_V@h
>   beq 1f
>   slbmte  r6,r5
> 1:addir8,r8,16
>   .endr
> 
> no_segments:
> 
Cool. Will make the change.

Thanks,
Shreyas

Re: [PATCH v7 07/11] powerpc/powernv: Add platform support for stop instruction

[Shreyas B Prabhu]

On 07/08/2016 07:50 AM, Michael Neuling wrote:
> 
>> diff --git a/arch/powerpc/include/asm/cpuidle.h 
>> b/arch/powerpc/include/asm/cpuidle.h
>> index d2f99ca..3d7fc06 100644
>> --- a/arch/powerpc/include/asm/cpuidle.h
>> +++ b/arch/powerpc/include/asm/cpuidle.h
>> @@ -13,6 +13,8 @@
>>  #ifndef __ASSEMBLY__
>>  extern u32 pnv_fastsleep_workaround_at_entry[];
>>  extern u32 pnv_fastsleep_workaround_at_exit[];
>> +
>> +extern u64 pnv_first_deep_stop_state;
> 
> mpe asked a question about this which you neither answered or addressed.
> "Should this have some safe initial value?"
> 
> I'm thinking we could do this which is what you have in the init call.
>u64 pnv_first_deep_stop_state = MAX_STOP_STATE;
> 

I missed the comment. I'll make the change.
> 
>> @@ -439,7 +540,18 @@ timebase_resync:
>>   */
>>  bne cr4,clear_lock
>>  
>> -/* Restore per core state */
>> +/*
>> + * First thread in the core to wake up and its waking up with
>> + * complete hypervisor state loss. Restore per core hypervisor
>> + * state.
>> + */
>> +BEGIN_FTR_SECTION
>> +ld  r4,_PTCR(r1)
>> +mtspr   SPRN_PTCR,r4
>> +ld  r4,_RPR(r1)
>> +mtspr   SPRN_RPR,r4
> 
> RPR looks wrong here.  This should be on POWER8 too.
> 
> This has changed since v6 and not noted in the v7 comments.  Why are you
> changing this now?
> 
RPR is a per-core resource in P9. So with this patch, RPR will continue
to be restored per-subcore in P8 and will restored once per core in P9.

>> +END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300)
>> +
>>  ld  r4,_TSCR(r1)
>>  mtspr   SPRN_TSCR,r4
>>  ld  r4,_WORC(r1)
>> @@ -461,9 +573,7 @@ common_exit:
>>  
>>  /* Waking up from winkle */
>>  
>> -/* Restore per thread state */
>> -bl  __restore_cpu_power8
>> -
>> +BEGIN_MMU_FTR_SECTION
>>  /* Restore SLB  from PACA */
>>  ld  r8,PACA_SLBSHADOWPTR(r13)
>>  
>> @@ -477,6 +587,9 @@ common_exit:
>>  slbmte  r6,r5
>>  1:  addir8,r8,16
>>  .endr
>> +END_MMU_FTR_SECTION_IFCLR(MMU_FTR_RADIX)
>> +
>> +/* Restore per thread state */
> 
> This FTR section is too big  It ends up at 25 instructions with the loop.
> Probably better like this:
> 
> BEGIN_MMU_FTR_SECTION
>   b   no_segments
> END_MMU_FTR_SECTION_IFSET(MMU_FTR_RADIX)
>   /* Restore SLB  from PACA */
>   ld  r8,PACA_SLBSHADOWPTR(r13)
> 
>   .rept   SLB_NUM_BOLTED
>   li  r3, SLBSHADOW_SAVEAREA
>   LDX_BE  r5, r8, r3
>   addir3, r3, 8
>   LDX_BE  r6, r8, r3
>   andis.  r7,r5,SLB_ESID_V@h
>   beq 1f
>   slbmte  r6,r5
> 1:addir8,r8,16
>   .endr
> 
> no_segments:
> 
Cool. Will make the change.

Thanks,
Shreyas

[PATCH v7 06/11] powerpc/powernv: abstraction for saving SPRs before entering deep idle states

[Shreyas B. Prabhu]

Create a function for saving SPRs before entering deep idle states.
This function can be reused for POWER9 deep idle states.

Reviewed-by: Gautham R. Shenoy <e...@linux.vnet.ibm.com>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
 - No changes since v3

Changes in v3:
=
 - Newly added in v3

 arch/powerpc/kernel/idle_book3s.S | 54 +++
 1 file changed, 32 insertions(+), 22 deletions(-)

diff --git a/arch/powerpc/kernel/idle_book3s.S 
b/arch/powerpc/kernel/idle_book3s.S
index a8397e3..2f909a1 100644
--- a/arch/powerpc/kernel/idle_book3s.S
+++ b/arch/powerpc/kernel/idle_book3s.S
@@ -53,6 +53,36 @@
.text
 
 /*
+ * Used by threads before entering deep idle states. Saves SPRs
+ * in interrupt stack frame
+ */
+save_sprs_to_stack:
+   /*
+* Note all register i.e per-core, per-subcore or per-thread is saved
+* here since any thread in the core might wake up first
+*/
+   mfspr   r3,SPRN_SDR1
+   std r3,_SDR1(r1)
+   mfspr   r3,SPRN_RPR
+   std r3,_RPR(r1)
+   mfspr   r3,SPRN_SPURR
+   std r3,_SPURR(r1)
+   mfspr   r3,SPRN_PURR
+   std r3,_PURR(r1)
+   mfspr   r3,SPRN_TSCR
+   std r3,_TSCR(r1)
+   mfspr   r3,SPRN_DSCR
+   std r3,_DSCR(r1)
+   mfspr   r3,SPRN_AMOR
+   std r3,_AMOR(r1)
+   mfspr   r3,SPRN_WORT
+   std r3,_WORT(r1)
+   mfspr   r3,SPRN_WORC
+   std r3,_WORC(r1)
+
+   blr
+
+/*
  * Used by threads when the lock bit of core_idle_state is set.
  * Threads will spin in HMT_LOW until the lock bit is cleared.
  * r14 - pointer to core_idle_state
@@ -209,28 +239,8 @@ fastsleep_workaround_at_entry:
b   common_enter
 
 enter_winkle:
-   /*
-* Note all register i.e per-core, per-subcore or per-thread is saved
-* here since any thread in the core might wake up first
-*/
-   mfspr   r3,SPRN_SDR1
-   std r3,_SDR1(r1)
-   mfspr   r3,SPRN_RPR
-   std r3,_RPR(r1)
-   mfspr   r3,SPRN_SPURR
-   std r3,_SPURR(r1)
-   mfspr   r3,SPRN_PURR
-   std r3,_PURR(r1)
-   mfspr   r3,SPRN_TSCR
-   std r3,_TSCR(r1)
-   mfspr   r3,SPRN_DSCR
-   std r3,_DSCR(r1)
-   mfspr   r3,SPRN_AMOR
-   std r3,_AMOR(r1)
-   mfspr   r3,SPRN_WORT
-   std r3,_WORT(r1)
-   mfspr   r3,SPRN_WORC
-   std r3,_WORC(r1)
+   bl  save_sprs_to_stack
+
IDLE_STATE_ENTER_SEQ(PPC_WINKLE)
 
 _GLOBAL(power7_idle)
-- 
2.4.11

[PATCH v7 05/11] powerpc/powernv: Make pnv_powersave_common more generic

[Shreyas B. Prabhu]

pnv_powersave_common does common steps needed before entering idle
state and eventually changes MSR to MSR_IDLE and does rfid to
pnv_enter_arch207_idle_mode.

Move the updation of HSTATE_HWTHREAD_STATE to pnv_powersave_common
from pnv_enter_arch207_idle_mode and make it more generic by passing the
rfid address as a function parameter.

Reviewed-by: Gautham R. Shenoy <e...@linux.vnet.ibm.com>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
 - No changes since v4

Changes in v4:
==
 - Moved renaming of power7_powersave_common to earlier patch

Changes in v3:
==
 - Moved HSTATE_HWTHREAD_STATE updation to power_powersave_common

 arch/powerpc/kernel/idle_book3s.S | 23 ++-
 1 file changed, 14 insertions(+), 9 deletions(-)

diff --git a/arch/powerpc/kernel/idle_book3s.S 
b/arch/powerpc/kernel/idle_book3s.S
index 34dbfc9..a8397e3 100644
--- a/arch/powerpc/kernel/idle_book3s.S
+++ b/arch/powerpc/kernel/idle_book3s.S
@@ -75,6 +75,8 @@ core_idle_lock_held:
  * To check IRQ_HAPPENED in r4
  * 0 - don't check
  * 1 - check
+ *
+ * Address to 'rfid' to in r5
  */
 _GLOBAL(pnv_powersave_common)
/* Use r3 to pass state nap/sleep/winkle */
@@ -127,28 +129,28 @@ _GLOBAL(pnv_powersave_common)
std r9,_MSR(r1)
std r1,PACAR1(r13)
 
+#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
+   /* Tell KVM we're entering idle */
+   li  r4,KVM_HWTHREAD_IN_NAP
+   stb r4,HSTATE_HWTHREAD_STATE(r13)
+#endif
+
/*
 * Go to real mode to do the nap, as required by the architecture.
 * Also, we need to be in real mode before setting hwthread_state,
 * because as soon as we do that, another thread can switch
 * the MMU context to the guest.
 */
-   LOAD_REG_IMMEDIATE(r5, MSR_IDLE)
+   LOAD_REG_IMMEDIATE(r7, MSR_IDLE)
li  r6, MSR_RI
andcr6, r9, r6
-   LOAD_REG_ADDR(r7, pnv_enter_arch207_idle_mode)
mtmsrd  r6, 1   /* clear RI before setting SRR0/1 */
-   mtspr   SPRN_SRR0, r7
-   mtspr   SPRN_SRR1, r5
+   mtspr   SPRN_SRR0, r5
+   mtspr   SPRN_SRR1, r7
rfid
 
.globl pnv_enter_arch207_idle_mode
 pnv_enter_arch207_idle_mode:
-#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
-   /* Tell KVM we're napping */
-   li  r4,KVM_HWTHREAD_IN_NAP
-   stb r4,HSTATE_HWTHREAD_STATE(r13)
-#endif
stb r3,PACA_THREAD_IDLE_STATE(r13)
cmpwi   cr3,r3,PNV_THREAD_SLEEP
bge cr3,2f
@@ -243,18 +245,21 @@ _GLOBAL(power7_idle)
 _GLOBAL(power7_nap)
mr  r4,r3
li  r3,PNV_THREAD_NAP
+   LOAD_REG_ADDR(r5, pnv_enter_arch207_idle_mode)
b   pnv_powersave_common
/* No return */
 
 _GLOBAL(power7_sleep)
li  r3,PNV_THREAD_SLEEP
li  r4,1
+   LOAD_REG_ADDR(r5, pnv_enter_arch207_idle_mode)
b   pnv_powersave_common
/* No return */
 
 _GLOBAL(power7_winkle)
li  r3,PNV_THREAD_WINKLE
li  r4,1
+   LOAD_REG_ADDR(r5, pnv_enter_arch207_idle_mode)
b   pnv_powersave_common
/* No return */
 
-- 
2.4.11

[PATCH v7 06/11] powerpc/powernv: abstraction for saving SPRs before entering deep idle states

[Shreyas B. Prabhu]

Create a function for saving SPRs before entering deep idle states.
This function can be reused for POWER9 deep idle states.

Reviewed-by: Gautham R. Shenoy 
Signed-off-by: Shreyas B. Prabhu 
---
 - No changes since v3

Changes in v3:
=
 - Newly added in v3

 arch/powerpc/kernel/idle_book3s.S | 54 +++
 1 file changed, 32 insertions(+), 22 deletions(-)

diff --git a/arch/powerpc/kernel/idle_book3s.S 
b/arch/powerpc/kernel/idle_book3s.S
index a8397e3..2f909a1 100644
--- a/arch/powerpc/kernel/idle_book3s.S
+++ b/arch/powerpc/kernel/idle_book3s.S
@@ -53,6 +53,36 @@
.text
 
 /*
+ * Used by threads before entering deep idle states. Saves SPRs
+ * in interrupt stack frame
+ */
+save_sprs_to_stack:
+   /*
+* Note all register i.e per-core, per-subcore or per-thread is saved
+* here since any thread in the core might wake up first
+*/
+   mfspr   r3,SPRN_SDR1
+   std r3,_SDR1(r1)
+   mfspr   r3,SPRN_RPR
+   std r3,_RPR(r1)
+   mfspr   r3,SPRN_SPURR
+   std r3,_SPURR(r1)
+   mfspr   r3,SPRN_PURR
+   std r3,_PURR(r1)
+   mfspr   r3,SPRN_TSCR
+   std r3,_TSCR(r1)
+   mfspr   r3,SPRN_DSCR
+   std r3,_DSCR(r1)
+   mfspr   r3,SPRN_AMOR
+   std r3,_AMOR(r1)
+   mfspr   r3,SPRN_WORT
+   std r3,_WORT(r1)
+   mfspr   r3,SPRN_WORC
+   std r3,_WORC(r1)
+
+   blr
+
+/*
  * Used by threads when the lock bit of core_idle_state is set.
  * Threads will spin in HMT_LOW until the lock bit is cleared.
  * r14 - pointer to core_idle_state
@@ -209,28 +239,8 @@ fastsleep_workaround_at_entry:
b   common_enter
 
 enter_winkle:
-   /*
-* Note all register i.e per-core, per-subcore or per-thread is saved
-* here since any thread in the core might wake up first
-*/
-   mfspr   r3,SPRN_SDR1
-   std r3,_SDR1(r1)
-   mfspr   r3,SPRN_RPR
-   std r3,_RPR(r1)
-   mfspr   r3,SPRN_SPURR
-   std r3,_SPURR(r1)
-   mfspr   r3,SPRN_PURR
-   std r3,_PURR(r1)
-   mfspr   r3,SPRN_TSCR
-   std r3,_TSCR(r1)
-   mfspr   r3,SPRN_DSCR
-   std r3,_DSCR(r1)
-   mfspr   r3,SPRN_AMOR
-   std r3,_AMOR(r1)
-   mfspr   r3,SPRN_WORT
-   std r3,_WORT(r1)
-   mfspr   r3,SPRN_WORC
-   std r3,_WORC(r1)
+   bl  save_sprs_to_stack
+
IDLE_STATE_ENTER_SEQ(PPC_WINKLE)
 
 _GLOBAL(power7_idle)
-- 
2.4.11

[PATCH v7 05/11] powerpc/powernv: Make pnv_powersave_common more generic

[Shreyas B. Prabhu]

pnv_powersave_common does common steps needed before entering idle
state and eventually changes MSR to MSR_IDLE and does rfid to
pnv_enter_arch207_idle_mode.

Move the updation of HSTATE_HWTHREAD_STATE to pnv_powersave_common
from pnv_enter_arch207_idle_mode and make it more generic by passing the
rfid address as a function parameter.

Reviewed-by: Gautham R. Shenoy 
Signed-off-by: Shreyas B. Prabhu 
---
 - No changes since v4

Changes in v4:
==
 - Moved renaming of power7_powersave_common to earlier patch

Changes in v3:
==
 - Moved HSTATE_HWTHREAD_STATE updation to power_powersave_common

 arch/powerpc/kernel/idle_book3s.S | 23 ++-
 1 file changed, 14 insertions(+), 9 deletions(-)

diff --git a/arch/powerpc/kernel/idle_book3s.S 
b/arch/powerpc/kernel/idle_book3s.S
index 34dbfc9..a8397e3 100644
--- a/arch/powerpc/kernel/idle_book3s.S
+++ b/arch/powerpc/kernel/idle_book3s.S
@@ -75,6 +75,8 @@ core_idle_lock_held:
  * To check IRQ_HAPPENED in r4
  * 0 - don't check
  * 1 - check
+ *
+ * Address to 'rfid' to in r5
  */
 _GLOBAL(pnv_powersave_common)
/* Use r3 to pass state nap/sleep/winkle */
@@ -127,28 +129,28 @@ _GLOBAL(pnv_powersave_common)
std r9,_MSR(r1)
std r1,PACAR1(r13)
 
+#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
+   /* Tell KVM we're entering idle */
+   li  r4,KVM_HWTHREAD_IN_NAP
+   stb r4,HSTATE_HWTHREAD_STATE(r13)
+#endif
+
/*
 * Go to real mode to do the nap, as required by the architecture.
 * Also, we need to be in real mode before setting hwthread_state,
 * because as soon as we do that, another thread can switch
 * the MMU context to the guest.
 */
-   LOAD_REG_IMMEDIATE(r5, MSR_IDLE)
+   LOAD_REG_IMMEDIATE(r7, MSR_IDLE)
li  r6, MSR_RI
andcr6, r9, r6
-   LOAD_REG_ADDR(r7, pnv_enter_arch207_idle_mode)
mtmsrd  r6, 1   /* clear RI before setting SRR0/1 */
-   mtspr   SPRN_SRR0, r7
-   mtspr   SPRN_SRR1, r5
+   mtspr   SPRN_SRR0, r5
+   mtspr   SPRN_SRR1, r7
rfid
 
.globl pnv_enter_arch207_idle_mode
 pnv_enter_arch207_idle_mode:
-#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
-   /* Tell KVM we're napping */
-   li  r4,KVM_HWTHREAD_IN_NAP
-   stb r4,HSTATE_HWTHREAD_STATE(r13)
-#endif
stb r3,PACA_THREAD_IDLE_STATE(r13)
cmpwi   cr3,r3,PNV_THREAD_SLEEP
bge cr3,2f
@@ -243,18 +245,21 @@ _GLOBAL(power7_idle)
 _GLOBAL(power7_nap)
mr  r4,r3
li  r3,PNV_THREAD_NAP
+   LOAD_REG_ADDR(r5, pnv_enter_arch207_idle_mode)
b   pnv_powersave_common
/* No return */
 
 _GLOBAL(power7_sleep)
li  r3,PNV_THREAD_SLEEP
li  r4,1
+   LOAD_REG_ADDR(r5, pnv_enter_arch207_idle_mode)
b   pnv_powersave_common
/* No return */
 
 _GLOBAL(power7_winkle)
li  r3,PNV_THREAD_WINKLE
li  r4,1
+   LOAD_REG_ADDR(r5, pnv_enter_arch207_idle_mode)
b   pnv_powersave_common
/* No return */
 
-- 
2.4.11

[PATCH v7 03/11] powerpc/powernv: Rename idle_power7.S to idle_book3s.S

[Shreyas B. Prabhu]

idle_power7.S handles idle entry/exit for POWER7, POWER8 and in next
patch for POWER9. Rename the file to a non-hardware specific
name.

Reviewed-by: Gautham R. Shenoy <e...@linux.vnet.ibm.com>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
Changes in v7:
=
 - File renamed to idle_book3s.S instead of idle_power_common.S
Changes in v3:
==
 - Instead of moving few common functions from idle_power7.S to
   idle_power_common.S, renaming idle_power7.S to idle_power_common.S

 arch/powerpc/kernel/Makefile  |   2 +-
 arch/powerpc/kernel/idle_book3s.S | 527 ++
 arch/powerpc/kernel/idle_power7.S | 527 --
 3 files changed, 528 insertions(+), 528 deletions(-)
 create mode 100644 arch/powerpc/kernel/idle_book3s.S
 delete mode 100644 arch/powerpc/kernel/idle_power7.S

diff --git a/arch/powerpc/kernel/Makefile b/arch/powerpc/kernel/Makefile
index 2da380f..9e7bfc32 100644
--- a/arch/powerpc/kernel/Makefile
+++ b/arch/powerpc/kernel/Makefile
@@ -47,7 +47,7 @@ obj-$(CONFIG_PPC_BOOK3E_64)   += exceptions-64e.o 
idle_book3e.o
 obj-$(CONFIG_PPC64)+= vdso64/
 obj-$(CONFIG_ALTIVEC)  += vecemu.o
 obj-$(CONFIG_PPC_970_NAP)  += idle_power4.o
-obj-$(CONFIG_PPC_P7_NAP)   += idle_power7.o
+obj-$(CONFIG_PPC_P7_NAP)   += idle_book3s.o
 procfs-y   := proc_powerpc.o
 obj-$(CONFIG_PROC_FS)  += $(procfs-y)
 rtaspci-$(CONFIG_PPC64)-$(CONFIG_PCI)  := rtas_pci.o
diff --git a/arch/powerpc/kernel/idle_book3s.S 
b/arch/powerpc/kernel/idle_book3s.S
new file mode 100644
index 000..d5def06
--- /dev/null
+++ b/arch/powerpc/kernel/idle_book3s.S
@@ -0,0 +1,527 @@
+/*
+ *  This file contains the power_save function for Power7 CPUs.
+ *
+ *  This program is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU General Public License
+ *  as published by the Free Software Foundation; either version
+ *  2 of the License, or (at your option) any later version.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#undef DEBUG
+
+/*
+ * Use unused space in the interrupt stack to save and restore
+ * registers for winkle support.
+ */
+#define _SDR1  GPR3
+#define _RPR   GPR4
+#define _SPURR GPR5
+#define _PURR  GPR6
+#define _TSCR  GPR7
+#define _DSCR  GPR8
+#define _AMOR  GPR9
+#define _WORT  GPR10
+#define _WORC  GPR11
+
+/* Idle state entry routines */
+
+#defineIDLE_STATE_ENTER_SEQ(IDLE_INST) \
+   /* Magic NAP/SLEEP/WINKLE mode enter sequence */\
+   std r0,0(r1);   \
+   ptesync;\
+   ld  r0,0(r1);   \
+1: cmp cr0,r0,r0;  \
+   bne 1b; \
+   IDLE_INST;  \
+   b   .
+
+   .text
+
+/*
+ * Used by threads when the lock bit of core_idle_state is set.
+ * Threads will spin in HMT_LOW until the lock bit is cleared.
+ * r14 - pointer to core_idle_state
+ * r15 - used to load contents of core_idle_state
+ */
+
+core_idle_lock_held:
+   HMT_LOW
+3: lwz r15,0(r14)
+   andi.   r15,r15,PNV_CORE_IDLE_LOCK_BIT
+   bne 3b
+   HMT_MEDIUM
+   lwarx   r15,0,r14
+   blr
+
+/*
+ * Pass requested state in r3:
+ * r3 - PNV_THREAD_NAP/SLEEP/WINKLE
+ *
+ * To check IRQ_HAPPENED in r4
+ * 0 - don't check
+ * 1 - check
+ */
+_GLOBAL(power7_powersave_common)
+   /* Use r3 to pass state nap/sleep/winkle */
+   /* NAP is a state loss, we create a regs frame on the
+* stack, fill it up with the state we care about and
+* stick a pointer to it in PACAR1. We really only
+* need to save PC, some CR bits and the NV GPRs,
+* but for now an interrupt frame will do.
+*/
+   mflrr0
+   std r0,16(r1)
+   stdur1,-INT_FRAME_SIZE(r1)
+   std r0,_LINK(r1)
+   std r0,_NIP(r1)
+
+   /* Hard disable interrupts */
+   mfmsr   r9
+   rldicl  r9,r9,48,1
+   rotldi  r9,r9,16
+   mtmsrd  r9,1/* hard-disable interrupts */
+
+   /* Check if something happened while soft-disabled */
+   lbz r0,PACAIRQHAPPENED(r13)
+   andi.   r0,r0,~PACA_IRQ_HARD_DIS@l
+   beq 1f
+   cmpwi   cr0,r4,0
+   beq 1f
+   addir1,r1,INT_FRAME_SIZE
+   ld  r0,16(r1)
+   li  r3,0/* Return 0 (no nap) */
+   mtlrr0
+   blr
+
+1: /* We mark irqs hard disabled as this is the state we'll
+* be in when returning and we need to tell arch_local_irq_restore()
+* about it
+*/
+ 

[PATCH v7 10/11] cpuidle/powernv: Add support for POWER ISA v3 idle states

[Shreyas B. Prabhu]

POWER ISA v3 defines a new idle processor core mechanism. In summary,
 a) new instruction named stop is added.
 b) new per thread SPR named PSSCR is added which controls the behavior
of stop instruction.

Supported idle states and value to be written to PSSCR register to enter
any idle state is exposed via ibm,cpu-idle-state-names and
ibm,cpu-idle-state-psscr respectively. To enter an idle state,
platform provided power_stop() needs to be invoked with the appropriate
PSSCR value.

This patch adds support for this new mechanism in cpuidle powernv driver.

Cc: Rafael J. Wysocki <rafael.j.wyso...@intel.com>
Cc: Daniel Lezcano <daniel.lezc...@linaro.org>
Cc: Rob Herring <robh...@kernel.org>
Cc: Lorenzo Pieralisi <lorenzo.pieral...@arm.com>
Cc: linux...@vger.kernel.org
Cc: Michael Ellerman <m...@ellerman.id.au>
Cc: Paul Mackerras <pau...@ozlabs.org>
Cc: linuxppc-...@lists.ozlabs.org
Reviewed-by: Gautham R. Shenoy <e...@linux.vnet.ibm.com>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
Note: Documentation for the device tree bindings is posted here-
http://patchwork.ozlabs.org/patch/629125/

Changes in v7
=
 - Using stack instead kzalloc/kcalloc 

Changes in v5
=
 - Use generic cpuidle constant CPUIDLE_NAME_LEN
 - Fix return code handling for of_property_read_string_array
 - Use DT flags to determine if are using stop instruction, instead of
   cpu_has_feature
 - Removed uncessary cast with names
 - _loop -> stop_loop
 - Added POWERNV_THRESHOLD_LATENCY_NS to filter out idle states with high 
latency

 drivers/cpuidle/cpuidle-powernv.c | 61 +++
 1 file changed, 61 insertions(+)

diff --git a/drivers/cpuidle/cpuidle-powernv.c 
b/drivers/cpuidle/cpuidle-powernv.c
index a89f546..d27e955 100644
--- a/drivers/cpuidle/cpuidle-powernv.c
+++ b/drivers/cpuidle/cpuidle-powernv.c
@@ -20,6 +20,8 @@
 #include 
 #include 
 
+#define POWERNV_THRESHOLD_LATENCY_NS 20
+
 struct cpuidle_driver powernv_idle_driver = {
.name = "powernv_idle",
.owner= THIS_MODULE,
@@ -27,6 +29,9 @@ struct cpuidle_driver powernv_idle_driver = {
 
 static int max_idle_state;
 static struct cpuidle_state *cpuidle_state_table;
+
+static u64 stop_psscr_table[CPUIDLE_STATE_MAX];
+
 static u64 snooze_timeout;
 static bool snooze_timeout_en;
 
@@ -91,6 +96,17 @@ static int fastsleep_loop(struct cpuidle_device *dev,
return index;
 }
 #endif
+
+static int stop_loop(struct cpuidle_device *dev,
+struct cpuidle_driver *drv,
+int index)
+{
+   ppc64_runlatch_off();
+   power9_idle_stop(stop_psscr_table[index]);
+   ppc64_runlatch_on();
+   return index;
+}
+
 /*
  * States for dedicated partition case.
  */
@@ -169,6 +185,8 @@ static int powernv_add_idle_states(void)
u32 latency_ns[CPUIDLE_STATE_MAX];
u32 residency_ns[CPUIDLE_STATE_MAX];
u32 flags[CPUIDLE_STATE_MAX];
+   u64 psscr_val[CPUIDLE_STATE_MAX];
+   const char *names[CPUIDLE_STATE_MAX];
int i, rc;
 
/* Currently we have snooze statically defined */
@@ -207,11 +225,34 @@ static int powernv_add_idle_states(void)
pr_warn("cpuidle-powernv: missing 
ibm,cpu-idle-state-latencies-ns in DT\n");
goto out;
}
+   if (of_property_read_string_array(power_mgt,
+   "ibm,cpu-idle-state-names", names, dt_idle_states)) {
+   pr_warn("cpuidle-powernv: missing ibm,cpu-idle-state-names in 
DT\n");
+   goto out;
+   }
+
+   /*
+* If the idle states use stop instruction, probe for psscr values
+* which are necessary to specify required stop level.
+*/
+   if (flags[0] & (OPAL_PM_STOP_INST_FAST | OPAL_PM_STOP_INST_DEEP))
+   if (of_property_read_u64_array(power_mgt,
+   "ibm,cpu-idle-state-psscr", psscr_val, dt_idle_states)) {
+   pr_warn("cpuidle-powernv: missing 
ibm,cpu-idle-states-psscr in DT\n");
+   goto out;
+   }
 
rc = of_property_read_u32_array(power_mgt,
"ibm,cpu-idle-state-residency-ns", residency_ns, 
dt_idle_states);
 
for (i = 0; i < dt_idle_states; i++) {
+   /*
+* If an idle state has exit latency beyond
+* POWERNV_THRESHOLD_LATENCY_NS then don't use it
+* in cpu-idle.
+*/
+   if (latency_ns[i] > POWERNV_THRESHOLD_LATENCY_NS)
+   continue;
 
/*
 * Cpuidle accepts exit_latency and target_residency in us.
@@ -224,6 +265,16 @@ static int powernv_add_idle_states(void)
powernv_states[nr_idle_states].flags = 0;
powernv_states[nr_idle_states].t

[PATCH v7 03/11] powerpc/powernv: Rename idle_power7.S to idle_book3s.S

[Shreyas B. Prabhu]

idle_power7.S handles idle entry/exit for POWER7, POWER8 and in next
patch for POWER9. Rename the file to a non-hardware specific
name.

Reviewed-by: Gautham R. Shenoy 
Signed-off-by: Shreyas B. Prabhu 
---
Changes in v7:
=
 - File renamed to idle_book3s.S instead of idle_power_common.S
Changes in v3:
==
 - Instead of moving few common functions from idle_power7.S to
   idle_power_common.S, renaming idle_power7.S to idle_power_common.S

 arch/powerpc/kernel/Makefile  |   2 +-
 arch/powerpc/kernel/idle_book3s.S | 527 ++
 arch/powerpc/kernel/idle_power7.S | 527 --
 3 files changed, 528 insertions(+), 528 deletions(-)
 create mode 100644 arch/powerpc/kernel/idle_book3s.S
 delete mode 100644 arch/powerpc/kernel/idle_power7.S

diff --git a/arch/powerpc/kernel/Makefile b/arch/powerpc/kernel/Makefile
index 2da380f..9e7bfc32 100644
--- a/arch/powerpc/kernel/Makefile
+++ b/arch/powerpc/kernel/Makefile
@@ -47,7 +47,7 @@ obj-$(CONFIG_PPC_BOOK3E_64)   += exceptions-64e.o 
idle_book3e.o
 obj-$(CONFIG_PPC64)+= vdso64/
 obj-$(CONFIG_ALTIVEC)  += vecemu.o
 obj-$(CONFIG_PPC_970_NAP)  += idle_power4.o
-obj-$(CONFIG_PPC_P7_NAP)   += idle_power7.o
+obj-$(CONFIG_PPC_P7_NAP)   += idle_book3s.o
 procfs-y   := proc_powerpc.o
 obj-$(CONFIG_PROC_FS)  += $(procfs-y)
 rtaspci-$(CONFIG_PPC64)-$(CONFIG_PCI)  := rtas_pci.o
diff --git a/arch/powerpc/kernel/idle_book3s.S 
b/arch/powerpc/kernel/idle_book3s.S
new file mode 100644
index 000..d5def06
--- /dev/null
+++ b/arch/powerpc/kernel/idle_book3s.S
@@ -0,0 +1,527 @@
+/*
+ *  This file contains the power_save function for Power7 CPUs.
+ *
+ *  This program is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU General Public License
+ *  as published by the Free Software Foundation; either version
+ *  2 of the License, or (at your option) any later version.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#undef DEBUG
+
+/*
+ * Use unused space in the interrupt stack to save and restore
+ * registers for winkle support.
+ */
+#define _SDR1  GPR3
+#define _RPR   GPR4
+#define _SPURR GPR5
+#define _PURR  GPR6
+#define _TSCR  GPR7
+#define _DSCR  GPR8
+#define _AMOR  GPR9
+#define _WORT  GPR10
+#define _WORC  GPR11
+
+/* Idle state entry routines */
+
+#defineIDLE_STATE_ENTER_SEQ(IDLE_INST) \
+   /* Magic NAP/SLEEP/WINKLE mode enter sequence */\
+   std r0,0(r1);   \
+   ptesync;\
+   ld  r0,0(r1);   \
+1: cmp cr0,r0,r0;  \
+   bne 1b; \
+   IDLE_INST;  \
+   b   .
+
+   .text
+
+/*
+ * Used by threads when the lock bit of core_idle_state is set.
+ * Threads will spin in HMT_LOW until the lock bit is cleared.
+ * r14 - pointer to core_idle_state
+ * r15 - used to load contents of core_idle_state
+ */
+
+core_idle_lock_held:
+   HMT_LOW
+3: lwz r15,0(r14)
+   andi.   r15,r15,PNV_CORE_IDLE_LOCK_BIT
+   bne 3b
+   HMT_MEDIUM
+   lwarx   r15,0,r14
+   blr
+
+/*
+ * Pass requested state in r3:
+ * r3 - PNV_THREAD_NAP/SLEEP/WINKLE
+ *
+ * To check IRQ_HAPPENED in r4
+ * 0 - don't check
+ * 1 - check
+ */
+_GLOBAL(power7_powersave_common)
+   /* Use r3 to pass state nap/sleep/winkle */
+   /* NAP is a state loss, we create a regs frame on the
+* stack, fill it up with the state we care about and
+* stick a pointer to it in PACAR1. We really only
+* need to save PC, some CR bits and the NV GPRs,
+* but for now an interrupt frame will do.
+*/
+   mflrr0
+   std r0,16(r1)
+   stdur1,-INT_FRAME_SIZE(r1)
+   std r0,_LINK(r1)
+   std r0,_NIP(r1)
+
+   /* Hard disable interrupts */
+   mfmsr   r9
+   rldicl  r9,r9,48,1
+   rotldi  r9,r9,16
+   mtmsrd  r9,1/* hard-disable interrupts */
+
+   /* Check if something happened while soft-disabled */
+   lbz r0,PACAIRQHAPPENED(r13)
+   andi.   r0,r0,~PACA_IRQ_HARD_DIS@l
+   beq 1f
+   cmpwi   cr0,r4,0
+   beq 1f
+   addir1,r1,INT_FRAME_SIZE
+   ld  r0,16(r1)
+   li  r3,0/* Return 0 (no nap) */
+   mtlrr0
+   blr
+
+1: /* We mark irqs hard disabled as this is the state we'll
+* be in when returning and we need to tell arch_local_irq_restore()
+* about it
+*/
+   li  r0,PACA_IRQ_HARD_DIS
+   stb r0,PACAIRQHAPPENED(r13

[PATCH v7 10/11] cpuidle/powernv: Add support for POWER ISA v3 idle states

[Shreyas B. Prabhu]

POWER ISA v3 defines a new idle processor core mechanism. In summary,
 a) new instruction named stop is added.
 b) new per thread SPR named PSSCR is added which controls the behavior
of stop instruction.

Supported idle states and value to be written to PSSCR register to enter
any idle state is exposed via ibm,cpu-idle-state-names and
ibm,cpu-idle-state-psscr respectively. To enter an idle state,
platform provided power_stop() needs to be invoked with the appropriate
PSSCR value.

This patch adds support for this new mechanism in cpuidle powernv driver.

Cc: Rafael J. Wysocki 
Cc: Daniel Lezcano 
Cc: Rob Herring 
Cc: Lorenzo Pieralisi 
Cc: linux...@vger.kernel.org
Cc: Michael Ellerman 
Cc: Paul Mackerras 
Cc: linuxppc-...@lists.ozlabs.org
Reviewed-by: Gautham R. Shenoy 
Signed-off-by: Shreyas B. Prabhu 
---
Note: Documentation for the device tree bindings is posted here-
http://patchwork.ozlabs.org/patch/629125/

Changes in v7
=
 - Using stack instead kzalloc/kcalloc 

Changes in v5
=
 - Use generic cpuidle constant CPUIDLE_NAME_LEN
 - Fix return code handling for of_property_read_string_array
 - Use DT flags to determine if are using stop instruction, instead of
   cpu_has_feature
 - Removed uncessary cast with names
 - _loop -> stop_loop
 - Added POWERNV_THRESHOLD_LATENCY_NS to filter out idle states with high 
latency

 drivers/cpuidle/cpuidle-powernv.c | 61 +++
 1 file changed, 61 insertions(+)

diff --git a/drivers/cpuidle/cpuidle-powernv.c 
b/drivers/cpuidle/cpuidle-powernv.c
index a89f546..d27e955 100644
--- a/drivers/cpuidle/cpuidle-powernv.c
+++ b/drivers/cpuidle/cpuidle-powernv.c
@@ -20,6 +20,8 @@
 #include 
 #include 
 
+#define POWERNV_THRESHOLD_LATENCY_NS 20
+
 struct cpuidle_driver powernv_idle_driver = {
.name = "powernv_idle",
.owner= THIS_MODULE,
@@ -27,6 +29,9 @@ struct cpuidle_driver powernv_idle_driver = {
 
 static int max_idle_state;
 static struct cpuidle_state *cpuidle_state_table;
+
+static u64 stop_psscr_table[CPUIDLE_STATE_MAX];
+
 static u64 snooze_timeout;
 static bool snooze_timeout_en;
 
@@ -91,6 +96,17 @@ static int fastsleep_loop(struct cpuidle_device *dev,
return index;
 }
 #endif
+
+static int stop_loop(struct cpuidle_device *dev,
+struct cpuidle_driver *drv,
+int index)
+{
+   ppc64_runlatch_off();
+   power9_idle_stop(stop_psscr_table[index]);
+   ppc64_runlatch_on();
+   return index;
+}
+
 /*
  * States for dedicated partition case.
  */
@@ -169,6 +185,8 @@ static int powernv_add_idle_states(void)
u32 latency_ns[CPUIDLE_STATE_MAX];
u32 residency_ns[CPUIDLE_STATE_MAX];
u32 flags[CPUIDLE_STATE_MAX];
+   u64 psscr_val[CPUIDLE_STATE_MAX];
+   const char *names[CPUIDLE_STATE_MAX];
int i, rc;
 
/* Currently we have snooze statically defined */
@@ -207,11 +225,34 @@ static int powernv_add_idle_states(void)
pr_warn("cpuidle-powernv: missing 
ibm,cpu-idle-state-latencies-ns in DT\n");
goto out;
}
+   if (of_property_read_string_array(power_mgt,
+   "ibm,cpu-idle-state-names", names, dt_idle_states)) {
+   pr_warn("cpuidle-powernv: missing ibm,cpu-idle-state-names in 
DT\n");
+   goto out;
+   }
+
+   /*
+* If the idle states use stop instruction, probe for psscr values
+* which are necessary to specify required stop level.
+*/
+   if (flags[0] & (OPAL_PM_STOP_INST_FAST | OPAL_PM_STOP_INST_DEEP))
+   if (of_property_read_u64_array(power_mgt,
+   "ibm,cpu-idle-state-psscr", psscr_val, dt_idle_states)) {
+   pr_warn("cpuidle-powernv: missing 
ibm,cpu-idle-states-psscr in DT\n");
+   goto out;
+   }
 
rc = of_property_read_u32_array(power_mgt,
"ibm,cpu-idle-state-residency-ns", residency_ns, 
dt_idle_states);
 
for (i = 0; i < dt_idle_states; i++) {
+   /*
+* If an idle state has exit latency beyond
+* POWERNV_THRESHOLD_LATENCY_NS then don't use it
+* in cpu-idle.
+*/
+   if (latency_ns[i] > POWERNV_THRESHOLD_LATENCY_NS)
+   continue;
 
/*
 * Cpuidle accepts exit_latency and target_residency in us.
@@ -224,6 +265,16 @@ static int powernv_add_idle_states(void)
powernv_states[nr_idle_states].flags = 0;
powernv_states[nr_idle_states].target_residency = 100;
powernv_states[nr_idle_states].enter = nap_loop;
+   } else if ((flags[i] & OPAL_PM_STOP_INST_FAST) &&
+   !(flags[i] 

[PATCH v7 09/11] cpuidle/powernv: cleanup powernv_add_idle_states

[Shreyas B. Prabhu]

 - Use stack instead of kzalloc'ed memory for variables while probing
   device tree for idle states.
 - Set cap for number of idle states that can be added to
   cpuidle_state_table
 - Minor change in way we check of_property_read_u32_array for error
   for sake of consistency
 - Drop unnecessary "&" while assigning function pointer

Cc: Rafael J. Wysocki <rafael.j.wyso...@intel.com>
Cc: Daniel Lezcano <daniel.lezc...@linaro.org>
Cc: linux...@vger.kernel.org
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
 - New in v7. This was mainly to make the existing code
   consistent with the review comments for new code

 drivers/cpuidle/cpuidle-powernv.c | 36 +++-
 1 file changed, 19 insertions(+), 17 deletions(-)

diff --git a/drivers/cpuidle/cpuidle-powernv.c 
b/drivers/cpuidle/cpuidle-powernv.c
index 3a763a8..a89f546 100644
--- a/drivers/cpuidle/cpuidle-powernv.c
+++ b/drivers/cpuidle/cpuidle-powernv.c
@@ -166,7 +166,9 @@ static int powernv_add_idle_states(void)
struct device_node *power_mgt;
int nr_idle_states = 1; /* Snooze */
int dt_idle_states;
-   u32 *latency_ns, *residency_ns, *flags;
+   u32 latency_ns[CPUIDLE_STATE_MAX];
+   u32 residency_ns[CPUIDLE_STATE_MAX];
+   u32 flags[CPUIDLE_STATE_MAX];
int i, rc;
 
/* Currently we have snooze statically defined */
@@ -184,22 +186,28 @@ static int powernv_add_idle_states(void)
goto out;
}
 
-   flags = kzalloc(sizeof(*flags) * dt_idle_states, GFP_KERNEL);
+   /*
+* Since snooze is used as first idle state, max idle states allowed is
+* CPUIDLE_STATE_MAX -1
+*/
+   if (dt_idle_states > CPUIDLE_STATE_MAX - 1) {
+   pr_warn("cpuidle-powernv: discovered idle states more than 
allowed");
+   dt_idle_states = CPUIDLE_STATE_MAX - 1;
+   }
+
if (of_property_read_u32_array(power_mgt,
"ibm,cpu-idle-state-flags", flags, dt_idle_states)) {
pr_warn("cpuidle-powernv : missing ibm,cpu-idle-state-flags in 
DT\n");
-   goto out_free_flags;
+   goto out;
}
 
-   latency_ns = kzalloc(sizeof(*latency_ns) * dt_idle_states, GFP_KERNEL);
-   rc = of_property_read_u32_array(power_mgt,
-   "ibm,cpu-idle-state-latencies-ns", latency_ns, dt_idle_states);
-   if (rc) {
+   if (of_property_read_u32_array(power_mgt,
+   "ibm,cpu-idle-state-latencies-ns", latency_ns,
+   dt_idle_states)) {
pr_warn("cpuidle-powernv: missing 
ibm,cpu-idle-state-latencies-ns in DT\n");
-   goto out_free_latency;
+   goto out;
}
 
-   residency_ns = kzalloc(sizeof(*residency_ns) * dt_idle_states, 
GFP_KERNEL);
rc = of_property_read_u32_array(power_mgt,
"ibm,cpu-idle-state-residency-ns", residency_ns, 
dt_idle_states);
 
@@ -215,7 +223,7 @@ static int powernv_add_idle_states(void)
strcpy(powernv_states[nr_idle_states].desc, "Nap");
powernv_states[nr_idle_states].flags = 0;
powernv_states[nr_idle_states].target_residency = 100;
-   powernv_states[nr_idle_states].enter = _loop;
+   powernv_states[nr_idle_states].enter = nap_loop;
}
 
/*
@@ -230,7 +238,7 @@ static int powernv_add_idle_states(void)
strcpy(powernv_states[nr_idle_states].desc, 
"FastSleep");
powernv_states[nr_idle_states].flags = 
CPUIDLE_FLAG_TIMER_STOP;
powernv_states[nr_idle_states].target_residency = 
30;
-   powernv_states[nr_idle_states].enter = _loop;
+   powernv_states[nr_idle_states].enter = fastsleep_loop;
}
 #endif
powernv_states[nr_idle_states].exit_latency =
@@ -243,12 +251,6 @@ static int powernv_add_idle_states(void)
 
nr_idle_states++;
}
-
-   kfree(residency_ns);
-out_free_latency:
-   kfree(latency_ns);
-out_free_flags:
-   kfree(flags);
 out:
return nr_idle_states;
 }
-- 
2.4.11

[PATCH v7 09/11] cpuidle/powernv: cleanup powernv_add_idle_states

[Shreyas B. Prabhu]

 - Use stack instead of kzalloc'ed memory for variables while probing
   device tree for idle states.
 - Set cap for number of idle states that can be added to
   cpuidle_state_table
 - Minor change in way we check of_property_read_u32_array for error
   for sake of consistency
 - Drop unnecessary "&" while assigning function pointer

Cc: Rafael J. Wysocki 
Cc: Daniel Lezcano 
Cc: linux...@vger.kernel.org
Signed-off-by: Shreyas B. Prabhu 
---
 - New in v7. This was mainly to make the existing code
   consistent with the review comments for new code

 drivers/cpuidle/cpuidle-powernv.c | 36 +++-
 1 file changed, 19 insertions(+), 17 deletions(-)

diff --git a/drivers/cpuidle/cpuidle-powernv.c 
b/drivers/cpuidle/cpuidle-powernv.c
index 3a763a8..a89f546 100644
--- a/drivers/cpuidle/cpuidle-powernv.c
+++ b/drivers/cpuidle/cpuidle-powernv.c
@@ -166,7 +166,9 @@ static int powernv_add_idle_states(void)
struct device_node *power_mgt;
int nr_idle_states = 1; /* Snooze */
int dt_idle_states;
-   u32 *latency_ns, *residency_ns, *flags;
+   u32 latency_ns[CPUIDLE_STATE_MAX];
+   u32 residency_ns[CPUIDLE_STATE_MAX];
+   u32 flags[CPUIDLE_STATE_MAX];
int i, rc;
 
/* Currently we have snooze statically defined */
@@ -184,22 +186,28 @@ static int powernv_add_idle_states(void)
goto out;
}
 
-   flags = kzalloc(sizeof(*flags) * dt_idle_states, GFP_KERNEL);
+   /*
+* Since snooze is used as first idle state, max idle states allowed is
+* CPUIDLE_STATE_MAX -1
+*/
+   if (dt_idle_states > CPUIDLE_STATE_MAX - 1) {
+   pr_warn("cpuidle-powernv: discovered idle states more than 
allowed");
+   dt_idle_states = CPUIDLE_STATE_MAX - 1;
+   }
+
if (of_property_read_u32_array(power_mgt,
"ibm,cpu-idle-state-flags", flags, dt_idle_states)) {
pr_warn("cpuidle-powernv : missing ibm,cpu-idle-state-flags in 
DT\n");
-   goto out_free_flags;
+   goto out;
}
 
-   latency_ns = kzalloc(sizeof(*latency_ns) * dt_idle_states, GFP_KERNEL);
-   rc = of_property_read_u32_array(power_mgt,
-   "ibm,cpu-idle-state-latencies-ns", latency_ns, dt_idle_states);
-   if (rc) {
+   if (of_property_read_u32_array(power_mgt,
+   "ibm,cpu-idle-state-latencies-ns", latency_ns,
+   dt_idle_states)) {
pr_warn("cpuidle-powernv: missing 
ibm,cpu-idle-state-latencies-ns in DT\n");
-   goto out_free_latency;
+   goto out;
}
 
-   residency_ns = kzalloc(sizeof(*residency_ns) * dt_idle_states, 
GFP_KERNEL);
rc = of_property_read_u32_array(power_mgt,
"ibm,cpu-idle-state-residency-ns", residency_ns, 
dt_idle_states);
 
@@ -215,7 +223,7 @@ static int powernv_add_idle_states(void)
strcpy(powernv_states[nr_idle_states].desc, "Nap");
powernv_states[nr_idle_states].flags = 0;
powernv_states[nr_idle_states].target_residency = 100;
-   powernv_states[nr_idle_states].enter = _loop;
+   powernv_states[nr_idle_states].enter = nap_loop;
}
 
/*
@@ -230,7 +238,7 @@ static int powernv_add_idle_states(void)
strcpy(powernv_states[nr_idle_states].desc, 
"FastSleep");
powernv_states[nr_idle_states].flags = 
CPUIDLE_FLAG_TIMER_STOP;
powernv_states[nr_idle_states].target_residency = 
30;
-   powernv_states[nr_idle_states].enter = _loop;
+   powernv_states[nr_idle_states].enter = fastsleep_loop;
}
 #endif
powernv_states[nr_idle_states].exit_latency =
@@ -243,12 +251,6 @@ static int powernv_add_idle_states(void)
 
nr_idle_states++;
}
-
-   kfree(residency_ns);
-out_free_latency:
-   kfree(latency_ns);
-out_free_flags:
-   kfree(flags);
 out:
return nr_idle_states;
 }
-- 
2.4.11

[PATCH v7 07/11] powerpc/powernv: Add platform support for stop instruction

[Shreyas B. Prabhu]

POWER ISA v3 defines a new idle processor core mechanism. In summary,
 a) new instruction named stop is added. This instruction replaces
instructions like nap, sleep, rvwinkle.
 b) new per thread SPR named Processor Stop Status and Control Register
(PSSCR) is added which controls the behavior of stop instruction.

PSSCR layout:
--
| PLS | /// | SD | ESL | EC | PSLL | /// | TR | MTL | RL |
--
0  4 41   4243   44 4854   5660

PSSCR key fields:
Bits 0:3  - Power-Saving Level Status. This field indicates the lowest
power-saving state the thread entered since stop instruction was last
executed.

Bit 42 - Enable State Loss
0 - No state is lost irrespective of other fields
1 - Allows state loss

Bits 44:47 - Power-Saving Level Limit
This limits the power-saving level that can be entered into.

Bits 60:63 - Requested Level
Used to specify which power-saving level must be entered on executing
stop instruction

This patch adds support for stop instruction and PSSCR handling.

Reviewed-by: Gautham R. Shenoy <e...@linux.vnet.ibm.com>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
Changes in v7
=
 - LMRR, LMSER and ADSR not restored since its not necessary
 - power_stop0, power_stop renamed to power9_idle and power_idle_stop
 - PSSCR template is now a macro instead of storing in paca
 - power9_idle in C file instead of assembly
 - Fixed TOC related bug
 - Handling subcore within FTR section
 - Functions in idle.c reordered and broken into multiple functions
 - calling __restore_cpu_power8/9 via cur_cpu_spec->cpu_restore 
 
Changes in v6
=
 - Save/restore new P9 SPRs when using deep idle states

Changes in v4:
==
 - Added PSSCR layout to commit message
 - Improved / Fixed comments
 - Fixed whitespace error in paca.h
 - Using MAX_POSSIBLE_STOP_STATE macro instead of hardcoding 0xF as 
   max possible stop state

Changes in v3:
==
 - Instead of introducing new file idle_power_stop.S, P9 idle support
   is added to idle_power_common.S using CPU_FTR sections.
 - Fixed r4 reg clobbering in power_stop0
 - Improved comments

Changes in v2:
==
 - Using CPU_FTR_ARCH_300 bit instead of CPU_FTR_STOP_INST

 arch/powerpc/include/asm/cpuidle.h|   2 +
 arch/powerpc/include/asm/kvm_book3s_asm.h |   2 +-
 arch/powerpc/include/asm/opal-api.h   |  11 +-
 arch/powerpc/include/asm/ppc-opcode.h |   4 +
 arch/powerpc/include/asm/processor.h  |   2 +
 arch/powerpc/include/asm/reg.h|  10 ++
 arch/powerpc/kernel/idle_book3s.S | 189 --
 arch/powerpc/platforms/powernv/idle.c | 174 ++-
 8 files changed, 328 insertions(+), 66 deletions(-)

diff --git a/arch/powerpc/include/asm/cpuidle.h 
b/arch/powerpc/include/asm/cpuidle.h
index d2f99ca..3d7fc06 100644
--- a/arch/powerpc/include/asm/cpuidle.h
+++ b/arch/powerpc/include/asm/cpuidle.h
@@ -13,6 +13,8 @@
 #ifndef __ASSEMBLY__
 extern u32 pnv_fastsleep_workaround_at_entry[];
 extern u32 pnv_fastsleep_workaround_at_exit[];
+
+extern u64 pnv_first_deep_stop_state;
 #endif
 
 #endif
diff --git a/arch/powerpc/include/asm/kvm_book3s_asm.h 
b/arch/powerpc/include/asm/kvm_book3s_asm.h
index 72b6225..d318d43 100644
--- a/arch/powerpc/include/asm/kvm_book3s_asm.h
+++ b/arch/powerpc/include/asm/kvm_book3s_asm.h
@@ -162,7 +162,7 @@ struct kvmppc_book3s_shadow_vcpu {
 
 /* Values for kvm_state */
 #define KVM_HWTHREAD_IN_KERNEL 0
-#define KVM_HWTHREAD_IN_NAP1
+#define KVM_HWTHREAD_IN_IDLE   1
 #define KVM_HWTHREAD_IN_KVM2
 
 #endif /* __ASM_KVM_BOOK3S_ASM_H__ */
diff --git a/arch/powerpc/include/asm/opal-api.h 
b/arch/powerpc/include/asm/opal-api.h
index 72b5f27..6de1e4e 100644
--- a/arch/powerpc/include/asm/opal-api.h
+++ b/arch/powerpc/include/asm/opal-api.h
@@ -166,13 +166,20 @@
 
 /* Device tree flags */
 
-/* Flags set in power-mgmt nodes in device tree if
- * respective idle states are supported in the platform.
+/*
+ * Flags set in power-mgmt nodes in device tree describing
+ * idle states that are supported in the platform.
  */
+
+#define OPAL_PM_TIMEBASE_STOP  0x0002
+#define OPAL_PM_LOSE_HYP_CONTEXT   0x2000
+#define OPAL_PM_LOSE_FULL_CONTEXT  0x4000
 #define OPAL_PM_NAP_ENABLED0x0001
 #define OPAL_PM_SLEEP_ENABLED  0x0002
 #define OPAL_PM_WINKLE_ENABLED 0x0004
 #define OPAL_PM_SLEEP_ENABLED_ER1  0x0008 /* with workaround */
+#define OPAL_PM_STOP_INST_FAST 0x0010
+#define OPAL_PM_STOP_INST_DEEP 0x0020
 
 /*
  * OPAL_CONFIG_CPU_IDLE_STATE parameters
diff --git a/arch/powerpc/include/asm/ppc-opcode.h 
b/arch/powerpc/include/asm/ppc-opcode.h
index 9de9df1..81657a1 100644
---

[PATCH v7 07/11] powerpc/powernv: Add platform support for stop instruction

[Shreyas B. Prabhu]

POWER ISA v3 defines a new idle processor core mechanism. In summary,
 a) new instruction named stop is added. This instruction replaces
instructions like nap, sleep, rvwinkle.
 b) new per thread SPR named Processor Stop Status and Control Register
(PSSCR) is added which controls the behavior of stop instruction.

PSSCR layout:
--
| PLS | /// | SD | ESL | EC | PSLL | /// | TR | MTL | RL |
--
0  4 41   4243   44 4854   5660

PSSCR key fields:
Bits 0:3  - Power-Saving Level Status. This field indicates the lowest
power-saving state the thread entered since stop instruction was last
executed.

Bit 42 - Enable State Loss
0 - No state is lost irrespective of other fields
1 - Allows state loss

Bits 44:47 - Power-Saving Level Limit
This limits the power-saving level that can be entered into.

Bits 60:63 - Requested Level
Used to specify which power-saving level must be entered on executing
stop instruction

This patch adds support for stop instruction and PSSCR handling.

Reviewed-by: Gautham R. Shenoy 
Signed-off-by: Shreyas B. Prabhu 
---
Changes in v7
=
 - LMRR, LMSER and ADSR not restored since its not necessary
 - power_stop0, power_stop renamed to power9_idle and power_idle_stop
 - PSSCR template is now a macro instead of storing in paca
 - power9_idle in C file instead of assembly
 - Fixed TOC related bug
 - Handling subcore within FTR section
 - Functions in idle.c reordered and broken into multiple functions
 - calling __restore_cpu_power8/9 via cur_cpu_spec->cpu_restore 
 
Changes in v6
=
 - Save/restore new P9 SPRs when using deep idle states

Changes in v4:
==
 - Added PSSCR layout to commit message
 - Improved / Fixed comments
 - Fixed whitespace error in paca.h
 - Using MAX_POSSIBLE_STOP_STATE macro instead of hardcoding 0xF as 
   max possible stop state

Changes in v3:
==
 - Instead of introducing new file idle_power_stop.S, P9 idle support
   is added to idle_power_common.S using CPU_FTR sections.
 - Fixed r4 reg clobbering in power_stop0
 - Improved comments

Changes in v2:
==
 - Using CPU_FTR_ARCH_300 bit instead of CPU_FTR_STOP_INST

 arch/powerpc/include/asm/cpuidle.h|   2 +
 arch/powerpc/include/asm/kvm_book3s_asm.h |   2 +-
 arch/powerpc/include/asm/opal-api.h   |  11 +-
 arch/powerpc/include/asm/ppc-opcode.h |   4 +
 arch/powerpc/include/asm/processor.h  |   2 +
 arch/powerpc/include/asm/reg.h|  10 ++
 arch/powerpc/kernel/idle_book3s.S | 189 --
 arch/powerpc/platforms/powernv/idle.c | 174 ++-
 8 files changed, 328 insertions(+), 66 deletions(-)

diff --git a/arch/powerpc/include/asm/cpuidle.h 
b/arch/powerpc/include/asm/cpuidle.h
index d2f99ca..3d7fc06 100644
--- a/arch/powerpc/include/asm/cpuidle.h
+++ b/arch/powerpc/include/asm/cpuidle.h
@@ -13,6 +13,8 @@
 #ifndef __ASSEMBLY__
 extern u32 pnv_fastsleep_workaround_at_entry[];
 extern u32 pnv_fastsleep_workaround_at_exit[];
+
+extern u64 pnv_first_deep_stop_state;
 #endif
 
 #endif
diff --git a/arch/powerpc/include/asm/kvm_book3s_asm.h 
b/arch/powerpc/include/asm/kvm_book3s_asm.h
index 72b6225..d318d43 100644
--- a/arch/powerpc/include/asm/kvm_book3s_asm.h
+++ b/arch/powerpc/include/asm/kvm_book3s_asm.h
@@ -162,7 +162,7 @@ struct kvmppc_book3s_shadow_vcpu {
 
 /* Values for kvm_state */
 #define KVM_HWTHREAD_IN_KERNEL 0
-#define KVM_HWTHREAD_IN_NAP1
+#define KVM_HWTHREAD_IN_IDLE   1
 #define KVM_HWTHREAD_IN_KVM2
 
 #endif /* __ASM_KVM_BOOK3S_ASM_H__ */
diff --git a/arch/powerpc/include/asm/opal-api.h 
b/arch/powerpc/include/asm/opal-api.h
index 72b5f27..6de1e4e 100644
--- a/arch/powerpc/include/asm/opal-api.h
+++ b/arch/powerpc/include/asm/opal-api.h
@@ -166,13 +166,20 @@
 
 /* Device tree flags */
 
-/* Flags set in power-mgmt nodes in device tree if
- * respective idle states are supported in the platform.
+/*
+ * Flags set in power-mgmt nodes in device tree describing
+ * idle states that are supported in the platform.
  */
+
+#define OPAL_PM_TIMEBASE_STOP  0x0002
+#define OPAL_PM_LOSE_HYP_CONTEXT   0x2000
+#define OPAL_PM_LOSE_FULL_CONTEXT  0x4000
 #define OPAL_PM_NAP_ENABLED0x0001
 #define OPAL_PM_SLEEP_ENABLED  0x0002
 #define OPAL_PM_WINKLE_ENABLED 0x0004
 #define OPAL_PM_SLEEP_ENABLED_ER1  0x0008 /* with workaround */
+#define OPAL_PM_STOP_INST_FAST 0x0010
+#define OPAL_PM_STOP_INST_DEEP 0x0020
 
 /*
  * OPAL_CONFIG_CPU_IDLE_STATE parameters
diff --git a/arch/powerpc/include/asm/ppc-opcode.h 
b/arch/powerpc/include/asm/ppc-opcode.h
index 9de9df1..81657a1 100644
--- a/arch/powerpc/include/asm/ppc-opcode.h
+++ b/arch/powerpc/include/

[PATCH v7 11/11] powerpc/powernv: Use deepest stop state when cpu is offlined

[Shreyas B. Prabhu]

If hardware supports stop state, use the deepest stop state when
the cpu is offlined.

Reviewed-by: Gautham R. Shenoy <e...@linux.vnet.ibm.com>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
 - No changes since v1
 arch/powerpc/platforms/powernv/idle.c| 15 +--
 arch/powerpc/platforms/powernv/powernv.h |  1 +
 arch/powerpc/platforms/powernv/smp.c |  4 +++-
 3 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 006e467..2f6968c 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -253,6 +253,11 @@ static void power9_idle(void)
 u64 pnv_first_deep_stop_state;
 
 /*
+ * Deepest stop idle state. Used when a cpu is offlined
+ */
+u64 pnv_deepest_stop_state;
+
+/*
  * Power ISA 3.0 idle initialization.
  *
  * POWER ISA 3.0 defines a new SPR Processor stop Status and Control
@@ -314,8 +319,11 @@ static int __init pnv_arch300_idle_init(struct device_node 
*np, u32 *flags,
}
 
/*
-* Set pnv_first_deep_stop_state to the first stop level
-* to cause hypervisor state loss
+* Set pnv_first_deep_stop_state and pnv_deepest_stop_state.
+* pnv_first_deep_stop_state should be set to the first stop
+* level to cause hypervisor state loss.
+* pnv_deepest_stop_state should be set to the deepest stop
+* stop state.
 */
pnv_first_deep_stop_state = MAX_STOP_STATE;
for (i = 0; i < dt_idle_states; i++) {
@@ -324,6 +332,9 @@ static int __init pnv_arch300_idle_init(struct device_node 
*np, u32 *flags,
if ((flags[i] & OPAL_PM_LOSE_FULL_CONTEXT) &&
 (pnv_first_deep_stop_state > psscr_rl))
pnv_first_deep_stop_state = psscr_rl;
+
+   if (pnv_deepest_stop_state < psscr_rl)
+   pnv_deepest_stop_state = psscr_rl;
}
 
 out:
diff --git a/arch/powerpc/platforms/powernv/powernv.h 
b/arch/powerpc/platforms/powernv/powernv.h
index 6dbc0a1..da7c843 100644
--- a/arch/powerpc/platforms/powernv/powernv.h
+++ b/arch/powerpc/platforms/powernv/powernv.h
@@ -18,6 +18,7 @@ static inline void pnv_pci_shutdown(void) { }
 #endif
 
 extern u32 pnv_get_supported_cpuidle_states(void);
+extern u64 pnv_deepest_stop_state;
 
 extern void pnv_lpc_init(void);
 
diff --git a/arch/powerpc/platforms/powernv/smp.c 
b/arch/powerpc/platforms/powernv/smp.c
index ad7b1a3..c789258 100644
--- a/arch/powerpc/platforms/powernv/smp.c
+++ b/arch/powerpc/platforms/powernv/smp.c
@@ -182,7 +182,9 @@ static void pnv_smp_cpu_kill_self(void)
 
ppc64_runlatch_off();
 
-   if (idle_states & OPAL_PM_WINKLE_ENABLED)
+   if (cpu_has_feature(CPU_FTR_ARCH_300))
+   srr1 = power9_idle_stop(pnv_deepest_stop_state);
+   else if (idle_states & OPAL_PM_WINKLE_ENABLED)
srr1 = power7_winkle();
else if ((idle_states & OPAL_PM_SLEEP_ENABLED) ||
(idle_states & OPAL_PM_SLEEP_ENABLED_ER1))
-- 
2.4.11

[PATCH v7 01/11] powerpc/powernv: Use PNV_THREAD_WINKLE macro while requesting for winkle

[Shreyas B. Prabhu]

Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
-No changes since v4

Changes in v4
=
- New in v4

 arch/powerpc/kernel/idle_power7.S | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/powerpc/kernel/idle_power7.S 
b/arch/powerpc/kernel/idle_power7.S
index 470ceeb..705c867 100644
--- a/arch/powerpc/kernel/idle_power7.S
+++ b/arch/powerpc/kernel/idle_power7.S
@@ -252,7 +252,7 @@ _GLOBAL(power7_sleep)
/* No return */
 
 _GLOBAL(power7_winkle)
-   li  r3,3
+   li  r3,PNV_THREAD_WINKLE
li  r4,1
b   power7_powersave_common
/* No return */
-- 
2.4.11

[PATCH v7 11/11] powerpc/powernv: Use deepest stop state when cpu is offlined

[Shreyas B. Prabhu]

If hardware supports stop state, use the deepest stop state when
the cpu is offlined.

Reviewed-by: Gautham R. Shenoy 
Signed-off-by: Shreyas B. Prabhu 
---
 - No changes since v1
 arch/powerpc/platforms/powernv/idle.c| 15 +--
 arch/powerpc/platforms/powernv/powernv.h |  1 +
 arch/powerpc/platforms/powernv/smp.c |  4 +++-
 3 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 006e467..2f6968c 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -253,6 +253,11 @@ static void power9_idle(void)
 u64 pnv_first_deep_stop_state;
 
 /*
+ * Deepest stop idle state. Used when a cpu is offlined
+ */
+u64 pnv_deepest_stop_state;
+
+/*
  * Power ISA 3.0 idle initialization.
  *
  * POWER ISA 3.0 defines a new SPR Processor stop Status and Control
@@ -314,8 +319,11 @@ static int __init pnv_arch300_idle_init(struct device_node 
*np, u32 *flags,
}
 
/*
-* Set pnv_first_deep_stop_state to the first stop level
-* to cause hypervisor state loss
+* Set pnv_first_deep_stop_state and pnv_deepest_stop_state.
+* pnv_first_deep_stop_state should be set to the first stop
+* level to cause hypervisor state loss.
+* pnv_deepest_stop_state should be set to the deepest stop
+* stop state.
 */
pnv_first_deep_stop_state = MAX_STOP_STATE;
for (i = 0; i < dt_idle_states; i++) {
@@ -324,6 +332,9 @@ static int __init pnv_arch300_idle_init(struct device_node 
*np, u32 *flags,
if ((flags[i] & OPAL_PM_LOSE_FULL_CONTEXT) &&
 (pnv_first_deep_stop_state > psscr_rl))
pnv_first_deep_stop_state = psscr_rl;
+
+   if (pnv_deepest_stop_state < psscr_rl)
+   pnv_deepest_stop_state = psscr_rl;
}
 
 out:
diff --git a/arch/powerpc/platforms/powernv/powernv.h 
b/arch/powerpc/platforms/powernv/powernv.h
index 6dbc0a1..da7c843 100644
--- a/arch/powerpc/platforms/powernv/powernv.h
+++ b/arch/powerpc/platforms/powernv/powernv.h
@@ -18,6 +18,7 @@ static inline void pnv_pci_shutdown(void) { }
 #endif
 
 extern u32 pnv_get_supported_cpuidle_states(void);
+extern u64 pnv_deepest_stop_state;
 
 extern void pnv_lpc_init(void);
 
diff --git a/arch/powerpc/platforms/powernv/smp.c 
b/arch/powerpc/platforms/powernv/smp.c
index ad7b1a3..c789258 100644
--- a/arch/powerpc/platforms/powernv/smp.c
+++ b/arch/powerpc/platforms/powernv/smp.c
@@ -182,7 +182,9 @@ static void pnv_smp_cpu_kill_self(void)
 
ppc64_runlatch_off();
 
-   if (idle_states & OPAL_PM_WINKLE_ENABLED)
+   if (cpu_has_feature(CPU_FTR_ARCH_300))
+   srr1 = power9_idle_stop(pnv_deepest_stop_state);
+   else if (idle_states & OPAL_PM_WINKLE_ENABLED)
srr1 = power7_winkle();
else if ((idle_states & OPAL_PM_SLEEP_ENABLED) ||
(idle_states & OPAL_PM_SLEEP_ENABLED_ER1))
-- 
2.4.11

[PATCH v7 01/11] powerpc/powernv: Use PNV_THREAD_WINKLE macro while requesting for winkle

[Shreyas B. Prabhu]

Signed-off-by: Shreyas B. Prabhu 
---
-No changes since v4

Changes in v4
=
- New in v4

 arch/powerpc/kernel/idle_power7.S | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/powerpc/kernel/idle_power7.S 
b/arch/powerpc/kernel/idle_power7.S
index 470ceeb..705c867 100644
--- a/arch/powerpc/kernel/idle_power7.S
+++ b/arch/powerpc/kernel/idle_power7.S
@@ -252,7 +252,7 @@ _GLOBAL(power7_sleep)
/* No return */
 
 _GLOBAL(power7_winkle)
-   li  r3,3
+   li  r3,PNV_THREAD_WINKLE
li  r4,1
b   power7_powersave_common
/* No return */
-- 
2.4.11

[PATCH v7 02/11] powerpc/kvm: make hypervisor state restore a function

[Shreyas B. Prabhu]

In the current code, when the thread wakes up in reset vector, some
of the state restore code and check for whether a thread needs to
branch to kvm is duplicated. Reorder the code such that this
duplication is avoided.

At a higher level this is what the change looks like-

Before this patch -
power7_wakeup_tb_loss:
restore hypervisor state
if (thread needed by kvm)
goto kvm_start_guest
restore nvgprs, cr, pc
rfid to process context

power7_wakeup_loss:
restore nvgprs, cr, pc
rfid to process context

reset vector:
if (waking from deep idle states)
goto power7_wakeup_tb_loss
else
if (thread needed by kvm)
goto kvm_start_guest
goto power7_wakeup_loss

After this patch -
power7_wakeup_tb_loss:
restore hypervisor state
return

power7_restore_hyp_resource():
if (waking from deep idle states)
goto power7_wakeup_tb_loss
return

power7_wakeup_loss:
restore nvgprs, cr, pc
rfid to process context

reset vector:
power7_restore_hyp_resource()
if (thread needed by kvm)
goto kvm_start_guest
goto power7_wakeup_loss

Reviewed-by: Paul Mackerras <pau...@samba.org>
Reviewed-by: Gautham R. Shenoy <e...@linux.vnet.ibm.com>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
- No changes since v3

Changes in v3:
=
- Retaining GET_PACA(r13) in System Reset vector instead of moving it
  to power7_restore_hyp_resource
- Added comments indicating entry conditions for power7_restore_hyp_resource
- Improved comments around return statements

 arch/powerpc/kernel/exceptions-64s.S | 28 ++
 arch/powerpc/kernel/idle_power7.S| 72 +---
 2 files changed, 46 insertions(+), 54 deletions(-)

diff --git a/arch/powerpc/kernel/exceptions-64s.S 
b/arch/powerpc/kernel/exceptions-64s.S
index 4c94406..4a74d6a 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -107,25 +107,9 @@ BEGIN_FTR_SECTION
beq 9f
 
cmpwi   cr3,r13,2
-
-   /*
-* Check if last bit of HSPGR0 is set. This indicates whether we are
-* waking up from winkle.
-*/
GET_PACA(r13)
-   clrldi  r5,r13,63
-   clrrdi  r13,r13,1
-   cmpwi   cr4,r5,1
-   mtspr   SPRN_HSPRG0,r13
+   bl  power7_restore_hyp_resource
 
-   lbz r0,PACA_THREAD_IDLE_STATE(r13)
-   cmpwi   cr2,r0,PNV_THREAD_NAP
-   bgt cr2,8f  /* Either sleep or Winkle */
-
-   /* Waking up from nap should not cause hypervisor state loss */
-   bgt cr3,.
-
-   /* Waking up from nap */
li  r0,PNV_THREAD_RUNNING
stb r0,PACA_THREAD_IDLE_STATE(r13)  /* Clear thread state */
 
@@ -143,13 +127,9 @@ BEGIN_FTR_SECTION
 
/* Return SRR1 from power7_nap() */
mfspr   r3,SPRN_SRR1
-   beq cr3,2f
-   b   power7_wakeup_noloss
-2: b   power7_wakeup_loss
-
-   /* Fast Sleep wakeup on PowerNV */
-8: GET_PACA(r13)
-   b   power7_wakeup_tb_loss
+   blt cr3,2f
+   b   power7_wakeup_loss
+2: b   power7_wakeup_noloss
 
 9:
 END_FTR_SECTION_IFSET(CPU_FTR_HVMODE | CPU_FTR_ARCH_206)
diff --git a/arch/powerpc/kernel/idle_power7.S 
b/arch/powerpc/kernel/idle_power7.S
index 705c867..d5def06 100644
--- a/arch/powerpc/kernel/idle_power7.S
+++ b/arch/powerpc/kernel/idle_power7.S
@@ -276,6 +276,39 @@ ALT_FTR_SECTION_END_NESTED_IFSET(CPU_FTR_ARCH_207S, 66);   
\
 20:nop;
 
 
+/*
+ * Called from reset vector. Check whether we have woken up with
+ * hypervisor state loss. If yes, restore hypervisor state and return
+ * back to reset vector.
+ *
+ * r13 - Contents of HSPRG0
+ * cr3 - set to gt if waking up with partial/complete hypervisor state loss
+ */
+_GLOBAL(power7_restore_hyp_resource)
+   /*
+* Check if last bit of HSPGR0 is set. This indicates whether we are
+* waking up from winkle.
+*/
+   clrldi  r5,r13,63
+   clrrdi  r13,r13,1
+   cmpwi   cr4,r5,1
+   mtspr   SPRN_HSPRG0,r13
+
+   lbz r0,PACA_THREAD_IDLE_STATE(r13)
+   cmpwi   cr2,r0,PNV_THREAD_NAP
+   bgt cr2,power7_wakeup_tb_loss   /* Either sleep or Winkle */
+
+   /*
+* We fall through here if PACA_THREAD_IDLE_STATE shows we are waking
+* up from nap. At this stage CR3 shouldn't contains 'gt' since that
+* indicates we are waking with hypervisor state loss from nap.
+*/
+   bgt cr3,.
+
+   blr /* Return back to System Reset vector from where
+  power7_restore_hyp_resource was invoked */
+
+
 _GLOBAL(power7_wakeup_tb_loss)
ld  r2,PACATOC(r13);
ld  r1,PACAR1(r13)
@@ -284,11 +317,13 @@ _GLOBAL(pow

[PATCH v7 02/11] powerpc/kvm: make hypervisor state restore a function

[Shreyas B. Prabhu]

In the current code, when the thread wakes up in reset vector, some
of the state restore code and check for whether a thread needs to
branch to kvm is duplicated. Reorder the code such that this
duplication is avoided.

At a higher level this is what the change looks like-

Before this patch -
power7_wakeup_tb_loss:
restore hypervisor state
if (thread needed by kvm)
goto kvm_start_guest
restore nvgprs, cr, pc
rfid to process context

power7_wakeup_loss:
restore nvgprs, cr, pc
rfid to process context

reset vector:
if (waking from deep idle states)
goto power7_wakeup_tb_loss
else
if (thread needed by kvm)
goto kvm_start_guest
goto power7_wakeup_loss

After this patch -
power7_wakeup_tb_loss:
restore hypervisor state
return

power7_restore_hyp_resource():
if (waking from deep idle states)
goto power7_wakeup_tb_loss
return

power7_wakeup_loss:
restore nvgprs, cr, pc
rfid to process context

reset vector:
power7_restore_hyp_resource()
if (thread needed by kvm)
goto kvm_start_guest
goto power7_wakeup_loss

Reviewed-by: Paul Mackerras 
Reviewed-by: Gautham R. Shenoy 
Signed-off-by: Shreyas B. Prabhu 
---
- No changes since v3

Changes in v3:
=
- Retaining GET_PACA(r13) in System Reset vector instead of moving it
  to power7_restore_hyp_resource
- Added comments indicating entry conditions for power7_restore_hyp_resource
- Improved comments around return statements

 arch/powerpc/kernel/exceptions-64s.S | 28 ++
 arch/powerpc/kernel/idle_power7.S| 72 +---
 2 files changed, 46 insertions(+), 54 deletions(-)

diff --git a/arch/powerpc/kernel/exceptions-64s.S 
b/arch/powerpc/kernel/exceptions-64s.S
index 4c94406..4a74d6a 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -107,25 +107,9 @@ BEGIN_FTR_SECTION
beq 9f
 
cmpwi   cr3,r13,2
-
-   /*
-* Check if last bit of HSPGR0 is set. This indicates whether we are
-* waking up from winkle.
-*/
GET_PACA(r13)
-   clrldi  r5,r13,63
-   clrrdi  r13,r13,1
-   cmpwi   cr4,r5,1
-   mtspr   SPRN_HSPRG0,r13
+   bl  power7_restore_hyp_resource
 
-   lbz r0,PACA_THREAD_IDLE_STATE(r13)
-   cmpwi   cr2,r0,PNV_THREAD_NAP
-   bgt cr2,8f  /* Either sleep or Winkle */
-
-   /* Waking up from nap should not cause hypervisor state loss */
-   bgt cr3,.
-
-   /* Waking up from nap */
li  r0,PNV_THREAD_RUNNING
stb r0,PACA_THREAD_IDLE_STATE(r13)  /* Clear thread state */
 
@@ -143,13 +127,9 @@ BEGIN_FTR_SECTION
 
/* Return SRR1 from power7_nap() */
mfspr   r3,SPRN_SRR1
-   beq cr3,2f
-   b   power7_wakeup_noloss
-2: b   power7_wakeup_loss
-
-   /* Fast Sleep wakeup on PowerNV */
-8: GET_PACA(r13)
-   b   power7_wakeup_tb_loss
+   blt cr3,2f
+   b   power7_wakeup_loss
+2: b   power7_wakeup_noloss
 
 9:
 END_FTR_SECTION_IFSET(CPU_FTR_HVMODE | CPU_FTR_ARCH_206)
diff --git a/arch/powerpc/kernel/idle_power7.S 
b/arch/powerpc/kernel/idle_power7.S
index 705c867..d5def06 100644
--- a/arch/powerpc/kernel/idle_power7.S
+++ b/arch/powerpc/kernel/idle_power7.S
@@ -276,6 +276,39 @@ ALT_FTR_SECTION_END_NESTED_IFSET(CPU_FTR_ARCH_207S, 66);   
\
 20:nop;
 
 
+/*
+ * Called from reset vector. Check whether we have woken up with
+ * hypervisor state loss. If yes, restore hypervisor state and return
+ * back to reset vector.
+ *
+ * r13 - Contents of HSPRG0
+ * cr3 - set to gt if waking up with partial/complete hypervisor state loss
+ */
+_GLOBAL(power7_restore_hyp_resource)
+   /*
+* Check if last bit of HSPGR0 is set. This indicates whether we are
+* waking up from winkle.
+*/
+   clrldi  r5,r13,63
+   clrrdi  r13,r13,1
+   cmpwi   cr4,r5,1
+   mtspr   SPRN_HSPRG0,r13
+
+   lbz r0,PACA_THREAD_IDLE_STATE(r13)
+   cmpwi   cr2,r0,PNV_THREAD_NAP
+   bgt cr2,power7_wakeup_tb_loss   /* Either sleep or Winkle */
+
+   /*
+* We fall through here if PACA_THREAD_IDLE_STATE shows we are waking
+* up from nap. At this stage CR3 shouldn't contains 'gt' since that
+* indicates we are waking with hypervisor state loss from nap.
+*/
+   bgt cr3,.
+
+   blr /* Return back to System Reset vector from where
+  power7_restore_hyp_resource was invoked */
+
+
 _GLOBAL(power7_wakeup_tb_loss)
ld  r2,PACATOC(r13);
ld  r1,PACAR1(r13)
@@ -284,11 +317,13 @@ _GLOBAL(power7_wakeup_tb_loss)
 * and they are restored before switching to the process context. Hence

[PATCH v7 04/11] powerpc/powernv: Rename reusable idle functions to hardware agnostic names

[Shreyas B. Prabhu]

Functions like power7_wakeup_loss, power7_wakeup_noloss,
power7_wakeup_tb_loss are used by POWER7 and POWER8 hardware. They can
also be used by POWER9. Hence rename these functions hardware agnostic
names.

Suggested-by: Gautham R. Shenoy <e...@linux.vnet.ibm.com>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
 - No changes since v4

Changes in v4:
==
 - renaming power7_powersave_common to pnv_powersave_common
 - renaming power7_enter_nap_mode to pnv_enter_arch207_idle_mode

 arch/powerpc/kernel/exceptions-64s.S|  8 
 arch/powerpc/kernel/idle_book3s.S   | 33 +
 arch/powerpc/kvm/book3s_hv_rmhandlers.S |  4 ++--
 3 files changed, 23 insertions(+), 22 deletions(-)

diff --git a/arch/powerpc/kernel/exceptions-64s.S 
b/arch/powerpc/kernel/exceptions-64s.S
index 4a74d6a..2a123cd 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -108,7 +108,7 @@ BEGIN_FTR_SECTION
 
cmpwi   cr3,r13,2
GET_PACA(r13)
-   bl  power7_restore_hyp_resource
+   bl  pnv_restore_hyp_resource
 
li  r0,PNV_THREAD_RUNNING
stb r0,PACA_THREAD_IDLE_STATE(r13)  /* Clear thread state */
@@ -128,8 +128,8 @@ BEGIN_FTR_SECTION
/* Return SRR1 from power7_nap() */
mfspr   r3,SPRN_SRR1
blt cr3,2f
-   b   power7_wakeup_loss
-2: b   power7_wakeup_noloss
+   b   pnv_wakeup_loss
+2: b   pnv_wakeup_noloss
 
 9:
 END_FTR_SECTION_IFSET(CPU_FTR_HVMODE | CPU_FTR_ARCH_206)
@@ -1269,7 +1269,7 @@ machine_check_handle_early:
GET_PACA(r13)
ld  r1,PACAR1(r13)
li  r3,PNV_THREAD_NAP
-   b   power7_enter_nap_mode
+   b   pnv_enter_arch207_idle_mode
 4:
 #endif
/*
diff --git a/arch/powerpc/kernel/idle_book3s.S 
b/arch/powerpc/kernel/idle_book3s.S
index d5def06..34dbfc9 100644
--- a/arch/powerpc/kernel/idle_book3s.S
+++ b/arch/powerpc/kernel/idle_book3s.S
@@ -1,5 +1,6 @@
 /*
- *  This file contains the power_save function for Power7 CPUs.
+ *  This file contains idle entry/exit functions for POWER7 and
+ *  POWER8 CPUs.
  *
  *  This program is free software; you can redistribute it and/or
  *  modify it under the terms of the GNU General Public License
@@ -75,7 +76,7 @@ core_idle_lock_held:
  * 0 - don't check
  * 1 - check
  */
-_GLOBAL(power7_powersave_common)
+_GLOBAL(pnv_powersave_common)
/* Use r3 to pass state nap/sleep/winkle */
/* NAP is a state loss, we create a regs frame on the
 * stack, fill it up with the state we care about and
@@ -135,14 +136,14 @@ _GLOBAL(power7_powersave_common)
LOAD_REG_IMMEDIATE(r5, MSR_IDLE)
li  r6, MSR_RI
andcr6, r9, r6
-   LOAD_REG_ADDR(r7, power7_enter_nap_mode)
+   LOAD_REG_ADDR(r7, pnv_enter_arch207_idle_mode)
mtmsrd  r6, 1   /* clear RI before setting SRR0/1 */
mtspr   SPRN_SRR0, r7
mtspr   SPRN_SRR1, r5
rfid
 
-   .globl  power7_enter_nap_mode
-power7_enter_nap_mode:
+   .globl pnv_enter_arch207_idle_mode
+pnv_enter_arch207_idle_mode:
 #ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
/* Tell KVM we're napping */
li  r4,KVM_HWTHREAD_IN_NAP
@@ -242,19 +243,19 @@ _GLOBAL(power7_idle)
 _GLOBAL(power7_nap)
mr  r4,r3
li  r3,PNV_THREAD_NAP
-   b   power7_powersave_common
+   b   pnv_powersave_common
/* No return */
 
 _GLOBAL(power7_sleep)
li  r3,PNV_THREAD_SLEEP
li  r4,1
-   b   power7_powersave_common
+   b   pnv_powersave_common
/* No return */
 
 _GLOBAL(power7_winkle)
li  r3,PNV_THREAD_WINKLE
li  r4,1
-   b   power7_powersave_common
+   b   pnv_powersave_common
/* No return */
 
 #define CHECK_HMI_INTERRUPT\
@@ -284,7 +285,7 @@ ALT_FTR_SECTION_END_NESTED_IFSET(CPU_FTR_ARCH_207S, 66);
\
  * r13 - Contents of HSPRG0
  * cr3 - set to gt if waking up with partial/complete hypervisor state loss
  */
-_GLOBAL(power7_restore_hyp_resource)
+_GLOBAL(pnv_restore_hyp_resource)
/*
 * Check if last bit of HSPGR0 is set. This indicates whether we are
 * waking up from winkle.
@@ -296,7 +297,7 @@ _GLOBAL(power7_restore_hyp_resource)
 
lbz r0,PACA_THREAD_IDLE_STATE(r13)
cmpwi   cr2,r0,PNV_THREAD_NAP
-   bgt cr2,power7_wakeup_tb_loss   /* Either sleep or Winkle */
+   bgt cr2,pnv_wakeup_tb_loss  /* Either sleep or Winkle */
 
/*
 * We fall through here if PACA_THREAD_IDLE_STATE shows we are waking
@@ -306,10 +307,10 @@ _GLOBAL(power7_restore_hyp_resource)
bgt cr3,.
 
blr /* Return back to System Reset vector from where
-  power7_restore_hyp_resource was invoked */
+  pnv_restore_

[PATCH v7 08/11] cpuidle/powernv: Use CPUIDLE_STATE_MAX instead of MAX_POWERNV_IDLE_STATES

[Shreyas B. Prabhu]

Use cpuidle's CPUIDLE_STATE_MAX macro instead of powernv specific
MAX_POWERNV_IDLE_STATES.

Cc: Rafael J. Wysocki <rafael.j.wyso...@intel.com>
Cc: Daniel Lezcano <daniel.lezc...@linaro.org>
Cc: linux...@vger.kernel.org
Acked-by: Daniel Lezcano <daniel.lezc...@linaro.org>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
 - No changes after v5

Changes in v5
=
 - New in v5

 drivers/cpuidle/cpuidle-powernv.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/drivers/cpuidle/cpuidle-powernv.c 
b/drivers/cpuidle/cpuidle-powernv.c
index e12dc30..3a763a8 100644
--- a/drivers/cpuidle/cpuidle-powernv.c
+++ b/drivers/cpuidle/cpuidle-powernv.c
@@ -20,8 +20,6 @@
 #include 
 #include 
 
-#define MAX_POWERNV_IDLE_STATES8
-
 struct cpuidle_driver powernv_idle_driver = {
.name = "powernv_idle",
.owner= THIS_MODULE,
@@ -96,7 +94,7 @@ static int fastsleep_loop(struct cpuidle_device *dev,
 /*
  * States for dedicated partition case.
  */
-static struct cpuidle_state powernv_states[MAX_POWERNV_IDLE_STATES] = {
+static struct cpuidle_state powernv_states[CPUIDLE_STATE_MAX] = {
{ /* Snooze */
.name = "snooze",
.desc = "snooze",
-- 
2.4.11

[PATCH v7 04/11] powerpc/powernv: Rename reusable idle functions to hardware agnostic names

[Shreyas B. Prabhu]

Functions like power7_wakeup_loss, power7_wakeup_noloss,
power7_wakeup_tb_loss are used by POWER7 and POWER8 hardware. They can
also be used by POWER9. Hence rename these functions hardware agnostic
names.

Suggested-by: Gautham R. Shenoy 
Signed-off-by: Shreyas B. Prabhu 
---
 - No changes since v4

Changes in v4:
==
 - renaming power7_powersave_common to pnv_powersave_common
 - renaming power7_enter_nap_mode to pnv_enter_arch207_idle_mode

 arch/powerpc/kernel/exceptions-64s.S|  8 
 arch/powerpc/kernel/idle_book3s.S   | 33 +
 arch/powerpc/kvm/book3s_hv_rmhandlers.S |  4 ++--
 3 files changed, 23 insertions(+), 22 deletions(-)

diff --git a/arch/powerpc/kernel/exceptions-64s.S 
b/arch/powerpc/kernel/exceptions-64s.S
index 4a74d6a..2a123cd 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -108,7 +108,7 @@ BEGIN_FTR_SECTION
 
cmpwi   cr3,r13,2
GET_PACA(r13)
-   bl  power7_restore_hyp_resource
+   bl  pnv_restore_hyp_resource
 
li  r0,PNV_THREAD_RUNNING
stb r0,PACA_THREAD_IDLE_STATE(r13)  /* Clear thread state */
@@ -128,8 +128,8 @@ BEGIN_FTR_SECTION
/* Return SRR1 from power7_nap() */
mfspr   r3,SPRN_SRR1
blt cr3,2f
-   b   power7_wakeup_loss
-2: b   power7_wakeup_noloss
+   b   pnv_wakeup_loss
+2: b   pnv_wakeup_noloss
 
 9:
 END_FTR_SECTION_IFSET(CPU_FTR_HVMODE | CPU_FTR_ARCH_206)
@@ -1269,7 +1269,7 @@ machine_check_handle_early:
GET_PACA(r13)
ld  r1,PACAR1(r13)
li  r3,PNV_THREAD_NAP
-   b   power7_enter_nap_mode
+   b   pnv_enter_arch207_idle_mode
 4:
 #endif
/*
diff --git a/arch/powerpc/kernel/idle_book3s.S 
b/arch/powerpc/kernel/idle_book3s.S
index d5def06..34dbfc9 100644
--- a/arch/powerpc/kernel/idle_book3s.S
+++ b/arch/powerpc/kernel/idle_book3s.S
@@ -1,5 +1,6 @@
 /*
- *  This file contains the power_save function for Power7 CPUs.
+ *  This file contains idle entry/exit functions for POWER7 and
+ *  POWER8 CPUs.
  *
  *  This program is free software; you can redistribute it and/or
  *  modify it under the terms of the GNU General Public License
@@ -75,7 +76,7 @@ core_idle_lock_held:
  * 0 - don't check
  * 1 - check
  */
-_GLOBAL(power7_powersave_common)
+_GLOBAL(pnv_powersave_common)
/* Use r3 to pass state nap/sleep/winkle */
/* NAP is a state loss, we create a regs frame on the
 * stack, fill it up with the state we care about and
@@ -135,14 +136,14 @@ _GLOBAL(power7_powersave_common)
LOAD_REG_IMMEDIATE(r5, MSR_IDLE)
li  r6, MSR_RI
andcr6, r9, r6
-   LOAD_REG_ADDR(r7, power7_enter_nap_mode)
+   LOAD_REG_ADDR(r7, pnv_enter_arch207_idle_mode)
mtmsrd  r6, 1   /* clear RI before setting SRR0/1 */
mtspr   SPRN_SRR0, r7
mtspr   SPRN_SRR1, r5
rfid
 
-   .globl  power7_enter_nap_mode
-power7_enter_nap_mode:
+   .globl pnv_enter_arch207_idle_mode
+pnv_enter_arch207_idle_mode:
 #ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
/* Tell KVM we're napping */
li  r4,KVM_HWTHREAD_IN_NAP
@@ -242,19 +243,19 @@ _GLOBAL(power7_idle)
 _GLOBAL(power7_nap)
mr  r4,r3
li  r3,PNV_THREAD_NAP
-   b   power7_powersave_common
+   b   pnv_powersave_common
/* No return */
 
 _GLOBAL(power7_sleep)
li  r3,PNV_THREAD_SLEEP
li  r4,1
-   b   power7_powersave_common
+   b   pnv_powersave_common
/* No return */
 
 _GLOBAL(power7_winkle)
li  r3,PNV_THREAD_WINKLE
li  r4,1
-   b   power7_powersave_common
+   b   pnv_powersave_common
/* No return */
 
 #define CHECK_HMI_INTERRUPT\
@@ -284,7 +285,7 @@ ALT_FTR_SECTION_END_NESTED_IFSET(CPU_FTR_ARCH_207S, 66);
\
  * r13 - Contents of HSPRG0
  * cr3 - set to gt if waking up with partial/complete hypervisor state loss
  */
-_GLOBAL(power7_restore_hyp_resource)
+_GLOBAL(pnv_restore_hyp_resource)
/*
 * Check if last bit of HSPGR0 is set. This indicates whether we are
 * waking up from winkle.
@@ -296,7 +297,7 @@ _GLOBAL(power7_restore_hyp_resource)
 
lbz r0,PACA_THREAD_IDLE_STATE(r13)
cmpwi   cr2,r0,PNV_THREAD_NAP
-   bgt cr2,power7_wakeup_tb_loss   /* Either sleep or Winkle */
+   bgt cr2,pnv_wakeup_tb_loss  /* Either sleep or Winkle */
 
/*
 * We fall through here if PACA_THREAD_IDLE_STATE shows we are waking
@@ -306,10 +307,10 @@ _GLOBAL(power7_restore_hyp_resource)
bgt cr3,.
 
blr /* Return back to System Reset vector from where
-  power7_restore_hyp_resource was invoked */
+  pnv_restore_hyp_resource was invoked */
 
 
-_GLOBAL(power7_wakeup_tb_loss)
+_GLOBAL

[PATCH v7 08/11] cpuidle/powernv: Use CPUIDLE_STATE_MAX instead of MAX_POWERNV_IDLE_STATES

[Shreyas B. Prabhu]

Use cpuidle's CPUIDLE_STATE_MAX macro instead of powernv specific
MAX_POWERNV_IDLE_STATES.

Cc: Rafael J. Wysocki 
Cc: Daniel Lezcano 
Cc: linux...@vger.kernel.org
Acked-by: Daniel Lezcano 
Signed-off-by: Shreyas B. Prabhu 
---
 - No changes after v5

Changes in v5
=
 - New in v5

 drivers/cpuidle/cpuidle-powernv.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/drivers/cpuidle/cpuidle-powernv.c 
b/drivers/cpuidle/cpuidle-powernv.c
index e12dc30..3a763a8 100644
--- a/drivers/cpuidle/cpuidle-powernv.c
+++ b/drivers/cpuidle/cpuidle-powernv.c
@@ -20,8 +20,6 @@
 #include 
 #include 
 
-#define MAX_POWERNV_IDLE_STATES8
-
 struct cpuidle_driver powernv_idle_driver = {
.name = "powernv_idle",
.owner= THIS_MODULE,
@@ -96,7 +94,7 @@ static int fastsleep_loop(struct cpuidle_device *dev,
 /*
  * States for dedicated partition case.
  */
-static struct cpuidle_state powernv_states[MAX_POWERNV_IDLE_STATES] = {
+static struct cpuidle_state powernv_states[CPUIDLE_STATE_MAX] = {
{ /* Snooze */
.name = "snooze",
.desc = "snooze",
-- 
2.4.11

[PATCH v7 00/11] powerpc/powernv/cpuidle: Add support for POWER ISA v3 idle states

[Shreyas B. Prabhu]

POWER ISA v3 defines a new idle processor core mechanism. In summary,
 a) new instruction named stop is added. This instruction replaces
instructions like nap, sleep, rvwinkle.
 b) new per thread SPR named PSSCR is added which controls the behavior
of stop instruction. 

PSSCR has following key fields
Bits 0:3  - Power-Saving Level Status. This field indicates the
lowest power-saving state the thread entered since stop
instruction was last executed.

Bit 42 - Enable State Loss  
0 - No state is lost irrespective of other fields  
1 - Allows state loss

Bits 44:47 - Power-Saving Level Limit  
This limits the power-saving level that can be entered into.

Bits 60:63 - Requested Level  
Used to specify which power-saving level must be entered on
executing stop instruction

Stop idle states and their properties like name, latency, target
residency, psscr value are exposed via device tree.

This patch series adds support for this new mechanism.

Patches 1-6 are cleanups and code movement.
Patch 7 adds platform specific support for stop and psscr handling.
Patch 8 and 9 are minor cleanup in cpuidle driver.
Patch 10 adds cpuidle driver support.
Patch 11 makes offlined cpu use deepest stop state.

Note: Documentation for the device tree bindings is posted here-
http://patchwork.ozlabs.org/patch/629125/

Changes in v7
=
 - File renamed to idle_book3s.S instead of idle_power_common.S
 - Comment changes
 - power_stop0, power_stop renamed to power9_idle and power_idle_stop
 - PSSCR template is now a macro instead of storing in paca
 - power9_idle in C file instead of assembly
 - Fixed TOC related bug
 - Handling subcore within FTR section
 - Functions in idle.c reordered and broken into multiple functions
 - calling __restore_cpu_power8/9 via cur_cpu_spec->cpu_restore 
 - Added a minor patch with minor cleanups in cpuidle-powernv.c . This
   was mainly to make the existing code consistent with the review
   comments for new code
 - Using stack for variables while probing for idle states instead of
   kzalloc/kcalloc

Changes in v6
=
 - Restore new POWER ISA v3 SPRS when waking up from deep idle

Changes in v5
=
 - Use generic cpuidle constant CPUIDLE_NAME_LEN
 - Fix return code handling for of_property_read_string_array
 - Use DT flags to determine if are using stop instruction, instead of
   cpu_has_feature
 - Removed uncessary cast with names
 - _loop -> stop_loop
 - Added POWERNV_THRESHOLD_LATENCY_NS to filter out idle states with high 
latency

Changes in v4
=
 - Added a patch to use PNV_THREAD_WINKLE macro while requesting for winkle
 - Moved power7_powersave_common rename to more appropriate patch
 - renaming power7_enter_nap_mode to pnv_enter_arch207_idle_mode
 - Added PSSCR layout to Patch 7's commit message
 - Improved / Fixed comments
 - Fixed whitespace error in paca.h
 - Using MAX_POSSIBLE_STOP_STATE macro instead of hardcoding 0xF has
   max possible stop state

Changes in v3
=
 - Rebased on powerpc-next
 - Dropping patch 1 since we are not adding a new file for P9 idle support
 - Improved comments in multiple places
 - Moved GET_PACA from power7_restore_hyp_resource to System Reset
 - Instead of moving few functions from idle_power7 to idle_power_common,
   renaming idle_power7.S to idle_power_common.S
 - Moved HSTATE_HWTHREAD_STATE updation to power_powersave_common
 - Dropped earlier patch 5 which moved few macros from idle_power_common to
   asm/cpuidle.h. 
 - Added a patch to rename reusable power7_* idle functions to pnv_*
 - Added new patch that creates abstraction for saving SPRs before
   entering deep idle states
 - Instead of introducing new file idle_power_stop.S, P9 idle support
   is added to idle_power_common.S using CPU_FTR sections.
 - Fixed r4 reg clobbering in power_stop0

Changes in v2
=
 - Rebased on v4.6-rc6
 - Using CPU_FTR_ARCH_300 bit instead of CPU_FTR_STOP_INST

Cc: Rafael J. Wysocki <rafael.j.wyso...@intel.com>
Cc: Daniel Lezcano <daniel.lezc...@linaro.org>
Cc: linux...@vger.kernel.org
Cc: Benjamin Herrenschmidt <b...@au1.ibm.com>
Cc: Michael Ellerman <m...@ellerman.id.au>
Cc: Paul Mackerras <pau...@ozlabs.org>
Cc: Michael Neuling <mi...@neuling.org>
Cc: linuxppc-...@lists.ozlabs.org
Cc: Rob Herring <robh...@kernel.org>
Cc: Lorenzo Pieralisi <lorenzo.pieral...@arm.com>

Shreyas B. Prabhu (11):
  powerpc/powernv: Use PNV_THREAD_WINKLE macro while requesting for
winkle
  powerpc/kvm: make hypervisor state restore a function
  powerpc/powernv: Rename idle_power7.S to idle_book3s.S
  powerpc/powernv: Rename reusable idle functions to hardware agnostic
names
  powerpc/powernv: Make pnv_powersave_common more generic
  powerpc/pow

[PATCH v7 00/11] powerpc/powernv/cpuidle: Add support for POWER ISA v3 idle states

[Shreyas B. Prabhu]

POWER ISA v3 defines a new idle processor core mechanism. In summary,
 a) new instruction named stop is added. This instruction replaces
instructions like nap, sleep, rvwinkle.
 b) new per thread SPR named PSSCR is added which controls the behavior
of stop instruction. 

PSSCR has following key fields
Bits 0:3  - Power-Saving Level Status. This field indicates the
lowest power-saving state the thread entered since stop
instruction was last executed.

Bit 42 - Enable State Loss  
0 - No state is lost irrespective of other fields  
1 - Allows state loss

Bits 44:47 - Power-Saving Level Limit  
This limits the power-saving level that can be entered into.

Bits 60:63 - Requested Level  
Used to specify which power-saving level must be entered on
executing stop instruction

Stop idle states and their properties like name, latency, target
residency, psscr value are exposed via device tree.

This patch series adds support for this new mechanism.

Patches 1-6 are cleanups and code movement.
Patch 7 adds platform specific support for stop and psscr handling.
Patch 8 and 9 are minor cleanup in cpuidle driver.
Patch 10 adds cpuidle driver support.
Patch 11 makes offlined cpu use deepest stop state.

Note: Documentation for the device tree bindings is posted here-
http://patchwork.ozlabs.org/patch/629125/

Changes in v7
=
 - File renamed to idle_book3s.S instead of idle_power_common.S
 - Comment changes
 - power_stop0, power_stop renamed to power9_idle and power_idle_stop
 - PSSCR template is now a macro instead of storing in paca
 - power9_idle in C file instead of assembly
 - Fixed TOC related bug
 - Handling subcore within FTR section
 - Functions in idle.c reordered and broken into multiple functions
 - calling __restore_cpu_power8/9 via cur_cpu_spec->cpu_restore 
 - Added a minor patch with minor cleanups in cpuidle-powernv.c . This
   was mainly to make the existing code consistent with the review
   comments for new code
 - Using stack for variables while probing for idle states instead of
   kzalloc/kcalloc

Changes in v6
=
 - Restore new POWER ISA v3 SPRS when waking up from deep idle

Changes in v5
=
 - Use generic cpuidle constant CPUIDLE_NAME_LEN
 - Fix return code handling for of_property_read_string_array
 - Use DT flags to determine if are using stop instruction, instead of
   cpu_has_feature
 - Removed uncessary cast with names
 - _loop -> stop_loop
 - Added POWERNV_THRESHOLD_LATENCY_NS to filter out idle states with high 
latency

Changes in v4
=
 - Added a patch to use PNV_THREAD_WINKLE macro while requesting for winkle
 - Moved power7_powersave_common rename to more appropriate patch
 - renaming power7_enter_nap_mode to pnv_enter_arch207_idle_mode
 - Added PSSCR layout to Patch 7's commit message
 - Improved / Fixed comments
 - Fixed whitespace error in paca.h
 - Using MAX_POSSIBLE_STOP_STATE macro instead of hardcoding 0xF has
   max possible stop state

Changes in v3
=
 - Rebased on powerpc-next
 - Dropping patch 1 since we are not adding a new file for P9 idle support
 - Improved comments in multiple places
 - Moved GET_PACA from power7_restore_hyp_resource to System Reset
 - Instead of moving few functions from idle_power7 to idle_power_common,
   renaming idle_power7.S to idle_power_common.S
 - Moved HSTATE_HWTHREAD_STATE updation to power_powersave_common
 - Dropped earlier patch 5 which moved few macros from idle_power_common to
   asm/cpuidle.h. 
 - Added a patch to rename reusable power7_* idle functions to pnv_*
 - Added new patch that creates abstraction for saving SPRs before
   entering deep idle states
 - Instead of introducing new file idle_power_stop.S, P9 idle support
   is added to idle_power_common.S using CPU_FTR sections.
 - Fixed r4 reg clobbering in power_stop0

Changes in v2
=
 - Rebased on v4.6-rc6
 - Using CPU_FTR_ARCH_300 bit instead of CPU_FTR_STOP_INST

Cc: Rafael J. Wysocki 
Cc: Daniel Lezcano 
Cc: linux...@vger.kernel.org
Cc: Benjamin Herrenschmidt 
Cc: Michael Ellerman 
Cc: Paul Mackerras 
Cc: Michael Neuling 
Cc: linuxppc-...@lists.ozlabs.org
Cc: Rob Herring 
Cc: Lorenzo Pieralisi 

Shreyas B. Prabhu (11):
  powerpc/powernv: Use PNV_THREAD_WINKLE macro while requesting for
winkle
  powerpc/kvm: make hypervisor state restore a function
  powerpc/powernv: Rename idle_power7.S to idle_book3s.S
  powerpc/powernv: Rename reusable idle functions to hardware agnostic
names
  powerpc/powernv: Make pnv_powersave_common more generic
  powerpc/powernv: abstraction for saving SPRs before entering deep idle
states
  powerpc/powernv: Add platform support for stop instruction
  cpuidle/powernv: Use CPUIDLE_STATE_MAX instead of
MAX_POWERNV_IDLE_STATES
  cpuidle/p

Re: [v6, 08/11] powerpc/powernv: Add platform support for stop instruction

[Shreyas B Prabhu]

Hi Michael,

On 06/15/2016 04:44 PM, Michael Ellerman wrote:
> Hi Shreyas,
> 
> Comments inline ...
> 
> On Wed, 2016-08-06 at 16:54:28 UTC, "Shreyas B. Prabhu" wrote:
>> POWER ISA v3 defines a new idle processor core mechanism. In summary,
>>  a) new instruction named stop is added. This instruction replaces
>>  instructions like nap, sleep, rvwinkle.
>>  b) new per thread SPR named Processor Stop Status and Control Register
>>  (PSSCR) is added which controls the behavior of stop instruction.
>>
>> PSSCR layout:
>> --
>> | PLS | /// | SD | ESL | EC | PSLL | /// | TR | MTL | RL |
>> --
>> 0  4 41   4243   44 4854   5660
>>
>> PSSCR key fields:
>>  Bits 0:3  - Power-Saving Level Status. This field indicates the lowest
>>  power-saving state the thread entered since stop instruction was last
>>  executed.
>>
>>  Bit 42 - Enable State Loss
>>  0 - No state is lost irrespective of other fields
>>  1 - Allows state loss
>>
>>  Bits 44:47 - Power-Saving Level Limit
>>  This limits the power-saving level that can be entered into.
>>
>>  Bits 60:63 - Requested Level
>>  Used to specify which power-saving level must be entered on executing
>>  stop instruction
> 
> That would probably be good as a comment somewhere too, maybe idle.c
> 

Ok. I'll add it there.

>> diff --git a/arch/powerpc/include/asm/cpuidle.h 
>> b/arch/powerpc/include/asm/cpuidle.h
>> index d2f99ca..3d7fc06 100644
>> --- a/arch/powerpc/include/asm/cpuidle.h
>> +++ b/arch/powerpc/include/asm/cpuidle.h
>> @@ -13,6 +13,8 @@
>>  #ifndef __ASSEMBLY__
>>  extern u32 pnv_fastsleep_workaround_at_entry[];
>>  extern u32 pnv_fastsleep_workaround_at_exit[];
>> +
>> +extern u64 pnv_first_deep_stop_state;
> 
> Should this have some safe initial value?
> 
>> diff --git a/arch/powerpc/include/asm/machdep.h 
>> b/arch/powerpc/include/asm/machdep.h
>> index 6bdcd0d..ae3b155 100644
>> --- a/arch/powerpc/include/asm/machdep.h
>> +++ b/arch/powerpc/include/asm/machdep.h
>> @@ -262,6 +262,7 @@ struct machdep_calls {
>>  extern void e500_idle(void);
>>  extern void power4_idle(void);
>>  extern void power7_idle(void);
>> +extern void power_stop0(void);
> 
> Can that have a better name please?

What do you have in mind?
power_arch300_idle0()?
power_arch300_stop0()?
or I can use power9_idle() here. But we will still need a better
replacement for power_stop() below.

> 
>> diff --git a/arch/powerpc/include/asm/opal-api.h 
>> b/arch/powerpc/include/asm/opal-api.h
>> index 9bb8ddf..7f3f8c6 100644
>> --- a/arch/powerpc/include/asm/opal-api.h
>> +++ b/arch/powerpc/include/asm/opal-api.h
>> @@ -162,13 +162,20 @@
>>  
>>  /* Device tree flags */
>>  
>> -/* Flags set in power-mgmt nodes in device tree if
>> - * respective idle states are supported in the platform.
>> +/*
>> + * Flags set in power-mgmt nodes in device tree describing
>> + * idle states that are supported in the platform.
>>   */
>> +
>> +#define OPAL_PM_TIMEBASE_STOP   0x0002
>> +#define OPAL_PM_LOSE_HYP_CONTEXT0x2000
>> +#define OPAL_PM_LOSE_FULL_CONTEXT   0x4000
>>  #define OPAL_PM_NAP_ENABLED 0x0001
>>  #define OPAL_PM_SLEEP_ENABLED   0x0002
>>  #define OPAL_PM_WINKLE_ENABLED  0x0004
>>  #define OPAL_PM_SLEEP_ENABLED_ER1   0x0008 /* with workaround */
>> +#define OPAL_PM_STOP_INST_FAST  0x0010
>> +#define OPAL_PM_STOP_INST_DEEP  0x0020
> 
> I don't see the above in skiboot yet?

I've posted it here -
http://patchwork.ozlabs.org/patch/617828/
> 
>> diff --git a/arch/powerpc/include/asm/paca.h 
>> b/arch/powerpc/include/asm/paca.h
>> index 546540b..ae91b44 100644
>> --- a/arch/powerpc/include/asm/paca.h
>> +++ b/arch/powerpc/include/asm/paca.h
>> @@ -171,6 +171,8 @@ struct paca_struct {
>>  /* Mask to denote subcore sibling threads */
>>  u8 subcore_sibling_mask;
>>  #endif
>> +/* Template for PSSCR with EC, ESL, TR, PSLL, MTL fields set */
>> +u64 thread_psscr;
> 
> I'm not entirely clear on why that needs to be in the paca. Could it not be 
> global?
> 

While we use Requested Level (RL) field of PSSCR to request a stop
level, other fields in the SPR like EC, ESL, TR, PSLL, MTL can be
modified by individual threads less fr

Re: [v6, 08/11] powerpc/powernv: Add platform support for stop instruction

[Shreyas B Prabhu]

Hi Michael,

On 06/15/2016 04:44 PM, Michael Ellerman wrote:
> Hi Shreyas,
> 
> Comments inline ...
> 
> On Wed, 2016-08-06 at 16:54:28 UTC, "Shreyas B. Prabhu" wrote:
>> POWER ISA v3 defines a new idle processor core mechanism. In summary,
>>  a) new instruction named stop is added. This instruction replaces
>>  instructions like nap, sleep, rvwinkle.
>>  b) new per thread SPR named Processor Stop Status and Control Register
>>  (PSSCR) is added which controls the behavior of stop instruction.
>>
>> PSSCR layout:
>> --
>> | PLS | /// | SD | ESL | EC | PSLL | /// | TR | MTL | RL |
>> --
>> 0  4 41   4243   44 4854   5660
>>
>> PSSCR key fields:
>>  Bits 0:3  - Power-Saving Level Status. This field indicates the lowest
>>  power-saving state the thread entered since stop instruction was last
>>  executed.
>>
>>  Bit 42 - Enable State Loss
>>  0 - No state is lost irrespective of other fields
>>  1 - Allows state loss
>>
>>  Bits 44:47 - Power-Saving Level Limit
>>  This limits the power-saving level that can be entered into.
>>
>>  Bits 60:63 - Requested Level
>>  Used to specify which power-saving level must be entered on executing
>>  stop instruction
> 
> That would probably be good as a comment somewhere too, maybe idle.c
> 

Ok. I'll add it there.

>> diff --git a/arch/powerpc/include/asm/cpuidle.h 
>> b/arch/powerpc/include/asm/cpuidle.h
>> index d2f99ca..3d7fc06 100644
>> --- a/arch/powerpc/include/asm/cpuidle.h
>> +++ b/arch/powerpc/include/asm/cpuidle.h
>> @@ -13,6 +13,8 @@
>>  #ifndef __ASSEMBLY__
>>  extern u32 pnv_fastsleep_workaround_at_entry[];
>>  extern u32 pnv_fastsleep_workaround_at_exit[];
>> +
>> +extern u64 pnv_first_deep_stop_state;
> 
> Should this have some safe initial value?
> 
>> diff --git a/arch/powerpc/include/asm/machdep.h 
>> b/arch/powerpc/include/asm/machdep.h
>> index 6bdcd0d..ae3b155 100644
>> --- a/arch/powerpc/include/asm/machdep.h
>> +++ b/arch/powerpc/include/asm/machdep.h
>> @@ -262,6 +262,7 @@ struct machdep_calls {
>>  extern void e500_idle(void);
>>  extern void power4_idle(void);
>>  extern void power7_idle(void);
>> +extern void power_stop0(void);
> 
> Can that have a better name please?

What do you have in mind?
power_arch300_idle0()?
power_arch300_stop0()?
or I can use power9_idle() here. But we will still need a better
replacement for power_stop() below.

> 
>> diff --git a/arch/powerpc/include/asm/opal-api.h 
>> b/arch/powerpc/include/asm/opal-api.h
>> index 9bb8ddf..7f3f8c6 100644
>> --- a/arch/powerpc/include/asm/opal-api.h
>> +++ b/arch/powerpc/include/asm/opal-api.h
>> @@ -162,13 +162,20 @@
>>  
>>  /* Device tree flags */
>>  
>> -/* Flags set in power-mgmt nodes in device tree if
>> - * respective idle states are supported in the platform.
>> +/*
>> + * Flags set in power-mgmt nodes in device tree describing
>> + * idle states that are supported in the platform.
>>   */
>> +
>> +#define OPAL_PM_TIMEBASE_STOP   0x0002
>> +#define OPAL_PM_LOSE_HYP_CONTEXT0x2000
>> +#define OPAL_PM_LOSE_FULL_CONTEXT   0x4000
>>  #define OPAL_PM_NAP_ENABLED 0x0001
>>  #define OPAL_PM_SLEEP_ENABLED   0x0002
>>  #define OPAL_PM_WINKLE_ENABLED  0x0004
>>  #define OPAL_PM_SLEEP_ENABLED_ER1   0x0008 /* with workaround */
>> +#define OPAL_PM_STOP_INST_FAST  0x0010
>> +#define OPAL_PM_STOP_INST_DEEP  0x0020
> 
> I don't see the above in skiboot yet?

I've posted it here -
http://patchwork.ozlabs.org/patch/617828/
> 
>> diff --git a/arch/powerpc/include/asm/paca.h 
>> b/arch/powerpc/include/asm/paca.h
>> index 546540b..ae91b44 100644
>> --- a/arch/powerpc/include/asm/paca.h
>> +++ b/arch/powerpc/include/asm/paca.h
>> @@ -171,6 +171,8 @@ struct paca_struct {
>>  /* Mask to denote subcore sibling threads */
>>  u8 subcore_sibling_mask;
>>  #endif
>> +/* Template for PSSCR with EC, ESL, TR, PSLL, MTL fields set */
>> +u64 thread_psscr;
> 
> I'm not entirely clear on why that needs to be in the paca. Could it not be 
> global?
> 

While we use Requested Level (RL) field of PSSCR to request a stop
level, other fields in the SPR like EC, ESL, TR, PSLL, MTL can be
modified by individual threads less fr

Re: [v6, 07/11] powerpc/powernv: set power_save func after the idle states are initialized

[Shreyas B Prabhu]

On 06/15/2016 11:11 AM, Michael Ellerman wrote:
> On Wed, 2016-08-06 at 16:54:27 UTC, "Shreyas B. Prabhu" wrote:
>> pnv_init_idle_states discovers supported idle states from the
>> device tree and does the required initialization. Set power_save
>> function pointer only after this initialization is done
> 
> This looks like a bug fix? Or is this not a concern in practice for some 
> reason
> (and if so what is that reason)?
> 

This isn't a concern currently because, all powernv machines so far
supported nap and nap does not need any initialization from kernel side.

Thanks,
Shreyas

Re: [v6, 07/11] powerpc/powernv: set power_save func after the idle states are initialized

[Shreyas B Prabhu]

On 06/15/2016 11:11 AM, Michael Ellerman wrote:
> On Wed, 2016-08-06 at 16:54:27 UTC, "Shreyas B. Prabhu" wrote:
>> pnv_init_idle_states discovers supported idle states from the
>> device tree and does the required initialization. Set power_save
>> function pointer only after this initialization is done
> 
> This looks like a bug fix? Or is this not a concern in practice for some 
> reason
> (and if so what is that reason)?
> 

This isn't a concern currently because, all powernv machines so far
supported nap and nap does not need any initialization from kernel side.

Thanks,
Shreyas

Re: [PATCH v6 10/11] cpuidle/powernv: Add support for POWER ISA v3 idle states

[Shreyas B Prabhu]

On 06/14/2016 04:59 PM, Benjamin Herrenschmidt wrote:
> On Tue, 2016-06-14 at 16:17 +0530, Shreyas B Prabhu wrote:
> 
>>
>> I ignored adding this check because this is part of initcall and we are
>> unlikely to run out of memory at this state. But I'll add the check in
>> next version.
> 
> Why do you malloc the u64 array and not the string pointer array ?
> Shouldn't you either have both on stack or both allocated ?
> 

Yes. I'll make this consistent.

Thanks,
Shreyas

Re: [PATCH v6 10/11] cpuidle/powernv: Add support for POWER ISA v3 idle states

[Shreyas B Prabhu]

On 06/14/2016 04:59 PM, Benjamin Herrenschmidt wrote:
> On Tue, 2016-06-14 at 16:17 +0530, Shreyas B Prabhu wrote:
> 
>>
>> I ignored adding this check because this is part of initcall and we are
>> unlikely to run out of memory at this state. But I'll add the check in
>> next version.
> 
> Why do you malloc the u64 array and not the string pointer array ?
> Shouldn't you either have both on stack or both allocated ?
> 

Yes. I'll make this consistent.

Thanks,
Shreyas

Re: [PATCH v6 10/11] cpuidle/powernv: Add support for POWER ISA v3 idle states

[Shreyas B Prabhu]

On 06/14/2016 03:18 AM, Benjamin Herrenschmidt wrote:
> On Wed, 2016-06-08 at 11:54 -0500, Shreyas B. Prabhu wrote:
>>
>>  /*
>>   * States for dedicated partition case.
>>   */
>> @@ -167,6 +183,8 @@ static int powernv_add_idle_states(void)
>>  int nr_idle_states = 1; /* Snooze */
>>  int dt_idle_states;
>>  u32 *latency_ns, *residency_ns, *flags;
>> +u64 *psscr_val = NULL;
>> +const char *names[CPUIDLE_STATE_MAX];
>>  int i, rc;
>>  
>>  /* Currently we have snooze statically defined */
>> @@ -199,12 +217,41 @@ static int powernv_add_idle_states(void)
>>  goto out_free_latency;
>>  }
>>  
>> +rc = of_property_read_string_array(power_mgt,
>> +   "ibm,cpu-idle-state-names", names,
>> +   dt_idle_states);
> 
> Ok so from this I assume that dt_idle_states is the number of entries,
> which has been checked properly to be < CPUIDLE_STATE_MAX correct ?
> 
> Beause ...
>

While dt_idle_states should not be > CPUIDLE_STATE_MAX, if that were the
case we will end up corrupting memory while updating powernv_states[].
I'll add a WARN_ON for such a case and
handle adding idle states to powernv_states accordingly. Thanks for
pointing this out.

>> +if (rc < 0) {
>> +pr_warn("cpuidle-powernv: missing ibm,cpu-idle-state-names in 
>> DT\n");
>> +goto out_free_latency;
>> +}
>> +
>> +/*
>> + * If the idle states use stop instruction, probe for psscr values
>> + * which are necessary to specify required stop level.
>> + */
>> +if (flags[0] & (OPAL_PM_STOP_INST_FAST | OPAL_PM_STOP_INST_DEEP)) {
>> +psscr_val = kcalloc(dt_idle_states, sizeof(*psscr_val),
>> +GFP_KERNEL);
>> +rc = of_property_read_u64_array(power_mgt,
>> +"ibm,cpu-idle-state-psscr",
>> +psscr_val, dt_idle_states);
> 
> Here, psscr val is only one u64 ... shouldn't you kmalloc sizeof(..) *
> dt_idle_states ?

I'm using kcalloc here since checkpatch script suggested kcalloc over
kzalloc for allocating memory for arrays.
I'll also include a patch to use kcalloc throughout the file for
uniformity in next version. I was originally planning to post that
cleanup separately.

Thanks,
Shreyas

Re: [PATCH v6 10/11] cpuidle/powernv: Add support for POWER ISA v3 idle states

[Shreyas B Prabhu]

On 06/14/2016 03:18 AM, Benjamin Herrenschmidt wrote:
> On Wed, 2016-06-08 at 11:54 -0500, Shreyas B. Prabhu wrote:
>>
>>  /*
>>   * States for dedicated partition case.
>>   */
>> @@ -167,6 +183,8 @@ static int powernv_add_idle_states(void)
>>  int nr_idle_states = 1; /* Snooze */
>>  int dt_idle_states;
>>  u32 *latency_ns, *residency_ns, *flags;
>> +u64 *psscr_val = NULL;
>> +const char *names[CPUIDLE_STATE_MAX];
>>  int i, rc;
>>  
>>  /* Currently we have snooze statically defined */
>> @@ -199,12 +217,41 @@ static int powernv_add_idle_states(void)
>>  goto out_free_latency;
>>  }
>>  
>> +rc = of_property_read_string_array(power_mgt,
>> +   "ibm,cpu-idle-state-names", names,
>> +   dt_idle_states);
> 
> Ok so from this I assume that dt_idle_states is the number of entries,
> which has been checked properly to be < CPUIDLE_STATE_MAX correct ?
> 
> Beause ...
>

While dt_idle_states should not be > CPUIDLE_STATE_MAX, if that were the
case we will end up corrupting memory while updating powernv_states[].
I'll add a WARN_ON for such a case and
handle adding idle states to powernv_states accordingly. Thanks for
pointing this out.

>> +if (rc < 0) {
>> +pr_warn("cpuidle-powernv: missing ibm,cpu-idle-state-names in 
>> DT\n");
>> +goto out_free_latency;
>> +}
>> +
>> +/*
>> + * If the idle states use stop instruction, probe for psscr values
>> + * which are necessary to specify required stop level.
>> + */
>> +if (flags[0] & (OPAL_PM_STOP_INST_FAST | OPAL_PM_STOP_INST_DEEP)) {
>> +psscr_val = kcalloc(dt_idle_states, sizeof(*psscr_val),
>> +GFP_KERNEL);
>> +rc = of_property_read_u64_array(power_mgt,
>> +"ibm,cpu-idle-state-psscr",
>> +psscr_val, dt_idle_states);
> 
> Here, psscr val is only one u64 ... shouldn't you kmalloc sizeof(..) *
> dt_idle_states ?

I'm using kcalloc here since checkpatch script suggested kcalloc over
kzalloc for allocating memory for arrays.
I'll also include a patch to use kcalloc throughout the file for
uniformity in next version. I was originally planning to post that
cleanup separately.

Thanks,
Shreyas

Re: [PATCH v6 10/11] cpuidle/powernv: Add support for POWER ISA v3 idle states

[Shreyas B Prabhu]

On 06/13/2016 09:04 PM, Daniel Lezcano wrote:
> On Wed, Jun 08, 2016 at 11:54:30AM -0500, Shreyas B. Prabhu wrote:
>> POWER ISA v3 defines a new idle processor core mechanism. In summary,
>>  a) new instruction named stop is added.
>>  b) new per thread SPR named PSSCR is added which controls the behavior
>>  of stop instruction.
>>
>> Supported idle states and value to be written to PSSCR register to enter
>> any idle state is exposed via ibm,cpu-idle-state-names and
>> ibm,cpu-idle-state-psscr respectively. To enter an idle state,
>> platform provided power_stop() needs to be invoked with the appropriate
>> PSSCR value.
>>
>> This patch adds support for this new mechanism in cpuidle powernv driver.
>>
>> Cc: Rafael J. Wysocki <rafael.j.wyso...@intel.com>
>> Cc: Daniel Lezcano <daniel.lezc...@linaro.org>
>> Cc: Rob Herring <robh...@kernel.org>
>> Cc: Lorenzo Pieralisi <lorenzo.pieral...@arm.com>
>> Cc: linux...@vger.kernel.org
>> Cc: Michael Ellerman <m...@ellerman.id.au>
>> Cc: Paul Mackerras <pau...@ozlabs.org>
>> Cc: linuxppc-...@lists.ozlabs.org
>> Reviewed-by: Gautham R. Shenoy <e...@linux.vnet.ibm.com>
>> Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
>> ---
> 
> [ ... ]
> 
>> +rc = of_property_read_string_array(power_mgt,
>> +   "ibm,cpu-idle-state-names", names,
>> +   dt_idle_states);
>> +if (rc < 0) {
>> +pr_warn("cpuidle-powernv: missing ibm,cpu-idle-state-names in 
>> DT\n");
>> +goto out_free_latency;
>> +}
>> +
>> +/*
>> + * If the idle states use stop instruction, probe for psscr values
>> + * which are necessary to specify required stop level.
>> + */
>> +if (flags[0] & (OPAL_PM_STOP_INST_FAST | OPAL_PM_STOP_INST_DEEP)) {
>> +psscr_val = kcalloc(dt_idle_states, sizeof(*psscr_val),
>> +GFP_KERNEL);
> 
> if (!psscr_val) check missing.

I ignored adding this check because this is part of initcall and we are
unlikely to run out of memory at this state. But I'll add the check in
next version.
> 
>> +rc = of_property_read_u64_array(power_mgt,
>> +"ibm,cpu-idle-state-psscr",
>> +psscr_val, dt_idle_states);
>> +if (rc) {
>> +pr_warn("cpuidle-powernv: missing 
>> ibm,cpu-idle-states-psscr in DT\n");
>> +goto out_free_psscr;
>> +}
>> +}
>>  residency_ns = kzalloc(sizeof(*residency_ns) * dt_idle_states, 
>> GFP_KERNEL);
> 
> if (!residency_ns) check missing.
> 
> I suppose the code is relying on 'of_property_read_u32_array' to check it, 
> right ?

I'll add the NULL check for existing kzalloc's in the file as well in
the next version.

Thanks,
Shreyas

Re: [PATCH v6 10/11] cpuidle/powernv: Add support for POWER ISA v3 idle states

[Shreyas B Prabhu]

On 06/13/2016 09:04 PM, Daniel Lezcano wrote:
> On Wed, Jun 08, 2016 at 11:54:30AM -0500, Shreyas B. Prabhu wrote:
>> POWER ISA v3 defines a new idle processor core mechanism. In summary,
>>  a) new instruction named stop is added.
>>  b) new per thread SPR named PSSCR is added which controls the behavior
>>  of stop instruction.
>>
>> Supported idle states and value to be written to PSSCR register to enter
>> any idle state is exposed via ibm,cpu-idle-state-names and
>> ibm,cpu-idle-state-psscr respectively. To enter an idle state,
>> platform provided power_stop() needs to be invoked with the appropriate
>> PSSCR value.
>>
>> This patch adds support for this new mechanism in cpuidle powernv driver.
>>
>> Cc: Rafael J. Wysocki 
>> Cc: Daniel Lezcano 
>> Cc: Rob Herring 
>> Cc: Lorenzo Pieralisi 
>> Cc: linux...@vger.kernel.org
>> Cc: Michael Ellerman 
>> Cc: Paul Mackerras 
>> Cc: linuxppc-...@lists.ozlabs.org
>> Reviewed-by: Gautham R. Shenoy 
>> Signed-off-by: Shreyas B. Prabhu 
>> ---
> 
> [ ... ]
> 
>> +rc = of_property_read_string_array(power_mgt,
>> +   "ibm,cpu-idle-state-names", names,
>> +   dt_idle_states);
>> +if (rc < 0) {
>> +pr_warn("cpuidle-powernv: missing ibm,cpu-idle-state-names in 
>> DT\n");
>> +goto out_free_latency;
>> +}
>> +
>> +/*
>> + * If the idle states use stop instruction, probe for psscr values
>> + * which are necessary to specify required stop level.
>> + */
>> +if (flags[0] & (OPAL_PM_STOP_INST_FAST | OPAL_PM_STOP_INST_DEEP)) {
>> +psscr_val = kcalloc(dt_idle_states, sizeof(*psscr_val),
>> +GFP_KERNEL);
> 
> if (!psscr_val) check missing.

I ignored adding this check because this is part of initcall and we are
unlikely to run out of memory at this state. But I'll add the check in
next version.
> 
>> +rc = of_property_read_u64_array(power_mgt,
>> +"ibm,cpu-idle-state-psscr",
>> +psscr_val, dt_idle_states);
>> +if (rc) {
>> +pr_warn("cpuidle-powernv: missing 
>> ibm,cpu-idle-states-psscr in DT\n");
>> +goto out_free_psscr;
>> +}
>> +}
>>  residency_ns = kzalloc(sizeof(*residency_ns) * dt_idle_states, 
>> GFP_KERNEL);
> 
> if (!residency_ns) check missing.
> 
> I suppose the code is relying on 'of_property_read_u32_array' to check it, 
> right ?

I'll add the NULL check for existing kzalloc's in the file as well in
the next version.

Thanks,
Shreyas

Re: [PATCH v5 08/11] powerpc/powernv: Add platform support for stop instruction

[Shreyas B Prabhu]

On 06/09/2016 10:12 AM, Sam Bobroff wrote:
> On Thu, Jun 02, 2016 at 07:38:58AM -0500, Shreyas B. Prabhu wrote:
> 
> ...
> 
>> +/* Power Management - PSSCR Fields */
> 
> It might be nice to give the full name of the register, as below with the 
> FPSCR.
> 

I'll make the change while posting the next rev.

Thanks,
Shreyas

>> +#define PSSCR_RL_MASK   0x000F
>> +#define PSSCR_MTL_MASK  0x00F0
>> +#define PSSCR_TR_MASK   0x0300
>> +#define PSSCR_PSLL_MASK 0x000F
>> +#define PSSCR_EC0x0010
>> +#define PSSCR_ESL   0x0020
>> +#define PSSCR_SD0x0040
>> +
>> +
>>  /* Floating Point Status and Control Register (FPSCR) Fields */
> 
> Cheers,
> Sam.
> 

Re: [PATCH v5 08/11] powerpc/powernv: Add platform support for stop instruction

[Shreyas B Prabhu]

On 06/09/2016 10:12 AM, Sam Bobroff wrote:
> On Thu, Jun 02, 2016 at 07:38:58AM -0500, Shreyas B. Prabhu wrote:
> 
> ...
> 
>> +/* Power Management - PSSCR Fields */
> 
> It might be nice to give the full name of the register, as below with the 
> FPSCR.
> 

I'll make the change while posting the next rev.

Thanks,
Shreyas

>> +#define PSSCR_RL_MASK   0x000F
>> +#define PSSCR_MTL_MASK  0x00F0
>> +#define PSSCR_TR_MASK   0x0300
>> +#define PSSCR_PSLL_MASK 0x000F
>> +#define PSSCR_EC0x0010
>> +#define PSSCR_ESL   0x0020
>> +#define PSSCR_SD0x0040
>> +
>> +
>>  /* Floating Point Status and Control Register (FPSCR) Fields */
> 
> Cheers,
> Sam.
> 

Re: [PATCH v5 08/11] powerpc/powernv: Add platform support for stop instruction

[Shreyas B Prabhu]

Hi Ben,

Sorry for the delayed response.

On 06/06/2016 03:58 AM, Benjamin Herrenschmidt wrote:
> On Thu, 2016-06-02 at 07:38 -0500, Shreyas B. Prabhu wrote:
>> @@ -61,8 +72,13 @@ save_sprs_to_stack:
>>  * Note all register i.e per-core, per-subcore or per-thread is saved
>>  * here since any thread in the core might wake up first
>>  */
>> +BEGIN_FTR_SECTION
>> +   mfspr   r3,SPRN_PTCR
>> +   std r3,_PTCR(r1)
>> +FTR_SECTION_ELSE
>> mfspr   r3,SPRN_SDR1
>> std r3,_SDR1(r1)
>> +ALT_FTR_SECTION_END_IFSET(CPU_FTR_ARCH_300)
> 
> This is the only new SPR we care about in P9 ?
> 
After reviewing ISA again, I've identified LMRR, LMSER and ASDR also
need to be restored. I've fixed this in v6.

Thanks,
Shreyas

Re: [PATCH v5 08/11] powerpc/powernv: Add platform support for stop instruction

[Shreyas B Prabhu]

Hi Ben,

Sorry for the delayed response.

On 06/06/2016 03:58 AM, Benjamin Herrenschmidt wrote:
> On Thu, 2016-06-02 at 07:38 -0500, Shreyas B. Prabhu wrote:
>> @@ -61,8 +72,13 @@ save_sprs_to_stack:
>>  * Note all register i.e per-core, per-subcore or per-thread is saved
>>  * here since any thread in the core might wake up first
>>  */
>> +BEGIN_FTR_SECTION
>> +   mfspr   r3,SPRN_PTCR
>> +   std r3,_PTCR(r1)
>> +FTR_SECTION_ELSE
>> mfspr   r3,SPRN_SDR1
>> std r3,_SDR1(r1)
>> +ALT_FTR_SECTION_END_IFSET(CPU_FTR_ARCH_300)
> 
> This is the only new SPR we care about in P9 ?
> 
After reviewing ISA again, I've identified LMRR, LMSER and ASDR also
need to be restored. I've fixed this in v6.

Thanks,
Shreyas

[PATCH v6 07/11] powerpc/powernv: set power_save func after the idle states are initialized

[Shreyas B. Prabhu]

pnv_init_idle_states discovers supported idle states from the
device tree and does the required initialization. Set power_save
function pointer only after this initialization is done

Reviewed-by: Gautham R. Shenoy <e...@linux.vnet.ibm.com>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
- No changes since v1

 arch/powerpc/platforms/powernv/idle.c  | 3 +++
 arch/powerpc/platforms/powernv/setup.c | 2 +-
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index fcc8b68..fbb09fb 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -285,6 +285,9 @@ static int __init pnv_init_idle_states(void)
}
 
pnv_alloc_idle_core_states();
+
+   if (supported_cpuidle_states & OPAL_PM_NAP_ENABLED)
+   ppc_md.power_save = power7_idle;
 out_free:
kfree(flags);
 out:
diff --git a/arch/powerpc/platforms/powernv/setup.c 
b/arch/powerpc/platforms/powernv/setup.c
index ee6430b..8492bbb 100644
--- a/arch/powerpc/platforms/powernv/setup.c
+++ b/arch/powerpc/platforms/powernv/setup.c
@@ -315,7 +315,7 @@ define_machine(powernv) {
.get_proc_freq  = pnv_get_proc_freq,
.progress   = pnv_progress,
.machine_shutdown   = pnv_shutdown,
-   .power_save = power7_idle,
+   .power_save = NULL,
.calibrate_decr = generic_calibrate_decr,
 #ifdef CONFIG_KEXEC
.kexec_cpu_down = pnv_kexec_cpu_down,
-- 
2.1.4

[PATCH v6 08/11] powerpc/powernv: Add platform support for stop instruction

[Shreyas B. Prabhu]

POWER ISA v3 defines a new idle processor core mechanism. In summary,
 a) new instruction named stop is added. This instruction replaces
instructions like nap, sleep, rvwinkle.
 b) new per thread SPR named Processor Stop Status and Control Register
(PSSCR) is added which controls the behavior of stop instruction.

PSSCR layout:
--
| PLS | /// | SD | ESL | EC | PSLL | /// | TR | MTL | RL |
--
0  4 41   4243   44 4854   5660

PSSCR key fields:
Bits 0:3  - Power-Saving Level Status. This field indicates the lowest
power-saving state the thread entered since stop instruction was last
executed.

Bit 42 - Enable State Loss
0 - No state is lost irrespective of other fields
1 - Allows state loss

Bits 44:47 - Power-Saving Level Limit
This limits the power-saving level that can be entered into.

Bits 60:63 - Requested Level
Used to specify which power-saving level must be entered on executing
stop instruction

This patch adds support for stop instruction and PSSCR handling.

Reviewed-by: Gautham R. Shenoy <e...@linux.vnet.ibm.com>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
Changes in v6
=
 - Save/restore new P9 SPRs when using deep idle states

Changes in v4:
==
 - Added PSSCR layout to commit message
 - Improved / Fixed comments
 - Fixed whitespace error in paca.h
 - Using MAX_POSSIBLE_STOP_STATE macro instead of hardcoding 0xF as 
   max possible stop state

Changes in v3:
==
 - Instead of introducing new file idle_power_stop.S, P9 idle support
   is added to idle_power_common.S using CPU_FTR sections.
 - Fixed r4 reg clobbering in power_stop0
 - Improved comments

Changes in v2:
==
 - Using CPU_FTR_ARCH_300 bit instead of CPU_FTR_STOP_INST

 arch/powerpc/include/asm/cpuidle.h|   2 +
 arch/powerpc/include/asm/kvm_book3s_asm.h |   2 +-
 arch/powerpc/include/asm/machdep.h|   1 +
 arch/powerpc/include/asm/opal-api.h   |  11 +-
 arch/powerpc/include/asm/paca.h   |   2 +
 arch/powerpc/include/asm/ppc-opcode.h |   4 +
 arch/powerpc/include/asm/processor.h  |   1 +
 arch/powerpc/include/asm/reg.h|  14 +++
 arch/powerpc/kernel/asm-offsets.c |   2 +
 arch/powerpc/kernel/idle_power_common.S   | 175 +++---
 arch/powerpc/platforms/powernv/idle.c |  84 --
 11 files changed, 265 insertions(+), 33 deletions(-)

diff --git a/arch/powerpc/include/asm/cpuidle.h 
b/arch/powerpc/include/asm/cpuidle.h
index d2f99ca..3d7fc06 100644
--- a/arch/powerpc/include/asm/cpuidle.h
+++ b/arch/powerpc/include/asm/cpuidle.h
@@ -13,6 +13,8 @@
 #ifndef __ASSEMBLY__
 extern u32 pnv_fastsleep_workaround_at_entry[];
 extern u32 pnv_fastsleep_workaround_at_exit[];
+
+extern u64 pnv_first_deep_stop_state;
 #endif
 
 #endif
diff --git a/arch/powerpc/include/asm/kvm_book3s_asm.h 
b/arch/powerpc/include/asm/kvm_book3s_asm.h
index 72b6225..d318d43 100644
--- a/arch/powerpc/include/asm/kvm_book3s_asm.h
+++ b/arch/powerpc/include/asm/kvm_book3s_asm.h
@@ -162,7 +162,7 @@ struct kvmppc_book3s_shadow_vcpu {
 
 /* Values for kvm_state */
 #define KVM_HWTHREAD_IN_KERNEL 0
-#define KVM_HWTHREAD_IN_NAP1
+#define KVM_HWTHREAD_IN_IDLE   1
 #define KVM_HWTHREAD_IN_KVM2
 
 #endif /* __ASM_KVM_BOOK3S_ASM_H__ */
diff --git a/arch/powerpc/include/asm/machdep.h 
b/arch/powerpc/include/asm/machdep.h
index 6bdcd0d..ae3b155 100644
--- a/arch/powerpc/include/asm/machdep.h
+++ b/arch/powerpc/include/asm/machdep.h
@@ -262,6 +262,7 @@ struct machdep_calls {
 extern void e500_idle(void);
 extern void power4_idle(void);
 extern void power7_idle(void);
+extern void power_stop0(void);
 extern void ppc6xx_idle(void);
 extern void book3e_idle(void);
 
diff --git a/arch/powerpc/include/asm/opal-api.h 
b/arch/powerpc/include/asm/opal-api.h
index 9bb8ddf..7f3f8c6 100644
--- a/arch/powerpc/include/asm/opal-api.h
+++ b/arch/powerpc/include/asm/opal-api.h
@@ -162,13 +162,20 @@
 
 /* Device tree flags */
 
-/* Flags set in power-mgmt nodes in device tree if
- * respective idle states are supported in the platform.
+/*
+ * Flags set in power-mgmt nodes in device tree describing
+ * idle states that are supported in the platform.
  */
+
+#define OPAL_PM_TIMEBASE_STOP  0x0002
+#define OPAL_PM_LOSE_HYP_CONTEXT   0x2000
+#define OPAL_PM_LOSE_FULL_CONTEXT  0x4000
 #define OPAL_PM_NAP_ENABLED0x0001
 #define OPAL_PM_SLEEP_ENABLED  0x0002
 #define OPAL_PM_WINKLE_ENABLED 0x0004
 #define OPAL_PM_SLEEP_ENABLED_ER1  0x0008 /* with workaround */
+#define OPAL_PM_STOP_INST_FAST 0x0010
+#define OPAL_PM_STOP_INST_DEEP 0x0020
 
 /*
  * OPAL_CONFIG_CPU_IDLE_STATE parameters
diff --git a/

[PATCH v6 06/11] powerpc/powernv: abstraction for saving SPRs before entering deep idle states

[Shreyas B. Prabhu]

Create a function for saving SPRs before entering deep idle states.
This function can be reused for POWER9 deep idle states.

Reviewed-by: Gautham R. Shenoy <e...@linux.vnet.ibm.com>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
 - No changes since v3

Changes in v3:
=
 - Newly added in v3

 arch/powerpc/kernel/idle_power_common.S | 54 +++--
 1 file changed, 32 insertions(+), 22 deletions(-)

diff --git a/arch/powerpc/kernel/idle_power_common.S 
b/arch/powerpc/kernel/idle_power_common.S
index a8397e3..2f909a1 100644
--- a/arch/powerpc/kernel/idle_power_common.S
+++ b/arch/powerpc/kernel/idle_power_common.S
@@ -53,6 +53,36 @@
.text
 
 /*
+ * Used by threads before entering deep idle states. Saves SPRs
+ * in interrupt stack frame
+ */
+save_sprs_to_stack:
+   /*
+* Note all register i.e per-core, per-subcore or per-thread is saved
+* here since any thread in the core might wake up first
+*/
+   mfspr   r3,SPRN_SDR1
+   std r3,_SDR1(r1)
+   mfspr   r3,SPRN_RPR
+   std r3,_RPR(r1)
+   mfspr   r3,SPRN_SPURR
+   std r3,_SPURR(r1)
+   mfspr   r3,SPRN_PURR
+   std r3,_PURR(r1)
+   mfspr   r3,SPRN_TSCR
+   std r3,_TSCR(r1)
+   mfspr   r3,SPRN_DSCR
+   std r3,_DSCR(r1)
+   mfspr   r3,SPRN_AMOR
+   std r3,_AMOR(r1)
+   mfspr   r3,SPRN_WORT
+   std r3,_WORT(r1)
+   mfspr   r3,SPRN_WORC
+   std r3,_WORC(r1)
+
+   blr
+
+/*
  * Used by threads when the lock bit of core_idle_state is set.
  * Threads will spin in HMT_LOW until the lock bit is cleared.
  * r14 - pointer to core_idle_state
@@ -209,28 +239,8 @@ fastsleep_workaround_at_entry:
b   common_enter
 
 enter_winkle:
-   /*
-* Note all register i.e per-core, per-subcore or per-thread is saved
-* here since any thread in the core might wake up first
-*/
-   mfspr   r3,SPRN_SDR1
-   std r3,_SDR1(r1)
-   mfspr   r3,SPRN_RPR
-   std r3,_RPR(r1)
-   mfspr   r3,SPRN_SPURR
-   std r3,_SPURR(r1)
-   mfspr   r3,SPRN_PURR
-   std r3,_PURR(r1)
-   mfspr   r3,SPRN_TSCR
-   std r3,_TSCR(r1)
-   mfspr   r3,SPRN_DSCR
-   std r3,_DSCR(r1)
-   mfspr   r3,SPRN_AMOR
-   std r3,_AMOR(r1)
-   mfspr   r3,SPRN_WORT
-   std r3,_WORT(r1)
-   mfspr   r3,SPRN_WORC
-   std r3,_WORC(r1)
+   bl  save_sprs_to_stack
+
IDLE_STATE_ENTER_SEQ(PPC_WINKLE)
 
 _GLOBAL(power7_idle)
-- 
2.1.4

[PATCH v6 07/11] powerpc/powernv: set power_save func after the idle states are initialized

[Shreyas B. Prabhu]

pnv_init_idle_states discovers supported idle states from the
device tree and does the required initialization. Set power_save
function pointer only after this initialization is done

Reviewed-by: Gautham R. Shenoy 
Signed-off-by: Shreyas B. Prabhu 
---
- No changes since v1

 arch/powerpc/platforms/powernv/idle.c  | 3 +++
 arch/powerpc/platforms/powernv/setup.c | 2 +-
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index fcc8b68..fbb09fb 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -285,6 +285,9 @@ static int __init pnv_init_idle_states(void)
}
 
pnv_alloc_idle_core_states();
+
+   if (supported_cpuidle_states & OPAL_PM_NAP_ENABLED)
+   ppc_md.power_save = power7_idle;
 out_free:
kfree(flags);
 out:
diff --git a/arch/powerpc/platforms/powernv/setup.c 
b/arch/powerpc/platforms/powernv/setup.c
index ee6430b..8492bbb 100644
--- a/arch/powerpc/platforms/powernv/setup.c
+++ b/arch/powerpc/platforms/powernv/setup.c
@@ -315,7 +315,7 @@ define_machine(powernv) {
.get_proc_freq  = pnv_get_proc_freq,
.progress   = pnv_progress,
.machine_shutdown   = pnv_shutdown,
-   .power_save = power7_idle,
+   .power_save = NULL,
.calibrate_decr = generic_calibrate_decr,
 #ifdef CONFIG_KEXEC
.kexec_cpu_down = pnv_kexec_cpu_down,
-- 
2.1.4

[PATCH v6 08/11] powerpc/powernv: Add platform support for stop instruction

[Shreyas B. Prabhu]

POWER ISA v3 defines a new idle processor core mechanism. In summary,
 a) new instruction named stop is added. This instruction replaces
instructions like nap, sleep, rvwinkle.
 b) new per thread SPR named Processor Stop Status and Control Register
(PSSCR) is added which controls the behavior of stop instruction.

PSSCR layout:
--
| PLS | /// | SD | ESL | EC | PSLL | /// | TR | MTL | RL |
--
0  4 41   4243   44 4854   5660

PSSCR key fields:
Bits 0:3  - Power-Saving Level Status. This field indicates the lowest
power-saving state the thread entered since stop instruction was last
executed.

Bit 42 - Enable State Loss
0 - No state is lost irrespective of other fields
1 - Allows state loss

Bits 44:47 - Power-Saving Level Limit
This limits the power-saving level that can be entered into.

Bits 60:63 - Requested Level
Used to specify which power-saving level must be entered on executing
stop instruction

This patch adds support for stop instruction and PSSCR handling.

Reviewed-by: Gautham R. Shenoy 
Signed-off-by: Shreyas B. Prabhu 
---
Changes in v6
=
 - Save/restore new P9 SPRs when using deep idle states

Changes in v4:
==
 - Added PSSCR layout to commit message
 - Improved / Fixed comments
 - Fixed whitespace error in paca.h
 - Using MAX_POSSIBLE_STOP_STATE macro instead of hardcoding 0xF as 
   max possible stop state

Changes in v3:
==
 - Instead of introducing new file idle_power_stop.S, P9 idle support
   is added to idle_power_common.S using CPU_FTR sections.
 - Fixed r4 reg clobbering in power_stop0
 - Improved comments

Changes in v2:
==
 - Using CPU_FTR_ARCH_300 bit instead of CPU_FTR_STOP_INST

 arch/powerpc/include/asm/cpuidle.h|   2 +
 arch/powerpc/include/asm/kvm_book3s_asm.h |   2 +-
 arch/powerpc/include/asm/machdep.h|   1 +
 arch/powerpc/include/asm/opal-api.h   |  11 +-
 arch/powerpc/include/asm/paca.h   |   2 +
 arch/powerpc/include/asm/ppc-opcode.h |   4 +
 arch/powerpc/include/asm/processor.h  |   1 +
 arch/powerpc/include/asm/reg.h|  14 +++
 arch/powerpc/kernel/asm-offsets.c |   2 +
 arch/powerpc/kernel/idle_power_common.S   | 175 +++---
 arch/powerpc/platforms/powernv/idle.c |  84 --
 11 files changed, 265 insertions(+), 33 deletions(-)

diff --git a/arch/powerpc/include/asm/cpuidle.h 
b/arch/powerpc/include/asm/cpuidle.h
index d2f99ca..3d7fc06 100644
--- a/arch/powerpc/include/asm/cpuidle.h
+++ b/arch/powerpc/include/asm/cpuidle.h
@@ -13,6 +13,8 @@
 #ifndef __ASSEMBLY__
 extern u32 pnv_fastsleep_workaround_at_entry[];
 extern u32 pnv_fastsleep_workaround_at_exit[];
+
+extern u64 pnv_first_deep_stop_state;
 #endif
 
 #endif
diff --git a/arch/powerpc/include/asm/kvm_book3s_asm.h 
b/arch/powerpc/include/asm/kvm_book3s_asm.h
index 72b6225..d318d43 100644
--- a/arch/powerpc/include/asm/kvm_book3s_asm.h
+++ b/arch/powerpc/include/asm/kvm_book3s_asm.h
@@ -162,7 +162,7 @@ struct kvmppc_book3s_shadow_vcpu {
 
 /* Values for kvm_state */
 #define KVM_HWTHREAD_IN_KERNEL 0
-#define KVM_HWTHREAD_IN_NAP1
+#define KVM_HWTHREAD_IN_IDLE   1
 #define KVM_HWTHREAD_IN_KVM2
 
 #endif /* __ASM_KVM_BOOK3S_ASM_H__ */
diff --git a/arch/powerpc/include/asm/machdep.h 
b/arch/powerpc/include/asm/machdep.h
index 6bdcd0d..ae3b155 100644
--- a/arch/powerpc/include/asm/machdep.h
+++ b/arch/powerpc/include/asm/machdep.h
@@ -262,6 +262,7 @@ struct machdep_calls {
 extern void e500_idle(void);
 extern void power4_idle(void);
 extern void power7_idle(void);
+extern void power_stop0(void);
 extern void ppc6xx_idle(void);
 extern void book3e_idle(void);
 
diff --git a/arch/powerpc/include/asm/opal-api.h 
b/arch/powerpc/include/asm/opal-api.h
index 9bb8ddf..7f3f8c6 100644
--- a/arch/powerpc/include/asm/opal-api.h
+++ b/arch/powerpc/include/asm/opal-api.h
@@ -162,13 +162,20 @@
 
 /* Device tree flags */
 
-/* Flags set in power-mgmt nodes in device tree if
- * respective idle states are supported in the platform.
+/*
+ * Flags set in power-mgmt nodes in device tree describing
+ * idle states that are supported in the platform.
  */
+
+#define OPAL_PM_TIMEBASE_STOP  0x0002
+#define OPAL_PM_LOSE_HYP_CONTEXT   0x2000
+#define OPAL_PM_LOSE_FULL_CONTEXT  0x4000
 #define OPAL_PM_NAP_ENABLED0x0001
 #define OPAL_PM_SLEEP_ENABLED  0x0002
 #define OPAL_PM_WINKLE_ENABLED 0x0004
 #define OPAL_PM_SLEEP_ENABLED_ER1  0x0008 /* with workaround */
+#define OPAL_PM_STOP_INST_FAST 0x0010
+#define OPAL_PM_STOP_INST_DEEP 0x0020
 
 /*
  * OPAL_CONFIG_CPU_IDLE_STATE parameters
diff --git a/arch/powerpc/include/asm/paca.h b/arch/powerpc/include/asm/paca.h
index

[PATCH v6 06/11] powerpc/powernv: abstraction for saving SPRs before entering deep idle states

[Shreyas B. Prabhu]

Create a function for saving SPRs before entering deep idle states.
This function can be reused for POWER9 deep idle states.

Reviewed-by: Gautham R. Shenoy 
Signed-off-by: Shreyas B. Prabhu 
---
 - No changes since v3

Changes in v3:
=
 - Newly added in v3

 arch/powerpc/kernel/idle_power_common.S | 54 +++--
 1 file changed, 32 insertions(+), 22 deletions(-)

diff --git a/arch/powerpc/kernel/idle_power_common.S 
b/arch/powerpc/kernel/idle_power_common.S
index a8397e3..2f909a1 100644
--- a/arch/powerpc/kernel/idle_power_common.S
+++ b/arch/powerpc/kernel/idle_power_common.S
@@ -53,6 +53,36 @@
.text
 
 /*
+ * Used by threads before entering deep idle states. Saves SPRs
+ * in interrupt stack frame
+ */
+save_sprs_to_stack:
+   /*
+* Note all register i.e per-core, per-subcore or per-thread is saved
+* here since any thread in the core might wake up first
+*/
+   mfspr   r3,SPRN_SDR1
+   std r3,_SDR1(r1)
+   mfspr   r3,SPRN_RPR
+   std r3,_RPR(r1)
+   mfspr   r3,SPRN_SPURR
+   std r3,_SPURR(r1)
+   mfspr   r3,SPRN_PURR
+   std r3,_PURR(r1)
+   mfspr   r3,SPRN_TSCR
+   std r3,_TSCR(r1)
+   mfspr   r3,SPRN_DSCR
+   std r3,_DSCR(r1)
+   mfspr   r3,SPRN_AMOR
+   std r3,_AMOR(r1)
+   mfspr   r3,SPRN_WORT
+   std r3,_WORT(r1)
+   mfspr   r3,SPRN_WORC
+   std r3,_WORC(r1)
+
+   blr
+
+/*
  * Used by threads when the lock bit of core_idle_state is set.
  * Threads will spin in HMT_LOW until the lock bit is cleared.
  * r14 - pointer to core_idle_state
@@ -209,28 +239,8 @@ fastsleep_workaround_at_entry:
b   common_enter
 
 enter_winkle:
-   /*
-* Note all register i.e per-core, per-subcore or per-thread is saved
-* here since any thread in the core might wake up first
-*/
-   mfspr   r3,SPRN_SDR1
-   std r3,_SDR1(r1)
-   mfspr   r3,SPRN_RPR
-   std r3,_RPR(r1)
-   mfspr   r3,SPRN_SPURR
-   std r3,_SPURR(r1)
-   mfspr   r3,SPRN_PURR
-   std r3,_PURR(r1)
-   mfspr   r3,SPRN_TSCR
-   std r3,_TSCR(r1)
-   mfspr   r3,SPRN_DSCR
-   std r3,_DSCR(r1)
-   mfspr   r3,SPRN_AMOR
-   std r3,_AMOR(r1)
-   mfspr   r3,SPRN_WORT
-   std r3,_WORT(r1)
-   mfspr   r3,SPRN_WORC
-   std r3,_WORC(r1)
+   bl  save_sprs_to_stack
+
IDLE_STATE_ENTER_SEQ(PPC_WINKLE)
 
 _GLOBAL(power7_idle)
-- 
2.1.4

[PATCH v6 02/11] powerpc/kvm: make hypervisor state restore a function

[Shreyas B. Prabhu]

In the current code, when the thread wakes up in reset vector, some
of the state restore code and check for whether a thread needs to
branch to kvm is duplicated. Reorder the code such that this
duplication is avoided.

At a higher level this is what the change looks like-

Before this patch -
power7_wakeup_tb_loss:
restore hypervisor state
if (thread needed by kvm)
goto kvm_start_guest
restore nvgprs, cr, pc
rfid to process context

power7_wakeup_loss:
restore nvgprs, cr, pc
rfid to process context

reset vector:
if (waking from deep idle states)
goto power7_wakeup_tb_loss
else
if (thread needed by kvm)
goto kvm_start_guest
goto power7_wakeup_loss

After this patch -
power7_wakeup_tb_loss:
restore hypervisor state
return

power7_restore_hyp_resource():
if (waking from deep idle states)
goto power7_wakeup_tb_loss
return

power7_wakeup_loss:
restore nvgprs, cr, pc
rfid to process context

reset vector:
power7_restore_hyp_resource()
if (thread needed by kvm)
goto kvm_start_guest
goto power7_wakeup_loss

Reviewed-by: Paul Mackerras <pau...@samba.org>
Reviewed-by: Gautham R. Shenoy <e...@linux.vnet.ibm.com>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
- No changes since v3

Changes in v3:
=
- Retaining GET_PACA(r13) in System Reset vector instead of moving it
  to power7_restore_hyp_resource
- Added comments indicating entry conditions for power7_restore_hyp_resource
- Improved comments around return statements

 arch/powerpc/kernel/exceptions-64s.S | 28 ++
 arch/powerpc/kernel/idle_power7.S| 72 +---
 2 files changed, 46 insertions(+), 54 deletions(-)

diff --git a/arch/powerpc/kernel/exceptions-64s.S 
b/arch/powerpc/kernel/exceptions-64s.S
index 4c94406..4a74d6a 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -107,25 +107,9 @@ BEGIN_FTR_SECTION
beq 9f
 
cmpwi   cr3,r13,2
-
-   /*
-* Check if last bit of HSPGR0 is set. This indicates whether we are
-* waking up from winkle.
-*/
GET_PACA(r13)
-   clrldi  r5,r13,63
-   clrrdi  r13,r13,1
-   cmpwi   cr4,r5,1
-   mtspr   SPRN_HSPRG0,r13
+   bl  power7_restore_hyp_resource
 
-   lbz r0,PACA_THREAD_IDLE_STATE(r13)
-   cmpwi   cr2,r0,PNV_THREAD_NAP
-   bgt cr2,8f  /* Either sleep or Winkle */
-
-   /* Waking up from nap should not cause hypervisor state loss */
-   bgt cr3,.
-
-   /* Waking up from nap */
li  r0,PNV_THREAD_RUNNING
stb r0,PACA_THREAD_IDLE_STATE(r13)  /* Clear thread state */
 
@@ -143,13 +127,9 @@ BEGIN_FTR_SECTION
 
/* Return SRR1 from power7_nap() */
mfspr   r3,SPRN_SRR1
-   beq cr3,2f
-   b   power7_wakeup_noloss
-2: b   power7_wakeup_loss
-
-   /* Fast Sleep wakeup on PowerNV */
-8: GET_PACA(r13)
-   b   power7_wakeup_tb_loss
+   blt cr3,2f
+   b   power7_wakeup_loss
+2: b   power7_wakeup_noloss
 
 9:
 END_FTR_SECTION_IFSET(CPU_FTR_HVMODE | CPU_FTR_ARCH_206)
diff --git a/arch/powerpc/kernel/idle_power7.S 
b/arch/powerpc/kernel/idle_power7.S
index 705c867..d5def06 100644
--- a/arch/powerpc/kernel/idle_power7.S
+++ b/arch/powerpc/kernel/idle_power7.S
@@ -276,6 +276,39 @@ ALT_FTR_SECTION_END_NESTED_IFSET(CPU_FTR_ARCH_207S, 66);   
\
 20:nop;
 
 
+/*
+ * Called from reset vector. Check whether we have woken up with
+ * hypervisor state loss. If yes, restore hypervisor state and return
+ * back to reset vector.
+ *
+ * r13 - Contents of HSPRG0
+ * cr3 - set to gt if waking up with partial/complete hypervisor state loss
+ */
+_GLOBAL(power7_restore_hyp_resource)
+   /*
+* Check if last bit of HSPGR0 is set. This indicates whether we are
+* waking up from winkle.
+*/
+   clrldi  r5,r13,63
+   clrrdi  r13,r13,1
+   cmpwi   cr4,r5,1
+   mtspr   SPRN_HSPRG0,r13
+
+   lbz r0,PACA_THREAD_IDLE_STATE(r13)
+   cmpwi   cr2,r0,PNV_THREAD_NAP
+   bgt cr2,power7_wakeup_tb_loss   /* Either sleep or Winkle */
+
+   /*
+* We fall through here if PACA_THREAD_IDLE_STATE shows we are waking
+* up from nap. At this stage CR3 shouldn't contains 'gt' since that
+* indicates we are waking with hypervisor state loss from nap.
+*/
+   bgt cr3,.
+
+   blr /* Return back to System Reset vector from where
+  power7_restore_hyp_resource was invoked */
+
+
 _GLOBAL(power7_wakeup_tb_loss)
ld  r2,PACATOC(r13);
ld  r1,PACAR1(r13)
@@ -284,11 +317,13 @@ _GLOBAL(pow

[PATCH v6 01/11] powerpc/powernv: Use PNV_THREAD_WINKLE macro while requesting for winkle

[Shreyas B. Prabhu]

Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
-No changes since v4

Changes in v4
=
- New in v4

 arch/powerpc/kernel/idle_power7.S | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/powerpc/kernel/idle_power7.S 
b/arch/powerpc/kernel/idle_power7.S
index 470ceeb..705c867 100644
--- a/arch/powerpc/kernel/idle_power7.S
+++ b/arch/powerpc/kernel/idle_power7.S
@@ -252,7 +252,7 @@ _GLOBAL(power7_sleep)
/* No return */
 
 _GLOBAL(power7_winkle)
-   li  r3,3
+   li  r3,PNV_THREAD_WINKLE
li  r4,1
b   power7_powersave_common
/* No return */
-- 
2.1.4

[PATCH v6 10/11] cpuidle/powernv: Add support for POWER ISA v3 idle states

[Shreyas B. Prabhu]

POWER ISA v3 defines a new idle processor core mechanism. In summary,
 a) new instruction named stop is added.
 b) new per thread SPR named PSSCR is added which controls the behavior
of stop instruction.

Supported idle states and value to be written to PSSCR register to enter
any idle state is exposed via ibm,cpu-idle-state-names and
ibm,cpu-idle-state-psscr respectively. To enter an idle state,
platform provided power_stop() needs to be invoked with the appropriate
PSSCR value.

This patch adds support for this new mechanism in cpuidle powernv driver.

Cc: Rafael J. Wysocki <rafael.j.wyso...@intel.com>
Cc: Daniel Lezcano <daniel.lezc...@linaro.org>
Cc: Rob Herring <robh...@kernel.org>
Cc: Lorenzo Pieralisi <lorenzo.pieral...@arm.com>
Cc: linux...@vger.kernel.org
Cc: Michael Ellerman <m...@ellerman.id.au>
Cc: Paul Mackerras <pau...@ozlabs.org>
Cc: linuxppc-...@lists.ozlabs.org
Reviewed-by: Gautham R. Shenoy <e...@linux.vnet.ibm.com>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
Note: Documentation for the device tree bindings is posted here-
http://patchwork.ozlabs.org/patch/629125/

 - No changes in v6

Changes in v5
=
 - Use generic cpuidle constant CPUIDLE_NAME_LEN
 - Fix return code handling for of_property_read_string_array
 - Use DT flags to determine if are using stop instruction, instead of
   cpu_has_feature
 - Removed uncessary cast with names
 - _loop -> stop_loop
 - Added POWERNV_THRESHOLD_LATENCY_NS to filter out idle states with high 
latency

 drivers/cpuidle/cpuidle-powernv.c | 71 ++-
 1 file changed, 70 insertions(+), 1 deletion(-)

diff --git a/drivers/cpuidle/cpuidle-powernv.c 
b/drivers/cpuidle/cpuidle-powernv.c
index 3a763a8..c74a020 100644
--- a/drivers/cpuidle/cpuidle-powernv.c
+++ b/drivers/cpuidle/cpuidle-powernv.c
@@ -20,6 +20,8 @@
 #include 
 #include 
 
+#define POWERNV_THRESHOLD_LATENCY_NS 20
+
 struct cpuidle_driver powernv_idle_driver = {
.name = "powernv_idle",
.owner= THIS_MODULE,
@@ -27,6 +29,9 @@ struct cpuidle_driver powernv_idle_driver = {
 
 static int max_idle_state;
 static struct cpuidle_state *cpuidle_state_table;
+
+static u64 stop_psscr_table[CPUIDLE_STATE_MAX];
+
 static u64 snooze_timeout;
 static bool snooze_timeout_en;
 
@@ -91,6 +96,17 @@ static int fastsleep_loop(struct cpuidle_device *dev,
return index;
 }
 #endif
+
+static int stop_loop(struct cpuidle_device *dev,
+struct cpuidle_driver *drv,
+int index)
+{
+   ppc64_runlatch_off();
+   power_stop(stop_psscr_table[index]);
+   ppc64_runlatch_on();
+   return index;
+}
+
 /*
  * States for dedicated partition case.
  */
@@ -167,6 +183,8 @@ static int powernv_add_idle_states(void)
int nr_idle_states = 1; /* Snooze */
int dt_idle_states;
u32 *latency_ns, *residency_ns, *flags;
+   u64 *psscr_val = NULL;
+   const char *names[CPUIDLE_STATE_MAX];
int i, rc;
 
/* Currently we have snooze statically defined */
@@ -199,12 +217,41 @@ static int powernv_add_idle_states(void)
goto out_free_latency;
}
 
+   rc = of_property_read_string_array(power_mgt,
+  "ibm,cpu-idle-state-names", names,
+  dt_idle_states);
+   if (rc < 0) {
+   pr_warn("cpuidle-powernv: missing ibm,cpu-idle-state-names in 
DT\n");
+   goto out_free_latency;
+   }
+
+   /*
+* If the idle states use stop instruction, probe for psscr values
+* which are necessary to specify required stop level.
+*/
+   if (flags[0] & (OPAL_PM_STOP_INST_FAST | OPAL_PM_STOP_INST_DEEP)) {
+   psscr_val = kcalloc(dt_idle_states, sizeof(*psscr_val),
+   GFP_KERNEL);
+   rc = of_property_read_u64_array(power_mgt,
+   "ibm,cpu-idle-state-psscr",
+   psscr_val, dt_idle_states);
+   if (rc) {
+   pr_warn("cpuidle-powernv: missing 
ibm,cpu-idle-states-psscr in DT\n");
+   goto out_free_psscr;
+   }
+   }
residency_ns = kzalloc(sizeof(*residency_ns) * dt_idle_states, 
GFP_KERNEL);
rc = of_property_read_u32_array(power_mgt,
"ibm,cpu-idle-state-residency-ns", residency_ns, 
dt_idle_states);
 
for (i = 0; i < dt_idle_states; i++) {
-
+   /*
+* If an idle state has exit latency beyond
+* POWERNV_THRESHOLD_LATENCY_NS then don't use it
+* in cpu-idle.
+*/
+   if (latency_ns[i] > POWERNV_THRESHOLD_LATENCY_NS)
+  

[PATCH v6 02/11] powerpc/kvm: make hypervisor state restore a function

[Shreyas B. Prabhu]

In the current code, when the thread wakes up in reset vector, some
of the state restore code and check for whether a thread needs to
branch to kvm is duplicated. Reorder the code such that this
duplication is avoided.

At a higher level this is what the change looks like-

Before this patch -
power7_wakeup_tb_loss:
restore hypervisor state
if (thread needed by kvm)
goto kvm_start_guest
restore nvgprs, cr, pc
rfid to process context

power7_wakeup_loss:
restore nvgprs, cr, pc
rfid to process context

reset vector:
if (waking from deep idle states)
goto power7_wakeup_tb_loss
else
if (thread needed by kvm)
goto kvm_start_guest
goto power7_wakeup_loss

After this patch -
power7_wakeup_tb_loss:
restore hypervisor state
return

power7_restore_hyp_resource():
if (waking from deep idle states)
goto power7_wakeup_tb_loss
return

power7_wakeup_loss:
restore nvgprs, cr, pc
rfid to process context

reset vector:
power7_restore_hyp_resource()
if (thread needed by kvm)
goto kvm_start_guest
goto power7_wakeup_loss

Reviewed-by: Paul Mackerras 
Reviewed-by: Gautham R. Shenoy 
Signed-off-by: Shreyas B. Prabhu 
---
- No changes since v3

Changes in v3:
=
- Retaining GET_PACA(r13) in System Reset vector instead of moving it
  to power7_restore_hyp_resource
- Added comments indicating entry conditions for power7_restore_hyp_resource
- Improved comments around return statements

 arch/powerpc/kernel/exceptions-64s.S | 28 ++
 arch/powerpc/kernel/idle_power7.S| 72 +---
 2 files changed, 46 insertions(+), 54 deletions(-)

diff --git a/arch/powerpc/kernel/exceptions-64s.S 
b/arch/powerpc/kernel/exceptions-64s.S
index 4c94406..4a74d6a 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -107,25 +107,9 @@ BEGIN_FTR_SECTION
beq 9f
 
cmpwi   cr3,r13,2
-
-   /*
-* Check if last bit of HSPGR0 is set. This indicates whether we are
-* waking up from winkle.
-*/
GET_PACA(r13)
-   clrldi  r5,r13,63
-   clrrdi  r13,r13,1
-   cmpwi   cr4,r5,1
-   mtspr   SPRN_HSPRG0,r13
+   bl  power7_restore_hyp_resource
 
-   lbz r0,PACA_THREAD_IDLE_STATE(r13)
-   cmpwi   cr2,r0,PNV_THREAD_NAP
-   bgt cr2,8f  /* Either sleep or Winkle */
-
-   /* Waking up from nap should not cause hypervisor state loss */
-   bgt cr3,.
-
-   /* Waking up from nap */
li  r0,PNV_THREAD_RUNNING
stb r0,PACA_THREAD_IDLE_STATE(r13)  /* Clear thread state */
 
@@ -143,13 +127,9 @@ BEGIN_FTR_SECTION
 
/* Return SRR1 from power7_nap() */
mfspr   r3,SPRN_SRR1
-   beq cr3,2f
-   b   power7_wakeup_noloss
-2: b   power7_wakeup_loss
-
-   /* Fast Sleep wakeup on PowerNV */
-8: GET_PACA(r13)
-   b   power7_wakeup_tb_loss
+   blt cr3,2f
+   b   power7_wakeup_loss
+2: b   power7_wakeup_noloss
 
 9:
 END_FTR_SECTION_IFSET(CPU_FTR_HVMODE | CPU_FTR_ARCH_206)
diff --git a/arch/powerpc/kernel/idle_power7.S 
b/arch/powerpc/kernel/idle_power7.S
index 705c867..d5def06 100644
--- a/arch/powerpc/kernel/idle_power7.S
+++ b/arch/powerpc/kernel/idle_power7.S
@@ -276,6 +276,39 @@ ALT_FTR_SECTION_END_NESTED_IFSET(CPU_FTR_ARCH_207S, 66);   
\
 20:nop;
 
 
+/*
+ * Called from reset vector. Check whether we have woken up with
+ * hypervisor state loss. If yes, restore hypervisor state and return
+ * back to reset vector.
+ *
+ * r13 - Contents of HSPRG0
+ * cr3 - set to gt if waking up with partial/complete hypervisor state loss
+ */
+_GLOBAL(power7_restore_hyp_resource)
+   /*
+* Check if last bit of HSPGR0 is set. This indicates whether we are
+* waking up from winkle.
+*/
+   clrldi  r5,r13,63
+   clrrdi  r13,r13,1
+   cmpwi   cr4,r5,1
+   mtspr   SPRN_HSPRG0,r13
+
+   lbz r0,PACA_THREAD_IDLE_STATE(r13)
+   cmpwi   cr2,r0,PNV_THREAD_NAP
+   bgt cr2,power7_wakeup_tb_loss   /* Either sleep or Winkle */
+
+   /*
+* We fall through here if PACA_THREAD_IDLE_STATE shows we are waking
+* up from nap. At this stage CR3 shouldn't contains 'gt' since that
+* indicates we are waking with hypervisor state loss from nap.
+*/
+   bgt cr3,.
+
+   blr /* Return back to System Reset vector from where
+  power7_restore_hyp_resource was invoked */
+
+
 _GLOBAL(power7_wakeup_tb_loss)
ld  r2,PACATOC(r13);
ld  r1,PACAR1(r13)
@@ -284,11 +317,13 @@ _GLOBAL(power7_wakeup_tb_loss)
 * and they are restored before switching to the process context. Hence

[PATCH v6 01/11] powerpc/powernv: Use PNV_THREAD_WINKLE macro while requesting for winkle

[Shreyas B. Prabhu]

Signed-off-by: Shreyas B. Prabhu 
---
-No changes since v4

Changes in v4
=
- New in v4

 arch/powerpc/kernel/idle_power7.S | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/powerpc/kernel/idle_power7.S 
b/arch/powerpc/kernel/idle_power7.S
index 470ceeb..705c867 100644
--- a/arch/powerpc/kernel/idle_power7.S
+++ b/arch/powerpc/kernel/idle_power7.S
@@ -252,7 +252,7 @@ _GLOBAL(power7_sleep)
/* No return */
 
 _GLOBAL(power7_winkle)
-   li  r3,3
+   li  r3,PNV_THREAD_WINKLE
li  r4,1
b   power7_powersave_common
/* No return */
-- 
2.1.4

[PATCH v6 10/11] cpuidle/powernv: Add support for POWER ISA v3 idle states

[Shreyas B. Prabhu]

POWER ISA v3 defines a new idle processor core mechanism. In summary,
 a) new instruction named stop is added.
 b) new per thread SPR named PSSCR is added which controls the behavior
of stop instruction.

Supported idle states and value to be written to PSSCR register to enter
any idle state is exposed via ibm,cpu-idle-state-names and
ibm,cpu-idle-state-psscr respectively. To enter an idle state,
platform provided power_stop() needs to be invoked with the appropriate
PSSCR value.

This patch adds support for this new mechanism in cpuidle powernv driver.

Cc: Rafael J. Wysocki 
Cc: Daniel Lezcano 
Cc: Rob Herring 
Cc: Lorenzo Pieralisi 
Cc: linux...@vger.kernel.org
Cc: Michael Ellerman 
Cc: Paul Mackerras 
Cc: linuxppc-...@lists.ozlabs.org
Reviewed-by: Gautham R. Shenoy 
Signed-off-by: Shreyas B. Prabhu 
---
Note: Documentation for the device tree bindings is posted here-
http://patchwork.ozlabs.org/patch/629125/

 - No changes in v6

Changes in v5
=
 - Use generic cpuidle constant CPUIDLE_NAME_LEN
 - Fix return code handling for of_property_read_string_array
 - Use DT flags to determine if are using stop instruction, instead of
   cpu_has_feature
 - Removed uncessary cast with names
 - _loop -> stop_loop
 - Added POWERNV_THRESHOLD_LATENCY_NS to filter out idle states with high 
latency

 drivers/cpuidle/cpuidle-powernv.c | 71 ++-
 1 file changed, 70 insertions(+), 1 deletion(-)

diff --git a/drivers/cpuidle/cpuidle-powernv.c 
b/drivers/cpuidle/cpuidle-powernv.c
index 3a763a8..c74a020 100644
--- a/drivers/cpuidle/cpuidle-powernv.c
+++ b/drivers/cpuidle/cpuidle-powernv.c
@@ -20,6 +20,8 @@
 #include 
 #include 
 
+#define POWERNV_THRESHOLD_LATENCY_NS 20
+
 struct cpuidle_driver powernv_idle_driver = {
.name = "powernv_idle",
.owner= THIS_MODULE,
@@ -27,6 +29,9 @@ struct cpuidle_driver powernv_idle_driver = {
 
 static int max_idle_state;
 static struct cpuidle_state *cpuidle_state_table;
+
+static u64 stop_psscr_table[CPUIDLE_STATE_MAX];
+
 static u64 snooze_timeout;
 static bool snooze_timeout_en;
 
@@ -91,6 +96,17 @@ static int fastsleep_loop(struct cpuidle_device *dev,
return index;
 }
 #endif
+
+static int stop_loop(struct cpuidle_device *dev,
+struct cpuidle_driver *drv,
+int index)
+{
+   ppc64_runlatch_off();
+   power_stop(stop_psscr_table[index]);
+   ppc64_runlatch_on();
+   return index;
+}
+
 /*
  * States for dedicated partition case.
  */
@@ -167,6 +183,8 @@ static int powernv_add_idle_states(void)
int nr_idle_states = 1; /* Snooze */
int dt_idle_states;
u32 *latency_ns, *residency_ns, *flags;
+   u64 *psscr_val = NULL;
+   const char *names[CPUIDLE_STATE_MAX];
int i, rc;
 
/* Currently we have snooze statically defined */
@@ -199,12 +217,41 @@ static int powernv_add_idle_states(void)
goto out_free_latency;
}
 
+   rc = of_property_read_string_array(power_mgt,
+  "ibm,cpu-idle-state-names", names,
+  dt_idle_states);
+   if (rc < 0) {
+   pr_warn("cpuidle-powernv: missing ibm,cpu-idle-state-names in 
DT\n");
+   goto out_free_latency;
+   }
+
+   /*
+* If the idle states use stop instruction, probe for psscr values
+* which are necessary to specify required stop level.
+*/
+   if (flags[0] & (OPAL_PM_STOP_INST_FAST | OPAL_PM_STOP_INST_DEEP)) {
+   psscr_val = kcalloc(dt_idle_states, sizeof(*psscr_val),
+   GFP_KERNEL);
+   rc = of_property_read_u64_array(power_mgt,
+   "ibm,cpu-idle-state-psscr",
+   psscr_val, dt_idle_states);
+   if (rc) {
+   pr_warn("cpuidle-powernv: missing 
ibm,cpu-idle-states-psscr in DT\n");
+   goto out_free_psscr;
+   }
+   }
residency_ns = kzalloc(sizeof(*residency_ns) * dt_idle_states, 
GFP_KERNEL);
rc = of_property_read_u32_array(power_mgt,
"ibm,cpu-idle-state-residency-ns", residency_ns, 
dt_idle_states);
 
for (i = 0; i < dt_idle_states; i++) {
-
+   /*
+* If an idle state has exit latency beyond
+* POWERNV_THRESHOLD_LATENCY_NS then don't use it
+* in cpu-idle.
+*/
+   if (latency_ns[i] > POWERNV_THRESHOLD_LATENCY_NS)
+   continue;
/*
 * Cpuidle accepts exit_latency and target_residency in us.
 * Use default target_residency values if f/w does not expose 
it.
@@ -216,6 +263,16 @@ static int powernv_add_idle_s

[PATCH v6 03/11] powerpc/powernv: Rename idle_power7.S to idle_power_common.S

[Shreyas B. Prabhu]

idle_power7.S handles idle entry/exit for POWER7, POWER8 and in next
patch for POWER9. Rename the file to a non-hardware specific
name.

Reviewed-by: Gautham R. Shenoy <e...@linux.vnet.ibm.com>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
 - No changes since v3

Changes in v3:
==
 - Instead of moving few common functions from idle_power7.S to
   idle_power_common.S, renaming idle_power7.S to idle_power_common.S

 arch/powerpc/kernel/Makefile|   2 +-
 arch/powerpc/kernel/idle_power7.S   | 527 
 arch/powerpc/kernel/idle_power_common.S | 527 
 3 files changed, 528 insertions(+), 528 deletions(-)
 delete mode 100644 arch/powerpc/kernel/idle_power7.S
 create mode 100644 arch/powerpc/kernel/idle_power_common.S

diff --git a/arch/powerpc/kernel/Makefile b/arch/powerpc/kernel/Makefile
index 2da380f..99116da 100644
--- a/arch/powerpc/kernel/Makefile
+++ b/arch/powerpc/kernel/Makefile
@@ -47,7 +47,7 @@ obj-$(CONFIG_PPC_BOOK3E_64)   += exceptions-64e.o 
idle_book3e.o
 obj-$(CONFIG_PPC64)+= vdso64/
 obj-$(CONFIG_ALTIVEC)  += vecemu.o
 obj-$(CONFIG_PPC_970_NAP)  += idle_power4.o
-obj-$(CONFIG_PPC_P7_NAP)   += idle_power7.o
+obj-$(CONFIG_PPC_P7_NAP)   += idle_power_common.o
 procfs-y   := proc_powerpc.o
 obj-$(CONFIG_PROC_FS)  += $(procfs-y)
 rtaspci-$(CONFIG_PPC64)-$(CONFIG_PCI)  := rtas_pci.o
diff --git a/arch/powerpc/kernel/idle_power7.S 
b/arch/powerpc/kernel/idle_power7.S
deleted file mode 100644
index d5def06..000
--- a/arch/powerpc/kernel/idle_power7.S
+++ /dev/null
@@ -1,527 +0,0 @@
-/*
- *  This file contains the power_save function for Power7 CPUs.
- *
- *  This program is free software; you can redistribute it and/or
- *  modify it under the terms of the GNU General Public License
- *  as published by the Free Software Foundation; either version
- *  2 of the License, or (at your option) any later version.
- */
-
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-
-#undef DEBUG
-
-/*
- * Use unused space in the interrupt stack to save and restore
- * registers for winkle support.
- */
-#define _SDR1  GPR3
-#define _RPR   GPR4
-#define _SPURR GPR5
-#define _PURR  GPR6
-#define _TSCR  GPR7
-#define _DSCR  GPR8
-#define _AMOR  GPR9
-#define _WORT  GPR10
-#define _WORC  GPR11
-
-/* Idle state entry routines */
-
-#defineIDLE_STATE_ENTER_SEQ(IDLE_INST) \
-   /* Magic NAP/SLEEP/WINKLE mode enter sequence */\
-   std r0,0(r1);   \
-   ptesync;\
-   ld  r0,0(r1);   \
-1: cmp cr0,r0,r0;  \
-   bne 1b; \
-   IDLE_INST;  \
-   b   .
-
-   .text
-
-/*
- * Used by threads when the lock bit of core_idle_state is set.
- * Threads will spin in HMT_LOW until the lock bit is cleared.
- * r14 - pointer to core_idle_state
- * r15 - used to load contents of core_idle_state
- */
-
-core_idle_lock_held:
-   HMT_LOW
-3: lwz r15,0(r14)
-   andi.   r15,r15,PNV_CORE_IDLE_LOCK_BIT
-   bne 3b
-   HMT_MEDIUM
-   lwarx   r15,0,r14
-   blr
-
-/*
- * Pass requested state in r3:
- * r3 - PNV_THREAD_NAP/SLEEP/WINKLE
- *
- * To check IRQ_HAPPENED in r4
- * 0 - don't check
- * 1 - check
- */
-_GLOBAL(power7_powersave_common)
-   /* Use r3 to pass state nap/sleep/winkle */
-   /* NAP is a state loss, we create a regs frame on the
-* stack, fill it up with the state we care about and
-* stick a pointer to it in PACAR1. We really only
-* need to save PC, some CR bits and the NV GPRs,
-* but for now an interrupt frame will do.
-*/
-   mflrr0
-   std r0,16(r1)
-   stdur1,-INT_FRAME_SIZE(r1)
-   std r0,_LINK(r1)
-   std r0,_NIP(r1)
-
-   /* Hard disable interrupts */
-   mfmsr   r9
-   rldicl  r9,r9,48,1
-   rotldi  r9,r9,16
-   mtmsrd  r9,1/* hard-disable interrupts */
-
-   /* Check if something happened while soft-disabled */
-   lbz r0,PACAIRQHAPPENED(r13)
-   andi.   r0,r0,~PACA_IRQ_HARD_DIS@l
-   beq 1f
-   cmpwi   cr0,r4,0
-   beq 1f
-   addir1,r1,INT_FRAME_SIZE
-   ld  r0,16(r1)
-   li  r3,0/* Return 0 (no nap) */
-   mtlrr0
-   blr
-
-1: /* We mark irqs hard disabled as this is the state we'll
-* be in when returning and we need to tell arch_local_irq_restore()
-* about it
-*/
-   li  r0,PACA_IRQ_HARD_DIS
-   stb r0,PACAIRQ

[PATCH v6 03/11] powerpc/powernv: Rename idle_power7.S to idle_power_common.S

[Shreyas B. Prabhu]

idle_power7.S handles idle entry/exit for POWER7, POWER8 and in next
patch for POWER9. Rename the file to a non-hardware specific
name.

Reviewed-by: Gautham R. Shenoy 
Signed-off-by: Shreyas B. Prabhu 
---
 - No changes since v3

Changes in v3:
==
 - Instead of moving few common functions from idle_power7.S to
   idle_power_common.S, renaming idle_power7.S to idle_power_common.S

 arch/powerpc/kernel/Makefile|   2 +-
 arch/powerpc/kernel/idle_power7.S   | 527 
 arch/powerpc/kernel/idle_power_common.S | 527 
 3 files changed, 528 insertions(+), 528 deletions(-)
 delete mode 100644 arch/powerpc/kernel/idle_power7.S
 create mode 100644 arch/powerpc/kernel/idle_power_common.S

diff --git a/arch/powerpc/kernel/Makefile b/arch/powerpc/kernel/Makefile
index 2da380f..99116da 100644
--- a/arch/powerpc/kernel/Makefile
+++ b/arch/powerpc/kernel/Makefile
@@ -47,7 +47,7 @@ obj-$(CONFIG_PPC_BOOK3E_64)   += exceptions-64e.o 
idle_book3e.o
 obj-$(CONFIG_PPC64)+= vdso64/
 obj-$(CONFIG_ALTIVEC)  += vecemu.o
 obj-$(CONFIG_PPC_970_NAP)  += idle_power4.o
-obj-$(CONFIG_PPC_P7_NAP)   += idle_power7.o
+obj-$(CONFIG_PPC_P7_NAP)   += idle_power_common.o
 procfs-y   := proc_powerpc.o
 obj-$(CONFIG_PROC_FS)  += $(procfs-y)
 rtaspci-$(CONFIG_PPC64)-$(CONFIG_PCI)  := rtas_pci.o
diff --git a/arch/powerpc/kernel/idle_power7.S 
b/arch/powerpc/kernel/idle_power7.S
deleted file mode 100644
index d5def06..000
--- a/arch/powerpc/kernel/idle_power7.S
+++ /dev/null
@@ -1,527 +0,0 @@
-/*
- *  This file contains the power_save function for Power7 CPUs.
- *
- *  This program is free software; you can redistribute it and/or
- *  modify it under the terms of the GNU General Public License
- *  as published by the Free Software Foundation; either version
- *  2 of the License, or (at your option) any later version.
- */
-
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-
-#undef DEBUG
-
-/*
- * Use unused space in the interrupt stack to save and restore
- * registers for winkle support.
- */
-#define _SDR1  GPR3
-#define _RPR   GPR4
-#define _SPURR GPR5
-#define _PURR  GPR6
-#define _TSCR  GPR7
-#define _DSCR  GPR8
-#define _AMOR  GPR9
-#define _WORT  GPR10
-#define _WORC  GPR11
-
-/* Idle state entry routines */
-
-#defineIDLE_STATE_ENTER_SEQ(IDLE_INST) \
-   /* Magic NAP/SLEEP/WINKLE mode enter sequence */\
-   std r0,0(r1);   \
-   ptesync;\
-   ld  r0,0(r1);   \
-1: cmp cr0,r0,r0;  \
-   bne 1b; \
-   IDLE_INST;  \
-   b   .
-
-   .text
-
-/*
- * Used by threads when the lock bit of core_idle_state is set.
- * Threads will spin in HMT_LOW until the lock bit is cleared.
- * r14 - pointer to core_idle_state
- * r15 - used to load contents of core_idle_state
- */
-
-core_idle_lock_held:
-   HMT_LOW
-3: lwz r15,0(r14)
-   andi.   r15,r15,PNV_CORE_IDLE_LOCK_BIT
-   bne 3b
-   HMT_MEDIUM
-   lwarx   r15,0,r14
-   blr
-
-/*
- * Pass requested state in r3:
- * r3 - PNV_THREAD_NAP/SLEEP/WINKLE
- *
- * To check IRQ_HAPPENED in r4
- * 0 - don't check
- * 1 - check
- */
-_GLOBAL(power7_powersave_common)
-   /* Use r3 to pass state nap/sleep/winkle */
-   /* NAP is a state loss, we create a regs frame on the
-* stack, fill it up with the state we care about and
-* stick a pointer to it in PACAR1. We really only
-* need to save PC, some CR bits and the NV GPRs,
-* but for now an interrupt frame will do.
-*/
-   mflrr0
-   std r0,16(r1)
-   stdur1,-INT_FRAME_SIZE(r1)
-   std r0,_LINK(r1)
-   std r0,_NIP(r1)
-
-   /* Hard disable interrupts */
-   mfmsr   r9
-   rldicl  r9,r9,48,1
-   rotldi  r9,r9,16
-   mtmsrd  r9,1/* hard-disable interrupts */
-
-   /* Check if something happened while soft-disabled */
-   lbz r0,PACAIRQHAPPENED(r13)
-   andi.   r0,r0,~PACA_IRQ_HARD_DIS@l
-   beq 1f
-   cmpwi   cr0,r4,0
-   beq 1f
-   addir1,r1,INT_FRAME_SIZE
-   ld  r0,16(r1)
-   li  r3,0/* Return 0 (no nap) */
-   mtlrr0
-   blr
-
-1: /* We mark irqs hard disabled as this is the state we'll
-* be in when returning and we need to tell arch_local_irq_restore()
-* about it
-*/
-   li  r0,PACA_IRQ_HARD_DIS
-   stb r0,PACAIRQHAPPENED(r13)
-
-   /* We haven't lost state ... yet

[PATCH v6 04/11] powerpc/powernv: Rename reusable idle functions to hardware agnostic names

[Shreyas B. Prabhu]

Functions like power7_wakeup_loss, power7_wakeup_noloss,
power7_wakeup_tb_loss are used by POWER7 and POWER8 hardware. They can
also be used by POWER9. Hence rename these functions hardware agnostic
names.

Suggested-by: Gautham R. Shenoy <e...@linux.vnet.ibm.com>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
 - No changes since v4

Changes in v4:
==
 - renaming power7_powersave_common to pnv_powersave_common
 - renaming power7_enter_nap_mode to pnv_enter_arch207_idle_mode

 arch/powerpc/kernel/exceptions-64s.S|  8 
 arch/powerpc/kernel/idle_power_common.S | 33 +
 arch/powerpc/kvm/book3s_hv_rmhandlers.S |  4 ++--
 3 files changed, 23 insertions(+), 22 deletions(-)

diff --git a/arch/powerpc/kernel/exceptions-64s.S 
b/arch/powerpc/kernel/exceptions-64s.S
index 4a74d6a..2a123cd 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -108,7 +108,7 @@ BEGIN_FTR_SECTION
 
cmpwi   cr3,r13,2
GET_PACA(r13)
-   bl  power7_restore_hyp_resource
+   bl  pnv_restore_hyp_resource
 
li  r0,PNV_THREAD_RUNNING
stb r0,PACA_THREAD_IDLE_STATE(r13)  /* Clear thread state */
@@ -128,8 +128,8 @@ BEGIN_FTR_SECTION
/* Return SRR1 from power7_nap() */
mfspr   r3,SPRN_SRR1
blt cr3,2f
-   b   power7_wakeup_loss
-2: b   power7_wakeup_noloss
+   b   pnv_wakeup_loss
+2: b   pnv_wakeup_noloss
 
 9:
 END_FTR_SECTION_IFSET(CPU_FTR_HVMODE | CPU_FTR_ARCH_206)
@@ -1269,7 +1269,7 @@ machine_check_handle_early:
GET_PACA(r13)
ld  r1,PACAR1(r13)
li  r3,PNV_THREAD_NAP
-   b   power7_enter_nap_mode
+   b   pnv_enter_arch207_idle_mode
 4:
 #endif
/*
diff --git a/arch/powerpc/kernel/idle_power_common.S 
b/arch/powerpc/kernel/idle_power_common.S
index d5def06..34dbfc9 100644
--- a/arch/powerpc/kernel/idle_power_common.S
+++ b/arch/powerpc/kernel/idle_power_common.S
@@ -1,5 +1,6 @@
 /*
- *  This file contains the power_save function for Power7 CPUs.
+ *  This file contains idle entry/exit functions for POWER7 and
+ *  POWER8 CPUs.
  *
  *  This program is free software; you can redistribute it and/or
  *  modify it under the terms of the GNU General Public License
@@ -75,7 +76,7 @@ core_idle_lock_held:
  * 0 - don't check
  * 1 - check
  */
-_GLOBAL(power7_powersave_common)
+_GLOBAL(pnv_powersave_common)
/* Use r3 to pass state nap/sleep/winkle */
/* NAP is a state loss, we create a regs frame on the
 * stack, fill it up with the state we care about and
@@ -135,14 +136,14 @@ _GLOBAL(power7_powersave_common)
LOAD_REG_IMMEDIATE(r5, MSR_IDLE)
li  r6, MSR_RI
andcr6, r9, r6
-   LOAD_REG_ADDR(r7, power7_enter_nap_mode)
+   LOAD_REG_ADDR(r7, pnv_enter_arch207_idle_mode)
mtmsrd  r6, 1   /* clear RI before setting SRR0/1 */
mtspr   SPRN_SRR0, r7
mtspr   SPRN_SRR1, r5
rfid
 
-   .globl  power7_enter_nap_mode
-power7_enter_nap_mode:
+   .globl pnv_enter_arch207_idle_mode
+pnv_enter_arch207_idle_mode:
 #ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
/* Tell KVM we're napping */
li  r4,KVM_HWTHREAD_IN_NAP
@@ -242,19 +243,19 @@ _GLOBAL(power7_idle)
 _GLOBAL(power7_nap)
mr  r4,r3
li  r3,PNV_THREAD_NAP
-   b   power7_powersave_common
+   b   pnv_powersave_common
/* No return */
 
 _GLOBAL(power7_sleep)
li  r3,PNV_THREAD_SLEEP
li  r4,1
-   b   power7_powersave_common
+   b   pnv_powersave_common
/* No return */
 
 _GLOBAL(power7_winkle)
li  r3,PNV_THREAD_WINKLE
li  r4,1
-   b   power7_powersave_common
+   b   pnv_powersave_common
/* No return */
 
 #define CHECK_HMI_INTERRUPT\
@@ -284,7 +285,7 @@ ALT_FTR_SECTION_END_NESTED_IFSET(CPU_FTR_ARCH_207S, 66);
\
  * r13 - Contents of HSPRG0
  * cr3 - set to gt if waking up with partial/complete hypervisor state loss
  */
-_GLOBAL(power7_restore_hyp_resource)
+_GLOBAL(pnv_restore_hyp_resource)
/*
 * Check if last bit of HSPGR0 is set. This indicates whether we are
 * waking up from winkle.
@@ -296,7 +297,7 @@ _GLOBAL(power7_restore_hyp_resource)
 
lbz r0,PACA_THREAD_IDLE_STATE(r13)
cmpwi   cr2,r0,PNV_THREAD_NAP
-   bgt cr2,power7_wakeup_tb_loss   /* Either sleep or Winkle */
+   bgt cr2,pnv_wakeup_tb_loss  /* Either sleep or Winkle */
 
/*
 * We fall through here if PACA_THREAD_IDLE_STATE shows we are waking
@@ -306,10 +307,10 @@ _GLOBAL(power7_restore_hyp_resource)
bgt cr3,.
 
blr /* Return back to System Reset vector from where
-  power7_restore_hyp_resource

[PATCH v6 04/11] powerpc/powernv: Rename reusable idle functions to hardware agnostic names

[Shreyas B. Prabhu]

Functions like power7_wakeup_loss, power7_wakeup_noloss,
power7_wakeup_tb_loss are used by POWER7 and POWER8 hardware. They can
also be used by POWER9. Hence rename these functions hardware agnostic
names.

Suggested-by: Gautham R. Shenoy 
Signed-off-by: Shreyas B. Prabhu 
---
 - No changes since v4

Changes in v4:
==
 - renaming power7_powersave_common to pnv_powersave_common
 - renaming power7_enter_nap_mode to pnv_enter_arch207_idle_mode

 arch/powerpc/kernel/exceptions-64s.S|  8 
 arch/powerpc/kernel/idle_power_common.S | 33 +
 arch/powerpc/kvm/book3s_hv_rmhandlers.S |  4 ++--
 3 files changed, 23 insertions(+), 22 deletions(-)

diff --git a/arch/powerpc/kernel/exceptions-64s.S 
b/arch/powerpc/kernel/exceptions-64s.S
index 4a74d6a..2a123cd 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -108,7 +108,7 @@ BEGIN_FTR_SECTION
 
cmpwi   cr3,r13,2
GET_PACA(r13)
-   bl  power7_restore_hyp_resource
+   bl  pnv_restore_hyp_resource
 
li  r0,PNV_THREAD_RUNNING
stb r0,PACA_THREAD_IDLE_STATE(r13)  /* Clear thread state */
@@ -128,8 +128,8 @@ BEGIN_FTR_SECTION
/* Return SRR1 from power7_nap() */
mfspr   r3,SPRN_SRR1
blt cr3,2f
-   b   power7_wakeup_loss
-2: b   power7_wakeup_noloss
+   b   pnv_wakeup_loss
+2: b   pnv_wakeup_noloss
 
 9:
 END_FTR_SECTION_IFSET(CPU_FTR_HVMODE | CPU_FTR_ARCH_206)
@@ -1269,7 +1269,7 @@ machine_check_handle_early:
GET_PACA(r13)
ld  r1,PACAR1(r13)
li  r3,PNV_THREAD_NAP
-   b   power7_enter_nap_mode
+   b   pnv_enter_arch207_idle_mode
 4:
 #endif
/*
diff --git a/arch/powerpc/kernel/idle_power_common.S 
b/arch/powerpc/kernel/idle_power_common.S
index d5def06..34dbfc9 100644
--- a/arch/powerpc/kernel/idle_power_common.S
+++ b/arch/powerpc/kernel/idle_power_common.S
@@ -1,5 +1,6 @@
 /*
- *  This file contains the power_save function for Power7 CPUs.
+ *  This file contains idle entry/exit functions for POWER7 and
+ *  POWER8 CPUs.
  *
  *  This program is free software; you can redistribute it and/or
  *  modify it under the terms of the GNU General Public License
@@ -75,7 +76,7 @@ core_idle_lock_held:
  * 0 - don't check
  * 1 - check
  */
-_GLOBAL(power7_powersave_common)
+_GLOBAL(pnv_powersave_common)
/* Use r3 to pass state nap/sleep/winkle */
/* NAP is a state loss, we create a regs frame on the
 * stack, fill it up with the state we care about and
@@ -135,14 +136,14 @@ _GLOBAL(power7_powersave_common)
LOAD_REG_IMMEDIATE(r5, MSR_IDLE)
li  r6, MSR_RI
andcr6, r9, r6
-   LOAD_REG_ADDR(r7, power7_enter_nap_mode)
+   LOAD_REG_ADDR(r7, pnv_enter_arch207_idle_mode)
mtmsrd  r6, 1   /* clear RI before setting SRR0/1 */
mtspr   SPRN_SRR0, r7
mtspr   SPRN_SRR1, r5
rfid
 
-   .globl  power7_enter_nap_mode
-power7_enter_nap_mode:
+   .globl pnv_enter_arch207_idle_mode
+pnv_enter_arch207_idle_mode:
 #ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
/* Tell KVM we're napping */
li  r4,KVM_HWTHREAD_IN_NAP
@@ -242,19 +243,19 @@ _GLOBAL(power7_idle)
 _GLOBAL(power7_nap)
mr  r4,r3
li  r3,PNV_THREAD_NAP
-   b   power7_powersave_common
+   b   pnv_powersave_common
/* No return */
 
 _GLOBAL(power7_sleep)
li  r3,PNV_THREAD_SLEEP
li  r4,1
-   b   power7_powersave_common
+   b   pnv_powersave_common
/* No return */
 
 _GLOBAL(power7_winkle)
li  r3,PNV_THREAD_WINKLE
li  r4,1
-   b   power7_powersave_common
+   b   pnv_powersave_common
/* No return */
 
 #define CHECK_HMI_INTERRUPT\
@@ -284,7 +285,7 @@ ALT_FTR_SECTION_END_NESTED_IFSET(CPU_FTR_ARCH_207S, 66);
\
  * r13 - Contents of HSPRG0
  * cr3 - set to gt if waking up with partial/complete hypervisor state loss
  */
-_GLOBAL(power7_restore_hyp_resource)
+_GLOBAL(pnv_restore_hyp_resource)
/*
 * Check if last bit of HSPGR0 is set. This indicates whether we are
 * waking up from winkle.
@@ -296,7 +297,7 @@ _GLOBAL(power7_restore_hyp_resource)
 
lbz r0,PACA_THREAD_IDLE_STATE(r13)
cmpwi   cr2,r0,PNV_THREAD_NAP
-   bgt cr2,power7_wakeup_tb_loss   /* Either sleep or Winkle */
+   bgt cr2,pnv_wakeup_tb_loss  /* Either sleep or Winkle */
 
/*
 * We fall through here if PACA_THREAD_IDLE_STATE shows we are waking
@@ -306,10 +307,10 @@ _GLOBAL(power7_restore_hyp_resource)
bgt cr3,.
 
blr /* Return back to System Reset vector from where
-  power7_restore_hyp_resource was invoked */
+  pnv_restore_hyp_resource was invoked */
 
 
-_GLOBAL

[PATCH v6 11/11] powerpc/powernv: Use deepest stop state when cpu is offlined

[Shreyas B. Prabhu]

If hardware supports stop state, use the deepest stop state when
the cpu is offlined.

Reviewed-by: Gautham R. Shenoy <e...@linux.vnet.ibm.com>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
 - No changes since v1

 arch/powerpc/platforms/powernv/idle.c| 15 +--
 arch/powerpc/platforms/powernv/powernv.h |  1 +
 arch/powerpc/platforms/powernv/smp.c |  4 +++-
 3 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index bfbd359..b38cb33 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -242,6 +242,11 @@ static DEVICE_ATTR(fastsleep_workaround_applyonce, 0600,
  */
 u64 pnv_first_deep_stop_state;
 
+/*
+ * Deepest stop idle state. Used when a cpu is offlined
+ */
+u64 pnv_deepest_stop_state;
+
 static int __init pnv_init_idle_states(void)
 {
struct device_node *power_mgt;
@@ -290,8 +295,11 @@ static int __init pnv_init_idle_states(void)
}
 
/*
-* Set pnv_first_deep_stop_state to the first stop level
-* to cause hypervisor state loss
+* Set pnv_first_deep_stop_state and pnv_deepest_stop_state.
+* pnv_first_deep_stop_state should be set to the first stop
+* level to cause hypervisor state loss.
+* pnv_deepest_stop_state should be set to the deepest stop
+* stop state.
 */
pnv_first_deep_stop_state = MAX_STOP_STATE;
for (i = 0; i < dt_idle_states; i++) {
@@ -300,6 +308,9 @@ static int __init pnv_init_idle_states(void)
if ((flags[i] & OPAL_PM_LOSE_FULL_CONTEXT) &&
 (pnv_first_deep_stop_state > psscr_rl))
pnv_first_deep_stop_state = psscr_rl;
+
+   if (pnv_deepest_stop_state < psscr_rl)
+   pnv_deepest_stop_state = psscr_rl;
}
}
 
diff --git a/arch/powerpc/platforms/powernv/powernv.h 
b/arch/powerpc/platforms/powernv/powernv.h
index 6dbc0a1..da7c843 100644
--- a/arch/powerpc/platforms/powernv/powernv.h
+++ b/arch/powerpc/platforms/powernv/powernv.h
@@ -18,6 +18,7 @@ static inline void pnv_pci_shutdown(void) { }
 #endif
 
 extern u32 pnv_get_supported_cpuidle_states(void);
+extern u64 pnv_deepest_stop_state;
 
 extern void pnv_lpc_init(void);
 
diff --git a/arch/powerpc/platforms/powernv/smp.c 
b/arch/powerpc/platforms/powernv/smp.c
index ad7b1a3..f69ceb6 100644
--- a/arch/powerpc/platforms/powernv/smp.c
+++ b/arch/powerpc/platforms/powernv/smp.c
@@ -182,7 +182,9 @@ static void pnv_smp_cpu_kill_self(void)
 
ppc64_runlatch_off();
 
-   if (idle_states & OPAL_PM_WINKLE_ENABLED)
+   if (cpu_has_feature(CPU_FTR_ARCH_300))
+   srr1 = power_stop(pnv_deepest_stop_state);
+   else if (idle_states & OPAL_PM_WINKLE_ENABLED)
srr1 = power7_winkle();
else if ((idle_states & OPAL_PM_SLEEP_ENABLED) ||
(idle_states & OPAL_PM_SLEEP_ENABLED_ER1))
-- 
2.1.4

[PATCH v6 11/11] powerpc/powernv: Use deepest stop state when cpu is offlined

[Shreyas B. Prabhu]

If hardware supports stop state, use the deepest stop state when
the cpu is offlined.

Reviewed-by: Gautham R. Shenoy 
Signed-off-by: Shreyas B. Prabhu 
---
 - No changes since v1

 arch/powerpc/platforms/powernv/idle.c| 15 +--
 arch/powerpc/platforms/powernv/powernv.h |  1 +
 arch/powerpc/platforms/powernv/smp.c |  4 +++-
 3 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index bfbd359..b38cb33 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -242,6 +242,11 @@ static DEVICE_ATTR(fastsleep_workaround_applyonce, 0600,
  */
 u64 pnv_first_deep_stop_state;
 
+/*
+ * Deepest stop idle state. Used when a cpu is offlined
+ */
+u64 pnv_deepest_stop_state;
+
 static int __init pnv_init_idle_states(void)
 {
struct device_node *power_mgt;
@@ -290,8 +295,11 @@ static int __init pnv_init_idle_states(void)
}
 
/*
-* Set pnv_first_deep_stop_state to the first stop level
-* to cause hypervisor state loss
+* Set pnv_first_deep_stop_state and pnv_deepest_stop_state.
+* pnv_first_deep_stop_state should be set to the first stop
+* level to cause hypervisor state loss.
+* pnv_deepest_stop_state should be set to the deepest stop
+* stop state.
 */
pnv_first_deep_stop_state = MAX_STOP_STATE;
for (i = 0; i < dt_idle_states; i++) {
@@ -300,6 +308,9 @@ static int __init pnv_init_idle_states(void)
if ((flags[i] & OPAL_PM_LOSE_FULL_CONTEXT) &&
 (pnv_first_deep_stop_state > psscr_rl))
pnv_first_deep_stop_state = psscr_rl;
+
+   if (pnv_deepest_stop_state < psscr_rl)
+   pnv_deepest_stop_state = psscr_rl;
}
}
 
diff --git a/arch/powerpc/platforms/powernv/powernv.h 
b/arch/powerpc/platforms/powernv/powernv.h
index 6dbc0a1..da7c843 100644
--- a/arch/powerpc/platforms/powernv/powernv.h
+++ b/arch/powerpc/platforms/powernv/powernv.h
@@ -18,6 +18,7 @@ static inline void pnv_pci_shutdown(void) { }
 #endif
 
 extern u32 pnv_get_supported_cpuidle_states(void);
+extern u64 pnv_deepest_stop_state;
 
 extern void pnv_lpc_init(void);
 
diff --git a/arch/powerpc/platforms/powernv/smp.c 
b/arch/powerpc/platforms/powernv/smp.c
index ad7b1a3..f69ceb6 100644
--- a/arch/powerpc/platforms/powernv/smp.c
+++ b/arch/powerpc/platforms/powernv/smp.c
@@ -182,7 +182,9 @@ static void pnv_smp_cpu_kill_self(void)
 
ppc64_runlatch_off();
 
-   if (idle_states & OPAL_PM_WINKLE_ENABLED)
+   if (cpu_has_feature(CPU_FTR_ARCH_300))
+   srr1 = power_stop(pnv_deepest_stop_state);
+   else if (idle_states & OPAL_PM_WINKLE_ENABLED)
srr1 = power7_winkle();
else if ((idle_states & OPAL_PM_SLEEP_ENABLED) ||
(idle_states & OPAL_PM_SLEEP_ENABLED_ER1))
-- 
2.1.4

[PATCH v6 09/11] cpuidle/powernv: Use CPUIDLE_STATE_MAX instead of MAX_POWERNV_IDLE_STATES

[Shreyas B. Prabhu]

Use cpuidle's CPUIDLE_STATE_MAX macro instead of powernv specific
MAX_POWERNV_IDLE_STATES.

Cc: Rafael J. Wysocki <rafael.j.wyso...@intel.com>
Cc: Daniel Lezcano <daniel.lezc...@linaro.org>
Cc: linux...@vger.kernel.org
Suggested-by: Daniel Lezcano <daniel.lezc...@linaro.org>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
 - No changes after v5

Changes in v5
=
 - New in v5

 drivers/cpuidle/cpuidle-powernv.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/drivers/cpuidle/cpuidle-powernv.c 
b/drivers/cpuidle/cpuidle-powernv.c
index e12dc30..3a763a8 100644
--- a/drivers/cpuidle/cpuidle-powernv.c
+++ b/drivers/cpuidle/cpuidle-powernv.c
@@ -20,8 +20,6 @@
 #include 
 #include 
 
-#define MAX_POWERNV_IDLE_STATES8
-
 struct cpuidle_driver powernv_idle_driver = {
.name = "powernv_idle",
.owner= THIS_MODULE,
@@ -96,7 +94,7 @@ static int fastsleep_loop(struct cpuidle_device *dev,
 /*
  * States for dedicated partition case.
  */
-static struct cpuidle_state powernv_states[MAX_POWERNV_IDLE_STATES] = {
+static struct cpuidle_state powernv_states[CPUIDLE_STATE_MAX] = {
{ /* Snooze */
.name = "snooze",
.desc = "snooze",
-- 
2.1.4

[PATCH v6 05/11] powerpc/powernv: Make pnv_powersave_common more generic

[Shreyas B. Prabhu]

pnv_powersave_common does common steps needed before entering idle
state and eventually changes MSR to MSR_IDLE and does rfid to
pnv_enter_arch207_idle_mode.

Move the updation of HSTATE_HWTHREAD_STATE to pnv_powersave_common
from pnv_enter_arch207_idle_mode and make it more generic by passing the rfid
address as a function parameter.

Reviewed-by: Gautham R. Shenoy <e...@linux.vnet.ibm.com>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
 - No changes since v4

Changes in v4:
==
 - Moved renaming of power7_powersave_common to earlier patch

Changes in v3:
==
 - Moved HSTATE_HWTHREAD_STATE updation to power_powersave_common

 arch/powerpc/kernel/idle_power_common.S | 23 ++-
 1 file changed, 14 insertions(+), 9 deletions(-)

diff --git a/arch/powerpc/kernel/idle_power_common.S 
b/arch/powerpc/kernel/idle_power_common.S
index 34dbfc9..a8397e3 100644
--- a/arch/powerpc/kernel/idle_power_common.S
+++ b/arch/powerpc/kernel/idle_power_common.S
@@ -75,6 +75,8 @@ core_idle_lock_held:
  * To check IRQ_HAPPENED in r4
  * 0 - don't check
  * 1 - check
+ *
+ * Address to 'rfid' to in r5
  */
 _GLOBAL(pnv_powersave_common)
/* Use r3 to pass state nap/sleep/winkle */
@@ -127,28 +129,28 @@ _GLOBAL(pnv_powersave_common)
std r9,_MSR(r1)
std r1,PACAR1(r13)
 
+#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
+   /* Tell KVM we're entering idle */
+   li  r4,KVM_HWTHREAD_IN_NAP
+   stb r4,HSTATE_HWTHREAD_STATE(r13)
+#endif
+
/*
 * Go to real mode to do the nap, as required by the architecture.
 * Also, we need to be in real mode before setting hwthread_state,
 * because as soon as we do that, another thread can switch
 * the MMU context to the guest.
 */
-   LOAD_REG_IMMEDIATE(r5, MSR_IDLE)
+   LOAD_REG_IMMEDIATE(r7, MSR_IDLE)
li  r6, MSR_RI
andcr6, r9, r6
-   LOAD_REG_ADDR(r7, pnv_enter_arch207_idle_mode)
mtmsrd  r6, 1   /* clear RI before setting SRR0/1 */
-   mtspr   SPRN_SRR0, r7
-   mtspr   SPRN_SRR1, r5
+   mtspr   SPRN_SRR0, r5
+   mtspr   SPRN_SRR1, r7
rfid
 
.globl pnv_enter_arch207_idle_mode
 pnv_enter_arch207_idle_mode:
-#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
-   /* Tell KVM we're napping */
-   li  r4,KVM_HWTHREAD_IN_NAP
-   stb r4,HSTATE_HWTHREAD_STATE(r13)
-#endif
stb r3,PACA_THREAD_IDLE_STATE(r13)
cmpwi   cr3,r3,PNV_THREAD_SLEEP
bge cr3,2f
@@ -243,18 +245,21 @@ _GLOBAL(power7_idle)
 _GLOBAL(power7_nap)
mr  r4,r3
li  r3,PNV_THREAD_NAP
+   LOAD_REG_ADDR(r5, pnv_enter_arch207_idle_mode)
b   pnv_powersave_common
/* No return */
 
 _GLOBAL(power7_sleep)
li  r3,PNV_THREAD_SLEEP
li  r4,1
+   LOAD_REG_ADDR(r5, pnv_enter_arch207_idle_mode)
b   pnv_powersave_common
/* No return */
 
 _GLOBAL(power7_winkle)
li  r3,PNV_THREAD_WINKLE
li  r4,1
+   LOAD_REG_ADDR(r5, pnv_enter_arch207_idle_mode)
b   pnv_powersave_common
/* No return */
 
-- 
2.1.4

[PATCH v6 09/11] cpuidle/powernv: Use CPUIDLE_STATE_MAX instead of MAX_POWERNV_IDLE_STATES

[Shreyas B. Prabhu]

Use cpuidle's CPUIDLE_STATE_MAX macro instead of powernv specific
MAX_POWERNV_IDLE_STATES.

Cc: Rafael J. Wysocki 
Cc: Daniel Lezcano 
Cc: linux...@vger.kernel.org
Suggested-by: Daniel Lezcano 
Signed-off-by: Shreyas B. Prabhu 
---
 - No changes after v5

Changes in v5
=
 - New in v5

 drivers/cpuidle/cpuidle-powernv.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/drivers/cpuidle/cpuidle-powernv.c 
b/drivers/cpuidle/cpuidle-powernv.c
index e12dc30..3a763a8 100644
--- a/drivers/cpuidle/cpuidle-powernv.c
+++ b/drivers/cpuidle/cpuidle-powernv.c
@@ -20,8 +20,6 @@
 #include 
 #include 
 
-#define MAX_POWERNV_IDLE_STATES8
-
 struct cpuidle_driver powernv_idle_driver = {
.name = "powernv_idle",
.owner= THIS_MODULE,
@@ -96,7 +94,7 @@ static int fastsleep_loop(struct cpuidle_device *dev,
 /*
  * States for dedicated partition case.
  */
-static struct cpuidle_state powernv_states[MAX_POWERNV_IDLE_STATES] = {
+static struct cpuidle_state powernv_states[CPUIDLE_STATE_MAX] = {
{ /* Snooze */
.name = "snooze",
.desc = "snooze",
-- 
2.1.4

[PATCH v6 05/11] powerpc/powernv: Make pnv_powersave_common more generic

[Shreyas B. Prabhu]

pnv_powersave_common does common steps needed before entering idle
state and eventually changes MSR to MSR_IDLE and does rfid to
pnv_enter_arch207_idle_mode.

Move the updation of HSTATE_HWTHREAD_STATE to pnv_powersave_common
from pnv_enter_arch207_idle_mode and make it more generic by passing the rfid
address as a function parameter.

Reviewed-by: Gautham R. Shenoy 
Signed-off-by: Shreyas B. Prabhu 
---
 - No changes since v4

Changes in v4:
==
 - Moved renaming of power7_powersave_common to earlier patch

Changes in v3:
==
 - Moved HSTATE_HWTHREAD_STATE updation to power_powersave_common

 arch/powerpc/kernel/idle_power_common.S | 23 ++-
 1 file changed, 14 insertions(+), 9 deletions(-)

diff --git a/arch/powerpc/kernel/idle_power_common.S 
b/arch/powerpc/kernel/idle_power_common.S
index 34dbfc9..a8397e3 100644
--- a/arch/powerpc/kernel/idle_power_common.S
+++ b/arch/powerpc/kernel/idle_power_common.S
@@ -75,6 +75,8 @@ core_idle_lock_held:
  * To check IRQ_HAPPENED in r4
  * 0 - don't check
  * 1 - check
+ *
+ * Address to 'rfid' to in r5
  */
 _GLOBAL(pnv_powersave_common)
/* Use r3 to pass state nap/sleep/winkle */
@@ -127,28 +129,28 @@ _GLOBAL(pnv_powersave_common)
std r9,_MSR(r1)
std r1,PACAR1(r13)
 
+#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
+   /* Tell KVM we're entering idle */
+   li  r4,KVM_HWTHREAD_IN_NAP
+   stb r4,HSTATE_HWTHREAD_STATE(r13)
+#endif
+
/*
 * Go to real mode to do the nap, as required by the architecture.
 * Also, we need to be in real mode before setting hwthread_state,
 * because as soon as we do that, another thread can switch
 * the MMU context to the guest.
 */
-   LOAD_REG_IMMEDIATE(r5, MSR_IDLE)
+   LOAD_REG_IMMEDIATE(r7, MSR_IDLE)
li  r6, MSR_RI
andcr6, r9, r6
-   LOAD_REG_ADDR(r7, pnv_enter_arch207_idle_mode)
mtmsrd  r6, 1   /* clear RI before setting SRR0/1 */
-   mtspr   SPRN_SRR0, r7
-   mtspr   SPRN_SRR1, r5
+   mtspr   SPRN_SRR0, r5
+   mtspr   SPRN_SRR1, r7
rfid
 
.globl pnv_enter_arch207_idle_mode
 pnv_enter_arch207_idle_mode:
-#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
-   /* Tell KVM we're napping */
-   li  r4,KVM_HWTHREAD_IN_NAP
-   stb r4,HSTATE_HWTHREAD_STATE(r13)
-#endif
stb r3,PACA_THREAD_IDLE_STATE(r13)
cmpwi   cr3,r3,PNV_THREAD_SLEEP
bge cr3,2f
@@ -243,18 +245,21 @@ _GLOBAL(power7_idle)
 _GLOBAL(power7_nap)
mr  r4,r3
li  r3,PNV_THREAD_NAP
+   LOAD_REG_ADDR(r5, pnv_enter_arch207_idle_mode)
b   pnv_powersave_common
/* No return */
 
 _GLOBAL(power7_sleep)
li  r3,PNV_THREAD_SLEEP
li  r4,1
+   LOAD_REG_ADDR(r5, pnv_enter_arch207_idle_mode)
b   pnv_powersave_common
/* No return */
 
 _GLOBAL(power7_winkle)
li  r3,PNV_THREAD_WINKLE
li  r4,1
+   LOAD_REG_ADDR(r5, pnv_enter_arch207_idle_mode)
b   pnv_powersave_common
/* No return */
 
-- 
2.1.4

[PATCH v6 00/11] powerpc/powernv/cpuidle: Add support for POWER ISA v3 idle states

[Shreyas B. Prabhu]

POWER ISA v3 defines a new idle processor core mechanism. In summary,
 a) new instruction named stop is added. This instruction replaces
instructions like nap, sleep, rvwinkle.
 b) new per thread SPR named PSSCR is added which controls the behavior
of stop instruction. 

PSSCR has following key fields
Bits 0:3  - Power-Saving Level Status. This field indicates the
lowest power-saving state the thread entered since stop
instruction was last executed.

Bit 42 - Enable State Loss  
0 - No state is lost irrespective of other fields  
1 - Allows state loss

Bits 44:47 - Power-Saving Level Limit  
This limits the power-saving level that can be entered into.

Bits 60:63 - Requested Level  
Used to specify which power-saving level must be entered on
executing stop instruction

Stop idle states and their properties like name, latency, target
residency, psscr value are exposed via device tree.

This patch series adds support for this new mechanism.

Patches 1-7 are cleanups and code movement.
Patch 8 adds platform specific support for stop and psscr handling.
Patch 9 is a minor cleanup in cpuidle driver.
Patch 10 adds cpuidle driver support.
Patch 11 makes offlined cpu use deepest stop state.

Note: Documentation for the device tree bindings is posted here-
http://patchwork.ozlabs.org/patch/629125/


Changes in v6
=
 - Restore new POWER ISA v3 SPRS when waking up from deep idle

Changes in v5
=
 - Use generic cpuidle constant CPUIDLE_NAME_LEN
 - Fix return code handling for of_property_read_string_array
 - Use DT flags to determine if are using stop instruction, instead of
   cpu_has_feature
 - Removed uncessary cast with names
 - _loop -> stop_loop
 - Added POWERNV_THRESHOLD_LATENCY_NS to filter out idle states with high 
latency

Changes in v4
=
 - Added a patch to use PNV_THREAD_WINKLE macro while requesting for winkle
 - Moved power7_powersave_common rename to more appropriate patch
 - renaming power7_enter_nap_mode to pnv_enter_arch207_idle_mode
 - Added PSSCR layout to Patch 7's commit message
 - Improved / Fixed comments
 - Fixed whitespace error in paca.h
 - Using MAX_POSSIBLE_STOP_STATE macro instead of hardcoding 0xF has
   max possible stop state

Changes in v3
=
 - Rebased on powerpc-next
 - Dropping patch 1 since we are not adding a new file for P9 idle support
 - Improved comments in multiple places
 - Moved GET_PACA from power7_restore_hyp_resource to System Reset
 - Instead of moving few functions from idle_power7 to idle_power_common,
   renaming idle_power7.S to idle_power_common.S
 - Moved HSTATE_HWTHREAD_STATE updation to power_powersave_common
 - Dropped earlier patch 5 which moved few macros from idle_power_common to
   asm/cpuidle.h. 
 - Added a patch to rename reusable power7_* idle functions to pnv_*
 - Added new patch that creates abstraction for saving SPRs before
   entering deep idle states
 - Instead of introducing new file idle_power_stop.S, P9 idle support
   is added to idle_power_common.S using CPU_FTR sections.
 - Fixed r4 reg clobbering in power_stop0

Changes in v2
=
 - Rebased on v4.6-rc6
 - Using CPU_FTR_ARCH_300 bit instead of CPU_FTR_STOP_INST

Cc: Rafael J. Wysocki <rafael.j.wyso...@intel.com>
Cc: Daniel Lezcano <daniel.lezc...@linaro.org>
Cc: linux...@vger.kernel.org
Cc: Benjamin Herrenschmidt <b...@au1.ibm.com>
Cc: Michael Ellerman <m...@ellerman.id.au>
Cc: Paul Mackerras <pau...@ozlabs.org>
Cc: Michael Neuling <mi...@neuling.org>
Cc: linuxppc-...@lists.ozlabs.org
Cc: Rob Herring <robh...@kernel.org>
Cc: Lorenzo Pieralisi <lorenzo.pieral...@arm.com>

Shreyas B. Prabhu (11):
  powerpc/powernv: Use PNV_THREAD_WINKLE macro while requesting for
winkle
  powerpc/kvm: make hypervisor state restore a function
  powerpc/powernv: Rename idle_power7.S to idle_power_common.S
  powerpc/powernv: Rename reusable idle functions to hardware agnostic
names
  powerpc/powernv: Make pnv_powersave_common more generic
  powerpc/powernv: abstraction for saving SPRs before entering deep idle
states
  powerpc/powernv: set power_save func after the idle states are
initialized
  powerpc/powernv: Add platform support for stop instruction
  cpuidle/powernv: Use CPUIDLE_STATE_MAX instead of
MAX_POWERNV_IDLE_STATES
  cpuidle/powernv: Add support for POWER ISA v3 idle states
  powerpc/powernv: Use deepest stop state when cpu is offlined

 arch/powerpc/include/asm/cpuidle.h|   2 +
 arch/powerpc/include/asm/kvm_book3s_asm.h |   2 +-
 arch/powerpc/include/asm/machdep.h|   1 +
 arch/powerpc/include/asm/opal-api.h   |  11 +-
 arch/powerpc/include/asm/paca.h   |   2 +
 arch/powerpc/include/asm/ppc-opcode.h |   4 +
 arch/powerpc

[PATCH v6 00/11] powerpc/powernv/cpuidle: Add support for POWER ISA v3 idle states

[Shreyas B. Prabhu]

POWER ISA v3 defines a new idle processor core mechanism. In summary,
 a) new instruction named stop is added. This instruction replaces
instructions like nap, sleep, rvwinkle.
 b) new per thread SPR named PSSCR is added which controls the behavior
of stop instruction. 

PSSCR has following key fields
Bits 0:3  - Power-Saving Level Status. This field indicates the
lowest power-saving state the thread entered since stop
instruction was last executed.

Bit 42 - Enable State Loss  
0 - No state is lost irrespective of other fields  
1 - Allows state loss

Bits 44:47 - Power-Saving Level Limit  
This limits the power-saving level that can be entered into.

Bits 60:63 - Requested Level  
Used to specify which power-saving level must be entered on
executing stop instruction

Stop idle states and their properties like name, latency, target
residency, psscr value are exposed via device tree.

This patch series adds support for this new mechanism.

Patches 1-7 are cleanups and code movement.
Patch 8 adds platform specific support for stop and psscr handling.
Patch 9 is a minor cleanup in cpuidle driver.
Patch 10 adds cpuidle driver support.
Patch 11 makes offlined cpu use deepest stop state.

Note: Documentation for the device tree bindings is posted here-
http://patchwork.ozlabs.org/patch/629125/


Changes in v6
=
 - Restore new POWER ISA v3 SPRS when waking up from deep idle

Changes in v5
=
 - Use generic cpuidle constant CPUIDLE_NAME_LEN
 - Fix return code handling for of_property_read_string_array
 - Use DT flags to determine if are using stop instruction, instead of
   cpu_has_feature
 - Removed uncessary cast with names
 - _loop -> stop_loop
 - Added POWERNV_THRESHOLD_LATENCY_NS to filter out idle states with high 
latency

Changes in v4
=
 - Added a patch to use PNV_THREAD_WINKLE macro while requesting for winkle
 - Moved power7_powersave_common rename to more appropriate patch
 - renaming power7_enter_nap_mode to pnv_enter_arch207_idle_mode
 - Added PSSCR layout to Patch 7's commit message
 - Improved / Fixed comments
 - Fixed whitespace error in paca.h
 - Using MAX_POSSIBLE_STOP_STATE macro instead of hardcoding 0xF has
   max possible stop state

Changes in v3
=
 - Rebased on powerpc-next
 - Dropping patch 1 since we are not adding a new file for P9 idle support
 - Improved comments in multiple places
 - Moved GET_PACA from power7_restore_hyp_resource to System Reset
 - Instead of moving few functions from idle_power7 to idle_power_common,
   renaming idle_power7.S to idle_power_common.S
 - Moved HSTATE_HWTHREAD_STATE updation to power_powersave_common
 - Dropped earlier patch 5 which moved few macros from idle_power_common to
   asm/cpuidle.h. 
 - Added a patch to rename reusable power7_* idle functions to pnv_*
 - Added new patch that creates abstraction for saving SPRs before
   entering deep idle states
 - Instead of introducing new file idle_power_stop.S, P9 idle support
   is added to idle_power_common.S using CPU_FTR sections.
 - Fixed r4 reg clobbering in power_stop0

Changes in v2
=
 - Rebased on v4.6-rc6
 - Using CPU_FTR_ARCH_300 bit instead of CPU_FTR_STOP_INST

Cc: Rafael J. Wysocki 
Cc: Daniel Lezcano 
Cc: linux...@vger.kernel.org
Cc: Benjamin Herrenschmidt 
Cc: Michael Ellerman 
Cc: Paul Mackerras 
Cc: Michael Neuling 
Cc: linuxppc-...@lists.ozlabs.org
Cc: Rob Herring 
Cc: Lorenzo Pieralisi 

Shreyas B. Prabhu (11):
  powerpc/powernv: Use PNV_THREAD_WINKLE macro while requesting for
winkle
  powerpc/kvm: make hypervisor state restore a function
  powerpc/powernv: Rename idle_power7.S to idle_power_common.S
  powerpc/powernv: Rename reusable idle functions to hardware agnostic
names
  powerpc/powernv: Make pnv_powersave_common more generic
  powerpc/powernv: abstraction for saving SPRs before entering deep idle
states
  powerpc/powernv: set power_save func after the idle states are
initialized
  powerpc/powernv: Add platform support for stop instruction
  cpuidle/powernv: Use CPUIDLE_STATE_MAX instead of
MAX_POWERNV_IDLE_STATES
  cpuidle/powernv: Add support for POWER ISA v3 idle states
  powerpc/powernv: Use deepest stop state when cpu is offlined

 arch/powerpc/include/asm/cpuidle.h|   2 +
 arch/powerpc/include/asm/kvm_book3s_asm.h |   2 +-
 arch/powerpc/include/asm/machdep.h|   1 +
 arch/powerpc/include/asm/opal-api.h   |  11 +-
 arch/powerpc/include/asm/paca.h   |   2 +
 arch/powerpc/include/asm/ppc-opcode.h |   4 +
 arch/powerpc/include/asm/processor.h  |   1 +
 arch/powerpc/include/asm/reg.h|  14 +
 arch/powerpc/kernel/Makefile  |   2 +-
 arch/powerpc/kernel/asm-offsets.c |   2 +
 arch/powerpc/kernel/exceptions-64

[PATCH v5 01/11] powerpc/powernv: Use PNV_THREAD_WINKLE macro while requesting for winkle

[Shreyas B. Prabhu]

Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
-No changes in v5

Changes in v4
=
- New in v4

 arch/powerpc/kernel/idle_power7.S | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/powerpc/kernel/idle_power7.S 
b/arch/powerpc/kernel/idle_power7.S
index 470ceeb..705c867 100644
--- a/arch/powerpc/kernel/idle_power7.S
+++ b/arch/powerpc/kernel/idle_power7.S
@@ -252,7 +252,7 @@ _GLOBAL(power7_sleep)
/* No return */
 
 _GLOBAL(power7_winkle)
-   li  r3,3
+   li  r3,PNV_THREAD_WINKLE
li  r4,1
b   power7_powersave_common
/* No return */
-- 
2.1.4

[PATCH v5 01/11] powerpc/powernv: Use PNV_THREAD_WINKLE macro while requesting for winkle

[Shreyas B. Prabhu]

Signed-off-by: Shreyas B. Prabhu 
---
-No changes in v5

Changes in v4
=
- New in v4

 arch/powerpc/kernel/idle_power7.S | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/powerpc/kernel/idle_power7.S 
b/arch/powerpc/kernel/idle_power7.S
index 470ceeb..705c867 100644
--- a/arch/powerpc/kernel/idle_power7.S
+++ b/arch/powerpc/kernel/idle_power7.S
@@ -252,7 +252,7 @@ _GLOBAL(power7_sleep)
/* No return */
 
 _GLOBAL(power7_winkle)
-   li  r3,3
+   li  r3,PNV_THREAD_WINKLE
li  r4,1
b   power7_powersave_common
/* No return */
-- 
2.1.4

[PATCH v5 07/11] powerpc/powernv: set power_save func after the idle states are initialized

[Shreyas B. Prabhu]

pnv_init_idle_states discovers supported idle states from the
device tree and does the required initialization. Set power_save
function pointer only after this initialization is done

Reviewed-by: Gautham R. Shenoy <e...@linux.vnet.ibm.com>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
- No changes since v1

 arch/powerpc/platforms/powernv/idle.c  | 3 +++
 arch/powerpc/platforms/powernv/setup.c | 2 +-
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index fcc8b68..fbb09fb 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -285,6 +285,9 @@ static int __init pnv_init_idle_states(void)
}
 
pnv_alloc_idle_core_states();
+
+   if (supported_cpuidle_states & OPAL_PM_NAP_ENABLED)
+   ppc_md.power_save = power7_idle;
 out_free:
kfree(flags);
 out:
diff --git a/arch/powerpc/platforms/powernv/setup.c 
b/arch/powerpc/platforms/powernv/setup.c
index ee6430b..8492bbb 100644
--- a/arch/powerpc/platforms/powernv/setup.c
+++ b/arch/powerpc/platforms/powernv/setup.c
@@ -315,7 +315,7 @@ define_machine(powernv) {
.get_proc_freq  = pnv_get_proc_freq,
.progress   = pnv_progress,
.machine_shutdown   = pnv_shutdown,
-   .power_save = power7_idle,
+   .power_save = NULL,
.calibrate_decr = generic_calibrate_decr,
 #ifdef CONFIG_KEXEC
.kexec_cpu_down = pnv_kexec_cpu_down,
-- 
2.1.4

[PATCH v5 04/11] powerpc/powernv: Rename reusable idle functions to hardware agnostic names

[Shreyas B. Prabhu]

Functions like power7_wakeup_loss, power7_wakeup_noloss,
power7_wakeup_tb_loss are used by POWER7 and POWER8 hardware. They can
also be used by POWER9. Hence rename these functions hardware agnostic
names.

Suggested-by: Gautham R. Shenoy <e...@linux.vnet.ibm.com>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
 - No changes in v5

Changes in v4:
==
 - renaming power7_powersave_common to pnv_powersave_common
 - renaming power7_enter_nap_mode to pnv_enter_arch207_idle_mode

 arch/powerpc/kernel/exceptions-64s.S|  8 
 arch/powerpc/kernel/idle_power_common.S | 33 +
 arch/powerpc/kvm/book3s_hv_rmhandlers.S |  4 ++--
 3 files changed, 23 insertions(+), 22 deletions(-)

diff --git a/arch/powerpc/kernel/exceptions-64s.S 
b/arch/powerpc/kernel/exceptions-64s.S
index 4a74d6a..2a123cd 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -108,7 +108,7 @@ BEGIN_FTR_SECTION
 
cmpwi   cr3,r13,2
GET_PACA(r13)
-   bl  power7_restore_hyp_resource
+   bl  pnv_restore_hyp_resource
 
li  r0,PNV_THREAD_RUNNING
stb r0,PACA_THREAD_IDLE_STATE(r13)  /* Clear thread state */
@@ -128,8 +128,8 @@ BEGIN_FTR_SECTION
/* Return SRR1 from power7_nap() */
mfspr   r3,SPRN_SRR1
blt cr3,2f
-   b   power7_wakeup_loss
-2: b   power7_wakeup_noloss
+   b   pnv_wakeup_loss
+2: b   pnv_wakeup_noloss
 
 9:
 END_FTR_SECTION_IFSET(CPU_FTR_HVMODE | CPU_FTR_ARCH_206)
@@ -1269,7 +1269,7 @@ machine_check_handle_early:
GET_PACA(r13)
ld  r1,PACAR1(r13)
li  r3,PNV_THREAD_NAP
-   b   power7_enter_nap_mode
+   b   pnv_enter_arch207_idle_mode
 4:
 #endif
/*
diff --git a/arch/powerpc/kernel/idle_power_common.S 
b/arch/powerpc/kernel/idle_power_common.S
index d5def06..34dbfc9 100644
--- a/arch/powerpc/kernel/idle_power_common.S
+++ b/arch/powerpc/kernel/idle_power_common.S
@@ -1,5 +1,6 @@
 /*
- *  This file contains the power_save function for Power7 CPUs.
+ *  This file contains idle entry/exit functions for POWER7 and
+ *  POWER8 CPUs.
  *
  *  This program is free software; you can redistribute it and/or
  *  modify it under the terms of the GNU General Public License
@@ -75,7 +76,7 @@ core_idle_lock_held:
  * 0 - don't check
  * 1 - check
  */
-_GLOBAL(power7_powersave_common)
+_GLOBAL(pnv_powersave_common)
/* Use r3 to pass state nap/sleep/winkle */
/* NAP is a state loss, we create a regs frame on the
 * stack, fill it up with the state we care about and
@@ -135,14 +136,14 @@ _GLOBAL(power7_powersave_common)
LOAD_REG_IMMEDIATE(r5, MSR_IDLE)
li  r6, MSR_RI
andcr6, r9, r6
-   LOAD_REG_ADDR(r7, power7_enter_nap_mode)
+   LOAD_REG_ADDR(r7, pnv_enter_arch207_idle_mode)
mtmsrd  r6, 1   /* clear RI before setting SRR0/1 */
mtspr   SPRN_SRR0, r7
mtspr   SPRN_SRR1, r5
rfid
 
-   .globl  power7_enter_nap_mode
-power7_enter_nap_mode:
+   .globl pnv_enter_arch207_idle_mode
+pnv_enter_arch207_idle_mode:
 #ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
/* Tell KVM we're napping */
li  r4,KVM_HWTHREAD_IN_NAP
@@ -242,19 +243,19 @@ _GLOBAL(power7_idle)
 _GLOBAL(power7_nap)
mr  r4,r3
li  r3,PNV_THREAD_NAP
-   b   power7_powersave_common
+   b   pnv_powersave_common
/* No return */
 
 _GLOBAL(power7_sleep)
li  r3,PNV_THREAD_SLEEP
li  r4,1
-   b   power7_powersave_common
+   b   pnv_powersave_common
/* No return */
 
 _GLOBAL(power7_winkle)
li  r3,PNV_THREAD_WINKLE
li  r4,1
-   b   power7_powersave_common
+   b   pnv_powersave_common
/* No return */
 
 #define CHECK_HMI_INTERRUPT\
@@ -284,7 +285,7 @@ ALT_FTR_SECTION_END_NESTED_IFSET(CPU_FTR_ARCH_207S, 66);
\
  * r13 - Contents of HSPRG0
  * cr3 - set to gt if waking up with partial/complete hypervisor state loss
  */
-_GLOBAL(power7_restore_hyp_resource)
+_GLOBAL(pnv_restore_hyp_resource)
/*
 * Check if last bit of HSPGR0 is set. This indicates whether we are
 * waking up from winkle.
@@ -296,7 +297,7 @@ _GLOBAL(power7_restore_hyp_resource)
 
lbz r0,PACA_THREAD_IDLE_STATE(r13)
cmpwi   cr2,r0,PNV_THREAD_NAP
-   bgt cr2,power7_wakeup_tb_loss   /* Either sleep or Winkle */
+   bgt cr2,pnv_wakeup_tb_loss  /* Either sleep or Winkle */
 
/*
 * We fall through here if PACA_THREAD_IDLE_STATE shows we are waking
@@ -306,10 +307,10 @@ _GLOBAL(power7_restore_hyp_resource)
bgt cr3,.
 
blr /* Return back to System Reset vector from where
-  power7_restore_hyp_resource

[PATCH v5 03/11] powerpc/powernv: Rename idle_power7.S to idle_power_common.S

[Shreyas B. Prabhu]

idle_power7.S handles idle entry/exit for POWER7, POWER8 and in next
patch for POWER9. Rename the file to a non-hardware specific
name.

Reviewed-by: Gautham R. Shenoy <e...@linux.vnet.ibm.com>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
 - No changes from v3

Changes in v3:
==
 - Instead of moving few common functions from idle_power7.S to
   idle_power_common.S, renaming idle_power7.S to idle_power_common.S

 arch/powerpc/kernel/Makefile|   2 +-
 arch/powerpc/kernel/idle_power7.S   | 527 
 arch/powerpc/kernel/idle_power_common.S | 527 
 3 files changed, 528 insertions(+), 528 deletions(-)
 delete mode 100644 arch/powerpc/kernel/idle_power7.S
 create mode 100644 arch/powerpc/kernel/idle_power_common.S

diff --git a/arch/powerpc/kernel/Makefile b/arch/powerpc/kernel/Makefile
index 2da380f..99116da 100644
--- a/arch/powerpc/kernel/Makefile
+++ b/arch/powerpc/kernel/Makefile
@@ -47,7 +47,7 @@ obj-$(CONFIG_PPC_BOOK3E_64)   += exceptions-64e.o 
idle_book3e.o
 obj-$(CONFIG_PPC64)+= vdso64/
 obj-$(CONFIG_ALTIVEC)  += vecemu.o
 obj-$(CONFIG_PPC_970_NAP)  += idle_power4.o
-obj-$(CONFIG_PPC_P7_NAP)   += idle_power7.o
+obj-$(CONFIG_PPC_P7_NAP)   += idle_power_common.o
 procfs-y   := proc_powerpc.o
 obj-$(CONFIG_PROC_FS)  += $(procfs-y)
 rtaspci-$(CONFIG_PPC64)-$(CONFIG_PCI)  := rtas_pci.o
diff --git a/arch/powerpc/kernel/idle_power7.S 
b/arch/powerpc/kernel/idle_power7.S
deleted file mode 100644
index d5def06..000
--- a/arch/powerpc/kernel/idle_power7.S
+++ /dev/null
@@ -1,527 +0,0 @@
-/*
- *  This file contains the power_save function for Power7 CPUs.
- *
- *  This program is free software; you can redistribute it and/or
- *  modify it under the terms of the GNU General Public License
- *  as published by the Free Software Foundation; either version
- *  2 of the License, or (at your option) any later version.
- */
-
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-
-#undef DEBUG
-
-/*
- * Use unused space in the interrupt stack to save and restore
- * registers for winkle support.
- */
-#define _SDR1  GPR3
-#define _RPR   GPR4
-#define _SPURR GPR5
-#define _PURR  GPR6
-#define _TSCR  GPR7
-#define _DSCR  GPR8
-#define _AMOR  GPR9
-#define _WORT  GPR10
-#define _WORC  GPR11
-
-/* Idle state entry routines */
-
-#defineIDLE_STATE_ENTER_SEQ(IDLE_INST) \
-   /* Magic NAP/SLEEP/WINKLE mode enter sequence */\
-   std r0,0(r1);   \
-   ptesync;\
-   ld  r0,0(r1);   \
-1: cmp cr0,r0,r0;  \
-   bne 1b; \
-   IDLE_INST;  \
-   b   .
-
-   .text
-
-/*
- * Used by threads when the lock bit of core_idle_state is set.
- * Threads will spin in HMT_LOW until the lock bit is cleared.
- * r14 - pointer to core_idle_state
- * r15 - used to load contents of core_idle_state
- */
-
-core_idle_lock_held:
-   HMT_LOW
-3: lwz r15,0(r14)
-   andi.   r15,r15,PNV_CORE_IDLE_LOCK_BIT
-   bne 3b
-   HMT_MEDIUM
-   lwarx   r15,0,r14
-   blr
-
-/*
- * Pass requested state in r3:
- * r3 - PNV_THREAD_NAP/SLEEP/WINKLE
- *
- * To check IRQ_HAPPENED in r4
- * 0 - don't check
- * 1 - check
- */
-_GLOBAL(power7_powersave_common)
-   /* Use r3 to pass state nap/sleep/winkle */
-   /* NAP is a state loss, we create a regs frame on the
-* stack, fill it up with the state we care about and
-* stick a pointer to it in PACAR1. We really only
-* need to save PC, some CR bits and the NV GPRs,
-* but for now an interrupt frame will do.
-*/
-   mflrr0
-   std r0,16(r1)
-   stdur1,-INT_FRAME_SIZE(r1)
-   std r0,_LINK(r1)
-   std r0,_NIP(r1)
-
-   /* Hard disable interrupts */
-   mfmsr   r9
-   rldicl  r9,r9,48,1
-   rotldi  r9,r9,16
-   mtmsrd  r9,1/* hard-disable interrupts */
-
-   /* Check if something happened while soft-disabled */
-   lbz r0,PACAIRQHAPPENED(r13)
-   andi.   r0,r0,~PACA_IRQ_HARD_DIS@l
-   beq 1f
-   cmpwi   cr0,r4,0
-   beq 1f
-   addir1,r1,INT_FRAME_SIZE
-   ld  r0,16(r1)
-   li  r3,0/* Return 0 (no nap) */
-   mtlrr0
-   blr
-
-1: /* We mark irqs hard disabled as this is the state we'll
-* be in when returning and we need to tell arch_local_irq_restore()
-* about it
-*/
-   li  r0,PACA_IRQ_HARD_DIS
-   stb r0,PACAIRQ

[PATCH v5 07/11] powerpc/powernv: set power_save func after the idle states are initialized

[Shreyas B. Prabhu]

pnv_init_idle_states discovers supported idle states from the
device tree and does the required initialization. Set power_save
function pointer only after this initialization is done

Reviewed-by: Gautham R. Shenoy 
Signed-off-by: Shreyas B. Prabhu 
---
- No changes since v1

 arch/powerpc/platforms/powernv/idle.c  | 3 +++
 arch/powerpc/platforms/powernv/setup.c | 2 +-
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index fcc8b68..fbb09fb 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -285,6 +285,9 @@ static int __init pnv_init_idle_states(void)
}
 
pnv_alloc_idle_core_states();
+
+   if (supported_cpuidle_states & OPAL_PM_NAP_ENABLED)
+   ppc_md.power_save = power7_idle;
 out_free:
kfree(flags);
 out:
diff --git a/arch/powerpc/platforms/powernv/setup.c 
b/arch/powerpc/platforms/powernv/setup.c
index ee6430b..8492bbb 100644
--- a/arch/powerpc/platforms/powernv/setup.c
+++ b/arch/powerpc/platforms/powernv/setup.c
@@ -315,7 +315,7 @@ define_machine(powernv) {
.get_proc_freq  = pnv_get_proc_freq,
.progress   = pnv_progress,
.machine_shutdown   = pnv_shutdown,
-   .power_save = power7_idle,
+   .power_save = NULL,
.calibrate_decr = generic_calibrate_decr,
 #ifdef CONFIG_KEXEC
.kexec_cpu_down = pnv_kexec_cpu_down,
-- 
2.1.4

[PATCH v5 04/11] powerpc/powernv: Rename reusable idle functions to hardware agnostic names

[Shreyas B. Prabhu]

Functions like power7_wakeup_loss, power7_wakeup_noloss,
power7_wakeup_tb_loss are used by POWER7 and POWER8 hardware. They can
also be used by POWER9. Hence rename these functions hardware agnostic
names.

Suggested-by: Gautham R. Shenoy 
Signed-off-by: Shreyas B. Prabhu 
---
 - No changes in v5

Changes in v4:
==
 - renaming power7_powersave_common to pnv_powersave_common
 - renaming power7_enter_nap_mode to pnv_enter_arch207_idle_mode

 arch/powerpc/kernel/exceptions-64s.S|  8 
 arch/powerpc/kernel/idle_power_common.S | 33 +
 arch/powerpc/kvm/book3s_hv_rmhandlers.S |  4 ++--
 3 files changed, 23 insertions(+), 22 deletions(-)

diff --git a/arch/powerpc/kernel/exceptions-64s.S 
b/arch/powerpc/kernel/exceptions-64s.S
index 4a74d6a..2a123cd 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -108,7 +108,7 @@ BEGIN_FTR_SECTION
 
cmpwi   cr3,r13,2
GET_PACA(r13)
-   bl  power7_restore_hyp_resource
+   bl  pnv_restore_hyp_resource
 
li  r0,PNV_THREAD_RUNNING
stb r0,PACA_THREAD_IDLE_STATE(r13)  /* Clear thread state */
@@ -128,8 +128,8 @@ BEGIN_FTR_SECTION
/* Return SRR1 from power7_nap() */
mfspr   r3,SPRN_SRR1
blt cr3,2f
-   b   power7_wakeup_loss
-2: b   power7_wakeup_noloss
+   b   pnv_wakeup_loss
+2: b   pnv_wakeup_noloss
 
 9:
 END_FTR_SECTION_IFSET(CPU_FTR_HVMODE | CPU_FTR_ARCH_206)
@@ -1269,7 +1269,7 @@ machine_check_handle_early:
GET_PACA(r13)
ld  r1,PACAR1(r13)
li  r3,PNV_THREAD_NAP
-   b   power7_enter_nap_mode
+   b   pnv_enter_arch207_idle_mode
 4:
 #endif
/*
diff --git a/arch/powerpc/kernel/idle_power_common.S 
b/arch/powerpc/kernel/idle_power_common.S
index d5def06..34dbfc9 100644
--- a/arch/powerpc/kernel/idle_power_common.S
+++ b/arch/powerpc/kernel/idle_power_common.S
@@ -1,5 +1,6 @@
 /*
- *  This file contains the power_save function for Power7 CPUs.
+ *  This file contains idle entry/exit functions for POWER7 and
+ *  POWER8 CPUs.
  *
  *  This program is free software; you can redistribute it and/or
  *  modify it under the terms of the GNU General Public License
@@ -75,7 +76,7 @@ core_idle_lock_held:
  * 0 - don't check
  * 1 - check
  */
-_GLOBAL(power7_powersave_common)
+_GLOBAL(pnv_powersave_common)
/* Use r3 to pass state nap/sleep/winkle */
/* NAP is a state loss, we create a regs frame on the
 * stack, fill it up with the state we care about and
@@ -135,14 +136,14 @@ _GLOBAL(power7_powersave_common)
LOAD_REG_IMMEDIATE(r5, MSR_IDLE)
li  r6, MSR_RI
andcr6, r9, r6
-   LOAD_REG_ADDR(r7, power7_enter_nap_mode)
+   LOAD_REG_ADDR(r7, pnv_enter_arch207_idle_mode)
mtmsrd  r6, 1   /* clear RI before setting SRR0/1 */
mtspr   SPRN_SRR0, r7
mtspr   SPRN_SRR1, r5
rfid
 
-   .globl  power7_enter_nap_mode
-power7_enter_nap_mode:
+   .globl pnv_enter_arch207_idle_mode
+pnv_enter_arch207_idle_mode:
 #ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
/* Tell KVM we're napping */
li  r4,KVM_HWTHREAD_IN_NAP
@@ -242,19 +243,19 @@ _GLOBAL(power7_idle)
 _GLOBAL(power7_nap)
mr  r4,r3
li  r3,PNV_THREAD_NAP
-   b   power7_powersave_common
+   b   pnv_powersave_common
/* No return */
 
 _GLOBAL(power7_sleep)
li  r3,PNV_THREAD_SLEEP
li  r4,1
-   b   power7_powersave_common
+   b   pnv_powersave_common
/* No return */
 
 _GLOBAL(power7_winkle)
li  r3,PNV_THREAD_WINKLE
li  r4,1
-   b   power7_powersave_common
+   b   pnv_powersave_common
/* No return */
 
 #define CHECK_HMI_INTERRUPT\
@@ -284,7 +285,7 @@ ALT_FTR_SECTION_END_NESTED_IFSET(CPU_FTR_ARCH_207S, 66);
\
  * r13 - Contents of HSPRG0
  * cr3 - set to gt if waking up with partial/complete hypervisor state loss
  */
-_GLOBAL(power7_restore_hyp_resource)
+_GLOBAL(pnv_restore_hyp_resource)
/*
 * Check if last bit of HSPGR0 is set. This indicates whether we are
 * waking up from winkle.
@@ -296,7 +297,7 @@ _GLOBAL(power7_restore_hyp_resource)
 
lbz r0,PACA_THREAD_IDLE_STATE(r13)
cmpwi   cr2,r0,PNV_THREAD_NAP
-   bgt cr2,power7_wakeup_tb_loss   /* Either sleep or Winkle */
+   bgt cr2,pnv_wakeup_tb_loss  /* Either sleep or Winkle */
 
/*
 * We fall through here if PACA_THREAD_IDLE_STATE shows we are waking
@@ -306,10 +307,10 @@ _GLOBAL(power7_restore_hyp_resource)
bgt cr3,.
 
blr /* Return back to System Reset vector from where
-  power7_restore_hyp_resource was invoked */
+  pnv_restore_hyp_resource was invoked */
 
 
-_GLOBAL

[PATCH v5 03/11] powerpc/powernv: Rename idle_power7.S to idle_power_common.S

[Shreyas B. Prabhu]

idle_power7.S handles idle entry/exit for POWER7, POWER8 and in next
patch for POWER9. Rename the file to a non-hardware specific
name.

Reviewed-by: Gautham R. Shenoy 
Signed-off-by: Shreyas B. Prabhu 
---
 - No changes from v3

Changes in v3:
==
 - Instead of moving few common functions from idle_power7.S to
   idle_power_common.S, renaming idle_power7.S to idle_power_common.S

 arch/powerpc/kernel/Makefile|   2 +-
 arch/powerpc/kernel/idle_power7.S   | 527 
 arch/powerpc/kernel/idle_power_common.S | 527 
 3 files changed, 528 insertions(+), 528 deletions(-)
 delete mode 100644 arch/powerpc/kernel/idle_power7.S
 create mode 100644 arch/powerpc/kernel/idle_power_common.S

diff --git a/arch/powerpc/kernel/Makefile b/arch/powerpc/kernel/Makefile
index 2da380f..99116da 100644
--- a/arch/powerpc/kernel/Makefile
+++ b/arch/powerpc/kernel/Makefile
@@ -47,7 +47,7 @@ obj-$(CONFIG_PPC_BOOK3E_64)   += exceptions-64e.o 
idle_book3e.o
 obj-$(CONFIG_PPC64)+= vdso64/
 obj-$(CONFIG_ALTIVEC)  += vecemu.o
 obj-$(CONFIG_PPC_970_NAP)  += idle_power4.o
-obj-$(CONFIG_PPC_P7_NAP)   += idle_power7.o
+obj-$(CONFIG_PPC_P7_NAP)   += idle_power_common.o
 procfs-y   := proc_powerpc.o
 obj-$(CONFIG_PROC_FS)  += $(procfs-y)
 rtaspci-$(CONFIG_PPC64)-$(CONFIG_PCI)  := rtas_pci.o
diff --git a/arch/powerpc/kernel/idle_power7.S 
b/arch/powerpc/kernel/idle_power7.S
deleted file mode 100644
index d5def06..000
--- a/arch/powerpc/kernel/idle_power7.S
+++ /dev/null
@@ -1,527 +0,0 @@
-/*
- *  This file contains the power_save function for Power7 CPUs.
- *
- *  This program is free software; you can redistribute it and/or
- *  modify it under the terms of the GNU General Public License
- *  as published by the Free Software Foundation; either version
- *  2 of the License, or (at your option) any later version.
- */
-
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-
-#undef DEBUG
-
-/*
- * Use unused space in the interrupt stack to save and restore
- * registers for winkle support.
- */
-#define _SDR1  GPR3
-#define _RPR   GPR4
-#define _SPURR GPR5
-#define _PURR  GPR6
-#define _TSCR  GPR7
-#define _DSCR  GPR8
-#define _AMOR  GPR9
-#define _WORT  GPR10
-#define _WORC  GPR11
-
-/* Idle state entry routines */
-
-#defineIDLE_STATE_ENTER_SEQ(IDLE_INST) \
-   /* Magic NAP/SLEEP/WINKLE mode enter sequence */\
-   std r0,0(r1);   \
-   ptesync;\
-   ld  r0,0(r1);   \
-1: cmp cr0,r0,r0;  \
-   bne 1b; \
-   IDLE_INST;  \
-   b   .
-
-   .text
-
-/*
- * Used by threads when the lock bit of core_idle_state is set.
- * Threads will spin in HMT_LOW until the lock bit is cleared.
- * r14 - pointer to core_idle_state
- * r15 - used to load contents of core_idle_state
- */
-
-core_idle_lock_held:
-   HMT_LOW
-3: lwz r15,0(r14)
-   andi.   r15,r15,PNV_CORE_IDLE_LOCK_BIT
-   bne 3b
-   HMT_MEDIUM
-   lwarx   r15,0,r14
-   blr
-
-/*
- * Pass requested state in r3:
- * r3 - PNV_THREAD_NAP/SLEEP/WINKLE
- *
- * To check IRQ_HAPPENED in r4
- * 0 - don't check
- * 1 - check
- */
-_GLOBAL(power7_powersave_common)
-   /* Use r3 to pass state nap/sleep/winkle */
-   /* NAP is a state loss, we create a regs frame on the
-* stack, fill it up with the state we care about and
-* stick a pointer to it in PACAR1. We really only
-* need to save PC, some CR bits and the NV GPRs,
-* but for now an interrupt frame will do.
-*/
-   mflrr0
-   std r0,16(r1)
-   stdur1,-INT_FRAME_SIZE(r1)
-   std r0,_LINK(r1)
-   std r0,_NIP(r1)
-
-   /* Hard disable interrupts */
-   mfmsr   r9
-   rldicl  r9,r9,48,1
-   rotldi  r9,r9,16
-   mtmsrd  r9,1/* hard-disable interrupts */
-
-   /* Check if something happened while soft-disabled */
-   lbz r0,PACAIRQHAPPENED(r13)
-   andi.   r0,r0,~PACA_IRQ_HARD_DIS@l
-   beq 1f
-   cmpwi   cr0,r4,0
-   beq 1f
-   addir1,r1,INT_FRAME_SIZE
-   ld  r0,16(r1)
-   li  r3,0/* Return 0 (no nap) */
-   mtlrr0
-   blr
-
-1: /* We mark irqs hard disabled as this is the state we'll
-* be in when returning and we need to tell arch_local_irq_restore()
-* about it
-*/
-   li  r0,PACA_IRQ_HARD_DIS
-   stb r0,PACAIRQHAPPENED(r13)
-
-   /* We haven't lost state ... yet

[PATCH v5 08/11] powerpc/powernv: Add platform support for stop instruction

[Shreyas B. Prabhu]

POWER ISA v3 defines a new idle processor core mechanism. In summary,
 a) new instruction named stop is added. This instruction replaces
instructions like nap, sleep, rvwinkle.
 b) new per thread SPR named Processor Stop Status and Control Register
(PSSCR) is added which controls the behavior of stop instruction.

PSSCR layout:
--
| PLS | /// | SD | ESL | EC | PSLL | /// | TR | MTL | RL |
--
0  4 41   4243   44 4854   5660

PSSCR key fields:
Bits 0:3  - Power-Saving Level Status. This field indicates the lowest
power-saving state the thread entered since stop instruction was last
executed.

Bit 42 - Enable State Loss
0 - No state is lost irrespective of other fields
1 - Allows state loss

Bits 44:47 - Power-Saving Level Limit
This limits the power-saving level that can be entered into.

Bits 60:63 - Requested Level
Used to specify which power-saving level must be entered on executing
stop instruction

This patch adds support for stop instruction and PSSCR handling.

Reviewed-by: Gautham R. Shenoy <e...@linux.vnet.ibm.com>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
 - No changes in v5

Changes in v4:
==
 - Added PSSCR layout to commit message
 - Improved / Fixed comments
 - Fixed whitespace error in paca.h
 - Using MAX_POSSIBLE_STOP_STATE macro instead of hardcoding 0xF has 
   max possible stop state

Changes in v3:
==
 - Instead of introducing new file idle_power_stop.S, P9 idle support
   is added to idle_power_common.S using CPU_FTR sections.
 - Fixed r4 reg clobbering in power_stop0
 - Improved comments

Changes in v2:
==
 - Using CPU_FTR_ARCH_300 bit instead of CPU_FTR_STOP_INST

 arch/powerpc/include/asm/cpuidle.h|   2 +
 arch/powerpc/include/asm/kvm_book3s_asm.h |   2 +-
 arch/powerpc/include/asm/machdep.h|   1 +
 arch/powerpc/include/asm/opal-api.h   |  11 ++-
 arch/powerpc/include/asm/paca.h   |   2 +
 arch/powerpc/include/asm/ppc-opcode.h |   4 +
 arch/powerpc/include/asm/processor.h  |   1 +
 arch/powerpc/include/asm/reg.h|  11 +++
 arch/powerpc/kernel/asm-offsets.c |   2 +
 arch/powerpc/kernel/idle_power_common.S   | 149 +++---
 arch/powerpc/platforms/powernv/idle.c |  84 ++---
 11 files changed, 239 insertions(+), 30 deletions(-)

diff --git a/arch/powerpc/include/asm/cpuidle.h 
b/arch/powerpc/include/asm/cpuidle.h
index d2f99ca..3d7fc06 100644
--- a/arch/powerpc/include/asm/cpuidle.h
+++ b/arch/powerpc/include/asm/cpuidle.h
@@ -13,6 +13,8 @@
 #ifndef __ASSEMBLY__
 extern u32 pnv_fastsleep_workaround_at_entry[];
 extern u32 pnv_fastsleep_workaround_at_exit[];
+
+extern u64 pnv_first_deep_stop_state;
 #endif
 
 #endif
diff --git a/arch/powerpc/include/asm/kvm_book3s_asm.h 
b/arch/powerpc/include/asm/kvm_book3s_asm.h
index 72b6225..d318d43 100644
--- a/arch/powerpc/include/asm/kvm_book3s_asm.h
+++ b/arch/powerpc/include/asm/kvm_book3s_asm.h
@@ -162,7 +162,7 @@ struct kvmppc_book3s_shadow_vcpu {
 
 /* Values for kvm_state */
 #define KVM_HWTHREAD_IN_KERNEL 0
-#define KVM_HWTHREAD_IN_NAP1
+#define KVM_HWTHREAD_IN_IDLE   1
 #define KVM_HWTHREAD_IN_KVM2
 
 #endif /* __ASM_KVM_BOOK3S_ASM_H__ */
diff --git a/arch/powerpc/include/asm/machdep.h 
b/arch/powerpc/include/asm/machdep.h
index 6bdcd0d..ae3b155 100644
--- a/arch/powerpc/include/asm/machdep.h
+++ b/arch/powerpc/include/asm/machdep.h
@@ -262,6 +262,7 @@ struct machdep_calls {
 extern void e500_idle(void);
 extern void power4_idle(void);
 extern void power7_idle(void);
+extern void power_stop0(void);
 extern void ppc6xx_idle(void);
 extern void book3e_idle(void);
 
diff --git a/arch/powerpc/include/asm/opal-api.h 
b/arch/powerpc/include/asm/opal-api.h
index 9bb8ddf..7f3f8c6 100644
--- a/arch/powerpc/include/asm/opal-api.h
+++ b/arch/powerpc/include/asm/opal-api.h
@@ -162,13 +162,20 @@
 
 /* Device tree flags */
 
-/* Flags set in power-mgmt nodes in device tree if
- * respective idle states are supported in the platform.
+/*
+ * Flags set in power-mgmt nodes in device tree describing
+ * idle states that are supported in the platform.
  */
+
+#define OPAL_PM_TIMEBASE_STOP  0x0002
+#define OPAL_PM_LOSE_HYP_CONTEXT   0x2000
+#define OPAL_PM_LOSE_FULL_CONTEXT  0x4000
 #define OPAL_PM_NAP_ENABLED0x0001
 #define OPAL_PM_SLEEP_ENABLED  0x0002
 #define OPAL_PM_WINKLE_ENABLED 0x0004
 #define OPAL_PM_SLEEP_ENABLED_ER1  0x0008 /* with workaround */
+#define OPAL_PM_STOP_INST_FAST 0x0010
+#define OPAL_PM_STOP_INST_DEEP 0x0020
 
 /*
  * OPAL_CONFIG_CPU_IDLE_STATE parameters
diff --git a/arch/powerpc/include/asm/paca.h b/arch/powerpc/include/asm/

[PATCH v5 09/11] cpuidle/powernv: Use CPUIDLE_STATE_MAX instead of MAX_POWERNV_IDLE_STATES

[Shreyas B. Prabhu]

Use cpuidle's CPUIDLE_STATE_MAX macro instead of powernv specific
MAX_POWERNV_IDLE_STATES.

Cc: Rafael J. Wysocki <rafael.j.wyso...@intel.com>
Cc: Daniel Lezcano <daniel.lezc...@linaro.org>
Cc: linux...@vger.kernel.org
Suggested-by: Daniel Lezcano <daniel.lezc...@linaro.org>
Signed-off-by: Shreyas B. Prabhu <shre...@linux.vnet.ibm.com>
---
 - New in v5

 drivers/cpuidle/cpuidle-powernv.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/drivers/cpuidle/cpuidle-powernv.c 
b/drivers/cpuidle/cpuidle-powernv.c
index e12dc30..3a763a8 100644
--- a/drivers/cpuidle/cpuidle-powernv.c
+++ b/drivers/cpuidle/cpuidle-powernv.c
@@ -20,8 +20,6 @@
 #include 
 #include 
 
-#define MAX_POWERNV_IDLE_STATES8
-
 struct cpuidle_driver powernv_idle_driver = {
.name = "powernv_idle",
.owner= THIS_MODULE,
@@ -96,7 +94,7 @@ static int fastsleep_loop(struct cpuidle_device *dev,
 /*
  * States for dedicated partition case.
  */
-static struct cpuidle_state powernv_states[MAX_POWERNV_IDLE_STATES] = {
+static struct cpuidle_state powernv_states[CPUIDLE_STATE_MAX] = {
{ /* Snooze */
.name = "snooze",
.desc = "snooze",
-- 
2.1.4