[PATCH] Revert "powerpc/powernv/idle: Replace CPU feature check with PVR check"

[Pratik Rajesh Sampat]

Cpuidle stop state implementation has minor optimizations for P10
where hardware preserves more SPR registers compared to P9.
The current P9 driver works for P10, although does few extra
save-restores. P9 driver can provide the required power management
features like SMT thread folding and core level power savings
on a P10 platform.

Until the P10 stop driver is available, revert the commit which
allows for only P9 systems to utilize cpuidle and blocks all
idle stop states for P10.
Cpu idle states are enabled and tested on the P10 platform
with this fix.

This reverts commit 8747bf36f312356f8a295a0c39ff092d65ce75ae.

Fixes: 8747bf36f312 ("powerpc/powernv/idle: Replace CPU feature check with PVR 
check")
Signed-off-by: Pratik Rajesh Sampat 
---
 @mpe: This revert would resolve a staging issue wherein the P10 stop
 driver is not yet ready while cpuidle stop states need not be blocked
 on 5.9 for Power10 systems which could cause SMT folding related
 performance issues.

 The P10 stop driver is in the works here:
 https://lists.ozlabs.org/pipermail/linuxppc-dev/2020-August/216773.html

 arch/powerpc/platforms/powernv/idle.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 77513a80cef9..345ab062b21a 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -1223,7 +1223,7 @@ static void __init pnv_probe_idle_states(void)
return;
}
 
-   if (pvr_version_is(PVR_POWER9))
+   if (cpu_has_feature(CPU_FTR_ARCH_300))
pnv_power9_idle_init();
 
for (i = 0; i < nr_pnv_idle_states; i++)
-- 
2.25.4

[PATCH v4 2/3] powerpc/powernv/idle: Rename pnv_first_spr_loss_level variable

[Pratik Rajesh Sampat]

Replace the variable name from using "pnv_first_spr_loss_level" to
"deep_spr_loss_state".

pnv_first_spr_loss_level is supposed to be the earliest state that
has OPAL_PM_LOSE_FULL_CONTEXT set, in other places the kernel uses the
"deep" states as terminology. Hence renaming the variable to be coherent
to its semantics.

Signed-off-by: Pratik Rajesh Sampat 
---
 arch/powerpc/platforms/powernv/idle.c | 18 +-
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 642abf0b8329..28462d59a8e1 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -48,7 +48,7 @@ static bool default_stop_found;
  * First stop state levels when SPR and TB loss can occur.
  */
 static u64 pnv_first_tb_loss_level = MAX_STOP_STATE + 1;
-static u64 pnv_first_spr_loss_level = MAX_STOP_STATE + 1;
+static u64 deep_spr_loss_state = MAX_STOP_STATE + 1;
 
 /*
  * psscr value and mask of the deepest stop idle state.
@@ -657,7 +657,7 @@ static unsigned long power9_idle_stop(unsigned long psscr, 
bool mmu_on)
  */
mmcr0   = mfspr(SPRN_MMCR0);
}
-   if ((psscr & PSSCR_RL_MASK) >= pnv_first_spr_loss_level) {
+   if ((psscr & PSSCR_RL_MASK) >= deep_spr_loss_state) {
sprs.lpcr   = mfspr(SPRN_LPCR);
sprs.hfscr  = mfspr(SPRN_HFSCR);
sprs.fscr   = mfspr(SPRN_FSCR);
@@ -741,7 +741,7 @@ static unsigned long power9_idle_stop(unsigned long psscr, 
bool mmu_on)
 * just always test PSSCR for SPR/TB state loss.
 */
pls = (psscr & PSSCR_PLS) >> PSSCR_PLS_SHIFT;
-   if (likely(pls < pnv_first_spr_loss_level)) {
+   if (likely(pls < deep_spr_loss_state)) {
if (sprs_saved)
atomic_stop_thread_idle();
goto out;
@@ -1088,7 +1088,7 @@ static void __init pnv_power9_idle_init(void)
 * the deepest loss-less (OPAL_PM_STOP_INST_FAST) stop state.
 */
pnv_first_tb_loss_level = MAX_STOP_STATE + 1;
-   pnv_first_spr_loss_level = MAX_STOP_STATE + 1;
+   deep_spr_loss_state = MAX_STOP_STATE + 1;
for (i = 0; i < nr_pnv_idle_states; i++) {
int err;
struct pnv_idle_states_t *state = _idle_states[i];
@@ -1099,8 +1099,8 @@ static void __init pnv_power9_idle_init(void)
pnv_first_tb_loss_level = psscr_rl;
 
if ((state->flags & OPAL_PM_LOSE_FULL_CONTEXT) &&
-(pnv_first_spr_loss_level > psscr_rl))
-   pnv_first_spr_loss_level = psscr_rl;
+(deep_spr_loss_state > psscr_rl))
+   deep_spr_loss_state = psscr_rl;
 
/*
 * The idle code does not deal with TB loss occurring
@@ -,8 +,8 @@ static void __init pnv_power9_idle_init(void)
 * compatibility.
 */
if ((state->flags & OPAL_PM_TIMEBASE_STOP) &&
-(pnv_first_spr_loss_level > psscr_rl))
-   pnv_first_spr_loss_level = psscr_rl;
+(deep_spr_loss_state > psscr_rl))
+   deep_spr_loss_state = psscr_rl;
 
err = validate_psscr_val_mask(>psscr_val,
  >psscr_mask,
@@ -1158,7 +1158,7 @@ static void __init pnv_power9_idle_init(void)
}
 
pr_info("cpuidle-powernv: First stop level that may lose SPRs = 
0x%llx\n",
-   pnv_first_spr_loss_level);
+   deep_spr_loss_state);
 
pr_info("cpuidle-powernv: First stop level that may lose timebase = 
0x%llx\n",
pnv_first_tb_loss_level);
-- 
2.25.4

[PATCH v4 3/3] powerpc/powernv/idle: Exclude mfspr on HID1, 4, 5 on P9 and above

[Pratik Rajesh Sampat]

POWER9 onwards the support for the registers HID1, HID4, HID5 has been
receded.
Although mfspr on the above registers worked in Power9, In Power10
simulator is unrecognized. Moving their assignment under the
check for machines lower than Power9

Signed-off-by: Pratik Rajesh Sampat 
Reviewed-by: Gautham R. Shenoy 
Reviewed-by: Nicholas Piggin 
---
 arch/powerpc/platforms/powernv/idle.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 28462d59a8e1..92098d6106be 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -73,9 +73,6 @@ static int pnv_save_sprs_for_deep_states(void)
 */
uint64_t lpcr_val   = mfspr(SPRN_LPCR);
uint64_t hid0_val   = mfspr(SPRN_HID0);
-   uint64_t hid1_val   = mfspr(SPRN_HID1);
-   uint64_t hid4_val   = mfspr(SPRN_HID4);
-   uint64_t hid5_val   = mfspr(SPRN_HID5);
uint64_t hmeer_val  = mfspr(SPRN_HMEER);
uint64_t msr_val = MSR_IDLE;
uint64_t psscr_val = pnv_deepest_stop_psscr_val;
@@ -117,6 +114,9 @@ static int pnv_save_sprs_for_deep_states(void)
 
/* Only p8 needs to set extra HID regiters */
if (!cpu_has_feature(CPU_FTR_ARCH_300)) {
+   uint64_t hid1_val = mfspr(SPRN_HID1);
+   uint64_t hid4_val = mfspr(SPRN_HID4);
+   uint64_t hid5_val = mfspr(SPRN_HID5);
 
rc = opal_slw_set_reg(pir, SPRN_HID1, hid1_val);
if (rc != 0)
-- 
2.25.4

[PATCH v4 1/3] powerpc/powernv/idle: Replace CPU features check with PVR check

[Pratik Rajesh Sampat]

As the idle framework's architecture is incomplete, hence instead of
checking for just the processor type advertised in the device tree CPU
features; check for the Processor Version Register (PVR) so that finer
granularity can be leveraged while making processor checks.

Hence, making the PVR check on the outer level function, subsequently in
the hierarchy keeping the CPU_FTR_ARCH_300 check intact as it is a
faster check to do because of static branches

Signed-off-by: Pratik Rajesh Sampat 
---
 arch/powerpc/platforms/powernv/idle.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 2dd467383a88..642abf0b8329 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -1205,7 +1205,7 @@ static void __init pnv_probe_idle_states(void)
return;
}
 
-   if (cpu_has_feature(CPU_FTR_ARCH_300))
+   if (pvr_version_is(PVR_POWER9))
pnv_power9_idle_init();
 
for (i = 0; i < nr_pnv_idle_states; i++)
-- 
2.25.4

[PATCH v4 0/3] powernv/idle: Power9 idle cleanup

[Pratik Rajesh Sampat]

v3: https://lkml.org/lkml/2020/7/17/1093
Changelog v3-->v4:
Based on comments from Nicholas Piggin and Gautham Shenoy,
1. Changed the naming of pnv_first_spr_loss_level from
pnv_first_fullstate_loss_level to deep_spr_loss_state
2. Make the P9 PVR check only on the top level function
pnv_probe_idle_states and let the rest of the checks be DT based because
it is faster to do so

Pratik Rajesh Sampat (3):
  powerpc/powernv/idle: Replace CPU features check with PVR check
  powerpc/powernv/idle: Rename pnv_first_spr_loss_level variable
  powerpc/powernv/idle: Exclude mfspr on HID1,4,5 on P9 and above

 arch/powerpc/platforms/powernv/idle.c | 26 +-
 1 file changed, 13 insertions(+), 13 deletions(-)

-- 
2.25.4

[PATCH v3 0/2] Selftest for cpuidle latency measurement

[Pratik Rajesh Sampat]

v2: https://lkml.org/lkml/2020/7/17/369
Changelog v2-->v3
Based on comments from Gautham R. Shenoy adding the following in the
selftest,
1. Grepping modules to determine if already loaded
2. Wrapper to enable/disable states
3. Preventing any operation/test on offlined CPUs 
---

The patch series introduces a mechanism to measure wakeup latency for
IPI and timer based interrupts
The motivation behind this series is to find significant deviations
behind advertised latency and resisdency values

To achieve this, we introduce a kernel module and expose its control
knobs through the debugfs interface that the selftests can engage with.

The kernel module provides the following interfaces within
/sys/kernel/debug/latency_test/ for,
1. IPI test:
  ipi_cpu_dest   # Destination CPU for the IPI
  ipi_cpu_src# Origin of the IPI
  ipi_latency_ns # Measured latency time in ns
2. Timeout test:
  timeout_cpu_src # CPU on which the timer to be queued
  timeout_expected_ns # Timer duration
  timeout_diff_ns # Difference of actual duration vs expected timer
To include the module, check option and include as module
kernel hacking -> Cpuidle latency selftests

The selftest inserts the module, disables all the idle states and
enables them one by one testing the following:
1. Keeping source CPU constant, iterates through all the CPUS measuring
   IPI latency for baseline (CPU is busy with
   "cat /dev/random > /dev/null" workload) and the when the CPU is
   allowed to be at rest
2. Iterating through all the CPUs, sending expected timer durations to
   be equivalent to the residency of the the deepest idle state
   enabled and extracting the difference in time between the time of
   wakeup and the expected timer duration

Usage
-
Can be used in conjuction to the rest of the selftests.
Default Output location in: tools/testing/cpuidle/cpuidle.log

To run this test specifically:
$ make -C tools/testing/selftests TARGETS="cpuidle" run_tests

There are a few optinal arguments too that the script can take
[-h ]
[-m ]
[-o ]

Sample output snippet
-
--IPI Latency Test---
--Baseline IPI Latency measurement: CPU Busy--
SRC_CPU   DEST_CPU IPI_Latency(ns)
...
08 1996
09 2125
0   10 1264
0   11 1788
0   12 2045
Baseline Average IPI latency(ns): 1843
---Enabling state: 5---
SRC_CPU   DEST_CPU IPI_Latency(ns)
08   621719
09   624752
0   10   622218
0   11   623968
0   12   621303
Expected IPI latency(ns): 10
Observed Average IPI latency(ns): 622792

--Timeout Latency Test--
--Baseline Timeout Latency measurement: CPU Busy--
Wakeup_src Baseline_delay(ns) 
...
82249
92226
10   2211
11   2183
12   2263
Baseline Average timeout diff(ns): 2226
---Enabling state: 5---
8   10749   
9   10911   
10  10912   
11  12100   
12  73276   
Expected timeout(ns): 1200
Observed Average timeout diff(ns): 23589

Pratik Rajesh Sampat (2):
  cpuidle: Trace IPI based and timer based wakeup latency from idle
states
  selftest/cpuidle: Add support for cpuidle latency measurement

 drivers/cpuidle/Makefile   |   1 +
 drivers/cpuidle/test-cpuidle_latency.c | 150 ++
 lib/Kconfig.debug  |  10 +
 tools/testing/selftests/Makefile   |   1 +
 tools/testing/selftests/cpuidle/Makefile   |   6 +
 tools/testing/selftests/cpuidle/cpuidle.sh | 310 +
 tools/testing/selftests/cpuidle/settings   |   1 +
 7 files changed, 479 insertions(+)
 create mode 100644 drivers/cpuidle/test-cpuidle_latency.c
 create mode 100644 tools/testing/selftests/cpuidle/Makefile
 create mode 100755 tools/testing/selftests/cpuidle/cpuidle.sh
 create mode 100644 tools/testing/selftests/cpuidle/settings

-- 
2.25.4

[PATCH v3 2/2] selftest/cpuidle: Add support for cpuidle latency measurement

[Pratik Rajesh Sampat]

This patch adds support to trace IPI based and timer based wakeup
latency from idle states

Latches onto the test-cpuidle_latency kernel module using the debugfs
interface to send IPIs or schedule a timer based event, which in-turn
populates the debugfs with the latency measurements.

Currently for the IPI and timer tests; first disable all idle states
and then test for latency measurements incrementally enabling each state

Signed-off-by: Pratik Rajesh Sampat 
---
 tools/testing/selftests/Makefile   |   1 +
 tools/testing/selftests/cpuidle/Makefile   |   6 +
 tools/testing/selftests/cpuidle/cpuidle.sh | 310 +
 tools/testing/selftests/cpuidle/settings   |   1 +
 4 files changed, 318 insertions(+)
 create mode 100644 tools/testing/selftests/cpuidle/Makefile
 create mode 100755 tools/testing/selftests/cpuidle/cpuidle.sh
 create mode 100644 tools/testing/selftests/cpuidle/settings

diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile
index 1195bd85af38..ab6cf51f3518 100644
--- a/tools/testing/selftests/Makefile
+++ b/tools/testing/selftests/Makefile
@@ -7,6 +7,7 @@ TARGETS += capabilities
 TARGETS += cgroup
 TARGETS += clone3
 TARGETS += cpufreq
+TARGETS += cpuidle
 TARGETS += cpu-hotplug
 TARGETS += drivers/dma-buf
 TARGETS += efivarfs
diff --git a/tools/testing/selftests/cpuidle/Makefile 
b/tools/testing/selftests/cpuidle/Makefile
new file mode 100644
index ..72fd5d2e974d
--- /dev/null
+++ b/tools/testing/selftests/cpuidle/Makefile
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+all:
+
+TEST_PROGS := cpuidle.sh
+
+include ../lib.mk
diff --git a/tools/testing/selftests/cpuidle/cpuidle.sh 
b/tools/testing/selftests/cpuidle/cpuidle.sh
new file mode 100755
index ..19cc24ccd4af
--- /dev/null
+++ b/tools/testing/selftests/cpuidle/cpuidle.sh
@@ -0,0 +1,310 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+LOG=cpuidle.log
+MODULE=/lib/modules/$(uname -r)/kernel/drivers/cpuidle/test-cpuidle_latency.ko
+
+# Kselftest framework requirement - SKIP code is 4.
+ksft_skip=4
+
+helpme()
+{
+   printf "Usage: $0 [-h] [-todg args]
+   [-h ]
+   [-m ]
+   [-o ]
+   \n"
+   exit 2
+}
+
+parse_arguments()
+{
+   while getopts ht:m:o: arg
+   do
+   case $arg in
+   h) # --help
+   helpme
+   ;;
+   m) # --mod-file
+   MODULE=$OPTARG
+   ;;
+   o) # output log files
+   LOG=$OPTARG
+   ;;
+   \?)
+   helpme
+   ;;
+   esac
+   done
+}
+
+ins_mod()
+{
+   if [ ! -f "$MODULE" ]; then
+   printf "$MODULE module does not exist. Exitting\n"
+   exit $ksft_skip
+   fi
+   # Check if the module is already loaded
+   grep "test_cpuidle_latency" /proc/modules
+   if [ $? == 0 ]; then
+   printf "Module: $MODULE already loaded\n\n"
+   return 0
+   fi
+   printf "Inserting $MODULE module\n\n"
+   insmod $MODULE
+   if [ $? != 0 ]; then
+   printf "Insmod $MODULE failed\n"
+   exit $ksft_skip
+   fi
+}
+
+compute_average()
+{
+   arr=("$@")
+   sum=0
+   size=${#arr[@]}
+   if [ $size == 0 ]; then
+   avg=0
+   return 1
+   fi
+   for i in "${arr[@]}"
+   do
+   sum=$((sum + i))
+   done
+   avg=$((sum/size))
+}
+
+# Disable all stop states
+disable_idle()
+{
+   for ((cpu=0; cpu 
/sys/devices/system/cpu/cpu$cpu/cpuidle/state$state/disable
+   done
+   done
+}
+
+# Perform operation on each CPU for the given state
+# $1 - Operation: enable (0) / disable (1)
+# $2 - State to enable
+op_state()
+{
+   for ((cpu=0; cpu 
/sys/devices/system/cpu/cpu$cpu/cpuidle/state$2/disable
+   done
+}
+
+cpuidle_enable_state()
+{
+   state=$1
+   op_state 0 $state
+}
+
+cpuidle_disable_state()
+{
+   state=$1
+   op_state 1 $state
+}
+
+cpu_is_online()
+{
+   cpu=$1
+   status=$(cat /sys/devices/system/cpu/cpu$cpu/online)
+   echo $status
+}
+
+# Extract latency in microseconds and convert to nanoseconds
+extract_latency()
+{
+   for ((state=0; state /dev/null &
+   task_pid=$!
+   # Wait for the workload to achieve 100% CPU usage
+   sleep 1
+   fi
+   taskset 0x1 echo $dest_cpu > /sys/kernel/debug/latency_test/ipi_cpu_dest
+   ipi_latency=$(cat /sys/kernel/debug/latency_test/ipi_latency_ns)
+   src_cpu=$(cat /sys/kernel/debug/latency_test/ipi_cpu_src)
+   if [ "$1" = "baseline" ]; then
+   

[PATCH v3 1/2] cpuidle: Trace IPI based and timer based wakeup latency from idle states

[Pratik Rajesh Sampat]

Fire directed smp_call_function_single IPIs from a specified source
CPU to the specified target CPU to reduce the noise we have to wade
through in the trace log.
The module is based on the idea written by Srivatsa Bhat and maintained
by Vaidyanathan Srinivasan internally.

Queue HR timer and measure jitter. Wakeup latency measurement for idle
states using hrtimer.  Echo a value in ns to timer_test_function and
watch trace. A HRtimer will be queued and when it fires the expected
wakeup vs actual wakeup is computes and delay printed in ns.

Implemented as a module which utilizes debugfs so that it can be
integrated with selftests.

To include the module, check option and include as module
kernel hacking -> Cpuidle latency selftests

[srivatsa.b...@linux.vnet.ibm.com: Initial implementation in
 cpidle/sysfs]

[sva...@linux.vnet.ibm.com: wakeup latency measurements using hrtimer
 and fix some of the time calculation]

[e...@linux.vnet.ibm.com: Fix some whitespace and tab errors and
 increase the resolution of IPI wakeup]

Signed-off-by: Pratik Rajesh Sampat 
Reviewed-by: Gautham R. Shenoy 
---
 drivers/cpuidle/Makefile   |   1 +
 drivers/cpuidle/test-cpuidle_latency.c | 150 +
 lib/Kconfig.debug  |  10 ++
 3 files changed, 161 insertions(+)
 create mode 100644 drivers/cpuidle/test-cpuidle_latency.c

diff --git a/drivers/cpuidle/Makefile b/drivers/cpuidle/Makefile
index f07800cbb43f..2ae05968078c 100644
--- a/drivers/cpuidle/Makefile
+++ b/drivers/cpuidle/Makefile
@@ -8,6 +8,7 @@ obj-$(CONFIG_ARCH_NEEDS_CPU_IDLE_COUPLED) += coupled.o
 obj-$(CONFIG_DT_IDLE_STATES) += dt_idle_states.o
 obj-$(CONFIG_ARCH_HAS_CPU_RELAX) += poll_state.o
 obj-$(CONFIG_HALTPOLL_CPUIDLE)   += cpuidle-haltpoll.o
+obj-$(CONFIG_IDLE_LATENCY_SELFTEST)  += test-cpuidle_latency.o
 
 
##
 # ARM SoC drivers
diff --git a/drivers/cpuidle/test-cpuidle_latency.c 
b/drivers/cpuidle/test-cpuidle_latency.c
new file mode 100644
index ..61574665e972
--- /dev/null
+++ b/drivers/cpuidle/test-cpuidle_latency.c
@@ -0,0 +1,150 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Module-based API test facility for cpuidle latency using IPIs and timers
+ */
+
+#include 
+#include 
+#include 
+
+/* IPI based wakeup latencies */
+struct latency {
+   unsigned int src_cpu;
+   unsigned int dest_cpu;
+   ktime_t time_start;
+   ktime_t time_end;
+   u64 latency_ns;
+} ipi_wakeup;
+
+static void measure_latency(void *info)
+{
+   struct latency *v;
+   ktime_t time_diff;
+
+   v = (struct latency *)info;
+   v->time_end = ktime_get();
+   time_diff = ktime_sub(v->time_end, v->time_start);
+   v->latency_ns = ktime_to_ns(time_diff);
+}
+
+void run_smp_call_function_test(unsigned int cpu)
+{
+   ipi_wakeup.src_cpu = smp_processor_id();
+   ipi_wakeup.dest_cpu = cpu;
+   ipi_wakeup.time_start = ktime_get();
+   smp_call_function_single(cpu, measure_latency, _wakeup, 1);
+}
+
+/* Timer based wakeup latencies */
+struct timer_data {
+   unsigned int src_cpu;
+   u64 timeout;
+   ktime_t time_start;
+   ktime_t time_end;
+   struct hrtimer timer;
+   u64 timeout_diff_ns;
+} timer_wakeup;
+
+static enum hrtimer_restart timer_called(struct hrtimer *hrtimer)
+{
+   struct timer_data *w;
+   ktime_t time_diff;
+
+   w = container_of(hrtimer, struct timer_data, timer);
+   w->time_end = ktime_get();
+
+   time_diff = ktime_sub(w->time_end, w->time_start);
+   time_diff = ktime_sub(time_diff, ns_to_ktime(w->timeout));
+   w->timeout_diff_ns = ktime_to_ns(time_diff);
+   return HRTIMER_NORESTART;
+}
+
+static void run_timer_test(unsigned int ns)
+{
+   hrtimer_init(_wakeup.timer, CLOCK_MONOTONIC,
+HRTIMER_MODE_REL);
+   timer_wakeup.timer.function = timer_called;
+   timer_wakeup.time_start = ktime_get();
+   timer_wakeup.src_cpu = smp_processor_id();
+   timer_wakeup.timeout = ns;
+
+   hrtimer_start(_wakeup.timer, ns_to_ktime(ns),
+ HRTIMER_MODE_REL_PINNED);
+}
+
+static struct dentry *dir;
+
+static int cpu_read_op(void *data, u64 *value)
+{
+   *value = ipi_wakeup.dest_cpu;
+   return 0;
+}
+
+static int cpu_write_op(void *data, u64 value)
+{
+   run_smp_call_function_test(value);
+   return 0;
+}
+DEFINE_SIMPLE_ATTRIBUTE(ipi_ops, cpu_read_op, cpu_write_op, "%llu\n");
+
+static int timeout_read_op(void *data, u64 *value)
+{
+   *value = timer_wakeup.timeout;
+   return 0;
+}
+
+static int timeout_write_op(void *data, u64 value)
+{
+   run_timer_test(value);
+   return 0;
+}
+DEFINE_SIMPLE_ATTRIBUTE(timeout_ops, timeout_read_op, timeout_write_op, 
"%llu\n");
+
+static int __init latency_init(void)
+{
+   struct dentry *temp;
+

[PATCH v3 3/3] powerpc/powernv/idle: Exclude mfspr on HID1, 4, 5 on P9 and above

[Pratik Rajesh Sampat]

POWER9 onwards the support for the registers HID1, HID4, HID5 has been
receded.
Although mfspr on the above registers worked in Power9, In Power10
simulator is unrecognized. Moving their assignment under the
check for machines lower than Power9

Signed-off-by: Pratik Rajesh Sampat 
Reviewed-by: Gautham R. Shenoy 
---
 arch/powerpc/platforms/powernv/idle.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index d439e11af101..d24d6671f3e8 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -73,9 +73,6 @@ static int pnv_save_sprs_for_deep_states(void)
 */
uint64_t lpcr_val   = mfspr(SPRN_LPCR);
uint64_t hid0_val   = mfspr(SPRN_HID0);
-   uint64_t hid1_val   = mfspr(SPRN_HID1);
-   uint64_t hid4_val   = mfspr(SPRN_HID4);
-   uint64_t hid5_val   = mfspr(SPRN_HID5);
uint64_t hmeer_val  = mfspr(SPRN_HMEER);
uint64_t msr_val = MSR_IDLE;
uint64_t psscr_val = pnv_deepest_stop_psscr_val;
@@ -117,6 +114,9 @@ static int pnv_save_sprs_for_deep_states(void)
 
/* Only p8 needs to set extra HID regiters */
if (!pvr_version_is(PVR_POWER9)) {
+   uint64_t hid1_val = mfspr(SPRN_HID1);
+   uint64_t hid4_val = mfspr(SPRN_HID4);
+   uint64_t hid5_val = mfspr(SPRN_HID5);
 
rc = opal_slw_set_reg(pir, SPRN_HID1, hid1_val);
if (rc != 0)
-- 
2.25.4

[PATCH v3 2/3] powerpc/powernv/idle: Rename pnv_first_spr_loss_level variable

[Pratik Rajesh Sampat]

Replace the variable name from using "pnv_first_spr_loss_level" to
"pnv_first_fullstate_loss_level".

As pnv_first_spr_loss_level is supposed to be the earliest state that
has OPAL_PM_LOSE_FULL_CONTEXT set, however as shallow states too loose
SPR values, render an incorrect terminology.

Signed-off-by: Pratik Rajesh Sampat 
---
 arch/powerpc/platforms/powernv/idle.c | 18 +-
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index f62904f70fc6..d439e11af101 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -48,7 +48,7 @@ static bool default_stop_found;
  * First stop state levels when SPR and TB loss can occur.
  */
 static u64 pnv_first_tb_loss_level = MAX_STOP_STATE + 1;
-static u64 pnv_first_spr_loss_level = MAX_STOP_STATE + 1;
+static u64 pnv_first_fullstate_loss_level = MAX_STOP_STATE + 1;
 
 /*
  * psscr value and mask of the deepest stop idle state.
@@ -657,7 +657,7 @@ static unsigned long power9_idle_stop(unsigned long psscr, 
bool mmu_on)
  */
mmcr0   = mfspr(SPRN_MMCR0);
}
-   if ((psscr & PSSCR_RL_MASK) >= pnv_first_spr_loss_level) {
+   if ((psscr & PSSCR_RL_MASK) >= pnv_first_fullstate_loss_level) {
sprs.lpcr   = mfspr(SPRN_LPCR);
sprs.hfscr  = mfspr(SPRN_HFSCR);
sprs.fscr   = mfspr(SPRN_FSCR);
@@ -741,7 +741,7 @@ static unsigned long power9_idle_stop(unsigned long psscr, 
bool mmu_on)
 * just always test PSSCR for SPR/TB state loss.
 */
pls = (psscr & PSSCR_PLS) >> PSSCR_PLS_SHIFT;
-   if (likely(pls < pnv_first_spr_loss_level)) {
+   if (likely(pls < pnv_first_fullstate_loss_level)) {
if (sprs_saved)
atomic_stop_thread_idle();
goto out;
@@ -1088,7 +1088,7 @@ static void __init pnv_power9_idle_init(void)
 * the deepest loss-less (OPAL_PM_STOP_INST_FAST) stop state.
 */
pnv_first_tb_loss_level = MAX_STOP_STATE + 1;
-   pnv_first_spr_loss_level = MAX_STOP_STATE + 1;
+   pnv_first_fullstate_loss_level = MAX_STOP_STATE + 1;
for (i = 0; i < nr_pnv_idle_states; i++) {
int err;
struct pnv_idle_states_t *state = _idle_states[i];
@@ -1099,8 +1099,8 @@ static void __init pnv_power9_idle_init(void)
pnv_first_tb_loss_level = psscr_rl;
 
if ((state->flags & OPAL_PM_LOSE_FULL_CONTEXT) &&
-(pnv_first_spr_loss_level > psscr_rl))
-   pnv_first_spr_loss_level = psscr_rl;
+(pnv_first_fullstate_loss_level > psscr_rl))
+   pnv_first_fullstate_loss_level = psscr_rl;
 
/*
 * The idle code does not deal with TB loss occurring
@@ -,8 +,8 @@ static void __init pnv_power9_idle_init(void)
 * compatibility.
 */
if ((state->flags & OPAL_PM_TIMEBASE_STOP) &&
-(pnv_first_spr_loss_level > psscr_rl))
-   pnv_first_spr_loss_level = psscr_rl;
+(pnv_first_fullstate_loss_level > psscr_rl))
+   pnv_first_fullstate_loss_level = psscr_rl;
 
err = validate_psscr_val_mask(>psscr_val,
  >psscr_mask,
@@ -1158,7 +1158,7 @@ static void __init pnv_power9_idle_init(void)
}
 
pr_info("cpuidle-powernv: First stop level that may lose SPRs = 
0x%llx\n",
-   pnv_first_spr_loss_level);
+   pnv_first_fullstate_loss_level);
 
pr_info("cpuidle-powernv: First stop level that may lose timebase = 
0x%llx\n",
pnv_first_tb_loss_level);
-- 
2.25.4

[PATCH v3 0/3] powernv/idle: Power9 idle cleanup

[Pratik Rajesh Sampat]

v2: https://lkml.org/lkml/2020/7/10/28
Changelog v2-->v3:
1. Based on comments from Nicholas Piggin, introducing a cleanup patch
   in which, instead of checking for CPU_FTR_ARCH_300 check for
   PVR_POWER9 to allow for a finer granularity of checks where one
   processor generation can have multiple ways to handling idle

2. Removed saving-restoring DAWR, DAWRX patch for P10 systems. Based on
   discussions it has become evident that checks based on PVR is the way
   to go; however, P10 PVR is yet to up-stream hence shelving this patch
   for later.

Pratik Rajesh Sampat (3):
  powerpc/powernv/idle: Replace CPU features checks with PVR checks
  powerpc/powernv/idle: Rename pnv_first_spr_loss_level variable
  powerpc/powernv/idle: Exclude mfspr on HID1,4,5 on P9 and above

 arch/powerpc/platforms/powernv/idle.c | 38 +--
 1 file changed, 19 insertions(+), 19 deletions(-)

-- 
2.25.4

[PATCH v3 1/3] powerpc/powernv/idle: Replace CPU features checks with PVR checks

[Pratik Rajesh Sampat]

As the idle framework's architecture is incomplete, hence instead of
checking for just the processor type advertised in the device tree CPU
features; check for the Processor Version Register (PVR) so that finer
granularity can be leveraged while making processor checks.

Signed-off-by: Pratik Rajesh Sampat 
---
 arch/powerpc/platforms/powernv/idle.c | 14 +++---
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 2dd467383a88..f62904f70fc6 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -92,7 +92,7 @@ static int pnv_save_sprs_for_deep_states(void)
if (rc != 0)
return rc;
 
-   if (cpu_has_feature(CPU_FTR_ARCH_300)) {
+   if (pvr_version_is(PVR_POWER9)) {
rc = opal_slw_set_reg(pir, P9_STOP_SPR_MSR, msr_val);
if (rc)
return rc;
@@ -116,7 +116,7 @@ static int pnv_save_sprs_for_deep_states(void)
return rc;
 
/* Only p8 needs to set extra HID regiters */
-   if (!cpu_has_feature(CPU_FTR_ARCH_300)) {
+   if (!pvr_version_is(PVR_POWER9)) {
 
rc = opal_slw_set_reg(pir, SPRN_HID1, hid1_val);
if (rc != 0)
@@ -971,7 +971,7 @@ unsigned long pnv_cpu_offline(unsigned int cpu)
 
__ppc64_runlatch_off();
 
-   if (cpu_has_feature(CPU_FTR_ARCH_300) && deepest_stop_found) {
+   if (pvr_version_is(PVR_POWER9) && deepest_stop_found) {
unsigned long psscr;
 
psscr = mfspr(SPRN_PSSCR);
@@ -1175,7 +1175,7 @@ static void __init pnv_disable_deep_states(void)
pr_warn("cpuidle-powernv: Disabling idle states that lose full 
context\n");
pr_warn("cpuidle-powernv: Idle power-savings, CPU-Hotplug affected\n");
 
-   if (cpu_has_feature(CPU_FTR_ARCH_300) &&
+   if (pvr_version_is(PVR_POWER9) &&
(pnv_deepest_stop_flag & OPAL_PM_LOSE_FULL_CONTEXT)) {
/*
 * Use the default stop state for CPU-Hotplug
@@ -1205,7 +1205,7 @@ static void __init pnv_probe_idle_states(void)
return;
}
 
-   if (cpu_has_feature(CPU_FTR_ARCH_300))
+   if (pvr_version_is(PVR_POWER9))
pnv_power9_idle_init();
 
for (i = 0; i < nr_pnv_idle_states; i++)
@@ -1278,7 +1278,7 @@ static int pnv_parse_cpuidle_dt(void)
pnv_idle_states[i].residency_ns = temp_u32[i];
 
/* For power9 */
-   if (cpu_has_feature(CPU_FTR_ARCH_300)) {
+   if (pvr_version_is(PVR_POWER9)) {
/* Read pm_crtl_val */
if (of_property_read_u64_array(np, "ibm,cpu-idle-state-psscr",
   temp_u64, nr_idle_states)) {
@@ -1337,7 +1337,7 @@ static int __init pnv_init_idle_states(void)
if (cpu == cpu_first_thread_sibling(cpu))
p->idle_state = (1 << threads_per_core) - 1;
 
-   if (!cpu_has_feature(CPU_FTR_ARCH_300)) {
+   if (!pvr_version_is(PVR_POWER9)) {
/* P7/P8 nap */
p->thread_idle_state = PNV_THREAD_RUNNING;
} else {
-- 
2.25.4

[PATCH v2 2/2] selftest/cpuidle: Add support for cpuidle latency measurement

[Pratik Rajesh Sampat]

This patch adds support to trace IPI based and timer based wakeup
latency from idle states

Latches onto the test-cpuidle_latency kernel module using the debugfs
interface to send IPIs or schedule a timer based event, which in-turn
populates the debugfs with the latency measurements.

Currently for the IPI and timer tests; first disable all idle states
and then test for latency measurements incrementally enabling each state

Signed-off-by: Pratik Rajesh Sampat 
---
 tools/testing/selftests/Makefile   |   1 +
 tools/testing/selftests/cpuidle/Makefile   |   6 +
 tools/testing/selftests/cpuidle/cpuidle.sh | 257 +
 tools/testing/selftests/cpuidle/settings   |   1 +
 4 files changed, 265 insertions(+)
 create mode 100644 tools/testing/selftests/cpuidle/Makefile
 create mode 100755 tools/testing/selftests/cpuidle/cpuidle.sh
 create mode 100644 tools/testing/selftests/cpuidle/settings

diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile
index 1195bd85af38..ab6cf51f3518 100644
--- a/tools/testing/selftests/Makefile
+++ b/tools/testing/selftests/Makefile
@@ -7,6 +7,7 @@ TARGETS += capabilities
 TARGETS += cgroup
 TARGETS += clone3
 TARGETS += cpufreq
+TARGETS += cpuidle
 TARGETS += cpu-hotplug
 TARGETS += drivers/dma-buf
 TARGETS += efivarfs
diff --git a/tools/testing/selftests/cpuidle/Makefile 
b/tools/testing/selftests/cpuidle/Makefile
new file mode 100644
index ..72fd5d2e974d
--- /dev/null
+++ b/tools/testing/selftests/cpuidle/Makefile
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+all:
+
+TEST_PROGS := cpuidle.sh
+
+include ../lib.mk
diff --git a/tools/testing/selftests/cpuidle/cpuidle.sh 
b/tools/testing/selftests/cpuidle/cpuidle.sh
new file mode 100755
index ..5c313a3abb0c
--- /dev/null
+++ b/tools/testing/selftests/cpuidle/cpuidle.sh
@@ -0,0 +1,257 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+LOG=cpuidle.log
+MODULE=/lib/modules/$(uname -r)/kernel/drivers/cpuidle/test-cpuidle_latency.ko
+
+# Kselftest framework requirement - SKIP code is 4.
+ksft_skip=4
+
+helpme()
+{
+   printf "Usage: $0 [-h] [-todg args]
+   [-h ]
+   [-m ]
+   [-o ]
+   \n"
+   exit 2
+}
+
+parse_arguments()
+{
+   while getopts ht:m:o: arg
+   do
+   case $arg in
+   h) # --help
+   helpme
+   ;;
+   m) # --mod-file
+   MODULE=$OPTARG
+   ;;
+   o) # output log files
+   LOG=$OPTARG
+   ;;
+   \?)
+   helpme
+   ;;
+   esac
+   done
+}
+
+ins_mod()
+{
+   if [ ! -f "$MODULE" ]; then
+   printf "$MODULE module does not exist. Exitting\n"
+   exit $ksft_skip
+   fi
+   printf "Inserting $MODULE module\n\n"
+   insmod $MODULE
+   if [ $? != 0 ]; then
+   printf "Insmod $MODULE failed\n"
+   exit $ksft_skip
+   fi
+}
+
+compute_average()
+{
+   arr=("$@")
+   sum=0
+   size=${#arr[@]}
+   for i in "${arr[@]}"
+   do
+   sum=$((sum + i))
+   done
+   avg=$((sum/size))
+}
+
+# Disable all stop states
+disable_idle()
+{
+   for ((cpu=0; cpu 
/sys/devices/system/cpu/cpu$cpu/cpuidle/state$state/disable
+   done
+   done
+}
+
+# Perform operation on each CPU for the given state
+# $1 - Operation: enable (0) / disable (1)
+# $2 - State to enable
+op_state()
+{
+   for ((cpu=0; cpu 
/sys/devices/system/cpu/cpu$cpu/cpuidle/state$2/disable
+   done
+}
+
+# Extract latency in microseconds and convert to nanoseconds
+extract_latency()
+{
+   for ((state=0; state /dev/null &
+   task_pid=$!
+   # Wait for the workload to achieve 100% CPU usage
+   sleep 1
+fi
+taskset 0x1 echo $dest_cpu > 
/sys/kernel/debug/latency_test/ipi_cpu_dest
+ipi_latency=$(cat /sys/kernel/debug/latency_test/ipi_latency_ns)
+src_cpu=$(cat /sys/kernel/debug/latency_test/ipi_cpu_src)
+if [ "$1" = "baseline" ]; then
+   kill $task_pid
+   wait $task_pid 2>/dev/null
+fi
+}
+
+# Incrementally Enable idle states one by one and compute the latency
+run_ipi_tests()
+{
+extract_latency
+disable_idle
+declare -a avg_arr
+echo -e "--IPI Latency Test---" >> $LOG
+
+   echo -e "--Baseline IPI Latency measurement: CPU Busy--" >> $LOG
+   printf "%s %10s %12s\n" "SRC_CPU" "DEST_CPU" "IPI_Latency(ns)" 
>> $LOG
+   

[PATCH v2 1/2] cpuidle: Trace IPI based and timer based wakeup latency from idle states

[Pratik Rajesh Sampat]

Fire directed smp_call_function_single IPIs from a specified source
CPU to the specified target CPU to reduce the noise we have to wade
through in the trace log.
The module is based on the idea written by Srivatsa Bhat and maintained
by Vaidyanathan Srinivasan internally.

Queue HR timer and measure jitter. Wakeup latency measurement for idle
states using hrtimer.  Echo a value in ns to timer_test_function and
watch trace. A HRtimer will be queued and when it fires the expected
wakeup vs actual wakeup is computes and delay printed in ns.

Implemented as a module which utilizes debugfs so that it can be
integrated with selftests.

To include the module, check option and include as module
kernel hacking -> Cpuidle latency selftests

[srivatsa.b...@linux.vnet.ibm.com: Initial implementation in
 cpidle/sysfs]

[sva...@linux.vnet.ibm.com: wakeup latency measurements using hrtimer
 and fix some of the time calculation]

[e...@linux.vnet.ibm.com: Fix some whitespace and tab errors and
 increase the resolution of IPI wakeup]

Signed-off-by: Pratik Rajesh Sampat 
---
 drivers/cpuidle/Makefile   |   1 +
 drivers/cpuidle/test-cpuidle_latency.c | 150 +
 lib/Kconfig.debug  |  10 ++
 3 files changed, 161 insertions(+)
 create mode 100644 drivers/cpuidle/test-cpuidle_latency.c

diff --git a/drivers/cpuidle/Makefile b/drivers/cpuidle/Makefile
index f07800cbb43f..2ae05968078c 100644
--- a/drivers/cpuidle/Makefile
+++ b/drivers/cpuidle/Makefile
@@ -8,6 +8,7 @@ obj-$(CONFIG_ARCH_NEEDS_CPU_IDLE_COUPLED) += coupled.o
 obj-$(CONFIG_DT_IDLE_STATES) += dt_idle_states.o
 obj-$(CONFIG_ARCH_HAS_CPU_RELAX) += poll_state.o
 obj-$(CONFIG_HALTPOLL_CPUIDLE)   += cpuidle-haltpoll.o
+obj-$(CONFIG_IDLE_LATENCY_SELFTEST)  += test-cpuidle_latency.o
 
 
##
 # ARM SoC drivers
diff --git a/drivers/cpuidle/test-cpuidle_latency.c 
b/drivers/cpuidle/test-cpuidle_latency.c
new file mode 100644
index ..61574665e972
--- /dev/null
+++ b/drivers/cpuidle/test-cpuidle_latency.c
@@ -0,0 +1,150 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Module-based API test facility for cpuidle latency using IPIs and timers
+ */
+
+#include 
+#include 
+#include 
+
+/* IPI based wakeup latencies */
+struct latency {
+   unsigned int src_cpu;
+   unsigned int dest_cpu;
+   ktime_t time_start;
+   ktime_t time_end;
+   u64 latency_ns;
+} ipi_wakeup;
+
+static void measure_latency(void *info)
+{
+   struct latency *v;
+   ktime_t time_diff;
+
+   v = (struct latency *)info;
+   v->time_end = ktime_get();
+   time_diff = ktime_sub(v->time_end, v->time_start);
+   v->latency_ns = ktime_to_ns(time_diff);
+}
+
+void run_smp_call_function_test(unsigned int cpu)
+{
+   ipi_wakeup.src_cpu = smp_processor_id();
+   ipi_wakeup.dest_cpu = cpu;
+   ipi_wakeup.time_start = ktime_get();
+   smp_call_function_single(cpu, measure_latency, _wakeup, 1);
+}
+
+/* Timer based wakeup latencies */
+struct timer_data {
+   unsigned int src_cpu;
+   u64 timeout;
+   ktime_t time_start;
+   ktime_t time_end;
+   struct hrtimer timer;
+   u64 timeout_diff_ns;
+} timer_wakeup;
+
+static enum hrtimer_restart timer_called(struct hrtimer *hrtimer)
+{
+   struct timer_data *w;
+   ktime_t time_diff;
+
+   w = container_of(hrtimer, struct timer_data, timer);
+   w->time_end = ktime_get();
+
+   time_diff = ktime_sub(w->time_end, w->time_start);
+   time_diff = ktime_sub(time_diff, ns_to_ktime(w->timeout));
+   w->timeout_diff_ns = ktime_to_ns(time_diff);
+   return HRTIMER_NORESTART;
+}
+
+static void run_timer_test(unsigned int ns)
+{
+   hrtimer_init(_wakeup.timer, CLOCK_MONOTONIC,
+HRTIMER_MODE_REL);
+   timer_wakeup.timer.function = timer_called;
+   timer_wakeup.time_start = ktime_get();
+   timer_wakeup.src_cpu = smp_processor_id();
+   timer_wakeup.timeout = ns;
+
+   hrtimer_start(_wakeup.timer, ns_to_ktime(ns),
+ HRTIMER_MODE_REL_PINNED);
+}
+
+static struct dentry *dir;
+
+static int cpu_read_op(void *data, u64 *value)
+{
+   *value = ipi_wakeup.dest_cpu;
+   return 0;
+}
+
+static int cpu_write_op(void *data, u64 value)
+{
+   run_smp_call_function_test(value);
+   return 0;
+}
+DEFINE_SIMPLE_ATTRIBUTE(ipi_ops, cpu_read_op, cpu_write_op, "%llu\n");
+
+static int timeout_read_op(void *data, u64 *value)
+{
+   *value = timer_wakeup.timeout;
+   return 0;
+}
+
+static int timeout_write_op(void *data, u64 value)
+{
+   run_timer_test(value);
+   return 0;
+}
+DEFINE_SIMPLE_ATTRIBUTE(timeout_ops, timeout_read_op, timeout_write_op, 
"%llu\n");
+
+static int __init latency_init(void)
+{
+   struct dentry *temp;
+
+   dir = debug

[PATCH v2 0/2] Selftest for cpuidle latency measurement

[Pratik Rajesh Sampat]

v1: https://lkml.org/lkml/2020/7/7/1036
Changelog v1 --> v2
1. Based on Shuah Khan's comment, changed exit code to ksft_skip to
   indicate the test is being skipped
2. Change the busy workload for baseline measurement from
   "yes > /dev/null" to "cat /dev/random to /dev/null", based on
   observed CPU utilization for "yes" consuming ~60% CPU while the
   latter consumes 100% of CPUs, giving more accurate baseline numbers
---

The patch series introduces a mechanism to measure wakeup latency for
IPI and timer based interrupts
The motivation behind this series is to find significant deviations
behind advertised latency and resisdency values

To achieve this, we introduce a kernel module and expose its control
knobs through the debugfs interface that the selftests can engage with.

The kernel module provides the following interfaces within
/sys/kernel/debug/latency_test/ for,
1. IPI test:
  ipi_cpu_dest   # Destination CPU for the IPI
  ipi_cpu_src# Origin of the IPI
  ipi_latency_ns # Measured latency time in ns
2. Timeout test:
  timeout_cpu_src # CPU on which the timer to be queued
  timeout_expected_ns # Timer duration
  timeout_diff_ns # Difference of actual duration vs expected timer
To include the module, check option and include as module
kernel hacking -> Cpuidle latency selftests

The selftest inserts the module, disables all the idle states and
enables them one by one testing the following:
1. Keeping source CPU constant, iterates through all the CPUS measuring
   IPI latency for baseline (CPU is busy with
   "cat /dev/random > /dev/null" workload) and the when the CPU is
   allowed to be at rest
2. Iterating through all the CPUs, sending expected timer durations to
   be equivalent to the residency of the the deepest idle state
   enabled and extracting the difference in time between the time of
   wakeup and the expected timer duration

Usage
-
Can be used in conjuction to the rest of the selftests.
Default Output location in: tools/testing/cpuidle/cpuidle.log

To run this test specifically:
$ make -C tools/testing/selftests TARGETS="cpuidle" run_tests

There are a few optinal arguments too that the script can take
[-h ]
[-m ]
[-o ]

Sample output snippet
-
--IPI Latency Test---
--Baseline IPI Latency measurement: CPU Busy--
SRC_CPU   DEST_CPU IPI_Latency(ns)
...
08 1996
09 2125
0   10 1264
0   11 1788
0   12 2045
Baseline Average IPI latency(ns): 1843
---Enabling state: 5---
SRC_CPU   DEST_CPU IPI_Latency(ns)
08   621719
09   624752
0   10   622218
0   11   623968
0   12   621303
Expected IPI latency(ns): 10
Observed Average IPI latency(ns): 622792

--Timeout Latency Test--
--Baseline Timeout Latency measurement: CPU Busy--
Wakeup_src Baseline_delay(ns) 
...
82249
92226
10   2211
11   2183
12   2263
Baseline Average timeout diff(ns): 2226
---Enabling state: 5---
8   10749   
9   10911   
10  10912   
11  12100   
12  73276   
Expected timeout(ns): 10000200
Observed Average timeout diff(ns): 23589

Pratik Rajesh Sampat (2):
  cpuidle: Trace IPI based and timer based wakeup latency from idle
states
  selftest/cpuidle: Add support for cpuidle latency measurement

 drivers/cpuidle/Makefile   |   1 +
 drivers/cpuidle/test-cpuidle_latency.c | 150 
 lib/Kconfig.debug  |  10 +
 tools/testing/selftests/Makefile   |   1 +
 tools/testing/selftests/cpuidle/Makefile   |   6 +
 tools/testing/selftests/cpuidle/cpuidle.sh | 257 +
 tools/testing/selftests/cpuidle/settings   |   1 +
 7 files changed, 426 insertions(+)
 create mode 100644 drivers/cpuidle/test-cpuidle_latency.c
 create mode 100644 tools/testing/selftests/cpuidle/Makefile
 create mode 100755 tools/testing/selftests/cpuidle/cpuidle.sh
 create mode 100644 tools/testing/selftests/cpuidle/settings

-- 
2.25.4

[PATCH v2 1/3] powerpc/powernv/idle: Exclude mfspr on HID1, 4, 5 on P9 and above

[Pratik Rajesh Sampat]

POWER9 onwards the support for the registers HID1, HID4, HID5 has been
receded.
Although mfspr on the above registers worked in Power9, In Power10
simulator is unrecognized. Moving their assignment under the
check for machines lower than Power9

Signed-off-by: Pratik Rajesh Sampat 
Reviewed-by: Gautham R. Shenoy 
---
 arch/powerpc/platforms/powernv/idle.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 2dd467383a88..19d94d021357 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -73,9 +73,6 @@ static int pnv_save_sprs_for_deep_states(void)
 */
uint64_t lpcr_val   = mfspr(SPRN_LPCR);
uint64_t hid0_val   = mfspr(SPRN_HID0);
-   uint64_t hid1_val   = mfspr(SPRN_HID1);
-   uint64_t hid4_val   = mfspr(SPRN_HID4);
-   uint64_t hid5_val   = mfspr(SPRN_HID5);
uint64_t hmeer_val  = mfspr(SPRN_HMEER);
uint64_t msr_val = MSR_IDLE;
uint64_t psscr_val = pnv_deepest_stop_psscr_val;
@@ -117,6 +114,9 @@ static int pnv_save_sprs_for_deep_states(void)
 
/* Only p8 needs to set extra HID regiters */
if (!cpu_has_feature(CPU_FTR_ARCH_300)) {
+   uint64_t hid1_val = mfspr(SPRN_HID1);
+   uint64_t hid4_val = mfspr(SPRN_HID4);
+   uint64_t hid5_val = mfspr(SPRN_HID5);
 
rc = opal_slw_set_reg(pir, SPRN_HID1, hid1_val);
if (rc != 0)
-- 
2.25.4

[PATCH v2 3/3] powerpc/powernv/idle: Rename pnv_first_spr_loss_level variable

[Pratik Rajesh Sampat]

Replace the variable name from using "pnv_first_spr_loss_level" to
"pnv_first_fullstate_loss_level".

As pnv_first_spr_loss_level is supposed to be the earliest state that
has OPAL_PM_LOSE_FULL_CONTEXT set, however as shallow states too loose
SPR values, render an incorrect terminology.

Signed-off-by: Pratik Rajesh Sampat 
---
 arch/powerpc/platforms/powernv/idle.c | 18 +-
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index f2e2a6a4c274..d54e7ef234e3 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -48,7 +48,7 @@ static bool default_stop_found;
  * First stop state levels when SPR and TB loss can occur.
  */
 static u64 pnv_first_tb_loss_level = MAX_STOP_STATE + 1;
-static u64 pnv_first_spr_loss_level = MAX_STOP_STATE + 1;
+static u64 pnv_first_fullstate_loss_level = MAX_STOP_STATE + 1;
 
 /*
  * psscr value and mask of the deepest stop idle state.
@@ -659,7 +659,7 @@ static unsigned long power9_idle_stop(unsigned long psscr, 
bool mmu_on)
  */
mmcr0   = mfspr(SPRN_MMCR0);
}
-   if ((psscr & PSSCR_RL_MASK) >= pnv_first_spr_loss_level) {
+   if ((psscr & PSSCR_RL_MASK) >= pnv_first_fullstate_loss_level) {
sprs.lpcr   = mfspr(SPRN_LPCR);
sprs.hfscr  = mfspr(SPRN_HFSCR);
sprs.fscr   = mfspr(SPRN_FSCR);
@@ -751,7 +751,7 @@ static unsigned long power9_idle_stop(unsigned long psscr, 
bool mmu_on)
 * just always test PSSCR for SPR/TB state loss.
 */
pls = (psscr & PSSCR_PLS) >> PSSCR_PLS_SHIFT;
-   if (likely(pls < pnv_first_spr_loss_level)) {
+   if (likely(pls < pnv_first_fullstate_loss_level)) {
if (sprs_saved)
atomic_stop_thread_idle();
goto out;
@@ -1098,7 +1098,7 @@ static void __init pnv_power9_idle_init(void)
 * the deepest loss-less (OPAL_PM_STOP_INST_FAST) stop state.
 */
pnv_first_tb_loss_level = MAX_STOP_STATE + 1;
-   pnv_first_spr_loss_level = MAX_STOP_STATE + 1;
+   pnv_first_fullstate_loss_level = MAX_STOP_STATE + 1;
for (i = 0; i < nr_pnv_idle_states; i++) {
int err;
struct pnv_idle_states_t *state = _idle_states[i];
@@ -1109,8 +1109,8 @@ static void __init pnv_power9_idle_init(void)
pnv_first_tb_loss_level = psscr_rl;
 
if ((state->flags & OPAL_PM_LOSE_FULL_CONTEXT) &&
-(pnv_first_spr_loss_level > psscr_rl))
-   pnv_first_spr_loss_level = psscr_rl;
+(pnv_first_fullstate_loss_level > psscr_rl))
+   pnv_first_fullstate_loss_level = psscr_rl;
 
/*
 * The idle code does not deal with TB loss occurring
@@ -1121,8 +1121,8 @@ static void __init pnv_power9_idle_init(void)
 * compatibility.
 */
if ((state->flags & OPAL_PM_TIMEBASE_STOP) &&
-(pnv_first_spr_loss_level > psscr_rl))
-   pnv_first_spr_loss_level = psscr_rl;
+(pnv_first_fullstate_loss_level > psscr_rl))
+   pnv_first_fullstate_loss_level = psscr_rl;
 
err = validate_psscr_val_mask(>psscr_val,
  >psscr_mask,
@@ -1168,7 +1168,7 @@ static void __init pnv_power9_idle_init(void)
}
 
pr_info("cpuidle-powernv: First stop level that may lose SPRs = 
0x%llx\n",
-   pnv_first_spr_loss_level);
+   pnv_first_fullstate_loss_level);
 
pr_info("cpuidle-powernv: First stop level that may lose timebase = 
0x%llx\n",
pnv_first_tb_loss_level);
-- 
2.25.4

[PATCH v2 2/3] powerpc/powernv/idle: save-restore DAWR0, DAWRX0 for P10

[Pratik Rajesh Sampat]

Additional registers DAWR0, DAWRX0 may be lost on Power 10 for
stop levels < 4.
Therefore save the values of these SPRs before entering a  "stop"
state and restore their values on wakeup.

Signed-off-by: Pratik Rajesh Sampat 
---
 arch/powerpc/platforms/powernv/idle.c | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 19d94d021357..f2e2a6a4c274 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -600,6 +600,8 @@ struct p9_sprs {
u64 iamr;
u64 amor;
u64 uamor;
+   u64 dawr0;
+   u64 dawrx0;
 };
 
 static unsigned long power9_idle_stop(unsigned long psscr, bool mmu_on)
@@ -687,6 +689,10 @@ static unsigned long power9_idle_stop(unsigned long psscr, 
bool mmu_on)
sprs.iamr   = mfspr(SPRN_IAMR);
sprs.amor   = mfspr(SPRN_AMOR);
sprs.uamor  = mfspr(SPRN_UAMOR);
+   if (cpu_has_feature(CPU_FTR_ARCH_31)) {
+   sprs.dawr0 = mfspr(SPRN_DAWR0);
+   sprs.dawrx0 = mfspr(SPRN_DAWRX0);
+   }
 
srr1 = isa300_idle_stop_mayloss(psscr); /* go idle */
 
@@ -710,6 +716,10 @@ static unsigned long power9_idle_stop(unsigned long psscr, 
bool mmu_on)
mtspr(SPRN_IAMR,sprs.iamr);
mtspr(SPRN_AMOR,sprs.amor);
mtspr(SPRN_UAMOR,   sprs.uamor);
+   if (cpu_has_feature(CPU_FTR_ARCH_31)) {
+   mtspr(SPRN_DAWR0, sprs.dawr0);
+   mtspr(SPRN_DAWRX0, sprs.dawrx0);
+   }
 
/*
 * Workaround for POWER9 DD2.0, if we lost resources, the ERAT
-- 
2.25.4

[PATCH v2 0/3] Power10 basic energy management

[Pratik Rajesh Sampat]

Changelog v1 --> v2:
1. Save-restore DAWR and DAWRX unconditionally as they are lost in
shallow idle states too
2. Rename pnv_first_spr_loss_level to pnv_first_fullstate_loss_level to
correct naming terminology

Pratik Rajesh Sampat (3):
  powerpc/powernv/idle: Exclude mfspr on HID1,4,5 on P9 and above
  powerpc/powernv/idle: save-restore DAWR0,DAWRX0 for P10
  powerpc/powernv/idle: Rename pnv_first_spr_loss_level variable

 arch/powerpc/platforms/powernv/idle.c | 34 +--
 1 file changed, 22 insertions(+), 12 deletions(-)

-- 
2.25.4

[PATCH 2/2] selftest/cpuidle: Add support for cpuidle latency measurement

[Pratik Rajesh Sampat]

This patch adds support to trace IPI based and timer based wakeup
latency from idle states

Latches onto the test-cpuidle_latency kernel module using the debugfs
interface to send IPIs or schedule a timer based event, which in-turn
populates the debugfs with the latency measurements.

Currently for the IPI and timer tests; first disable all idle states
and then test for latency measurements incrementally enabling each state

Signed-off-by: Pratik Rajesh Sampat 
---
 tools/testing/selftests/Makefile   |   1 +
 tools/testing/selftests/cpuidle/Makefile   |   6 +
 tools/testing/selftests/cpuidle/cpuidle.sh | 240 +
 tools/testing/selftests/cpuidle/settings   |   1 +
 4 files changed, 248 insertions(+)
 create mode 100644 tools/testing/selftests/cpuidle/Makefile
 create mode 100755 tools/testing/selftests/cpuidle/cpuidle.sh
 create mode 100644 tools/testing/selftests/cpuidle/settings

diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile
index 1195bd85af38..ab6cf51f3518 100644
--- a/tools/testing/selftests/Makefile
+++ b/tools/testing/selftests/Makefile
@@ -7,6 +7,7 @@ TARGETS += capabilities
 TARGETS += cgroup
 TARGETS += clone3
 TARGETS += cpufreq
+TARGETS += cpuidle
 TARGETS += cpu-hotplug
 TARGETS += drivers/dma-buf
 TARGETS += efivarfs
diff --git a/tools/testing/selftests/cpuidle/Makefile 
b/tools/testing/selftests/cpuidle/Makefile
new file mode 100644
index ..72fd5d2e974d
--- /dev/null
+++ b/tools/testing/selftests/cpuidle/Makefile
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+all:
+
+TEST_PROGS := cpuidle.sh
+
+include ../lib.mk
diff --git a/tools/testing/selftests/cpuidle/cpuidle.sh 
b/tools/testing/selftests/cpuidle/cpuidle.sh
new file mode 100755
index ..11666fe47c34
--- /dev/null
+++ b/tools/testing/selftests/cpuidle/cpuidle.sh
@@ -0,0 +1,240 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+LOG=cpuidle.log
+MODULE=/lib/modules/$(uname -r)/kernel/drivers/cpuidle/test-cpuidle_latency.ko
+
+helpme()
+{
+   printf "Usage: $0 [-h] [-todg args]
+   [-h ]
+   [-m ]
+   [-o ]
+   \n"
+   exit 2
+}
+
+parse_arguments()
+{
+   while getopts ht:m:o: arg
+   do
+   case $arg in
+   h) # --help
+   helpme
+   ;;
+   m) # --mod-file
+   MODULE=$OPTARG
+   ;;
+   o) # output log files
+   LOG=$OPTARG
+   ;;
+   \?)
+   helpme
+   ;;
+   esac
+   done
+}
+
+ins_mod()
+{
+   if [ ! -f "$MODULE" ]; then
+   printf "$MODULE module does not exist. Exitting\n"
+   exit 2
+   fi
+   printf "Inserting $MODULE module\n\n"
+   insmod $MODULE
+   if [ $? != 0 ]; then
+   printf "Insmod $MODULE failed\n"
+   exit 2
+   fi
+}
+
+compute_average()
+{
+   arr=("$@")
+   sum=0
+   size=${#arr[@]}
+   for i in "${arr[@]}"
+   do
+   sum=$((sum + i))
+   done
+   avg=$((sum/size))
+}
+
+# Disable all stop states
+disable_idle()
+{
+   for ((cpu=0; cpu 
/sys/devices/system/cpu/cpu$cpu/cpuidle/state$state/disable
+   done
+   done
+}
+
+# Enable the idle state supplied
+# $1: State to enable
+enable_state()
+{
+   for ((cpu=0; cpu /sys/devices/system/cpu/cpu$cpu/cpuidle/state$1/disable
+   done
+}
+
+# Extract latency in microseconds and convert to nanoseconds
+extract_latency()
+{
+   for ((state=0; state /dev/null &
+   yes_pid=$!
+fi
+taskset 0x1 echo $dest_cpu > 
/sys/kernel/debug/latency_test/ipi_cpu_dest
+ipi_latency=$(cat /sys/kernel/debug/latency_test/ipi_latency_ns)
+src_cpu=$(cat /sys/kernel/debug/latency_test/ipi_cpu_src)
+if [ "$1" = "baseline" ]; then
+   kill $yes_pid
+   wait $yes_pid 2>/dev/null
+fi
+}
+
+# Incrementally Enable idle states one by one and compute the latency
+run_ipi_tests()
+{
+extract_latency
+disable_idle
+declare -a avg_arr
+declare -a baseline_avg_array
+
+echo -e "--IPI Latency Test---" >> $LOG
+for ((state=0; state> $LOG
+   enable_state $state
+   printf "%s %10s %12s %12s\n" "SRC_CPU" "DEST_CPU" 
"Base_IPI_Latency(ns)" "IPI_Latency(ns)" >> $LOG
+   unset avg_arr
+   unset baseline_avg_arr
+   for ((cpu=0; cpu> $LOG
+   avg_arr+=($ipi_latency)
+ 

[PATCH 1/2] cpuidle: Trace IPI based and timer based wakeup latency from idle states

[Pratik Rajesh Sampat]

Fire directed smp_call_function_single IPIs from a specified source
CPU to the specified target CPU to reduce the noise we have to wade
through in the trace log.
The module is based on the idea written by Srivatsa Bhat and maintained
by Vaidyanathan Srinivasan internally.

Queue HR timer and measure jitter. Wakeup latency measurement for idle
states using hrtimer.  Echo a value in ns to timer_test_function and
watch trace. A HRtimer will be queued and when it fires the expected
wakeup vs actual wakeup is computes and delay printed in ns.

Implemented as a module which utilizes debugfs so that it can be
integrated with selftests.

To include the module, check option and include as module
kernel hacking -> Cpuidle latency selftests

[srivatsa.b...@linux.vnet.ibm.com: Initial implementation in
 cpidle/sysfs]

[sva...@linux.vnet.ibm.com: wakeup latency measurements using hrtimer
 and fix some of the time calculation]

[e...@linux.vnet.ibm.com: Fix some whitespace and tab errors and
 increase the resolution of IPI wakeup]

Signed-off-by: Pratik Rajesh Sampat 
---
 drivers/cpuidle/Makefile   |   1 +
 drivers/cpuidle/test-cpuidle_latency.c | 150 +
 lib/Kconfig.debug  |  10 ++
 3 files changed, 161 insertions(+)
 create mode 100644 drivers/cpuidle/test-cpuidle_latency.c

diff --git a/drivers/cpuidle/Makefile b/drivers/cpuidle/Makefile
index f07800cbb43f..2ae05968078c 100644
--- a/drivers/cpuidle/Makefile
+++ b/drivers/cpuidle/Makefile
@@ -8,6 +8,7 @@ obj-$(CONFIG_ARCH_NEEDS_CPU_IDLE_COUPLED) += coupled.o
 obj-$(CONFIG_DT_IDLE_STATES) += dt_idle_states.o
 obj-$(CONFIG_ARCH_HAS_CPU_RELAX) += poll_state.o
 obj-$(CONFIG_HALTPOLL_CPUIDLE)   += cpuidle-haltpoll.o
+obj-$(CONFIG_IDLE_LATENCY_SELFTEST)  += test-cpuidle_latency.o
 
 
##
 # ARM SoC drivers
diff --git a/drivers/cpuidle/test-cpuidle_latency.c 
b/drivers/cpuidle/test-cpuidle_latency.c
new file mode 100644
index ..61574665e972
--- /dev/null
+++ b/drivers/cpuidle/test-cpuidle_latency.c
@@ -0,0 +1,150 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Module-based API test facility for cpuidle latency using IPIs and timers
+ */
+
+#include 
+#include 
+#include 
+
+/* IPI based wakeup latencies */
+struct latency {
+   unsigned int src_cpu;
+   unsigned int dest_cpu;
+   ktime_t time_start;
+   ktime_t time_end;
+   u64 latency_ns;
+} ipi_wakeup;
+
+static void measure_latency(void *info)
+{
+   struct latency *v;
+   ktime_t time_diff;
+
+   v = (struct latency *)info;
+   v->time_end = ktime_get();
+   time_diff = ktime_sub(v->time_end, v->time_start);
+   v->latency_ns = ktime_to_ns(time_diff);
+}
+
+void run_smp_call_function_test(unsigned int cpu)
+{
+   ipi_wakeup.src_cpu = smp_processor_id();
+   ipi_wakeup.dest_cpu = cpu;
+   ipi_wakeup.time_start = ktime_get();
+   smp_call_function_single(cpu, measure_latency, _wakeup, 1);
+}
+
+/* Timer based wakeup latencies */
+struct timer_data {
+   unsigned int src_cpu;
+   u64 timeout;
+   ktime_t time_start;
+   ktime_t time_end;
+   struct hrtimer timer;
+   u64 timeout_diff_ns;
+} timer_wakeup;
+
+static enum hrtimer_restart timer_called(struct hrtimer *hrtimer)
+{
+   struct timer_data *w;
+   ktime_t time_diff;
+
+   w = container_of(hrtimer, struct timer_data, timer);
+   w->time_end = ktime_get();
+
+   time_diff = ktime_sub(w->time_end, w->time_start);
+   time_diff = ktime_sub(time_diff, ns_to_ktime(w->timeout));
+   w->timeout_diff_ns = ktime_to_ns(time_diff);
+   return HRTIMER_NORESTART;
+}
+
+static void run_timer_test(unsigned int ns)
+{
+   hrtimer_init(_wakeup.timer, CLOCK_MONOTONIC,
+HRTIMER_MODE_REL);
+   timer_wakeup.timer.function = timer_called;
+   timer_wakeup.time_start = ktime_get();
+   timer_wakeup.src_cpu = smp_processor_id();
+   timer_wakeup.timeout = ns;
+
+   hrtimer_start(_wakeup.timer, ns_to_ktime(ns),
+ HRTIMER_MODE_REL_PINNED);
+}
+
+static struct dentry *dir;
+
+static int cpu_read_op(void *data, u64 *value)
+{
+   *value = ipi_wakeup.dest_cpu;
+   return 0;
+}
+
+static int cpu_write_op(void *data, u64 value)
+{
+   run_smp_call_function_test(value);
+   return 0;
+}
+DEFINE_SIMPLE_ATTRIBUTE(ipi_ops, cpu_read_op, cpu_write_op, "%llu\n");
+
+static int timeout_read_op(void *data, u64 *value)
+{
+   *value = timer_wakeup.timeout;
+   return 0;
+}
+
+static int timeout_write_op(void *data, u64 value)
+{
+   run_timer_test(value);
+   return 0;
+}
+DEFINE_SIMPLE_ATTRIBUTE(timeout_ops, timeout_read_op, timeout_write_op, 
"%llu\n");
+
+static int __init latency_init(void)
+{
+   struct dentry *temp;
+
+   dir = debug

[PATCH 0/2] Selftest for cpuidle latency measurement

[Pratik Rajesh Sampat]

The patch series introduces a mechanism to measure wakeup latency for
IPI and timer based interrupts
The motivation behind this series is to find significant deviations
behind advertised latency and resisdency values

To achieve this, we introduce a kernel module and expose its control
knobs through the debugfs interface that the selftests can engage with.

The kernel module provides the following interfaces within
/sys/kernel/debug/latency_test/ for,
1. IPI test:
  ipi_cpu_dest   # Destination CPU for the IPI
  ipi_cpu_src# Origin of the IPI
  ipi_latency_ns # Measured latency time in ns
2. Timeout test:
  timeout_cpu_src # CPU on which the timer to be queued
  timeout_expected_ns # Timer duration
  timeout_diff_ns # Difference of actual duration vs expected timer
To include the module, check option and include as module
kernel hacking -> Cpuidle latency selftests

The selftest inserts the module, disables all the idle states and
enables them one by one testing:
1. Keeping source CPU constant, iterates through all the CPUS measuring
   IPI latency for baseline (CPU is busy with "yes" workload) and the
   when the CPU is at rest
2. Iterating through all the CPUs, sending expected timer durations to
   be equivalent to the residency of the the deepest idle state
   enabled and extracting the difference in time between the time of
   wakeup and the expected timer duration

Usage
-
Can be used in conjuction to the rest of the selftests.
Default Output location in: tools/testing/cpuidle/cpuidle.log

To run this test specifically:
$ make -C tools/testing/selftests TARGETS="cpuidle" run_tests

There are a few optinal arguments too that the script can take
[-h ]
[-m ]
[-o ]

Sample output snippet
-
--IPI Latency Test---
---Enabling state: 0---
SRC_CPU   DEST_CPU Base_IPI_Latency(ns) IPI_Latency(ns)
00  328291
01 1500   1071
02 1070   1062
03 1557   1668
. . . .
Expected IPI latency(ns): 1000
Baseline Average IPI latency(ns): 1113
Observed Average IPI latency(ns): 1023
--Timeout Latency Test--
---Enabling state: 0---
Wakeup_src Baseline_delay(ns)  Delay(ns)
03134   2128
12275   2107
2   2198
32421   2325
. . . .
Expected timeout(ns): 200
Baseline Average timeout diff(ns): 2513
Observed Average timeout diff(ns): 2189

Pratik Rajesh Sampat (2):
  cpuidle: Trace IPI based and timer based wakeup latency from idle
states
  selftest/cpuidle: Add support for cpuidle latency measurement

 drivers/cpuidle/Makefile   |   1 +
 drivers/cpuidle/test-cpuidle_latency.c | 150 +
 lib/Kconfig.debug  |  10 +
 tools/testing/selftests/Makefile   |   1 +
 tools/testing/selftests/cpuidle/Makefile   |   6 +
 tools/testing/selftests/cpuidle/cpuidle.sh | 240 +
 tools/testing/selftests/cpuidle/settings   |   1 +
 7 files changed, 409 insertions(+)
 create mode 100644 drivers/cpuidle/test-cpuidle_latency.c
 create mode 100644 tools/testing/selftests/cpuidle/Makefile
 create mode 100755 tools/testing/selftests/cpuidle/cpuidle.sh
 create mode 100644 tools/testing/selftests/cpuidle/settings

-- 
2.25.4

[PATCH 2/2] powerpc/powernv/idle: save-restore DAWR0,DAWRX0 for P10

[Pratik Rajesh Sampat]

Additional registers DAWR0, DAWRX0 may be lost on Power 10 for
stop levels < 4.
Therefore save the values of these SPRs before entering a  "stop"
state and restore their values on wakeup.

Signed-off-by: Pratik Rajesh Sampat 
---
 arch/powerpc/platforms/powernv/idle.c | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 19d94d021357..471d4a65b1fa 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -600,6 +600,8 @@ struct p9_sprs {
u64 iamr;
u64 amor;
u64 uamor;
+   u64 dawr0;
+   u64 dawrx0;
 };
 
 static unsigned long power9_idle_stop(unsigned long psscr, bool mmu_on)
@@ -677,6 +679,10 @@ static unsigned long power9_idle_stop(unsigned long psscr, 
bool mmu_on)
sprs.tscr   = mfspr(SPRN_TSCR);
if (!firmware_has_feature(FW_FEATURE_ULTRAVISOR))
sprs.ldbar = mfspr(SPRN_LDBAR);
+   if (cpu_has_feature(CPU_FTR_ARCH_31)) {
+   sprs.dawr0 = mfspr(SPRN_DAWR0);
+   sprs.dawrx0 = mfspr(SPRN_DAWRX0);
+   }
 
sprs_saved = true;
 
@@ -792,6 +798,10 @@ static unsigned long power9_idle_stop(unsigned long psscr, 
bool mmu_on)
mtspr(SPRN_MMCR2,   sprs.mmcr2);
if (!firmware_has_feature(FW_FEATURE_ULTRAVISOR))
mtspr(SPRN_LDBAR, sprs.ldbar);
+   if (cpu_has_feature(CPU_FTR_ARCH_31)) {
+   mtspr(SPRN_DAWR0, sprs.dawr0);
+   mtspr(SPRN_DAWRX0, sprs.dawrx0);
+   }
 
mtspr(SPRN_SPRG3,   local_paca->sprg_vdso);
 
-- 
2.25.4

[PATCH 1/2] powerpc/powernv/idle: Exclude mfspr on HID1, 4, 5 on P9 and above

[Pratik Rajesh Sampat]

POWER9 onwards the support for the registers HID1, HID4, HID5 has been
receded.
Although mfspr on the above registers worked in Power9, In Power10
simulator is unrecognized. Moving their assignment under the
check for machines lower than Power9

Signed-off-by: Pratik Rajesh Sampat 
---
 arch/powerpc/platforms/powernv/idle.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 2dd467383a88..19d94d021357 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -73,9 +73,6 @@ static int pnv_save_sprs_for_deep_states(void)
 */
uint64_t lpcr_val   = mfspr(SPRN_LPCR);
uint64_t hid0_val   = mfspr(SPRN_HID0);
-   uint64_t hid1_val   = mfspr(SPRN_HID1);
-   uint64_t hid4_val   = mfspr(SPRN_HID4);
-   uint64_t hid5_val   = mfspr(SPRN_HID5);
uint64_t hmeer_val  = mfspr(SPRN_HMEER);
uint64_t msr_val = MSR_IDLE;
uint64_t psscr_val = pnv_deepest_stop_psscr_val;
@@ -117,6 +114,9 @@ static int pnv_save_sprs_for_deep_states(void)
 
/* Only p8 needs to set extra HID regiters */
if (!cpu_has_feature(CPU_FTR_ARCH_300)) {
+   uint64_t hid1_val = mfspr(SPRN_HID1);
+   uint64_t hid4_val = mfspr(SPRN_HID4);
+   uint64_t hid5_val = mfspr(SPRN_HID5);
 
rc = opal_slw_set_reg(pir, SPRN_HID1, hid1_val);
if (rc != 0)
-- 
2.25.4

[PATCH v8 3/3] Self save API integration

[Pratik Rajesh Sampat]

The commit makes the self save API available outside the firmware by defining
an OPAL wrapper.
This wrapper has a similar interface to that of self restore and expects the
cpu pir, SPR number, minus the value of that SPR to be passed in its
paramters and returns OPAL_SUCCESS on success. It adds a device-tree
node signifying support for self-save after verifying the stop API
version compatibility.

The commit also documents both the self-save and the self-restore API
calls along with their working and usage.

Signed-off-by: Pratik Rajesh Sampat 
---
 doc/opal-api/opal-slw-self-save-reg-181.rst |  51 ++
 doc/opal-api/opal-slw-set-reg-100.rst   |   5 +
 doc/power-management.rst|  48 +
 hw/slw.c| 106 
 include/opal-api.h  |   3 +-
 include/p9_stop_api.H   |  18 
 include/skiboot.h   |   3 +
 7 files changed, 233 insertions(+), 1 deletion(-)
 create mode 100644 doc/opal-api/opal-slw-self-save-reg-181.rst

diff --git a/doc/opal-api/opal-slw-self-save-reg-181.rst 
b/doc/opal-api/opal-slw-self-save-reg-181.rst
new file mode 100644
index ..f20e9b81
--- /dev/null
+++ b/doc/opal-api/opal-slw-self-save-reg-181.rst
@@ -0,0 +1,51 @@
+.. OPAL_SLW_SELF_SAVE_REG:
+
+OPAL_SLW_SELF_SAVE_REG
+==
+
+.. code-block:: c
+
+   #define OPAL_SLW_SELF_SAVE_REG  181
+
+   int64_t opal_slw_self_save_reg(uint64_t cpu_pir, uint64_t sprn);
+
+:ref:`OPAL_SLW_SELF_SAVE_REG` is used to inform low-level firmware to save
+the current contents of the SPR before entering a state of loss and
+also restore the content back on waking up from a deep stop state.
+
+An OPAL call `OPAL_SLW_SET_REG` exists which is similar in function as
+saving and restoring the SPR, with one difference being that the value of the
+SPR must also be supplied in the parameters.
+Complete reference: doc/opal-api/opal-slw-set-reg-100.rst
+
+Parameters
+--
+
+``uint64_t cpu_pir``
+  This parameter specifies the pir of the cpu for which the call is being made.
+``uint64_t sprn``
+  This parameter specifies the spr number as mentioned in p9_stop_api.H
+  The list of SPRs supported is as follows.
+   P9_STOP_SPR_DAWR,
+   P9_STOP_SPR_HSPRG0,
+   P9_STOP_SPR_LDBAR,
+   P9_STOP_SPR_LPCR,
+   P9_STOP_SPR_PSSCR,
+   P9_STOP_SPR_MSR,
+   P9_STOP_SPR_HRMOR,
+   P9_STOP_SPR_HMEER,
+   P9_STOP_SPR_PMCR,
+   P9_STOP_SPR_PTCR
+
+  The property "ibm,opal-self-save" is supplied to the device tree to advterise
+  support.
+
+Returns
+---
+
+:ref:`OPAL_UNSUPPORTED`
+  If spr restore is not supported by pore engine.
+:ref:`OPAL_PARAMETER`
+  Invalid handle for the pir/chip
+:ref:`OPAL_SUCCESS`
+  On success
diff --git a/doc/opal-api/opal-slw-set-reg-100.rst 
b/doc/opal-api/opal-slw-set-reg-100.rst
index 2e8f1bd6..ee3e68ce 100644
--- a/doc/opal-api/opal-slw-set-reg-100.rst
+++ b/doc/opal-api/opal-slw-set-reg-100.rst
@@ -21,6 +21,11 @@ In Power 9, it uses p9_stop_save_cpureg(), api provided by 
self restore code,
 to inform the spr with their corresponding values with which they
 must be restored.
 
+An OPAL call `OPAL_SLW_SELF_SAVE_REG` exists which is similar in function
+saving and restoring the SPR, with one difference being that the value of the
+SPR doesn't need to be passed in the parameters, only with the SPR number
+the firmware can identify, save and restore the values for the same.
+Complete reference: doc/opal-api/opal-slw-self-save-reg-181.rst
 
 Parameters
 --
diff --git a/doc/power-management.rst b/doc/power-management.rst
index 76491a71..d6bd5358 100644
--- a/doc/power-management.rst
+++ b/doc/power-management.rst
@@ -15,3 +15,51 @@ On boot, specific stop states can be disabled via setting a 
mask. For example,
 to disable all but stop 0,1,2, use ~0xE000. ::
 
   nvram -p ibm,skiboot --update-config opal-stop-state-disable-mask=0x1FFF
+
+Saving and restoring Special Purpose Registers(SPRs)
+
+
+When a CPU wakes up from a deep stop state which can result in
+hypervisor state loss, all the SPRs are lost. The Linux Kernel expects
+a small set of SPRs to contain an expected value when the CPU wakes up
+from such a deep stop state. The microcode firmware provides the
+following two APIs, collectively known as the stop-APIs, to allow the
+kernel/OPAL to specify this set of SPRs and the value that they need
+to be restored with on waking up from a deep stop state.
+
+Self-restore:
+int64_t opal_slw_set_reg(uint64_t cpu_pir, uint64_t sprn, uint64_t val);
+The SPR number and the value of the that SPR must be restored with on
+wakeup from the deep-stop state must be specified. When this call is
+made, the microcode inserts instruction into the HOMER region to
+restore the content of the SPR to the specified value on wakeup from a
+deep-

[PATCH v8 1/1] powerpc/powernv: Introduce support and parsing for self-save API

[Pratik Rajesh Sampat]

This commit introduces and leverages the Self save API. The difference
between self-save and self-restore is that the value to be saved for the
SPR does not need to be passed to the call.

Add the new Self Save OPAL API call in the list of OPAL calls.

The device tree is parsed looking for the property "ibm,opal-self-save"
If self-save is supported then for all SPRs self-save is invoked for all
P9 supported registers. In the case self-save fails corresponding
self-restore call is invoked as a fallback.

Signed-off-by: Pratik Rajesh Sampat 
---
 arch/powerpc/include/asm/opal-api.h|  3 +-
 arch/powerpc/include/asm/opal.h|  1 +
 arch/powerpc/platforms/powernv/idle.c  | 73 ++
 arch/powerpc/platforms/powernv/opal-call.c |  1 +
 4 files changed, 64 insertions(+), 14 deletions(-)

diff --git a/arch/powerpc/include/asm/opal-api.h 
b/arch/powerpc/include/asm/opal-api.h
index 1dffa3cb16ba..7ba698369083 100644
--- a/arch/powerpc/include/asm/opal-api.h
+++ b/arch/powerpc/include/asm/opal-api.h
@@ -214,7 +214,8 @@
 #define OPAL_SECVAR_GET176
 #define OPAL_SECVAR_GET_NEXT   177
 #define OPAL_SECVAR_ENQUEUE_UPDATE 178
-#define OPAL_LAST  178
+#define OPAL_SLW_SELF_SAVE_REG 181
+#define OPAL_LAST  181
 
 #define QUIESCE_HOLD   1 /* Spin all calls at entry */
 #define QUIESCE_REJECT 2 /* Fail all calls with OPAL_BUSY */
diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h
index 9986ac34b8e2..a370b0e8d899 100644
--- a/arch/powerpc/include/asm/opal.h
+++ b/arch/powerpc/include/asm/opal.h
@@ -204,6 +204,7 @@ int64_t opal_handle_hmi2(__be64 *out_flags);
 int64_t opal_register_dump_region(uint32_t id, uint64_t start, uint64_t end);
 int64_t opal_unregister_dump_region(uint32_t id);
 int64_t opal_slw_set_reg(uint64_t cpu_pir, uint64_t sprn, uint64_t val);
+int64_t opal_slw_self_save_reg(uint64_t cpu_pir, uint64_t sprn);
 int64_t opal_config_cpu_idle_state(uint64_t state, uint64_t flag);
 int64_t opal_pci_set_phb_cxl_mode(uint64_t phb_id, uint64_t mode, uint64_t 
pe_number);
 int64_t opal_pci_get_pbcq_tunnel_bar(uint64_t phb_id, uint64_t *addr);
diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 78599bca66c2..ada7ece24521 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -32,6 +32,11 @@
 #define P9_STOP_SPR_MSR 2000
 #define P9_STOP_SPR_PSSCR  855
 
+/* Caching the self-save functionality, lpcr, ptcr support */
+DEFINE_STATIC_KEY_FALSE(self_save_available);
+DEFINE_STATIC_KEY_FALSE(is_lpcr_self_save);
+DEFINE_STATIC_KEY_FALSE(is_ptcr_self_save);
+
 static u32 supported_cpuidle_states;
 struct pnv_idle_states_t *pnv_idle_states;
 int nr_pnv_idle_states;
@@ -61,6 +66,35 @@ static bool deepest_stop_found;
 
 static unsigned long power7_offline_type;
 
+/*
+ * Cache support for SPRs that support self-save as well as kernel save restore
+ * so that kernel does not duplicate efforts in saving and restoring SPRs
+ */
+static void cache_spr_self_save_support(u64 sprn)
+{
+   switch (sprn) {
+   case SPRN_LPCR:
+   static_branch_enable(_lpcr_self_save);
+   break;
+   case SPRN_PTCR:
+   static_branch_enable(_ptcr_self_save);
+   break;
+   }
+}
+
+static int pnv_save_one_spr(u64 pir, u64 sprn, u64 val)
+{
+   if (static_branch_likely(_save_available)) {
+   int rc = opal_slw_self_save_reg(pir, sprn);
+
+   if (!rc) {
+   cache_spr_self_save_support(sprn);
+   return rc;
+   }
+   }
+   return opal_slw_set_reg(pir, sprn, val);
+}
+
 static int pnv_save_sprs_for_deep_states(void)
 {
int cpu;
@@ -72,6 +106,7 @@ static int pnv_save_sprs_for_deep_states(void)
 * same across all cpus.
 */
uint64_t lpcr_val   = mfspr(SPRN_LPCR);
+   uint64_t ptcr_val   = mfspr(SPRN_PTCR);
uint64_t hid0_val   = mfspr(SPRN_HID0);
uint64_t hid1_val   = mfspr(SPRN_HID1);
uint64_t hid4_val   = mfspr(SPRN_HID4);
@@ -84,30 +119,34 @@ static int pnv_save_sprs_for_deep_states(void)
uint64_t pir = get_hard_smp_processor_id(cpu);
uint64_t hsprg0_val = (uint64_t)paca_ptrs[cpu];
 
-   rc = opal_slw_set_reg(pir, SPRN_HSPRG0, hsprg0_val);
+   rc = pnv_save_one_spr(pir, SPRN_HSPRG0, hsprg0_val);
if (rc != 0)
return rc;
 
-   rc = opal_slw_set_reg(pir, SPRN_LPCR, lpcr_val);
+   rc = pnv_save_one_spr(pir, SPRN_LPCR, lpcr_val);
if (rc != 0)
return rc;
 
+   /*
+* No need to check for failure, if firmware fai

[PATCH v8 0/1] powerpc/powernv: Introduce support and parsing for self-save API

[Pratik Rajesh Sampat]

re are complementary features since,
self-restore can help in restoring a different value in the SPR on
wakeup from a deep-idle state than what it had before entering the
deep idle state. This was used in POWER8 for HSPRG0 to distinguish a
wakeup from Winkle vs Fastsleep.

Limitations of self-save

Ideally all SPRs should be available for self-save, but HID0 is very
tricky to implement in microcode due to various endianess quirks.
Couple of implementation schemes were buggy and hence HID0 was left
out to be self-restore only.

The fallout of this limitation is as follows:

* In Non PEF environment, no issue. Linux will use self-restore for
  HID0 as it does today and no functional impact.

* In PEF environment, the HID0 restore value is decided by OPAL during
  boot and it is setup for LE hypervisor with radix MMU. This is the
  default and current working configuration of a PEF environment.
  However if there is a change, then HV Linux will try to change the
  HID0 value to something different than what OPAL decided, at which
  time deep-stop states will be disabled under this new PEF
  environment.

A simple and workable design is achieved by scoping the power
management deep-stop state support only to a known default PEF
environment. Any deviation will affect *only* deep stop-state support
(stop4,5) in that environment and not have any functional impediment
to the environment itself.

In future, if there is a need to support changing of HID0 to various
values under PEF environment and support deep-stop states, it can be
worked out via an ultravisor call or improve the microcode design to
include HID0 in self-save.  These future scheme would be an extension
and does not break or make the current implementation scheme
redundant.

Design Choices
==

Presenting the design choices in front of us:

Design-Choice 1:

A simple implementation is to just replace self-restore calls with
self-save as it is direct super-set.

Pros:
A simple design, quick to implement


Cons:
* Breaks backward compatibility. Self-restore has historically been
  supported in the firmware and an old firmware running on an new
  kernel will be incompatible and deep stop states will be cut.
* Furthermore, critical SPRs which need to be restored
  before 0x100 vector like HID0 are not supported by self-save.

Design-Choice 2:

Advertise both self-restore and self-save from OPAL including the set
of registers that each support. The kernel can then choose which API
to go with.
For the sake of simplicity, in case both modes are supported for an
SPR by default self-save would be called for it.

Pros:
* Backwards compatible

Cons:
Overhead in parsing device tree with the SPR list

Design Choice 3:

The presence of self-save feature is indicated by the
"ibm,opal-self-save" in the device-tree.

Since self-save API supports most of the SPRs supported by
self-restore, when self-save feature is available, we first attempt
a self-save call for an SPR. If that fails, then we fallback to the
self-restore API.

Pros:
* Backwards compatible
* Minimal complexity

Cons:
* The kernel does not know which registers are supported by the
  self-save API, so a try-catch model is used. Although this model is
  also employed by the current self-restore API.

The patch chooses design choice 3 as an implementation.
The device tree is parsed looking for the property "ibm,opal-self-save"
If self-save is supported then for all SPRs self-save is invoked for all
P9 supported registers. In the case self-save fails corresponding
self-restore call is invoked as a fallback.

Pratik Rajesh Sampat (1):
  powerpc/powernv: Introduce support and parsing for self-save API

 arch/powerpc/include/asm/opal-api.h|  3 +-
 arch/powerpc/include/asm/opal.h|  1 +
 arch/powerpc/platforms/powernv/idle.c  | 73 ++
 arch/powerpc/platforms/powernv/opal-call.c |  1 +
 4 files changed, 64 insertions(+), 14 deletions(-)

-- 
2.17.1

[PATCH v8 1/3] Self Save: Introducing Support for SPR Self Save

[Pratik Rajesh Sampat]

From: Prem Shanker Jha 

The commit is a merger of commits that makes the following changes:
1. Commit fixes some issues with code found during integration test
  -  replacement of addi with xor instruction during self save API.
  -  fixing instruction generation for MFMSR during self save
  -  data struct updates in STOP API
  -  error RC updates for hcode image build
  -  HOMER parser updates.
  -  removed self save support for URMOR and HRMOR
  -  code changes for compilation with OPAL
  -  populating CME Image header with unsecure HOMER address.

Key_Cronus_Test=PM_REGRESS

Change-Id: I7cedcc466267c4245255d8d75c01ed695e316720
Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/66580
Tested-by: FSP CI Jenkins 
Tested-by: HWSV CI 
Tested-by: PPE CI 
Tested-by: Jenkins Server 
Tested-by: Cronus HW CI 
Tested-by: Hostboot CI 
Reviewed-by: Gregory S. Still 
Reviewed-by: RAHUL BATRA 
Reviewed-by: Jennifer A. Stofer 
Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/66587
Reviewed-by: Christian R. Geddes 
Signed-off-by: Prem Shanker Jha 
Signed-off-by: Akshay Adiga 
Signed-off-by: Pratik Rajesh Sampat 

2. The commit also incorporates changes that make STOP API project
agnostic changes include defining wrapper functions which call legacy
API. It also adds duplicate enum members which start with prefix PROC
instead of P9.

Key_Cronus_Test=PM_REGRESS

Change-Id: If87970f3e8cf9b507f33eb1be249e03eb3836a5e
RTC: 201128
Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/71307
Tested-by: FSP CI Jenkins 
Tested-by: Jenkins Server 
Tested-by: Hostboot CI 
Tested-by: Cronus HW CI 
Reviewed-by: RANGANATHPRASAD G. BRAHMASAMUDRA 
Reviewed-by: Gregory S. Still 
Reviewed-by: Jennifer A Stofer 
Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/71314
Tested-by: Jenkins OP Build CI 
Tested-by: Jenkins OP HW 
Reviewed-by: Daniel M. Crowell 
Signed-off-by: Prem Shanker Jha 
Signed-off-by: Pratik Rajesh Sampat 
---
 include/p9_stop_api.H|  79 +-
 libpore/p9_cpu_reg_restore_instruction.H |   4 +
 libpore/p9_stop_api.C| 954 +--
 libpore/p9_stop_api.H| 115 ++-
 libpore/p9_stop_data_struct.H|   4 +-
 libpore/p9_stop_util.H   |   7 +-
 6 files changed, 721 insertions(+), 442 deletions(-)

diff --git a/include/p9_stop_api.H b/include/p9_stop_api.H
index 79abd000..9d3bc1e5 100644
--- a/include/p9_stop_api.H
+++ b/include/p9_stop_api.H
@@ -63,6 +63,26 @@ typedef enum
 P9_STOP_SPR_PMCR=884,   // core register
 P9_STOP_SPR_HID =   1008,   // core register
 P9_STOP_SPR_MSR =   2000,   // thread register
+
+//enum members which are project agnostic
+PROC_STOP_SPR_DAWR=180,   // thread register
+PROC_STOP_SPR_CIABR   =187,   // thread register
+PROC_STOP_SPR_DAWRX   =188,   // thread register
+PROC_STOP_SPR_HSPRG0  =304,   // thread register
+PROC_STOP_SPR_HRMOR   =313,   // core register
+PROC_STOP_SPR_LPCR=318,   // thread register
+PROC_STOP_SPR_HMEER   =337,   // core register
+PROC_STOP_SPR_PTCR=464,   // core register
+PROC_STOP_SPR_USPRG0  =496,   // thread register
+PROC_STOP_SPR_USPRG1  =497,   // thread register
+PROC_STOP_SPR_URMOR   =505,   // core register
+PROC_STOP_SPR_SMFCTRL =511,   // thread register
+PROC_STOP_SPR_LDBAR   =850,   // thread register
+PROC_STOP_SPR_PSSCR   =855,   // thread register
+PROC_STOP_SPR_PMCR=884,   // core register
+PROC_STOP_SPR_HID =   1008,   // core register
+PROC_STOP_SPR_MSR =   2000,   // thread register
+
 } CpuReg_t;
 
 /**
@@ -85,6 +105,8 @@ typedef enum
 STOP_SAVE_SCOM_ENTRY_UPDATE_FAILED   = 12,
 STOP_SAVE_INVALID_FUSED_CORE_STATUS  = 13,
 STOP_SAVE_FAIL   = 14,  // for internal failure within 
firmware.
+STOP_SAVE_SPR_ENTRY_MISSING  =  15,
+STOP_SAVE_SPR_BIT_POS_RESERVE=  16,
 } StopReturnCode_t;
 
 /**
@@ -101,7 +123,20 @@ typedef enum
 P9_STOP_SCOM_RESET  = 6,
 P9_STOP_SCOM_OR_APPEND  = 7,
 P9_STOP_SCOM_AND_APPEND = 8,
-P9_STOP_SCOM_OP_MAX = 9
+P9_STOP_SCOM_OP_MAX = 9,
+
+//enum members which are project agnostic
+PROC_STOP_SCOM_OP_MIN =   0,
+PROC_STOP_SCOM_APPEND =   1,
+PROC_STOP_SCOM_REPLACE=   2,
+PROC_STOP_SCOM_OR =   3,
+PROC_STOP_SCOM_AND=   4,
+PROC_STOP_SCOM_NOOP   =   5,
+PROC_STOP_SCOM_RESET  =   6,
+PROC_STOP_SCOM_OR_APPEND  =   7,
+PROC_STOP_SCOM_AND_APPEND =   8,
+PROC_STOP_SCOM_OP_MAX =   9,
+
 } ScomOperation_t;
 
 /**
@@ -114,9 +149,49 @@ typedef enum
 P9_STOP_SECTION_EQ_SCOM = 2,
 P9_STOP_SECTION_L2  = 3,
 P9_STOP_SECTION_L3  = 4,
-P9_STOP_SECTION_MAX = 5
+P9_STOP_SECTION_MAX = 5,
+
+//enum members which are project agnostic

[PATCH v8 2/3] API to verify the STOP API and image compatibility

[Pratik Rajesh Sampat]

From: Prem Shanker Jha 

Commit defines a new API primarily intended for OPAL to determine
cpu register save API's compatibility with HOMER layout and
self save restore. It can help OPAL determine if version of
API integrated with OPAL is different from hostboot.

Change-Id: Ic0de45a336cfb8b6b6096a10ac1cd3ffbaa44fc0
Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/77612
Tested-by: FSP CI Jenkins 
Tested-by: Jenkins Server 
Tested-by: Hostboot CI 
Reviewed-by: RANGANATHPRASAD G. BRAHMASAMUDRA 
Reviewed-by: Gregory S Still 
Reviewed-by: Jennifer A Stofer 
Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/77614
Tested-by: Jenkins OP Build CI 
Tested-by: Jenkins OP HW 
Reviewed-by: Daniel M Crowell 
Signed-off-by: Pratik Rajesh Sampat 
---
 include/p9_stop_api.H| 25 ++
 libpore/p9_cpu_reg_restore_instruction.H |  7 ++-
 libpore/p9_hcd_memmap_base.H |  7 +++
 libpore/p9_stop_api.C| 58 +++-
 libpore/p9_stop_api.H| 26 ++-
 libpore/p9_stop_util.H   | 20 
 6 files changed, 130 insertions(+), 13 deletions(-)

diff --git a/include/p9_stop_api.H b/include/p9_stop_api.H
index 9d3bc1e5..cb5ffd6f 100644
--- a/include/p9_stop_api.H
+++ b/include/p9_stop_api.H
@@ -107,6 +107,7 @@ typedef enum
 STOP_SAVE_FAIL   = 14,  // for internal failure within 
firmware.
 STOP_SAVE_SPR_ENTRY_MISSING  =  15,
 STOP_SAVE_SPR_BIT_POS_RESERVE=  16,
+STOP_SAVE_API_IMG_INCOMPATIBLE   =  18,
 } StopReturnCode_t;
 
 /**
@@ -161,6 +162,14 @@ typedef enum
 
 } ScomSection_t;
 
+/**
+ * @brief   versions pertaining relvant to STOP API.
+ */
+typedef enum
+{
+STOP_API_VER=   0x00,
+STOP_API_VER_CONTROL=   0x02,
+} VersionList_t;
 
 
 /**
@@ -192,6 +201,14 @@ typedef enum
 BIT_POS_USPRG1  =   30,
 } SprBitPositionList_t;
 
+typedef enum
+{
+SMF_SUPPORT_MISSING_IN_HOMER =   0x01,
+SELF_SUPPORT_MISSING_FOR_LE_HYP  =   0x02,
+IPL_RUNTIME_CPU_SAVE_VER_MISMATCH=   0x04,
+SELF_RESTORE_VER_MISMATCH=   0x08,
+} VersionIncompList_t;
+
 #ifdef __cplusplus
 extern "C" {
 #endif
@@ -230,6 +247,14 @@ StopReturnCode_t p9_stop_save_scom( void* const   i_pImage,
 const ScomOperation_t i_operation,
 const ScomSection_t i_section );
 
+/**
+ * @brief   verifies if API is compatible of current HOMER image.
+ * @param[in]   i_pImagepoints to the start of HOMER image of P9 chip.
+ * @param[out]  o_inCompVector  list of incompatibilities found.
+ * @return  STOP_SAVE_SUCCESS if if API succeeds, error code otherwise.
+ */
+StopReturnCode_t proc_stop_api_discover_capability( void* const i_pImage, 
uint64_t* o_inCompVector );
+
 #ifdef __cplusplus
 } // extern "C"
 };  // namespace stopImageSection ends
diff --git a/libpore/p9_cpu_reg_restore_instruction.H 
b/libpore/p9_cpu_reg_restore_instruction.H
index d69a4212..5f168855 100644
--- a/libpore/p9_cpu_reg_restore_instruction.H
+++ b/libpore/p9_cpu_reg_restore_instruction.H
@@ -5,7 +5,7 @@
 /**/
 /* OpenPOWER HostBoot Project */
 /**/
-/* Contributors Listed Below - COPYRIGHT 2015,2018*/
+/* Contributors Listed Below - COPYRIGHT 2015,2020*/
 /* [+] International Business Machines Corp.  */
 /**/
 /**/
@@ -69,6 +69,11 @@ enum
 OPCODE_18   =   18,
 SELF_SAVE_FUNC_ADD  =   0x2300,
 SELF_SAVE_OFFSET=   0x180,
+SKIP_SPR_REST_INST  =   0x481c, //b . +0x01c
+MFLR_R30=   0x7fc802a6,
+SKIP_SPR_SELF_SAVE  =   0x3bff0020, //addi r31 r31, 0x20
+MTLR_INST   =   0x7fc803a6,  //mtlr r30
+BRANCH_BE_INST  =   0x4820,
 };
 
 #define MR_R0_TO_R100x7c0a0378UL //mr r10 r0
diff --git a/libpore/p9_hcd_memmap_base.H b/libpore/p9_hcd_memmap_base.H
index 000fafef..ddb56728 100644
--- a/libpore/p9_hcd_memmap_base.H
+++ b/libpore/p9_hcd_memmap_base.H
@@ -444,6 +444,13 @@ HCD_CONST(CME_QUAD_PSTATE_SIZE, HALF_KB)
 
 HCD_CONST(CME_REGION_SIZE,  (64 * ONE_KB))
 
+
+// HOMER compatibility
+
+HCD_CONST(STOP_API_CPU_SAVE_VER,0x02)
+HCD_CONST(SELF_SAVE_RESTORE_VER,0x02)
+HCD_CONST(SMF_SUPPORT_SIGNATURE_OFFSET, 0x1300)
+HCD_CONST(SMF_SELF_SIGNATURE,   (0x5f534d46))
 // Debug
 
 HCD_CONST(CPMR_TRACE_REGION_OFFSET, (512 * ONE_KB))
diff --git a/libpore/p9_stop_api.C b/libpore/p9_stop_api.C
index 2d9bb

[PATCH v8 0/3] Support for Self Save API in OPAL

[Pratik Rajesh Sampat]

en chosen as an implementation to demonstrate in
this patch series.

Patch 1:
Commit adds support calling into the self save firmware API.
Also adds abstraction for making platform agnostic calls.

Patch 2:
Commit adds API to determine the version of the STOP API. This helps
to identify support for self save in the firmware

Patch 3:
commit adds wrappers for the self save api for which an opal call can
be made. The commit also advertises the self-save feature if the STOP
API versions are consistent with the firmware.


Pratik Rajesh Sampat (1):
  Self save API integration

Prem Shanker Jha (2):
  Self Save: Introducing Support for SPR Self Save
  API to verify the STOP API and image compatibility

 doc/opal-api/opal-slw-self-save-reg-181.rst |  51 ++
 doc/opal-api/opal-slw-set-reg-100.rst   |   5 +
 doc/power-management.rst|  48 +
 hw/slw.c| 106 +++
 include/opal-api.h  |   3 +-
 include/p9_stop_api.H   | 122 ++-
 include/skiboot.h   |   3 +
 libpore/p9_cpu_reg_restore_instruction.H|  11 +-
 libpore/p9_hcd_memmap_base.H|   7 +
 libpore/p9_stop_api.C   | 964 +++-
 libpore/p9_stop_api.H   | 141 ++-
 libpore/p9_stop_data_struct.H   |   4 +-
 libpore/p9_stop_util.H  |  27 +-
 13 files changed, 1060 insertions(+), 432 deletions(-)
 create mode 100644 doc/opal-api/opal-slw-self-save-reg-181.rst

-- 
2.25.1

[PATCH v7 3/3] powerpc/powernv: Preference optimization for SPRs with constant values

[Pratik Rajesh Sampat]

There are SPRs whose values don't tend to change over time and invoking
self-save on them, where the values are gotten each time may turn out to
be inefficient. In that case calling a self-restore where passing the
value makes more sense as, if the value is same the memory location
is not updated.
SPRs that dont change are as follows:
SPRN_HSPRG0,
SPRN_LPCR,
SPRN_PTCR,
SPRN_HMEER,
SPRN_HID0,

There are also SPRs whose values change and/or their value may not be
correcty determinable in the kernel. Eg: MSR and PSSCR

The value of LPCR is dynamic based on if the CPU is entered a stop
state during cpu idle versus cpu hotplug.

Therefore in this optimization patch, introducing the concept of
preference for each SPR to choose from in the case both self-save and
self-restore is supported.

The preference bitmask is shown as below:

|... | 2nd pref | 1st pref |

MSB   LSB

The preference from higher to lower is from LSB to MSB with a shift of 8
bits.
Example:
Prefer self save first, if not available then prefer self
restore
The preference mask for this scenario will be seen as below.
((FIRMWARE_RESTORE << PREFERENCE_SHIFT) | FIRMWARE_SELF_SAVE)
-
|... | Self restore | Self save |
-
MSB LSB

Signed-off-by: Pratik Rajesh Sampat 
---
 arch/powerpc/platforms/powernv/idle.c | 88 +--
 1 file changed, 70 insertions(+), 18 deletions(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index fdcb18a8a05b..daa2f920bd05 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -43,6 +43,31 @@
 #define FIRMWARE_SELF_SAVE0x2
 #define KERNEL_SAVE_RESTORE   0x4
 
+#define NR_PREFERENCES2
+#define PREFERENCE_SHIFT  4
+#define PREFERENCE_MASK   0xf
+/*
+ * Bitmask defining the kind of preferences available.
+ * Note : The higher to lower preference is from LSB to MSB, with a shift of
+ * 4 bits.
+ * 
+ * || 2nd pref | 1st pref |
+ * 
+ * MSB   LSB
+ */
+/* Prefer Restore if available, otherwise unsupported */
+#define PREFER_SELF_RESTORE_ONLY   FIRMWARE_RESTORE
+/* Prefer Save if available, otherwise unsupported */
+#define PREFER_SELF_SAVE_ONLY  FIRMWARE_SELF_SAVE
+/* Prefer Restore when available, otherwise prefer Save */
+#define PREFER_RESTORE_SAVE((FIRMWARE_SELF_SAVE << \
+ PREFERENCE_SHIFT)\
+ | FIRMWARE_RESTORE)
+/* Prefer Save when available, otherwise prefer Restore*/
+#define PREFER_SAVE_RESTORE((FIRMWARE_RESTORE <<\
+ PREFERENCE_SHIFT)\
+ | FIRMWARE_SELF_SAVE)
+
 static u32 supported_cpuidle_states;
 struct pnv_idle_states_t *pnv_idle_states;
 int nr_pnv_idle_states;
@@ -52,6 +77,7 @@ DEFINE_STATIC_KEY_FALSE(is_ptcr_self_save);
 
 struct preferred_sprs {
u64 spr;
+   u32 preferred_mode;
u32 supported_mode;
 };
 
@@ -66,42 +92,52 @@ struct preferred_sprs {
 struct preferred_sprs preferred_sprs[] = {
{
.spr = SPRN_HSPRG0,
+   .preferred_mode = PREFER_RESTORE_SAVE,
.supported_mode = FIRMWARE_RESTORE,
},
{
.spr = SPRN_LPCR,
+   .preferred_mode = PREFER_SAVE_RESTORE,
.supported_mode = FIRMWARE_RESTORE,
},
{
.spr = SPRN_PTCR,
+   .preferred_mode = PREFER_RESTORE_SAVE,
.supported_mode = KERNEL_SAVE_RESTORE,
},
{
.spr = SPRN_HMEER,
+   .preferred_mode = PREFER_RESTORE_SAVE,
.supported_mode = FIRMWARE_RESTORE,
},
{
.spr = SPRN_HID0,
+   .preferred_mode = PREFER_RESTORE_SAVE,
.supported_mode = FIRMWARE_RESTORE,
},
{
.spr = P9_STOP_SPR_MSR,
+   .preferred_mode = PREFER_SAVE_RESTORE,
.supported_mode = FIRMWARE_RESTORE,
},
{
.spr = P9_STOP_SPR_PSSCR,
+   .preferred_mode = PREFER_SAVE_RESTORE,
.supported_mode = FIRMWARE_RESTORE,
},
{
.spr = SPRN_HID1,
+   .preferred_mode = PREFER_RESTORE_SAVE,
.supported_mode = FIRMWARE_RESTORE,
},
{
.spr = SPRN_HID4,
+   .preferred_mode = PREFER_SELF_RESTORE_ONLY,
.supported_mode = FIRMWARE_RESTORE,
},
{
.spr = SPRN_HID5,
+   .preferred_mode = PREFER_SELF_RESTORE_ONLY,
.supported_mode = FIRMWARE_RESTORE,
}
 };
@@ -218,7 +2

[PATCH v7 2/3] powerpc/powernv: Introduce support and parsing for self-save API

[Pratik Rajesh Sampat]

This commit introduces and leverages the Self save API. The difference
between self-save and self-restore is that the value to be saved for the
SPR does not need to be passed to the call.

Add the new Self Save OPAL API call in the list of OPAL calls.
Implement the self saving of the SPRs based on the support populated.
This commit prefers the self-save over self-restore in case both are
supported for a particular SPR.

Along with support for self-save, kernel supported save restore is also
populated in the list. This property is only populated for those SPRs
which encapsulate support from the kernel and have the possibility to
garner support from a firmware mode too.

In addition, the commit also parses the device tree for nodes self-save,
self-restore and populate support for the preferred SPRs based on what
was advertised by the device tree.

In the case a SPR is supported by the firmware self-save, self-restore
and kernel save restore then the preference of execution is also in the
same order as above.

Signed-off-by: Pratik Rajesh Sampat 
---
 .../bindings/powerpc/opal/power-mgt.txt   |  18 +++
 arch/powerpc/include/asm/opal-api.h   |   3 +-
 arch/powerpc/include/asm/opal.h   |   1 +
 arch/powerpc/platforms/powernv/idle.c | 135 +-
 arch/powerpc/platforms/powernv/opal-call.c|   1 +
 5 files changed, 150 insertions(+), 8 deletions(-)

diff --git a/Documentation/devicetree/bindings/powerpc/opal/power-mgt.txt 
b/Documentation/devicetree/bindings/powerpc/opal/power-mgt.txt
index 9d619e955576..5fb03c6d7de9 100644
--- a/Documentation/devicetree/bindings/powerpc/opal/power-mgt.txt
+++ b/Documentation/devicetree/bindings/powerpc/opal/power-mgt.txt
@@ -116,3 +116,21 @@ otherwise. The length of all the property arrays must be 
the same.
which of the fields of the PMICR are set in the corresponding
entries in ibm,cpu-idle-state-pmicr. This is an optional
property on POWER8 and is absent on POWER9.
+
+- self-restore:
+ Array of unsigned 64-bit values containing a property for sprn-mask
+ with each bit indicating the index of the supported SPR for the
+ functionality. This is an optional property for both Power8 and Power9
+
+- self-save:
+  Array of unsigned 64-bit values containing a property for sprn-mask
+  with each bit indicating the index of the supported SPR for the
+  functionality. This is an optional property for both Power8 and Power9
+
+Example of arrangement of self-restore and self-save arrays:
+For instance if PSSCR is supported, the value is 0x357 = 855.
+Since the array is of 64 bit values, the index of the array is determined by
+855 / 64 = 13th element. Within that index, the bit number is determined by
+855 % 64 = 23rd bit.
+This means that if the 23rd bit in array[13] is set, then that SPR is supported
+by the corresponding self-save or self-restore API.
diff --git a/arch/powerpc/include/asm/opal-api.h 
b/arch/powerpc/include/asm/opal-api.h
index 1dffa3cb16ba..7ba698369083 100644
--- a/arch/powerpc/include/asm/opal-api.h
+++ b/arch/powerpc/include/asm/opal-api.h
@@ -214,7 +214,8 @@
 #define OPAL_SECVAR_GET176
 #define OPAL_SECVAR_GET_NEXT   177
 #define OPAL_SECVAR_ENQUEUE_UPDATE 178
-#define OPAL_LAST  178
+#define OPAL_SLW_SELF_SAVE_REG 181
+#define OPAL_LAST  181
 
 #define QUIESCE_HOLD   1 /* Spin all calls at entry */
 #define QUIESCE_REJECT 2 /* Fail all calls with OPAL_BUSY */
diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h
index 9986ac34b8e2..a370b0e8d899 100644
--- a/arch/powerpc/include/asm/opal.h
+++ b/arch/powerpc/include/asm/opal.h
@@ -204,6 +204,7 @@ int64_t opal_handle_hmi2(__be64 *out_flags);
 int64_t opal_register_dump_region(uint32_t id, uint64_t start, uint64_t end);
 int64_t opal_unregister_dump_region(uint32_t id);
 int64_t opal_slw_set_reg(uint64_t cpu_pir, uint64_t sprn, uint64_t val);
+int64_t opal_slw_self_save_reg(uint64_t cpu_pir, uint64_t sprn);
 int64_t opal_config_cpu_idle_state(uint64_t state, uint64_t flag);
 int64_t opal_pci_set_phb_cxl_mode(uint64_t phb_id, uint64_t mode, uint64_t 
pe_number);
 int64_t opal_pci_get_pbcq_tunnel_bar(uint64_t phb_id, uint64_t *addr);
diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 858ceb86394d..fdcb18a8a05b 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -35,13 +35,20 @@
 /*
  * Type of support for each SPR
  * FIRMWARE_RESTORE: firmware restoration supported: calls self-restore OPAL 
API
+ * FIRMWARE_SELF_SAVE: firmware save and restore: calls self-save OPAL API
+ * KERNEL_SAVE_RESTORE: kernel handles the saving and restoring of SPR
  */
 #define UNSUPPORTED   0x0
 #define FIRMWARE_RESTORE  0x1
+#define FIRMWARE_SELF_SAVE0x2
+#define

[PATCH v7 1/3] powerpc/powernv: Introduce interface for self-restore support

[Pratik Rajesh Sampat]

Introduces an interface that helps determine support for the
self-restore API. The commit is isomorphic to the original interface of
declaring SPRs to self-restore.

Signed-off-by: Pratik Rajesh Sampat 
Reviewed-by: Gautham R. Shenoy 
---
 arch/powerpc/platforms/powernv/idle.c | 200 +++---
 1 file changed, 152 insertions(+), 48 deletions(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 78599bca66c2..858ceb86394d 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -32,10 +32,67 @@
 #define P9_STOP_SPR_MSR 2000
 #define P9_STOP_SPR_PSSCR  855
 
+/*
+ * Type of support for each SPR
+ * FIRMWARE_RESTORE: firmware restoration supported: calls self-restore OPAL 
API
+ */
+#define UNSUPPORTED   0x0
+#define FIRMWARE_RESTORE  0x1
+
 static u32 supported_cpuidle_states;
 struct pnv_idle_states_t *pnv_idle_states;
 int nr_pnv_idle_states;
 
+struct preferred_sprs {
+   u64 spr;
+   u32 supported_mode;
+};
+
+/*
+ * Supported mode: Default support. Can be overwritten during system
+ *initialization
+ */
+struct preferred_sprs preferred_sprs[] = {
+   {
+   .spr = SPRN_HSPRG0,
+   .supported_mode = FIRMWARE_RESTORE,
+   },
+   {
+   .spr = SPRN_LPCR,
+   .supported_mode = FIRMWARE_RESTORE,
+   },
+   {
+   .spr = SPRN_HMEER,
+   .supported_mode = FIRMWARE_RESTORE,
+   },
+   {
+   .spr = SPRN_HID0,
+   .supported_mode = FIRMWARE_RESTORE,
+   },
+   {
+   .spr = P9_STOP_SPR_MSR,
+   .supported_mode = FIRMWARE_RESTORE,
+   },
+   {
+   .spr = P9_STOP_SPR_PSSCR,
+   .supported_mode = FIRMWARE_RESTORE,
+   },
+   {
+   .spr = SPRN_HID1,
+   .supported_mode = FIRMWARE_RESTORE,
+   },
+   {
+   .spr = SPRN_HID4,
+   .supported_mode = FIRMWARE_RESTORE,
+   },
+   {
+   .spr = SPRN_HID5,
+   .supported_mode = FIRMWARE_RESTORE,
+   }
+};
+
+const int nr_preferred_sprs = ARRAY_SIZE(preferred_sprs);
+
 /*
  * The default stop state that will be used by ppc_md.power_save
  * function on platforms that support stop instruction.
@@ -61,78 +118,125 @@ static bool deepest_stop_found;
 
 static unsigned long power7_offline_type;
 
-static int pnv_save_sprs_for_deep_states(void)
+static int pnv_self_restore_sprs(u64 pir, int cpu, u64 spr)
 {
-   int cpu;
+   u64 reg_val;
int rc;
 
-   /*
-* hid0, hid1, hid4, hid5, hmeer and lpcr values are symmetric across
-* all cpus at boot. Get these reg values of current cpu and use the
-* same across all cpus.
-*/
-   uint64_t lpcr_val   = mfspr(SPRN_LPCR);
-   uint64_t hid0_val   = mfspr(SPRN_HID0);
-   uint64_t hid1_val   = mfspr(SPRN_HID1);
-   uint64_t hid4_val   = mfspr(SPRN_HID4);
-   uint64_t hid5_val   = mfspr(SPRN_HID5);
-   uint64_t hmeer_val  = mfspr(SPRN_HMEER);
-   uint64_t msr_val = MSR_IDLE;
-   uint64_t psscr_val = pnv_deepest_stop_psscr_val;
-
-   for_each_present_cpu(cpu) {
-   uint64_t pir = get_hard_smp_processor_id(cpu);
-   uint64_t hsprg0_val = (uint64_t)paca_ptrs[cpu];
-
-   rc = opal_slw_set_reg(pir, SPRN_HSPRG0, hsprg0_val);
+   switch (spr) {
+   case SPRN_HSPRG0:
+   reg_val = (uint64_t)paca_ptrs[cpu];
+   rc = opal_slw_set_reg(pir, SPRN_HSPRG0, reg_val);
if (rc != 0)
return rc;
-
-   rc = opal_slw_set_reg(pir, SPRN_LPCR, lpcr_val);
+   break;
+   case SPRN_LPCR:
+   reg_val = mfspr(SPRN_LPCR);
+   rc = opal_slw_set_reg(pir, SPRN_LPCR, reg_val);
if (rc != 0)
return rc;
-
+   break;
+   case P9_STOP_SPR_MSR:
+   reg_val = MSR_IDLE;
if (cpu_has_feature(CPU_FTR_ARCH_300)) {
-   rc = opal_slw_set_reg(pir, P9_STOP_SPR_MSR, msr_val);
+   rc = opal_slw_set_reg(pir, P9_STOP_SPR_MSR, reg_val);
if (rc)
return rc;
-
-   rc = opal_slw_set_reg(pir,
- P9_STOP_SPR_PSSCR, psscr_val);
-
+   }
+   break;
+   case P9_STOP_SPR_PSSCR:
+   reg_val = pnv_deepest_stop_psscr_val;
+   if (cpu_has_feature(CPU_FTR_ARCH_300)) {
+   rc = opal_slw_set_reg(pir, P9_STOP_SPR_PSSCR, reg_val);
if (rc)
return rc;
}
-
-   /* HIDs are per core registers */
+   break;
+   case SPRN_HMEER

[PATCH v7 0/3] powerpc/powernv: Introduce interface for self-restore support

[Pratik Rajesh Sampat]

tate. This was used in POWER8 for HSPRG0 to distinguish a
wakeup from Winkle vs Fastsleep.

Limitations of self-save

Ideally all SPRs should be available for self-save, but HID0 is very
tricky to implement in microcode due to various endianess quirks.
Couple of implementation schemes were buggy and hence HID0 was left
out to be self-restore only.

The fallout of this limitation is as follows:

* In Non PEF environment, no issue. Linux will use self-restore for
  HID0 as it does today and no functional impact.

* In PEF environment, the HID0 restore value is decided by OPAL during
  boot and it is setup for LE hypervisor with radix MMU. This is the
  default and current working configuration of a PEF environment.
  However if there is a change, then HV Linux will try to change the
  HID0 value to something different than what OPAL decided, at which
  time deep-stop states will be disabled under this new PEF
  environment.

A simple and workable design is achieved by scoping the power
management deep-stop state support only to a known default PEF
environment. Any deviation will affect *only* deep stop-state support
(stop4,5) in that environment and not have any functional impediment
to the environment itself.

In future, if there is a need to support changing of HID0 to various
values under PEF environment and support deep-stop states, it can be
worked out via an ultravisor call or improve the microcode design to
include HID0 in self-save.  These future scheme would be an extension
and does not break or make the current implementation scheme
redundant.

Design Choices
==

Presenting the design choices in front of us:

Design-Choice 1:

A simple implementation is to just replace self-restore calls with
self-save as it is direct super-set.

Pros:
A simple design, quick to implement


Cons:
* Breaks backward compatibility. Self-restore has historically been
  supported in the firmware and an old firmware running on an new
  kernel will be incompatible and deep stop states will be cut.
* Furthermore, critical SPRs which need to be restored
  before 0x100 vector like HID0 are not supported by self-save.

Design-Choice 2:

Advertise both self-restore and self-save from OPAL including the set
of registers that each support. The kernel can then choose which API
to go with.
For the sake of simplicity, in case both modes are supported for an
SPR by default self-save would be called for it.

Pros:
* Backwards compatible

Cons:
Overhead in parsing device tree with the SPR list

Possible optimization with Approach2:
-
There are SPRs whose values don't tend to change over time and invoking
self-save on them, where the values are gotten each time may turn out to
be inefficient. In that case calling a self-restore where passing the
value makes more sense as, if the value is same, the memory location
is not updated.
SPRs that dont change are as follows:
SPRN_HSPRG0,
SPRN_LPCR,
SPRN_PTCR,
SPRN_HMEER,
SPRN_HID0,

The values of PSSCR and MSR change at runtime and hence, the kernel
cannot determine during boot time what their values will be before
entering a particular deep-stop state.

Therefore, a preference based interface is introduced for choosing
between self-save or self-restore between for each SPR.
The per-SPR preference is only a refinement of
approach 2 purely for performance reasons. It can be dropped if the
complexity is not deemed worth the returns.

Patches Organization

Design Choice 2 has been chosen as an implementation to demonstrate in
the patch series.

Patch1:
Devises an interface which lists all the interested SPRs, along with
highlighting the support of mode.
It is an isomorphic patch to replicate the functionality of the older
self-restore firmware for the new interface

Patch2:
Introduces the self-save API and leverages upon the struct interface to
add another supported mode in the mix of saving and restoring. It also
enforces that in case both modes are supported self-save is chosen over
self-restore

The commit also parses the device-tree and populate support for
self-save and self-restore in the supported mask

Patch3:
Introduce an optimization to allow preference to choose between one more
over the one when both both modes are supported. This optimization can
allow for better performance for the SPRs that don't change in value and
hence self-restore is a better alternative, and in cases when it is
known for values to change self-save is more convenient.

Pratik Rajesh Sampat (3):
  powerpc/powernv: Introduce interface for self-restore support
  powerpc/powernv: Introduce support and parsing for self-save API
  powerpc/powernv: Preference optimization for SPRs with constant values

 .../bindings/powerpc/opal/power-mgt.txt   |  18 +
 arch/powerpc/include/asm/opal-api.h   |   3 +-
 arch/powerpc/include/asm/opal.h   |   1 +
 arch/powerpc/platforms/powernv/idle.c  

[PATCH v7 4/4] Advertise the self-save and self-restore attributes in the device tree

[Pratik Rajesh Sampat]

Support for self save and self restore interface is advertised in the
device tree, along with the list of SPRs it supports for each.

The Special Purpose Register identification is encoded in a 2048 bitmask
structure, where each bit signifies the identification key of that SPR
which is consistent with that of the POWER architecture set for that
register.

Signed-off-by: Pratik Rajesh Sampat 
Reviewed-by: Gautham R. Shenoy 
---
 .../ibm,opal/power-mgt/self-restore.rst   |  27 
 .../ibm,opal/power-mgt/self-save.rst  |  27 
 hw/slw.c  | 118 ++
 include/skiboot.h |   1 +
 4 files changed, 173 insertions(+)
 create mode 100644 doc/device-tree/ibm,opal/power-mgt/self-restore.rst
 create mode 100644 doc/device-tree/ibm,opal/power-mgt/self-save.rst

diff --git a/doc/device-tree/ibm,opal/power-mgt/self-restore.rst 
b/doc/device-tree/ibm,opal/power-mgt/self-restore.rst
new file mode 100644
index ..2a2269f7
--- /dev/null
+++ b/doc/device-tree/ibm,opal/power-mgt/self-restore.rst
@@ -0,0 +1,27 @@
+ibm,opal/power-mgt/self-restore device tree entries
+===
+
+This node exports the bitmask representing the special purpose registers that
+the self-restore API currently supports.
+
+Example:
+
+.. code-block:: dts
+
+  self-restore {
+sprn-bitmask = <0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x4201 0x0 0x0
+0x2 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
+0x0 0x0 0x10 0x90 0x0 0x0 0x53 0x0 0x0 0x0
+0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
+0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
+0x0 0x1>;
+phandle = <0x1c7>;
+  };
+
+sprn-bitmask
+
+
+This property is a bitmask of of all the existing SPRs and if the SPR is
+supported, the corresponding bit of the SPR number is set to 1.
+The representation of the bits are left-right, i.e the MSB of the first
+doubleword represants the 0th bit.
diff --git a/doc/device-tree/ibm,opal/power-mgt/self-save.rst 
b/doc/device-tree/ibm,opal/power-mgt/self-save.rst
new file mode 100644
index ..c367720e
--- /dev/null
+++ b/doc/device-tree/ibm,opal/power-mgt/self-save.rst
@@ -0,0 +1,27 @@
+ibm,opal/power-mgt/self-save device tree entries
+===
+
+This node exports the bitmask representing the special purpose registers that
+the self-save API currently supports.
+
+Example:
+
+.. code-block:: dts
+
+  self-save {
+sprn-bitmask = <0x0 0x0 0x0 0x0 0x10 0x0 0x0 0x0 0x4201 0x0 0x0
+0x2 0x0 0x0 0x0 0x1 0x0 0x0 0x0 0x0 0x0 0x0 0x0
+0x0 0x0 0x0 0x10 0x84 0x0 0x0 0x0 0x0 0x0 0x0
+0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
+0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
+0x0 0x1>;
+phandle = <0x1c8>;
+  };
+
+sprn-bitmask
+
+
+This property is a bitmask of of all the existing SPRs and if the SPR is
+supported, the corresponding bit of the SPR number is set to 1.
+The representation of the bits are left-right, i.e the MSB of the first
+doubleword represants the 0th bit.
diff --git a/hw/slw.c b/hw/slw.c
index 9751c04f..fb14bd0c 100644
--- a/hw/slw.c
+++ b/hw/slw.c
@@ -29,6 +29,7 @@
 #include 
 
 static uint32_t slw_saved_reset[0x100];
+#define SPR_BITMAP_LENGTH  2048
 
 static bool slw_current_le = false;
 
@@ -750,6 +751,121 @@ static void slw_late_init_p9(struct proc_chip *chip)
}
 }
 
+/* Add device tree properties to determine self-save | restore */
+void add_cpu_self_save_restore_properties(void)
+{
+   struct dt_node *self_restore, *self_save, *power_mgt;
+   uint64_t *self_save_mask, *self_restore_mask;
+   bool self_save_supported = true;
+   uint64_t compVector = -1;
+   struct proc_chip *chip;
+   int i, rc;
+
+   const uint64_t self_restore_regs[] = {
+   P8_SPR_HRMOR,
+   P8_SPR_HMEER,
+   P8_SPR_PMICR,
+   P8_SPR_PMCR,
+   P8_SPR_HID0,
+   P8_SPR_HID1,
+   P8_SPR_HID4,
+   P8_SPR_HID5,
+   P8_SPR_HSPRG0,
+   P8_SPR_LPCR,
+   P8_MSR_MSR
+   };
+
+   const uint64_t self_save_regs[] = {
+   P9_STOP_SPR_DAWR,
+   P9_STOP_SPR_HSPRG0,
+   P9_STOP_SPR_LDBAR,
+   P9_STOP_SPR_LPCR,
+   P9_STOP_SPR_PSSCR,
+   P9_STOP_SPR_MSR,
+   P9_STOP_SPR_HRMOR,
+   P9_STOP_SPR_HMEER,
+   P9_STOP_SPR_PMCR,
+   P9_STOP_SPR_PTCR
+   };
+
+   chip = next_chip(NULL);
+   

[PATCH v7 3/4] API to verify the STOP API and image compatibility

[Pratik Rajesh Sampat]

From: Prem Shanker Jha 

Commit defines a new API primarily intended for OPAL to determine
cpu register save API's compatibility with HOMER layout and
self save restore. It can help OPAL determine if version of
API integrated with OPAL is different from hostboot.

Change-Id: Ic0de45a336cfb8b6b6096a10ac1cd3ffbaa44fc0
Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/77612
Tested-by: FSP CI Jenkins 
Tested-by: Jenkins Server 
Tested-by: Hostboot CI 
Reviewed-by: RANGANATHPRASAD G. BRAHMASAMUDRA 
Reviewed-by: Gregory S Still 
Reviewed-by: Jennifer A Stofer 
Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/77614
Tested-by: Jenkins OP Build CI 
Tested-by: Jenkins OP HW 
Reviewed-by: Daniel M Crowell 
Signed-off-by: Pratik Rajesh Sampat 
---
 include/p9_stop_api.H| 26 +++
 libpore/p9_cpu_reg_restore_instruction.H |  7 ++-
 libpore/p9_hcd_memmap_base.H |  7 +++
 libpore/p9_stop_api.C| 58 +++-
 libpore/p9_stop_api.H| 26 ++-
 libpore/p9_stop_util.H   | 20 
 6 files changed, 131 insertions(+), 13 deletions(-)

diff --git a/include/p9_stop_api.H b/include/p9_stop_api.H
index c304f70f..09ce3dc1 100644
--- a/include/p9_stop_api.H
+++ b/include/p9_stop_api.H
@@ -110,6 +110,7 @@ typedef enum
 STOP_SAVE_FAIL   = 14,  // for internal failure within 
firmware.
 STOP_SAVE_SPR_ENTRY_MISSING  =  15,
 STOP_SAVE_SPR_BIT_POS_RESERVE=  16,
+STOP_SAVE_API_IMG_INCOMPATIBLE   =  18,
 } StopReturnCode_t;
 
 /**
@@ -164,6 +165,14 @@ typedef enum
 
 } ScomSection_t;
 
+/**
+ * @brief   versions pertaining relvant to STOP API.
+ */
+typedef enum
+{
+STOP_API_VER=   0x00,
+STOP_API_VER_CONTROL=   0x02,
+} VersionList_t;
 
 
 /**
@@ -195,6 +204,14 @@ typedef enum
 BIT_POS_USPRG1  =   30,
 } SprBitPositionList_t;
 
+typedef enum
+{
+SMF_SUPPORT_MISSING_IN_HOMER =   0x01,
+SELF_SUPPORT_MISSING_FOR_LE_HYP  =   0x02,
+IPL_RUNTIME_CPU_SAVE_VER_MISMATCH=   0x04,
+SELF_RESTORE_VER_MISMATCH=   0x08,
+} VersionIncompList_t;
+
 #ifdef __cplusplus
 extern "C" {
 #endif
@@ -247,6 +264,15 @@ StopReturnCode_t p9_stop_save_scom( void* const   i_pImage,
 StopReturnCode_t
 p9_stop_save_cpureg_control( void* i_pImage, const uint64_t i_pir,
  const uint32_t  i_saveRegVector );
+
+/**
+ * @brief   verifies if API is compatible of current HOMER image.
+ * @param[in]   i_pImagepoints to the start of HOMER image of P9 chip.
+ * @param[out]  o_inCompVector  list of incompatibilities found.
+ * @return  STOP_SAVE_SUCCESS if if API succeeds, error code otherwise.
+ */
+StopReturnCode_t proc_stop_api_discover_capability( void* const i_pImage, 
uint64_t* o_inCompVector );
+
 #ifdef __cplusplus
 } // extern "C"
 };  // namespace stopImageSection ends
diff --git a/libpore/p9_cpu_reg_restore_instruction.H 
b/libpore/p9_cpu_reg_restore_instruction.H
index d69a4212..5f168855 100644
--- a/libpore/p9_cpu_reg_restore_instruction.H
+++ b/libpore/p9_cpu_reg_restore_instruction.H
@@ -5,7 +5,7 @@
 /**/
 /* OpenPOWER HostBoot Project */
 /**/
-/* Contributors Listed Below - COPYRIGHT 2015,2018*/
+/* Contributors Listed Below - COPYRIGHT 2015,2020*/
 /* [+] International Business Machines Corp.  */
 /**/
 /**/
@@ -69,6 +69,11 @@ enum
 OPCODE_18   =   18,
 SELF_SAVE_FUNC_ADD  =   0x2300,
 SELF_SAVE_OFFSET=   0x180,
+SKIP_SPR_REST_INST  =   0x481c, //b . +0x01c
+MFLR_R30=   0x7fc802a6,
+SKIP_SPR_SELF_SAVE  =   0x3bff0020, //addi r31 r31, 0x20
+MTLR_INST   =   0x7fc803a6,  //mtlr r30
+BRANCH_BE_INST  =   0x4820,
 };
 
 #define MR_R0_TO_R100x7c0a0378UL //mr r10 r0
diff --git a/libpore/p9_hcd_memmap_base.H b/libpore/p9_hcd_memmap_base.H
index 000fafef..ddb56728 100644
--- a/libpore/p9_hcd_memmap_base.H
+++ b/libpore/p9_hcd_memmap_base.H
@@ -444,6 +444,13 @@ HCD_CONST(CME_QUAD_PSTATE_SIZE, HALF_KB)
 
 HCD_CONST(CME_REGION_SIZE,  (64 * ONE_KB))
 
+
+// HOMER compatibility
+
+HCD_CONST(STOP_API_CPU_SAVE_VER,0x02)
+HCD_CONST(SELF_SAVE_RESTORE_VER,0x02)
+HCD_CONST(SMF_SUPPORT_SIGNATURE_OFFSET, 0x1300)
+HCD_CONST(SMF_SELF_SIGNATURE,   (0x5f534d46))
 // Debug
 
 HCD_CONST(CPMR_TRACE_REGION_OFFSET, (512 * ONE_KB))
diff --git a/libpore/p9_stop_api.C b/libpore/p9_stop_

[PATCH v7 1/4] Self Save: Introducing Support for SPR Self Save

[Pratik Rajesh Sampat]

From: Prem Shanker Jha 

The commit is a merger of commits that makes the following changes:
1. Commit fixes some issues with code found during integration test
  -  replacement of addi with xor instruction during self save API.
  -  fixing instruction generation for MFMSR during self save
  -  data struct updates in STOP API
  -  error RC updates for hcode image build
  -  HOMER parser updates.
  -  removed self save support for URMOR and HRMOR
  -  code changes for compilation with OPAL
  -  populating CME Image header with unsecure HOMER address.

Key_Cronus_Test=PM_REGRESS

Change-Id: I7cedcc466267c4245255d8d75c01ed695e316720
Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/66580
Tested-by: FSP CI Jenkins 
Tested-by: HWSV CI 
Tested-by: PPE CI 
Tested-by: Jenkins Server 
Tested-by: Cronus HW CI 
Tested-by: Hostboot CI 
Reviewed-by: Gregory S. Still 
Reviewed-by: RAHUL BATRA 
Reviewed-by: Jennifer A. Stofer 
Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/66587
Reviewed-by: Christian R. Geddes 
Signed-off-by: Prem Shanker Jha 
Signed-off-by: Akshay Adiga 
Signed-off-by: Pratik Rajesh Sampat 

2. The commit also incorporates changes that make STOP API project
agnostic changes include defining wrapper functions which call legacy
API. It also adds duplicate enum members which start with prefix PROC
instead of P9.

Key_Cronus_Test=PM_REGRESS

Change-Id: If87970f3e8cf9b507f33eb1be249e03eb3836a5e
RTC: 201128
Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/71307
Tested-by: FSP CI Jenkins 
Tested-by: Jenkins Server 
Tested-by: Hostboot CI 
Tested-by: Cronus HW CI 
Reviewed-by: RANGANATHPRASAD G. BRAHMASAMUDRA 
Reviewed-by: Gregory S. Still 
Reviewed-by: Jennifer A Stofer 
Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/71314
Tested-by: Jenkins OP Build CI 
Tested-by: Jenkins OP HW 
Reviewed-by: Daniel M. Crowell 
Signed-off-by: Prem Shanker Jha 
Signed-off-by: Pratik Rajesh Sampat 
---
 include/p9_stop_api.H|  79 +-
 libpore/p9_cpu_reg_restore_instruction.H |   4 +
 libpore/p9_stop_api.C| 954 +--
 libpore/p9_stop_api.H| 115 ++-
 libpore/p9_stop_data_struct.H|   4 +-
 libpore/p9_stop_util.H   |   7 +-
 6 files changed, 721 insertions(+), 442 deletions(-)

diff --git a/include/p9_stop_api.H b/include/p9_stop_api.H
index 79abd000..9d3bc1e5 100644
--- a/include/p9_stop_api.H
+++ b/include/p9_stop_api.H
@@ -63,6 +63,26 @@ typedef enum
 P9_STOP_SPR_PMCR=884,   // core register
 P9_STOP_SPR_HID =   1008,   // core register
 P9_STOP_SPR_MSR =   2000,   // thread register
+
+//enum members which are project agnostic
+PROC_STOP_SPR_DAWR=180,   // thread register
+PROC_STOP_SPR_CIABR   =187,   // thread register
+PROC_STOP_SPR_DAWRX   =188,   // thread register
+PROC_STOP_SPR_HSPRG0  =304,   // thread register
+PROC_STOP_SPR_HRMOR   =313,   // core register
+PROC_STOP_SPR_LPCR=318,   // thread register
+PROC_STOP_SPR_HMEER   =337,   // core register
+PROC_STOP_SPR_PTCR=464,   // core register
+PROC_STOP_SPR_USPRG0  =496,   // thread register
+PROC_STOP_SPR_USPRG1  =497,   // thread register
+PROC_STOP_SPR_URMOR   =505,   // core register
+PROC_STOP_SPR_SMFCTRL =511,   // thread register
+PROC_STOP_SPR_LDBAR   =850,   // thread register
+PROC_STOP_SPR_PSSCR   =855,   // thread register
+PROC_STOP_SPR_PMCR=884,   // core register
+PROC_STOP_SPR_HID =   1008,   // core register
+PROC_STOP_SPR_MSR =   2000,   // thread register
+
 } CpuReg_t;
 
 /**
@@ -85,6 +105,8 @@ typedef enum
 STOP_SAVE_SCOM_ENTRY_UPDATE_FAILED   = 12,
 STOP_SAVE_INVALID_FUSED_CORE_STATUS  = 13,
 STOP_SAVE_FAIL   = 14,  // for internal failure within 
firmware.
+STOP_SAVE_SPR_ENTRY_MISSING  =  15,
+STOP_SAVE_SPR_BIT_POS_RESERVE=  16,
 } StopReturnCode_t;
 
 /**
@@ -101,7 +123,20 @@ typedef enum
 P9_STOP_SCOM_RESET  = 6,
 P9_STOP_SCOM_OR_APPEND  = 7,
 P9_STOP_SCOM_AND_APPEND = 8,
-P9_STOP_SCOM_OP_MAX = 9
+P9_STOP_SCOM_OP_MAX = 9,
+
+//enum members which are project agnostic
+PROC_STOP_SCOM_OP_MIN =   0,
+PROC_STOP_SCOM_APPEND =   1,
+PROC_STOP_SCOM_REPLACE=   2,
+PROC_STOP_SCOM_OR =   3,
+PROC_STOP_SCOM_AND=   4,
+PROC_STOP_SCOM_NOOP   =   5,
+PROC_STOP_SCOM_RESET  =   6,
+PROC_STOP_SCOM_OR_APPEND  =   7,
+PROC_STOP_SCOM_AND_APPEND =   8,
+PROC_STOP_SCOM_OP_MAX =   9,
+
 } ScomOperation_t;
 
 /**
@@ -114,9 +149,49 @@ typedef enum
 P9_STOP_SECTION_EQ_SCOM = 2,
 P9_STOP_SECTION_L2  = 3,
 P9_STOP_SECTION_L3  = 4,
-P9_STOP_SECTION_MAX = 5
+P9_STOP_SECTION_MAX = 5,
+
+//enum members which are project agnostic

[PATCH v7 2/4] Self save API integration

[Pratik Rajesh Sampat]

The commit makes the self save API available outside the firmware by defining
an OPAL wrapper.
This wrapper has a similar interface to that of self restore and expects the
cpu pir, SPR number, minus the value of that SPR to be passed in its
paramters and returns OPAL_SUCCESS on success.
The commit also documents both the self-save and the self-restore API
calls along with their working and usage.

Signed-off-by: Pratik Rajesh Sampat 
Reviewed-by: Gautham R. Shenoy 
---
 doc/opal-api/opal-slw-self-save-reg-181.rst | 49 
 doc/opal-api/opal-slw-set-reg-100.rst   |  5 ++
 doc/power-management.rst| 48 +++
 hw/slw.c| 89 +
 include/opal-api.h  |  3 +-
 include/p9_stop_api.H   | 17 
 include/skiboot.h   |  3 +
 7 files changed, 213 insertions(+), 1 deletion(-)
 create mode 100644 doc/opal-api/opal-slw-self-save-reg-181.rst

diff --git a/doc/opal-api/opal-slw-self-save-reg-181.rst 
b/doc/opal-api/opal-slw-self-save-reg-181.rst
new file mode 100644
index ..5aa4c930
--- /dev/null
+++ b/doc/opal-api/opal-slw-self-save-reg-181.rst
@@ -0,0 +1,49 @@
+.. OPAL_SLW_SELF_SAVE_REG:
+
+OPAL_SLW_SELF_SAVE_REG
+==
+
+.. code-block:: c
+
+   #define OPAL_SLW_SELF_SAVE_REG  181
+
+   int64_t opal_slw_self_save_reg(uint64_t cpu_pir, uint64_t sprn);
+
+:ref:`OPAL_SLW_SELF_SAVE_REG` is used to inform low-level firmware to save
+the current contents of the SPR before entering a state of loss and
+also restore the content back on waking up from a deep stop state.
+
+An OPAL call `OPAL_SLW_SET_REG` exists which is similar in function as
+saving and restoring the SPR, with one difference being that the value of the
+SPR must also be supplied in the parameters.
+Complete reference: doc/opal-api/opal-slw-set-reg-100.rst
+
+Parameters
+--
+
+``uint64_t cpu_pir``
+  This parameter specifies the pir of the cpu for which the call is being made.
+``uint64_t sprn``
+  This parameter specifies the spr number as mentioned in p9_stop_api.H
+  The list of SPRs supported is as follows. This list is suppiled through the
+  device tree:
+   P9_STOP_SPR_DAWR,
+   P9_STOP_SPR_HSPRG0,
+   P9_STOP_SPR_LDBAR,
+   P9_STOP_SPR_LPCR,
+   P9_STOP_SPR_PSSCR,
+   P9_STOP_SPR_MSR,
+   P9_STOP_SPR_HRMOR,
+   P9_STOP_SPR_HMEER,
+   P9_STOP_SPR_PMCR,
+   P9_STOP_SPR_PTCR
+
+Returns
+---
+
+:ref:`OPAL_UNSUPPORTED`
+  If spr restore is not supported by pore engine.
+:ref:`OPAL_PARAMETER`
+  Invalid handle for the pir/chip
+:ref:`OPAL_SUCCESS`
+  On success
diff --git a/doc/opal-api/opal-slw-set-reg-100.rst 
b/doc/opal-api/opal-slw-set-reg-100.rst
index 2e8f1bd6..ee3e68ce 100644
--- a/doc/opal-api/opal-slw-set-reg-100.rst
+++ b/doc/opal-api/opal-slw-set-reg-100.rst
@@ -21,6 +21,11 @@ In Power 9, it uses p9_stop_save_cpureg(), api provided by 
self restore code,
 to inform the spr with their corresponding values with which they
 must be restored.
 
+An OPAL call `OPAL_SLW_SELF_SAVE_REG` exists which is similar in function
+saving and restoring the SPR, with one difference being that the value of the
+SPR doesn't need to be passed in the parameters, only with the SPR number
+the firmware can identify, save and restore the values for the same.
+Complete reference: doc/opal-api/opal-slw-self-save-reg-181.rst
 
 Parameters
 --
diff --git a/doc/power-management.rst b/doc/power-management.rst
index 76491a71..d6bd5358 100644
--- a/doc/power-management.rst
+++ b/doc/power-management.rst
@@ -15,3 +15,51 @@ On boot, specific stop states can be disabled via setting a 
mask. For example,
 to disable all but stop 0,1,2, use ~0xE000. ::
 
   nvram -p ibm,skiboot --update-config opal-stop-state-disable-mask=0x1FFF
+
+Saving and restoring Special Purpose Registers(SPRs)
+
+
+When a CPU wakes up from a deep stop state which can result in
+hypervisor state loss, all the SPRs are lost. The Linux Kernel expects
+a small set of SPRs to contain an expected value when the CPU wakes up
+from such a deep stop state. The microcode firmware provides the
+following two APIs, collectively known as the stop-APIs, to allow the
+kernel/OPAL to specify this set of SPRs and the value that they need
+to be restored with on waking up from a deep stop state.
+
+Self-restore:
+int64_t opal_slw_set_reg(uint64_t cpu_pir, uint64_t sprn, uint64_t val);
+The SPR number and the value of the that SPR must be restored with on
+wakeup from the deep-stop state must be specified. When this call is
+made, the microcode inserts instruction into the HOMER region to
+restore the content of the SPR to the specified value on wakeup from a
+deep-stop state. These instructions are executed by the CPU as soon as
+it wakes up from a deep stop state. The call is to be made once per
+SPR.
+
+Self

[PATCH v7 0/4] Support for Self Save API in OPAL

[Pratik Rajesh Sampat]

d self
restore functionality along with the register set as a bitmask
currently supported in the firmware. It also uses the versioning API
to determine support for the self-save feature as a whole.

Pratik Rajesh Sampat (2):
  Self save API integration
  Advertise the self-save and self-restore attributes in the device tree

Prem Shanker Jha (2):
  Self Save: Introducing Support for SPR Self Save
  API to verify the STOP API and image compatibility

 .../ibm,opal/power-mgt/self-restore.rst   |  27 +
 .../ibm,opal/power-mgt/self-save.rst  |  27 +
 doc/opal-api/opal-slw-self-save-reg-181.rst   |  49 +
 doc/opal-api/opal-slw-set-reg-100.rst |   5 +
 doc/power-management.rst  |  48 +
 hw/slw.c  | 207 
 include/opal-api.h|   3 +-
 include/p9_stop_api.H | 122 ++-
 include/skiboot.h |   4 +
 libpore/p9_cpu_reg_restore_instruction.H  |  11 +-
 libpore/p9_hcd_memmap_base.H  |   7 +
 libpore/p9_stop_api.C | 964 ++
 libpore/p9_stop_api.H | 141 ++-
 libpore/p9_stop_data_struct.H |   4 +-
 libpore/p9_stop_util.H|  27 +-
 15 files changed, 1214 insertions(+), 432 deletions(-)
 create mode 100644 doc/device-tree/ibm,opal/power-mgt/self-restore.rst
 create mode 100644 doc/device-tree/ibm,opal/power-mgt/self-save.rst
 create mode 100644 doc/opal-api/opal-slw-self-save-reg-181.rst

-- 
2.25.1

[PATCH v6 3/3] powerpc/powernv: Preference optimization for SPRs with constant values

[Pratik Rajesh Sampat]

There are SPRs whose values don't tend to change over time and invoking
self-save on them, where the values are gotten each time may turn out to
be inefficient. In that case calling a self-restore where passing the
value makes more sense as, if the value is same the memory location
is not updated.
SPRs that dont change are as follows:
SPRN_HSPRG0,
SPRN_LPCR,
SPRN_PTCR,
SPRN_HMEER,
SPRN_HID0,

There are also SPRs whose values change and/or their value may not be
correcty determinable in the kernel. Eg: MSR and PSSCR

The value of LPCR is dynamic based on if the CPU is entered a stop
state during cpu idle versus cpu hotplug.

Therefore in this optimization patch, introducing the concept of
preference for each SPR to choose from in the case both self-save and
self-restore is supported.

The preference bitmask is shown as below:

|... | 2nd pref | 1st pref |

MSB   LSB

The preference from higher to lower is from LSB to MSB with a shift of 8
bits.
Example:
Prefer self save first, if not available then prefer self
restore
The preference mask for this scenario will be seen as below.
((FIRMWARE_RESTORE << PREFERENCE_SHIFT) | FIRMWARE_SELF_SAVE)
-
|... | Self restore | Self save |
-
MSB LSB

Signed-off-by: Pratik Rajesh Sampat 
---
 arch/powerpc/platforms/powernv/idle.c | 88 +--
 1 file changed, 70 insertions(+), 18 deletions(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index e77b31621081..4d896df51582 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -43,6 +43,31 @@
 #define FIRMWARE_SELF_SAVE0x2
 #define KERNEL_SAVE_RESTORE   0x4
 
+#define NR_PREFERENCES2
+#define PREFERENCE_SHIFT  4
+#define PREFERENCE_MASK   0xf
+/*
+ * Bitmask defining the kind of preferences available.
+ * Note : The higher to lower preference is from LSB to MSB, with a shift of
+ * 4 bits.
+ * 
+ * || 2nd pref | 1st pref |
+ * 
+ * MSB   LSB
+ */
+/* Prefer Restore if available, otherwise unsupported */
+#define PREFER_SELF_RESTORE_ONLY   FIRMWARE_RESTORE
+/* Prefer Save if available, otherwise unsupported */
+#define PREFER_SELF_SAVE_ONLY  FIRMWARE_SELF_SAVE
+/* Prefer Restore when available, otherwise prefer Save */
+#define PREFER_RESTORE_SAVE((FIRMWARE_SELF_SAVE << \
+ PREFERENCE_SHIFT)\
+ | FIRMWARE_RESTORE)
+/* Prefer Save when available, otherwise prefer Restore*/
+#define PREFER_SAVE_RESTORE((FIRMWARE_RESTORE <<\
+ PREFERENCE_SHIFT)\
+ | FIRMWARE_SELF_SAVE)
+
 static u32 supported_cpuidle_states;
 struct pnv_idle_states_t *pnv_idle_states;
 int nr_pnv_idle_states;
@@ -52,6 +77,7 @@ static bool is_ptcr_self_save;
 
 struct preferred_sprs {
u64 spr;
+   u32 preferred_mode;
u32 supported_mode;
 };
 
@@ -66,42 +92,52 @@ struct preferred_sprs {
 struct preferred_sprs preferred_sprs[] = {
{
.spr = SPRN_HSPRG0,
+   .preferred_mode = PREFER_RESTORE_SAVE,
.supported_mode = FIRMWARE_RESTORE,
},
{
.spr = SPRN_LPCR,
+   .preferred_mode = PREFER_SAVE_RESTORE,
.supported_mode = FIRMWARE_RESTORE,
},
{
.spr = SPRN_PTCR,
+   .preferred_mode = PREFER_RESTORE_SAVE,
.supported_mode = KERNEL_SAVE_RESTORE,
},
{
.spr = SPRN_HMEER,
+   .preferred_mode = PREFER_RESTORE_SAVE,
.supported_mode = FIRMWARE_RESTORE,
},
{
.spr = SPRN_HID0,
+   .preferred_mode = PREFER_RESTORE_SAVE,
.supported_mode = FIRMWARE_RESTORE,
},
{
.spr = P9_STOP_SPR_MSR,
+   .preferred_mode = PREFER_SAVE_RESTORE,
.supported_mode = FIRMWARE_RESTORE,
},
{
.spr = P9_STOP_SPR_PSSCR,
+   .preferred_mode = PREFER_SAVE_RESTORE,
.supported_mode = FIRMWARE_RESTORE,
},
{
.spr = SPRN_HID1,
+   .preferred_mode = PREFER_RESTORE_SAVE,
.supported_mode = FIRMWARE_RESTORE,
},
{
.spr = SPRN_HID4,
+   .preferred_mode = PREFER_SELF_RESTORE_ONLY,
.supported_mode = FIRMWARE_RESTORE,
},
{
.spr = SPRN_HID5,
+   .preferred_mode = PREFER_SELF_RESTORE_ONLY,
.supported_mode = FIRMWARE_RESTORE,
}
 };
@@ -218,7 +2

[PATCH v6 2/3] powerpc/powernv: Introduce support and parsing for self-save API

[Pratik Rajesh Sampat]

This commit introduces and leverages the Self save API. The difference
between self-save and self-restore is that the value to be saved for the
SPR does not need to be passed to the call.

Add the new Self Save OPAL API call in the list of OPAL calls.
Implement the self saving of the SPRs based on the support populated.
This commit imposes the self-save over self-restore in case both are
supported for a particular SPR.

Along with support for self-save, kernel supported save restore is also
populated in the list. This property is only populated for those SPRs
which encapsulate support from the kernel and hav ethe possibility to
garner support from a firmware mode too.

In addition, the commit also parses the device tree for nodes self-save,
self-restore and populate support for the preferred SPRs based on what
was advertised by the device tree.

In the case a SPR is supported by the firmware self-save, self-restore
and kernel save restore then the preference of execution is also in the
same order as above.

Signed-off-by: Pratik Rajesh Sampat 
---
 .../bindings/powerpc/opal/power-mgt.txt   |  18 +++
 arch/powerpc/include/asm/opal-api.h   |   3 +-
 arch/powerpc/include/asm/opal.h   |   1 +
 arch/powerpc/platforms/powernv/idle.c | 131 +-
 arch/powerpc/platforms/powernv/opal-call.c|   1 +
 5 files changed, 146 insertions(+), 8 deletions(-)

diff --git a/Documentation/devicetree/bindings/powerpc/opal/power-mgt.txt 
b/Documentation/devicetree/bindings/powerpc/opal/power-mgt.txt
index 9d619e955576..5fb03c6d7de9 100644
--- a/Documentation/devicetree/bindings/powerpc/opal/power-mgt.txt
+++ b/Documentation/devicetree/bindings/powerpc/opal/power-mgt.txt
@@ -116,3 +116,21 @@ otherwise. The length of all the property arrays must be 
the same.
which of the fields of the PMICR are set in the corresponding
entries in ibm,cpu-idle-state-pmicr. This is an optional
property on POWER8 and is absent on POWER9.
+
+- self-restore:
+ Array of unsigned 64-bit values containing a property for sprn-mask
+ with each bit indicating the index of the supported SPR for the
+ functionality. This is an optional property for both Power8 and Power9
+
+- self-save:
+  Array of unsigned 64-bit values containing a property for sprn-mask
+  with each bit indicating the index of the supported SPR for the
+  functionality. This is an optional property for both Power8 and Power9
+
+Example of arrangement of self-restore and self-save arrays:
+For instance if PSSCR is supported, the value is 0x357 = 855.
+Since the array is of 64 bit values, the index of the array is determined by
+855 / 64 = 13th element. Within that index, the bit number is determined by
+855 % 64 = 23rd bit.
+This means that if the 23rd bit in array[13] is set, then that SPR is supported
+by the corresponding self-save or self-restore API.
diff --git a/arch/powerpc/include/asm/opal-api.h 
b/arch/powerpc/include/asm/opal-api.h
index c1f25a760eb1..1b6e1a68d431 100644
--- a/arch/powerpc/include/asm/opal-api.h
+++ b/arch/powerpc/include/asm/opal-api.h
@@ -214,7 +214,8 @@
 #define OPAL_SECVAR_GET176
 #define OPAL_SECVAR_GET_NEXT   177
 #define OPAL_SECVAR_ENQUEUE_UPDATE 178
-#define OPAL_LAST  178
+#define OPAL_SLW_SELF_SAVE_REG 181
+#define OPAL_LAST  181
 
 #define QUIESCE_HOLD   1 /* Spin all calls at entry */
 #define QUIESCE_REJECT 2 /* Fail all calls with OPAL_BUSY */
diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h
index 9986ac34b8e2..a370b0e8d899 100644
--- a/arch/powerpc/include/asm/opal.h
+++ b/arch/powerpc/include/asm/opal.h
@@ -204,6 +204,7 @@ int64_t opal_handle_hmi2(__be64 *out_flags);
 int64_t opal_register_dump_region(uint32_t id, uint64_t start, uint64_t end);
 int64_t opal_unregister_dump_region(uint32_t id);
 int64_t opal_slw_set_reg(uint64_t cpu_pir, uint64_t sprn, uint64_t val);
+int64_t opal_slw_self_save_reg(uint64_t cpu_pir, uint64_t sprn);
 int64_t opal_config_cpu_idle_state(uint64_t state, uint64_t flag);
 int64_t opal_pci_set_phb_cxl_mode(uint64_t phb_id, uint64_t mode, uint64_t 
pe_number);
 int64_t opal_pci_get_pbcq_tunnel_bar(uint64_t phb_id, uint64_t *addr);
diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 858ceb86394d..e77b31621081 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -35,13 +35,20 @@
 /*
  * Type of support for each SPR
  * FIRMWARE_RESTORE: firmware restoration supported: calls self-restore OPAL 
API
+ * FIRMWARE_SELF_SAVE: firmware save and restore: calls self-save OPAL API
+ * KERNEL_SAVE_RESTORE: kernel handles the saving and restoring of SPR
  */
 #define UNSUPPORTED   0x0
 #define FIRMWARE_RESTORE  0x1
+#define FIRMWARE_SELF_SAVE0x2
+#define

[PATCH v6 1/3] powerpc/powernv: Introduce interface for self-restore support

[Pratik Rajesh Sampat]

Introduces an interface that helps determine support for the
self-restore API. The commit is isomorphic to the original interface of
declaring SPRs to self-restore.

Signed-off-by: Pratik Rajesh Sampat 
---
 arch/powerpc/platforms/powernv/idle.c | 200 +++---
 1 file changed, 152 insertions(+), 48 deletions(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 78599bca66c2..858ceb86394d 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -32,10 +32,67 @@
 #define P9_STOP_SPR_MSR 2000
 #define P9_STOP_SPR_PSSCR  855
 
+/*
+ * Type of support for each SPR
+ * FIRMWARE_RESTORE: firmware restoration supported: calls self-restore OPAL 
API
+ */
+#define UNSUPPORTED   0x0
+#define FIRMWARE_RESTORE  0x1
+
 static u32 supported_cpuidle_states;
 struct pnv_idle_states_t *pnv_idle_states;
 int nr_pnv_idle_states;
 
+struct preferred_sprs {
+   u64 spr;
+   u32 supported_mode;
+};
+
+/*
+ * Supported mode: Default support. Can be overwritten during system
+ *initialization
+ */
+struct preferred_sprs preferred_sprs[] = {
+   {
+   .spr = SPRN_HSPRG0,
+   .supported_mode = FIRMWARE_RESTORE,
+   },
+   {
+   .spr = SPRN_LPCR,
+   .supported_mode = FIRMWARE_RESTORE,
+   },
+   {
+   .spr = SPRN_HMEER,
+   .supported_mode = FIRMWARE_RESTORE,
+   },
+   {
+   .spr = SPRN_HID0,
+   .supported_mode = FIRMWARE_RESTORE,
+   },
+   {
+   .spr = P9_STOP_SPR_MSR,
+   .supported_mode = FIRMWARE_RESTORE,
+   },
+   {
+   .spr = P9_STOP_SPR_PSSCR,
+   .supported_mode = FIRMWARE_RESTORE,
+   },
+   {
+   .spr = SPRN_HID1,
+   .supported_mode = FIRMWARE_RESTORE,
+   },
+   {
+   .spr = SPRN_HID4,
+   .supported_mode = FIRMWARE_RESTORE,
+   },
+   {
+   .spr = SPRN_HID5,
+   .supported_mode = FIRMWARE_RESTORE,
+   }
+};
+
+const int nr_preferred_sprs = ARRAY_SIZE(preferred_sprs);
+
 /*
  * The default stop state that will be used by ppc_md.power_save
  * function on platforms that support stop instruction.
@@ -61,78 +118,125 @@ static bool deepest_stop_found;
 
 static unsigned long power7_offline_type;
 
-static int pnv_save_sprs_for_deep_states(void)
+static int pnv_self_restore_sprs(u64 pir, int cpu, u64 spr)
 {
-   int cpu;
+   u64 reg_val;
int rc;
 
-   /*
-* hid0, hid1, hid4, hid5, hmeer and lpcr values are symmetric across
-* all cpus at boot. Get these reg values of current cpu and use the
-* same across all cpus.
-*/
-   uint64_t lpcr_val   = mfspr(SPRN_LPCR);
-   uint64_t hid0_val   = mfspr(SPRN_HID0);
-   uint64_t hid1_val   = mfspr(SPRN_HID1);
-   uint64_t hid4_val   = mfspr(SPRN_HID4);
-   uint64_t hid5_val   = mfspr(SPRN_HID5);
-   uint64_t hmeer_val  = mfspr(SPRN_HMEER);
-   uint64_t msr_val = MSR_IDLE;
-   uint64_t psscr_val = pnv_deepest_stop_psscr_val;
-
-   for_each_present_cpu(cpu) {
-   uint64_t pir = get_hard_smp_processor_id(cpu);
-   uint64_t hsprg0_val = (uint64_t)paca_ptrs[cpu];
-
-   rc = opal_slw_set_reg(pir, SPRN_HSPRG0, hsprg0_val);
+   switch (spr) {
+   case SPRN_HSPRG0:
+   reg_val = (uint64_t)paca_ptrs[cpu];
+   rc = opal_slw_set_reg(pir, SPRN_HSPRG0, reg_val);
if (rc != 0)
return rc;
-
-   rc = opal_slw_set_reg(pir, SPRN_LPCR, lpcr_val);
+   break;
+   case SPRN_LPCR:
+   reg_val = mfspr(SPRN_LPCR);
+   rc = opal_slw_set_reg(pir, SPRN_LPCR, reg_val);
if (rc != 0)
return rc;
-
+   break;
+   case P9_STOP_SPR_MSR:
+   reg_val = MSR_IDLE;
if (cpu_has_feature(CPU_FTR_ARCH_300)) {
-   rc = opal_slw_set_reg(pir, P9_STOP_SPR_MSR, msr_val);
+   rc = opal_slw_set_reg(pir, P9_STOP_SPR_MSR, reg_val);
if (rc)
return rc;
-
-   rc = opal_slw_set_reg(pir,
- P9_STOP_SPR_PSSCR, psscr_val);
-
+   }
+   break;
+   case P9_STOP_SPR_PSSCR:
+   reg_val = pnv_deepest_stop_psscr_val;
+   if (cpu_has_feature(CPU_FTR_ARCH_300)) {
+   rc = opal_slw_set_reg(pir, P9_STOP_SPR_PSSCR, reg_val);
if (rc)
return rc;
}
-
-   /* HIDs are per core registers */
+   break;
+   case SPRN_HMEER:
+   reg_val = mfspr

[PATCH v6 0/3] powerpc/powernv: Introduce interface for self-restore support

[Pratik Rajesh Sampat]

e is
restored.

Self-save and self-restore are complementary features since,
self-restore can help in restoring a different value in the SPR on
wakeup from a deep-idle state than what it had before entering the
deep idle state. This was used in POWER8 for HSPRG0 to distinguish a
wakeup from Winkle vs Fastsleep.

Limitations of self-save

Ideally all SPRs should be available for self-save, but HID0 is very
tricky to implement in microcode due to various endianess quirks.
Couple of implementation schemes were buggy and hence HID0 was left
out to be self-restore only.

The fallout of this limitation is as follows:

* In Non PEF environment, no issue. Linux will use self-restore for
  HID0 as it does today and no functional impact.

* In PEF environment, the HID0 restore value is decided by OPAL during
  boot and it is setup for LE hypervisor with radix MMU. This is the
  default and current working configuration of a PEF environment.
  However if there is a change, then HV Linux will try to change the
  HID0 value to something different than what OPAL decided, at which
  time deep-stop states will be disabled under this new PEF
  environment.

A simple and workable design is achieved by scoping the power
management deep-stop state support only to a known default PEF
environment. Any deviation will affect *only* deep stop-state support
(stop4,5) in that environment and not have any functional impediment
to the environment itself.

In future, if there is a need to support changing of HID0 to various
values under PEF environment and support deep-stop states, it can be
worked out via an ultravisor call or improve the microcode design to
include HID0 in self-save.  These future scheme would be an extension
and does not break or make the current implementation scheme
redundant.

Design Choices
==

Presenting the design choices in front of us:

Design-Choice 1:

A simple implementation is to just replace self-restore calls with
self-save as it is direct super-set.

Pros:
A simple design, quick to implement


Cons:
* Breaks backward compatibility. Self-restore has historically been
  supported in the firmware and an old firmware running on an new
  kernel will be incompatible and deep stop states will be cut.
* Furthermore, critical SPRs which need to be restored
  before 0x100 vector like HID0 are not supported by self-save.

Design-Choice 2:

Advertise both self-restore and self-save from OPAL including the set
of registers that each support. The kernel can then choose which API
to go with.
For the sake of simplicity, in case both modes are supported for an
SPR by default self-save would be called for it.

Pros:
* Backwards compatible

Cons:
Overhead in parsing device tree with the SPR list

Possible optimization with Approach2:
-
There are SPRs whose values don't tend to change over time and invoking
self-save on them, where the values are gotten each time may turn out to
be inefficient. In that case calling a self-restore where passing the
value makes more sense as, if the value is same, the memory location
is not updated.
SPRs that dont change are as follows:
SPRN_HSPRG0,
SPRN_LPCR,
SPRN_PTCR,
SPRN_HMEER,
SPRN_HID0,

The values of PSSCR and MSR change at runtime and hence, the kernel
cannot determine during boot time what their values will be before
entering a particular deep-stop state.

Therefore, a preference based interface is introduced for choosing
between self-save or self-restore between for each SPR.
The per-SPR preference is only a refinement of
approach 2 purely for performance reasons. It can be dropped if the
complexity is not deemed worth the returns.

Patches Organization

Design Choice 2 has been chosen as an implementation to demonstrate in
the patch series.

Patch1:
Devises an interface which lists all the interested SPRs, along with
highlighting the support of mode.
It is an isomorphic patch to replicate the functionality of the older
self-restore firmware for the new interface

Patch2:
Introduces the self-save API and leverages upon the struct interface to
add another supported mode in the mix of saving and restoring. It also
enforces that in case both modes are supported self-save is chosen over
self-restore

The commit also parses the device-tree and populate support for
self-save and self-restore in the supported mask

Patch3:
Introduce an optimization to allow preference to choose between one more
over the one when both both modes are supported. This optimization can
allow for better performance for the SPRs that don't change in value and
hence self-restore is a better alternative, and in cases when it is
known for values to change self-save is more convenient.

Pratik Rajesh Sampat (3):
  powerpc/powernv: Introduce interface for self-restore support
  powerpc/powernv: Introduce support and parsing for self-save API
  powerpc/powernv: Preference optimization for SPRs w

[PATCH v6 4/4] Advertise the self-save and self-restore attributes in the device tree

[Pratik Rajesh Sampat]

Support for self save and self restore interface is advertised in the
device tree, along with the list of SPRs it supports for each.

The Special Purpose Register identification is encoded in a 2048 bitmask
structure, where each bit signifies the identification key of that SPR
which is consistent with that of the POWER architecture set for that
register.

Signed-off-by: Pratik Rajesh Sampat 
---
 .../ibm,opal/power-mgt/self-restore.rst   |  27 
 .../ibm,opal/power-mgt/self-save.rst  |  27 
 hw/slw.c  | 116 ++
 include/skiboot.h |   1 +
 4 files changed, 171 insertions(+)
 create mode 100644 doc/device-tree/ibm,opal/power-mgt/self-restore.rst
 create mode 100644 doc/device-tree/ibm,opal/power-mgt/self-save.rst

diff --git a/doc/device-tree/ibm,opal/power-mgt/self-restore.rst 
b/doc/device-tree/ibm,opal/power-mgt/self-restore.rst
new file mode 100644
index ..2a2269f7
--- /dev/null
+++ b/doc/device-tree/ibm,opal/power-mgt/self-restore.rst
@@ -0,0 +1,27 @@
+ibm,opal/power-mgt/self-restore device tree entries
+===
+
+This node exports the bitmask representing the special purpose registers that
+the self-restore API currently supports.
+
+Example:
+
+.. code-block:: dts
+
+  self-restore {
+sprn-bitmask = <0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x4201 0x0 0x0
+0x2 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
+0x0 0x0 0x10 0x90 0x0 0x0 0x53 0x0 0x0 0x0
+0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
+0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
+0x0 0x1>;
+phandle = <0x1c7>;
+  };
+
+sprn-bitmask
+
+
+This property is a bitmask of of all the existing SPRs and if the SPR is
+supported, the corresponding bit of the SPR number is set to 1.
+The representation of the bits are left-right, i.e the MSB of the first
+doubleword represants the 0th bit.
diff --git a/doc/device-tree/ibm,opal/power-mgt/self-save.rst 
b/doc/device-tree/ibm,opal/power-mgt/self-save.rst
new file mode 100644
index ..c367720e
--- /dev/null
+++ b/doc/device-tree/ibm,opal/power-mgt/self-save.rst
@@ -0,0 +1,27 @@
+ibm,opal/power-mgt/self-save device tree entries
+===
+
+This node exports the bitmask representing the special purpose registers that
+the self-save API currently supports.
+
+Example:
+
+.. code-block:: dts
+
+  self-save {
+sprn-bitmask = <0x0 0x0 0x0 0x0 0x10 0x0 0x0 0x0 0x4201 0x0 0x0
+0x2 0x0 0x0 0x0 0x1 0x0 0x0 0x0 0x0 0x0 0x0 0x0
+0x0 0x0 0x0 0x10 0x84 0x0 0x0 0x0 0x0 0x0 0x0
+0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
+0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
+0x0 0x1>;
+phandle = <0x1c8>;
+  };
+
+sprn-bitmask
+
+
+This property is a bitmask of of all the existing SPRs and if the SPR is
+supported, the corresponding bit of the SPR number is set to 1.
+The representation of the bits are left-right, i.e the MSB of the first
+doubleword represants the 0th bit.
diff --git a/hw/slw.c b/hw/slw.c
index 6a09cc2c..9d1fe2c5 100644
--- a/hw/slw.c
+++ b/hw/slw.c
@@ -29,6 +29,7 @@
 #include 
 
 static uint32_t slw_saved_reset[0x100];
+#define SPR_BITMAP_LENGTH  2048
 
 static bool slw_current_le = false;
 
@@ -750,6 +751,119 @@ static void slw_late_init_p9(struct proc_chip *chip)
}
 }
 
+/* Add device tree properties to determine self-save | restore */
+void add_cpu_self_save_restore_properties(void)
+{
+   struct dt_node *self_restore, *self_save, *power_mgt;
+   uint64_t *self_save_mask, *self_restore_mask;
+   bool self_save_supported = true;
+   uint64_t compVector = -1;
+   struct proc_chip *chip;
+   int i, rc;
+
+   const uint64_t self_restore_regs[] = {
+   P8_SPR_HRMOR,
+   P8_SPR_HMEER,
+   P8_SPR_PMICR,
+   P8_SPR_PMCR,
+   P8_SPR_HID0,
+   P8_SPR_HID1,
+   P8_SPR_HID4,
+   P8_SPR_HID5,
+   P8_SPR_HSPRG0,
+   P8_SPR_LPCR,
+   P8_MSR_MSR
+   };
+
+   const uint64_t self_save_regs[] = {
+   P9_STOP_SPR_DAWR,
+   P9_STOP_SPR_HSPRG0,
+   P9_STOP_SPR_LDBAR,
+   P9_STOP_SPR_LPCR,
+   P9_STOP_SPR_PSSCR,
+   P9_STOP_SPR_MSR,
+   P9_STOP_SPR_HRMOR,
+   P9_STOP_SPR_HMEER,
+   P9_STOP_SPR_PMCR,
+   P9_STOP_SPR_PTCR
+   };
+
+   chip = next_chip(NULL);
+   assert(chip);
+   rc = proc_stop_api_discover_capability((

[PATCH v6 3/4] API to verify the STOP API and image compatibility

[Pratik Rajesh Sampat]

From: Prem Shanker Jha 

Commit defines a new API primarily intended for OPAL to determine
cpu register save API's compatibility with HOMER layout and
self save restore. It can help OPAL determine if version of
API integrated with OPAL is different from hostboot.

Change-Id: Ic0de45a336cfb8b6b6096a10ac1cd3ffbaa44fc0
Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/77612
Tested-by: FSP CI Jenkins 
Tested-by: Jenkins Server 
Tested-by: Hostboot CI 
Reviewed-by: RANGANATHPRASAD G. BRAHMASAMUDRA 
Reviewed-by: Gregory S Still 
Reviewed-by: Jennifer A Stofer 
Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/77614
Tested-by: Jenkins OP Build CI 
Tested-by: Jenkins OP HW 
Reviewed-by: Daniel M Crowell 
Signed-off-by: Pratik Rajesh Sampat 
---
 include/p9_stop_api.H| 26 +++
 libpore/p9_cpu_reg_restore_instruction.H |  7 ++-
 libpore/p9_hcd_memmap_base.H |  7 +++
 libpore/p9_stop_api.C| 58 +++-
 libpore/p9_stop_api.H| 26 ++-
 libpore/p9_stop_util.H   | 20 
 6 files changed, 131 insertions(+), 13 deletions(-)

diff --git a/include/p9_stop_api.H b/include/p9_stop_api.H
index c304f70f..09ce3dc1 100644
--- a/include/p9_stop_api.H
+++ b/include/p9_stop_api.H
@@ -110,6 +110,7 @@ typedef enum
 STOP_SAVE_FAIL   = 14,  // for internal failure within 
firmware.
 STOP_SAVE_SPR_ENTRY_MISSING  =  15,
 STOP_SAVE_SPR_BIT_POS_RESERVE=  16,
+STOP_SAVE_API_IMG_INCOMPATIBLE   =  18,
 } StopReturnCode_t;
 
 /**
@@ -164,6 +165,14 @@ typedef enum
 
 } ScomSection_t;
 
+/**
+ * @brief   versions pertaining relvant to STOP API.
+ */
+typedef enum
+{
+STOP_API_VER=   0x00,
+STOP_API_VER_CONTROL=   0x02,
+} VersionList_t;
 
 
 /**
@@ -195,6 +204,14 @@ typedef enum
 BIT_POS_USPRG1  =   30,
 } SprBitPositionList_t;
 
+typedef enum
+{
+SMF_SUPPORT_MISSING_IN_HOMER =   0x01,
+SELF_SUPPORT_MISSING_FOR_LE_HYP  =   0x02,
+IPL_RUNTIME_CPU_SAVE_VER_MISMATCH=   0x04,
+SELF_RESTORE_VER_MISMATCH=   0x08,
+} VersionIncompList_t;
+
 #ifdef __cplusplus
 extern "C" {
 #endif
@@ -247,6 +264,15 @@ StopReturnCode_t p9_stop_save_scom( void* const   i_pImage,
 StopReturnCode_t
 p9_stop_save_cpureg_control( void* i_pImage, const uint64_t i_pir,
  const uint32_t  i_saveRegVector );
+
+/**
+ * @brief   verifies if API is compatible of current HOMER image.
+ * @param[in]   i_pImagepoints to the start of HOMER image of P9 chip.
+ * @param[out]  o_inCompVector  list of incompatibilities found.
+ * @return  STOP_SAVE_SUCCESS if if API succeeds, error code otherwise.
+ */
+StopReturnCode_t proc_stop_api_discover_capability( void* const i_pImage, 
uint64_t* o_inCompVector );
+
 #ifdef __cplusplus
 } // extern "C"
 };  // namespace stopImageSection ends
diff --git a/libpore/p9_cpu_reg_restore_instruction.H 
b/libpore/p9_cpu_reg_restore_instruction.H
index d69a4212..5f168855 100644
--- a/libpore/p9_cpu_reg_restore_instruction.H
+++ b/libpore/p9_cpu_reg_restore_instruction.H
@@ -5,7 +5,7 @@
 /**/
 /* OpenPOWER HostBoot Project */
 /**/
-/* Contributors Listed Below - COPYRIGHT 2015,2018*/
+/* Contributors Listed Below - COPYRIGHT 2015,2020*/
 /* [+] International Business Machines Corp.  */
 /**/
 /**/
@@ -69,6 +69,11 @@ enum
 OPCODE_18   =   18,
 SELF_SAVE_FUNC_ADD  =   0x2300,
 SELF_SAVE_OFFSET=   0x180,
+SKIP_SPR_REST_INST  =   0x481c, //b . +0x01c
+MFLR_R30=   0x7fc802a6,
+SKIP_SPR_SELF_SAVE  =   0x3bff0020, //addi r31 r31, 0x20
+MTLR_INST   =   0x7fc803a6,  //mtlr r30
+BRANCH_BE_INST  =   0x4820,
 };
 
 #define MR_R0_TO_R100x7c0a0378UL //mr r10 r0
diff --git a/libpore/p9_hcd_memmap_base.H b/libpore/p9_hcd_memmap_base.H
index 000fafef..ddb56728 100644
--- a/libpore/p9_hcd_memmap_base.H
+++ b/libpore/p9_hcd_memmap_base.H
@@ -444,6 +444,13 @@ HCD_CONST(CME_QUAD_PSTATE_SIZE, HALF_KB)
 
 HCD_CONST(CME_REGION_SIZE,  (64 * ONE_KB))
 
+
+// HOMER compatibility
+
+HCD_CONST(STOP_API_CPU_SAVE_VER,0x02)
+HCD_CONST(SELF_SAVE_RESTORE_VER,0x02)
+HCD_CONST(SMF_SUPPORT_SIGNATURE_OFFSET, 0x1300)
+HCD_CONST(SMF_SELF_SIGNATURE,   (0x5f534d46))
 // Debug
 
 HCD_CONST(CPMR_TRACE_REGION_OFFSET, (512 * ONE_KB))
diff --git a/libpore/p9_stop_api.C b/libpore/p9_stop_

[PATCH v6 2/4] Self save API integration

[Pratik Rajesh Sampat]

The commit makes the self save API available outside the firmware by defining
an OPAL wrapper.
This wrapper has a similar interface to that of self restore and expects the
cpu pir, SPR number, minus the value of that SPR to be passed in its
paramters and returns OPAL_SUCCESS on success.
The commit also documents both the self-save and the self-restore API
calls along with their working and usage.

Signed-off-by: Pratik Rajesh Sampat 
---
 doc/opal-api/opal-slw-self-save-reg-181.rst | 49 
 doc/opal-api/opal-slw-set-reg-100.rst   |  5 ++
 doc/power-management.rst| 44 ++
 hw/slw.c| 89 +
 include/opal-api.h  |  3 +-
 include/p9_stop_api.H   | 17 
 include/skiboot.h   |  3 +
 7 files changed, 209 insertions(+), 1 deletion(-)
 create mode 100644 doc/opal-api/opal-slw-self-save-reg-181.rst

diff --git a/doc/opal-api/opal-slw-self-save-reg-181.rst 
b/doc/opal-api/opal-slw-self-save-reg-181.rst
new file mode 100644
index ..5aa4c930
--- /dev/null
+++ b/doc/opal-api/opal-slw-self-save-reg-181.rst
@@ -0,0 +1,49 @@
+.. OPAL_SLW_SELF_SAVE_REG:
+
+OPAL_SLW_SELF_SAVE_REG
+==
+
+.. code-block:: c
+
+   #define OPAL_SLW_SELF_SAVE_REG  181
+
+   int64_t opal_slw_self_save_reg(uint64_t cpu_pir, uint64_t sprn);
+
+:ref:`OPAL_SLW_SELF_SAVE_REG` is used to inform low-level firmware to save
+the current contents of the SPR before entering a state of loss and
+also restore the content back on waking up from a deep stop state.
+
+An OPAL call `OPAL_SLW_SET_REG` exists which is similar in function as
+saving and restoring the SPR, with one difference being that the value of the
+SPR must also be supplied in the parameters.
+Complete reference: doc/opal-api/opal-slw-set-reg-100.rst
+
+Parameters
+--
+
+``uint64_t cpu_pir``
+  This parameter specifies the pir of the cpu for which the call is being made.
+``uint64_t sprn``
+  This parameter specifies the spr number as mentioned in p9_stop_api.H
+  The list of SPRs supported is as follows. This list is suppiled through the
+  device tree:
+   P9_STOP_SPR_DAWR,
+   P9_STOP_SPR_HSPRG0,
+   P9_STOP_SPR_LDBAR,
+   P9_STOP_SPR_LPCR,
+   P9_STOP_SPR_PSSCR,
+   P9_STOP_SPR_MSR,
+   P9_STOP_SPR_HRMOR,
+   P9_STOP_SPR_HMEER,
+   P9_STOP_SPR_PMCR,
+   P9_STOP_SPR_PTCR
+
+Returns
+---
+
+:ref:`OPAL_UNSUPPORTED`
+  If spr restore is not supported by pore engine.
+:ref:`OPAL_PARAMETER`
+  Invalid handle for the pir/chip
+:ref:`OPAL_SUCCESS`
+  On success
diff --git a/doc/opal-api/opal-slw-set-reg-100.rst 
b/doc/opal-api/opal-slw-set-reg-100.rst
index 2e8f1bd6..ee3e68ce 100644
--- a/doc/opal-api/opal-slw-set-reg-100.rst
+++ b/doc/opal-api/opal-slw-set-reg-100.rst
@@ -21,6 +21,11 @@ In Power 9, it uses p9_stop_save_cpureg(), api provided by 
self restore code,
 to inform the spr with their corresponding values with which they
 must be restored.
 
+An OPAL call `OPAL_SLW_SELF_SAVE_REG` exists which is similar in function
+saving and restoring the SPR, with one difference being that the value of the
+SPR doesn't need to be passed in the parameters, only with the SPR number
+the firmware can identify, save and restore the values for the same.
+Complete reference: doc/opal-api/opal-slw-self-save-reg-181.rst
 
 Parameters
 --
diff --git a/doc/power-management.rst b/doc/power-management.rst
index 76491a71..992a18d0 100644
--- a/doc/power-management.rst
+++ b/doc/power-management.rst
@@ -15,3 +15,47 @@ On boot, specific stop states can be disabled via setting a 
mask. For example,
 to disable all but stop 0,1,2, use ~0xE000. ::
 
   nvram -p ibm,skiboot --update-config opal-stop-state-disable-mask=0x1FFF
+
+Saving and restoring Special Purpose Registers(SPRs)
+
+
+When a CPU wakes up from a deep stop state which can result in
+hypervisor state loss, all the SPRs are lost. The Linux Kernel expects
+a small set of SPRs to contain an expected value when the CPU wakes up
+from such a deep stop state. The microcode firmware provides the
+following two APIs, collectively known as the stop-APIs, to allow the
+kernel/OPAL to specify this set of SPRs and the value that they need
+to be restored with on waking up from a deep stop state.
+
+Self-restore:
+int64_t opal_slw_set_reg(uint64_t cpu_pir, uint64_t sprn, uint64_t val);
+The SPR number and the value of the that SPR must be restored with on
+wakeup from the deep-stop state must be specified. When this call is
+made, the microcode inserts instruction into the HOMER region to
+restore the content of the SPR to the specified value on wakeup from a
+deep-stop state. These instructions are executed by the CPU as soon as
+it wakes up from a deep stop state. The call is to be made once per
+SPR.
+
+Self-Save:
+int64_t

[PATCH v6 1/4] Self Save: Introducing Support for SPR Self Save

[Pratik Rajesh Sampat]

From: Prem Shanker Jha 

The commit is a merger of commits that makes the following changes:
1. Commit fixes some issues with code found during integration test
  -  replacement of addi with xor instruction during self save API.
  -  fixing instruction generation for MFMSR during self save
  -  data struct updates in STOP API
  -  error RC updates for hcode image build
  -  HOMER parser updates.
  -  removed self save support for URMOR and HRMOR
  -  code changes for compilation with OPAL
  -  populating CME Image header with unsecure HOMER address.

Key_Cronus_Test=PM_REGRESS

Change-Id: I7cedcc466267c4245255d8d75c01ed695e316720
Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/66580
Tested-by: FSP CI Jenkins 
Tested-by: HWSV CI 
Tested-by: PPE CI 
Tested-by: Jenkins Server 
Tested-by: Cronus HW CI 
Tested-by: Hostboot CI 
Reviewed-by: Gregory S. Still 
Reviewed-by: RAHUL BATRA 
Reviewed-by: Jennifer A. Stofer 
Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/66587
Reviewed-by: Christian R. Geddes 
Signed-off-by: Prem Shanker Jha 
Signed-off-by: Akshay Adiga 
Signed-off-by: Pratik Rajesh Sampat 

2. The commit also incorporates changes that make STOP API project
agnostic changes include defining wrapper functions which call legacy
API. It also adds duplicate enum members which start with prefix PROC
instead of P9.

Key_Cronus_Test=PM_REGRESS

Change-Id: If87970f3e8cf9b507f33eb1be249e03eb3836a5e
RTC: 201128
Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/71307
Tested-by: FSP CI Jenkins 
Tested-by: Jenkins Server 
Tested-by: Hostboot CI 
Tested-by: Cronus HW CI 
Reviewed-by: RANGANATHPRASAD G. BRAHMASAMUDRA 
Reviewed-by: Gregory S. Still 
Reviewed-by: Jennifer A Stofer 
Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/71314
Tested-by: Jenkins OP Build CI 
Tested-by: Jenkins OP HW 
Reviewed-by: Daniel M. Crowell 
Signed-off-by: Prem Shanker Jha 
Signed-off-by: Pratik Rajesh Sampat 
---
 include/p9_stop_api.H|  79 +-
 libpore/p9_cpu_reg_restore_instruction.H |   4 +
 libpore/p9_stop_api.C| 954 +--
 libpore/p9_stop_api.H| 115 ++-
 libpore/p9_stop_data_struct.H|   4 +-
 libpore/p9_stop_util.H   |   7 +-
 6 files changed, 721 insertions(+), 442 deletions(-)

diff --git a/include/p9_stop_api.H b/include/p9_stop_api.H
index 79abd000..9d3bc1e5 100644
--- a/include/p9_stop_api.H
+++ b/include/p9_stop_api.H
@@ -63,6 +63,26 @@ typedef enum
 P9_STOP_SPR_PMCR=884,   // core register
 P9_STOP_SPR_HID =   1008,   // core register
 P9_STOP_SPR_MSR =   2000,   // thread register
+
+//enum members which are project agnostic
+PROC_STOP_SPR_DAWR=180,   // thread register
+PROC_STOP_SPR_CIABR   =187,   // thread register
+PROC_STOP_SPR_DAWRX   =188,   // thread register
+PROC_STOP_SPR_HSPRG0  =304,   // thread register
+PROC_STOP_SPR_HRMOR   =313,   // core register
+PROC_STOP_SPR_LPCR=318,   // thread register
+PROC_STOP_SPR_HMEER   =337,   // core register
+PROC_STOP_SPR_PTCR=464,   // core register
+PROC_STOP_SPR_USPRG0  =496,   // thread register
+PROC_STOP_SPR_USPRG1  =497,   // thread register
+PROC_STOP_SPR_URMOR   =505,   // core register
+PROC_STOP_SPR_SMFCTRL =511,   // thread register
+PROC_STOP_SPR_LDBAR   =850,   // thread register
+PROC_STOP_SPR_PSSCR   =855,   // thread register
+PROC_STOP_SPR_PMCR=884,   // core register
+PROC_STOP_SPR_HID =   1008,   // core register
+PROC_STOP_SPR_MSR =   2000,   // thread register
+
 } CpuReg_t;
 
 /**
@@ -85,6 +105,8 @@ typedef enum
 STOP_SAVE_SCOM_ENTRY_UPDATE_FAILED   = 12,
 STOP_SAVE_INVALID_FUSED_CORE_STATUS  = 13,
 STOP_SAVE_FAIL   = 14,  // for internal failure within 
firmware.
+STOP_SAVE_SPR_ENTRY_MISSING  =  15,
+STOP_SAVE_SPR_BIT_POS_RESERVE=  16,
 } StopReturnCode_t;
 
 /**
@@ -101,7 +123,20 @@ typedef enum
 P9_STOP_SCOM_RESET  = 6,
 P9_STOP_SCOM_OR_APPEND  = 7,
 P9_STOP_SCOM_AND_APPEND = 8,
-P9_STOP_SCOM_OP_MAX = 9
+P9_STOP_SCOM_OP_MAX = 9,
+
+//enum members which are project agnostic
+PROC_STOP_SCOM_OP_MIN =   0,
+PROC_STOP_SCOM_APPEND =   1,
+PROC_STOP_SCOM_REPLACE=   2,
+PROC_STOP_SCOM_OR =   3,
+PROC_STOP_SCOM_AND=   4,
+PROC_STOP_SCOM_NOOP   =   5,
+PROC_STOP_SCOM_RESET  =   6,
+PROC_STOP_SCOM_OR_APPEND  =   7,
+PROC_STOP_SCOM_AND_APPEND =   8,
+PROC_STOP_SCOM_OP_MAX =   9,
+
 } ScomOperation_t;
 
 /**
@@ -114,9 +149,49 @@ typedef enum
 P9_STOP_SECTION_EQ_SCOM = 2,
 P9_STOP_SECTION_L2  = 3,
 P9_STOP_SECTION_L3  = 4,
-P9_STOP_SECTION_MAX = 5
+P9_STOP_SECTION_MAX = 5,
+
+//enum members which are project agnostic

[PATCH v6 0/4] Support for Self Save API in OPAL

[Pratik Rajesh Sampat]

R8 for HSPRG0 to distinguish a
wakeup from Winkle vs Fastsleep.

Limitations of self-save

Ideally all SPRs should be available for self-save, but HID0 is very
tricky to implement in microcode due to various endianess quirks.
Couple of implementation schemes were buggy and hence HID0 was left
out to be self-restore only.

The fallout of this limitation is as follows:

* In Non PEF environment, no issue. Linux will use self-restore for
  HID0 as it does today and no functional impact.

* In PEF environment, the HID0 restore value is decided by OPAL during
  boot and it is setup for LE hypervisor with radix MMU. This is the
  default and current working configuration of a PEF environment.
  However if there is a change, then HV Linux will try to change the
  HID0 value to something different than what OPAL decided, at which
  time deep-stop states will be disabled under this new PEF
  environment.

A simple and workable design is achieved by scoping the power
management deep-stop state support only to a known default PEF
environment. Any deviation will affect *only* deep stop-state support
(stop4,5) in that environment and not have any functional impediment
to the environment itself.

In future, if there is a need to support changing of HID0 to various
values under PEF environment and support deep-stop states, it can be
worked out via an ultravisor call or improve the microcode design to
include HID0 in self-save.  These future scheme would be an extension
and does not break or make the current implementation scheme
redundant.

Design Choices
==

Presenting the design choices in front of us:

Design-Choice 1:

Only expose one of self-save or self-restore for all the SPRs. Prefer
Self-save

Pros:
   - Simplifies the design heavily, since the Kernel can unambiguously
   make one API call for all the SPRs on discovering the presence of
   the API type.

Cons:
- Breaks backward compatibility if OPAL always chooses to expose
  only the self-save API as the older kernels assume the existence
  of self-restore.

- The set of SPRs supported by self-save and self-restore are not
  identical. Eg: HID0 is not supported by self-save API. PSSCR
  support via self-restore is not robust during special-wakeup.

- As discussed above, self-save and self-restore are
  complementary. Thus OPAL apriory choosing one over the other for
  all SPRs takes away the flexibility from the kernel.


Design-Choice 2:

Expose two arrays of SPRs: One set of SPRs that are supported by
self-save. Another set of SPRs supported by self-restore. These two
sets do not intersect. Further, if an SPR is supported by both
self-save and self-restore APIs, expose it only via self-save.

Pros:
 - For an SPR the choice for the kernel is unambiguous.

Cons:
- Breaks backward compatibility if OPAL always chooses to expose
  the legacy SPRs only via the self-save API as the older kernels
  assume the existence of self-restore.

- By making the decision early on, we take away the flexibility
   from the kernel to use an API of its choice for an SPR.


Design-Choice 3
---
Expose two arrays of SPRs. One set of SPRs that are supported by
self-save API. Another set of SPRs supported by self-restore API. Let
the kernel choose which API to invoke. Even if it wants to always
prefer self-save over self-restore, let that be kernel's choice.

Pros:
 - Keeps the design flexible to allow the kernel to take a
   decision based on its functional and performance requirements.
   Thus, the kernel for instance can make a choice to invoke
   self-restore API (when available) for SPRs whose values do not
   evolve at runtime, and invoke the self-save API (when
   available)
   for SPRs whose values will change during runtime.

 - Design is backward compatible with older kernels.

Cons:
 - The Kernel code will have additional complexity for parsing two
 lists of SPRs and making a choice w.r.t invocation of a specific
 stop-api.



Patches Organization

Design choice 3 has been chosen as an implementation to demonstrate in
this patch series.

Patch 1:
Commit adds support calling into the self save firmware API.
Also adds abstraction for making platform agnostic calls.

Patch 2:
Commit adds wrappers for the Self Save API for which an OPAL call can
be made.

Patch 3:
Commit adds API to determine the version of the STOP API. This helps
to identify support for self save in the firmware

Patch 4:
Commit adds device tree attributes to advertise self save and self
restore functionality along with the register set as a bitmask
currently supported in the firmware. It also uses the versioning API
to determine support for the self-save feature as a whole.

Pratik Rajesh Sampat (2):
  Self save API integration
  Advertise the self-save and self-restore attributes in the device tree

Prem Shanker

[PATCH v5 0/3] Introduce Self-Save API for deep stop states

[Pratik Rajesh Sampat]

Skiboot patches v4: 
https://lists.ozlabs.org/pipermail/skiboot/2020-February/016404.html
Linux patches v4: https://lkml.org/lkml/2020/2/12/446
Changelog
v4 --> v5
Based on Michael Ellerman's comments, in patch3:
1. Added documentation in power-mgt.txt for self-save and self-restore
2. Used bitmap abstractions while parsing device tree
3. Fixed scenario where zero or less SPRs were treated as a success
4. Removed redundant device_node pointer
5. Removed unnecessary pr_warn messages
6. Changed the old of_find_node_by_path calls to of_find_compatible_node
7. Fixed leaking reference of the parsed device_node pointer 

Currently the stop-API supports a mechanism called as self-restore
which allows us to restore the values of certain SPRs on wakeup from a
deep-stop state to a desired value. To use this, the Kernel makes an
OPAL call passing the PIR of the CPU, the SPR number and the value to
which the SPR should be restored when that CPU wakes up from a deep
stop state.

Recently, a new feature, named self-save has been enabled in the
stop-api, which is an alternative mechanism to do the same, except
that self-save will save the current content of the SPR before
entering a deep stop state and also restore the content back on
waking up from a deep stop state.

This patch series aims at introducing and leveraging the self-save feature in
the kernel.

Now, as the kernel has a choice to prefer one mode over the other and
there can be registers in both the save/restore SPR list which are sent
from the device tree, a new interface has been defined for the seamless
handing of the modes for each SPR.

A list of preferred SPRs are maintained in the kernel which contains two
properties:
1. supported_mode: Helps in identifying if it strictly supports self
   save or restore or both.
   Initialized using the information from device tree.
2. preferred_mode: Calls out what mode is preferred for each SPR. It
   could be strictly self save or restore, or it can also
   determine the preference of  mode over the other if both
   are present by encapsulating the other in bitmask from
   LSB to MSB.
   Initialized statically.

Below is a table to show the Scenario::Consequence when the self save and
self restore modes are available or disabled in different combinations as
perceived from the device tree thus giving complete backwards compatibly
regardless of an older firmware running a newer kernel or vise-versa.
Support for self save or self-restore is embedded in the device tree,
along with the set of registers it supports.

SR = Self restore; SS = Self save

.---..
| Scenario  |Consequence |
:---+:
| Legacy Firmware. No SS or SR node | Self restore is called for all |
|   | supported SPRs |
:---+:
| SR: !active SS: !active   | Deep stop states disabled  |
:---+:
| SR: active SS: !active| Self restore is called for all |
|   | supported SPRs |
:---+:
| SR: active SS: active | Goes through the preferences for each  |
|   | SPR and executes of the modes  |
|   | accordingly. Currently, Self restore is|
|   | called for all the SPRs except PSSCR   |
|   | which is self saved|
:---+:
| SR: active(only HID0) SS: active  | Self save called for all supported |
|   | registers expect HID0 (as HID0 cannot  |
|   | be self saved currently)   |
:---+:
| SR: !active SS: active| currently will disable deep states as  |
|   | HID0 is needed to be self restored and |
|   | cannot be self saved   |
'---''

Pratik Rajesh Sampat (3):
  powerpc/powernv: Interface to define support and preference for a SPR
  powerpc/powernv: Introduce Self save support
  powerpc/powernv: Parse device tree, population of SPR support

 .../bindings/powerpc/opal/power-mgt.txt   |  10 +
 arch/powerpc/include/asm/opal-api.h   |   3 +-
 a

[PATCH v5 3/3] powerpc/powernv: Parse device tree, population of SPR support

[Pratik Rajesh Sampat]

Parse the device tree for nodes self-save, self-restore and populate
support for the preferred SPRs based what was advertised by the device
tree.

Signed-off-by: Pratik Rajesh Sampat 
Reviewed-by: Ram Pai 
---
 .../bindings/powerpc/opal/power-mgt.txt   | 10 +++
 arch/powerpc/platforms/powernv/idle.c | 78 +++
 2 files changed, 88 insertions(+)

diff --git a/Documentation/devicetree/bindings/powerpc/opal/power-mgt.txt 
b/Documentation/devicetree/bindings/powerpc/opal/power-mgt.txt
index 9d619e955576..093cb5fe3d2d 100644
--- a/Documentation/devicetree/bindings/powerpc/opal/power-mgt.txt
+++ b/Documentation/devicetree/bindings/powerpc/opal/power-mgt.txt
@@ -116,3 +116,13 @@ otherwise. The length of all the property arrays must be 
the same.
which of the fields of the PMICR are set in the corresponding
entries in ibm,cpu-idle-state-pmicr. This is an optional
property on POWER8 and is absent on POWER9.
+
+- self-restore:
+ Array of unsigned 64-bit values containing a property for sprn-mask
+ with each bit indicating the index of the supported SPR for the
+ functionality. This is an optional property for both Power8 and Power9
+
+- self-save:
+  Array of unsigned 64-bit values containing a property for sprn-mask
+  with each bit indicating the index of the supported SPR for the
+  functionality. This is an optional property for both Power8 and Power9
diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 97aeb45e897b..c39111b338ff 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -1436,6 +1436,81 @@ static void __init pnv_probe_idle_states(void)
supported_cpuidle_states |= pnv_idle_states[i].flags;
 }
 
+/*
+ * Extracts and populates the self save or restore capabilities
+ * passed from the device tree node
+ */
+static int extract_save_restore_state_dt(struct device_node *np, int type)
+{
+   int nr_sprns = 0, i, bitmask_index;
+   u64 *temp_u64;
+   u64 bit_pos;
+
+   nr_sprns = of_property_count_u64_elems(np, "sprn-bitmask");
+   if (nr_sprns <= 0)
+   return -EINVAL;
+   temp_u64 = kcalloc(nr_sprns, sizeof(u64), GFP_KERNEL);
+   if (of_property_read_u64_array(np, "sprn-bitmask",
+  temp_u64, nr_sprns)) {
+   pr_warn("cpuidle-powernv: failed to find registers in DT\n");
+   kfree(temp_u64);
+   return -EINVAL;
+   }
+   /*
+* Populate acknowledgment of support for the sprs in the global vector
+* gotten by the registers supplied by the firmware.
+* The registers are in a bitmask, bit index within
+* that specifies the SPR
+*/
+   for (i = 0; i < nr_preferred_sprs; i++) {
+   bitmask_index = BIT_WORD(preferred_sprs[i].spr);
+   bit_pos = BIT_MASK(preferred_sprs[i].spr);
+   if ((temp_u64[bitmask_index] & bit_pos) == 0) {
+   if (type == SELF_RESTORE_TYPE)
+   preferred_sprs[i].supported_mode &=
+   ~SELF_RESTORE_STRICT;
+   else
+   preferred_sprs[i].supported_mode &=
+   ~SELF_SAVE_STRICT;
+   continue;
+   }
+   if (type == SELF_RESTORE_TYPE) {
+   preferred_sprs[i].supported_mode |=
+   SELF_RESTORE_STRICT;
+   } else {
+   preferred_sprs[i].supported_mode |=
+   SELF_SAVE_STRICT;
+   }
+   }
+
+   kfree(temp_u64);
+   return 0;
+}
+
+static int pnv_parse_deepstate_dt(void)
+{
+   struct device_node *np;
+   int rc = 0, i;
+
+   /* Self restore register population */
+   np = of_find_compatible_node(NULL, NULL, "ibm,opal-self-restore");
+   if (np) {
+   rc = extract_save_restore_state_dt(np, SELF_RESTORE_TYPE);
+   if (rc != 0)
+   return rc;
+   }
+   /* Self save register population */
+   np = of_find_compatible_node(NULL, NULL, "ibm,opal-self-save");
+   if (!np) {
+   for (i = 0; i < nr_preferred_sprs; i++)
+   preferred_sprs[i].supported_mode &= ~SELF_SAVE_STRICT;
+   } else {
+   rc = extract_save_restore_state_dt(np, SELF_SAVE_TYPE);
+   }
+   of_node_put(np);
+   return rc;
+}
+
 /*
  * This function parses device-tree and populates all the information
  * into pnv_idle_states structure. It also sets up nr_pnv_idle_states
@@ -1584,6 +1659,9 @@ static int __init pnv_init_idle_states(void)
return rc;
pnv_probe_idle_states();
 
+   rc = pnv_parse_deepstate_dt

[PATCH v5 2/3] powerpc/powernv: Introduce Self save support

[Pratik Rajesh Sampat]

This commit introduces and leverages the Self save API which OPAL now
supports.

Add the new Self Save OPAL API call in the list of OPAL calls.
Implement the self saving of the SPRs based on the support populated
while respecting it's preferences.

This implementation allows mixing of support for the SPRs, which
means that a SPR can be self restored while another SPR be self saved if
they support and prefer it to be so.

Signed-off-by: Pratik Rajesh Sampat 
Reviewed-by: Ram Pai 
---
 arch/powerpc/include/asm/opal-api.h|  3 ++-
 arch/powerpc/include/asm/opal.h|  1 +
 arch/powerpc/platforms/powernv/idle.c  | 22 ++
 arch/powerpc/platforms/powernv/opal-call.c |  1 +
 4 files changed, 26 insertions(+), 1 deletion(-)

diff --git a/arch/powerpc/include/asm/opal-api.h 
b/arch/powerpc/include/asm/opal-api.h
index c1f25a760eb1..1b6e1a68d431 100644
--- a/arch/powerpc/include/asm/opal-api.h
+++ b/arch/powerpc/include/asm/opal-api.h
@@ -214,7 +214,8 @@
 #define OPAL_SECVAR_GET176
 #define OPAL_SECVAR_GET_NEXT   177
 #define OPAL_SECVAR_ENQUEUE_UPDATE 178
-#define OPAL_LAST  178
+#define OPAL_SLW_SELF_SAVE_REG 181
+#define OPAL_LAST  181
 
 #define QUIESCE_HOLD   1 /* Spin all calls at entry */
 #define QUIESCE_REJECT 2 /* Fail all calls with OPAL_BUSY */
diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h
index 9986ac34b8e2..389a85b63805 100644
--- a/arch/powerpc/include/asm/opal.h
+++ b/arch/powerpc/include/asm/opal.h
@@ -203,6 +203,7 @@ int64_t opal_handle_hmi(void);
 int64_t opal_handle_hmi2(__be64 *out_flags);
 int64_t opal_register_dump_region(uint32_t id, uint64_t start, uint64_t end);
 int64_t opal_unregister_dump_region(uint32_t id);
+int64_t opal_slw_self_save_reg(uint64_t cpu_pir, uint64_t sprn);
 int64_t opal_slw_set_reg(uint64_t cpu_pir, uint64_t sprn, uint64_t val);
 int64_t opal_config_cpu_idle_state(uint64_t state, uint64_t flag);
 int64_t opal_pci_set_phb_cxl_mode(uint64_t phb_id, uint64_t mode, uint64_t 
pe_number);
diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 03fe835aadd1..97aeb45e897b 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -279,6 +279,26 @@ static int pnv_self_save_restore_sprs(void)
if (rc != 0)
return rc;
break;
+   } else if (preferred & curr_spr.supported_mode
+  & SELF_SAVE_STRICT) {
+   is_initialized = true;
+   if (curr_spr.spr == SPRN_HMEER &&
+   cpu_thread_in_core(cpu) != 0) {
+   continue;
+   }
+   rc = opal_slw_self_save_reg(pir,
+   curr_spr.spr);
+   if (rc != 0)
+   return rc;
+   switch (curr_spr.spr) {
+   case SPRN_LPCR:
+   is_lpcr_self_save = true;
+   break;
+   case SPRN_PTCR:
+   is_ptcr_self_save = true;
+   break;
+   }
+   break;
}
preferred_sprs[index].preferred_mode =
preferred_sprs[index].preferred_mode >>
@@ -1159,6 +1179,8 @@ void pnv_program_cpu_hotplug_lpcr(unsigned int cpu, u64 
lpcr_val)
if (!is_lpcr_self_save)
opal_slw_set_reg(pir, SPRN_LPCR,
 lpcr_val);
+   else
+   opal_slw_self_save_reg(pir, SPRN_LPCR);
}
 }
 
diff --git a/arch/powerpc/platforms/powernv/opal-call.c 
b/arch/powerpc/platforms/powernv/opal-call.c
index 5cd0f52d258f..11e0ceb90de0 100644
--- a/arch/powerpc/platforms/powernv/opal-call.c
+++ b/arch/powerpc/platforms/powernv/opal-call.c
@@ -223,6 +223,7 @@ OPAL_CALL(opal_handle_hmi,  
OPAL_HANDLE_HMI);
 OPAL_CALL(opal_handle_hmi2,OPAL_HANDLE_HMI2);
 OPAL_CALL(opal_config_cpu_idle_state,  OPAL_CONFIG_CPU_IDLE_STATE);
 OPAL_CALL(opal_slw_set_reg,OPAL_SLW_SET_REG);
+OPAL_CALL(

[PATCH v5 1/3] powerpc/powernv: Interface to define support and preference for a SPR

[Pratik Rajesh Sampat]

Define a bitmask interface to determine support for the Self Restore,
Self Save or both.

Also define an interface to determine the preference of that SPR to
be strictly saved or restored or encapsulated with an order of preference.

The preference bitmask is shown as below:

|... | 2nd pref | 1st pref |

MSB   LSB

The preference from higher to lower is from LSB to MSB with a shift of 8
bits.
Example:
Prefer self save first, if not available then prefer self
restore
The preference mask for this scenario will be seen as below.
((SELF_RESTORE_STRICT << PREFERENCE_SHIFT) | SELF_SAVE_STRICT)
-
|... | Self restore | Self save |
-
MSB LSB

Finally, declare a list of preferred SPRs which encapsulate the bitmaks
for preferred and supported with defaults of both being set to support
legacy firmware.

This commit also implements using the above interface and retains the
legacy functionality of self restore.

Signed-off-by: Pratik Rajesh Sampat 
Reviewed-by: Ram Pai 
---
 arch/powerpc/platforms/powernv/idle.c | 316 +-
 1 file changed, 259 insertions(+), 57 deletions(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 78599bca66c2..03fe835aadd1 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -32,9 +32,112 @@
 #define P9_STOP_SPR_MSR 2000
 #define P9_STOP_SPR_PSSCR  855
 
+/* Interface for the stop state supported and preference */
+#define SELF_RESTORE_TYPE0
+#define SELF_SAVE_TYPE   1
+
+#define NR_PREFERENCES2
+#define PREFERENCE_SHIFT  4
+#define PREFERENCE_MASK   0xf
+
+#define UNSUPPORTED 0x0
+#define SELF_RESTORE_STRICT 0x1
+#define SELF_SAVE_STRICT0x2
+
+/*
+ * Bitmask defining the kind of preferences available.
+ * Note : The higher to lower preference is from LSB to MSB, with a shift of
+ * 4 bits.
+ * 
+ * || 2nd pref | 1st pref |
+ * 
+ * MSB   LSB
+ */
+/* Prefer Restore if available, otherwise unsupported */
+#define PREFER_SELF_RESTORE_ONLY   SELF_RESTORE_STRICT
+/* Prefer Save if available, otherwise unsupported */
+#define PREFER_SELF_SAVE_ONLY  SELF_SAVE_STRICT
+/* Prefer Restore when available, otherwise prefer Save */
+#define PREFER_RESTORE_SAVE((SELF_SAVE_STRICT << \
+ PREFERENCE_SHIFT)\
+ | SELF_RESTORE_STRICT)
+/* Prefer Save when available, otherwise prefer Restore*/
+#define PREFER_SAVE_RESTORE((SELF_RESTORE_STRICT <<\
+ PREFERENCE_SHIFT)\
+ | SELF_SAVE_STRICT)
 static u32 supported_cpuidle_states;
 struct pnv_idle_states_t *pnv_idle_states;
 int nr_pnv_idle_states;
+/* Caching the lpcr & ptcr support to use later */
+static bool is_lpcr_self_save;
+static bool is_ptcr_self_save;
+
+struct preferred_sprs {
+   u64 spr;
+   u32 preferred_mode;
+   u32 supported_mode;
+};
+
+/*
+ * Preferred mode: Order of precedence when both self-save and self-restore
+ *supported
+ * Supported mode: Default support. Can be overwritten during system
+ *initialization
+ */
+struct preferred_sprs preferred_sprs[] = {
+   {
+   .spr = SPRN_HSPRG0,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_LPCR,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_PTCR,
+   .preferred_mode = PREFER_SAVE_RESTORE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HMEER,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HID0,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = P9_STOP_SPR_MSR,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = P9_STOP_SPR_PSSCR,
+   .preferred_mode = PREFER_SAVE_RESTORE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HID1,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HID4,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode

[PATCH] cpufreq: powernv: Fix frame-size-overflow in powernv_cpufreq_work_fn

[Pratik Rajesh Sampat]

The patch avoids allocating cpufreq_policy on stack hence fixing frame
size overflow in 'powernv_cpufreq_work_fn'

Fixes: 227942809b52 ("cpufreq: powernv: Restore cpu frequency to policy->cur on 
unthrottling")
Signed-off-by: Pratik Rajesh Sampat 
---
 drivers/cpufreq/powernv-cpufreq.c | 13 -
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/drivers/cpufreq/powernv-cpufreq.c 
b/drivers/cpufreq/powernv-cpufreq.c
index 56f4bc0d209e..20ee0661555a 100644
--- a/drivers/cpufreq/powernv-cpufreq.c
+++ b/drivers/cpufreq/powernv-cpufreq.c
@@ -902,6 +902,7 @@ static struct notifier_block powernv_cpufreq_reboot_nb = {
 void powernv_cpufreq_work_fn(struct work_struct *work)
 {
struct chip *chip = container_of(work, struct chip, throttle);
+   struct cpufreq_policy *policy;
unsigned int cpu;
cpumask_t mask;
 
@@ -916,12 +917,14 @@ void powernv_cpufreq_work_fn(struct work_struct *work)
chip->restore = false;
for_each_cpu(cpu, ) {
int index;
-   struct cpufreq_policy policy;
 
-   cpufreq_get_policy(, cpu);
-   index = cpufreq_table_find_index_c(, policy.cur);
-   powernv_cpufreq_target_index(, index);
-   cpumask_andnot(, , policy.cpus);
+   policy = cpufreq_cpu_get(cpu);
+   if (!policy)
+   continue;
+   index = cpufreq_table_find_index_c(policy, policy->cur);
+   powernv_cpufreq_target_index(policy, index);
+   cpumask_andnot(, , policy->cpus);
+   cpufreq_cpu_put(policy);
}
 out:
put_online_cpus();
-- 
2.24.1

[PATCH] Fixes: 227942809b52 ("cpufreq: powernv: Restore cpu frequency to policy->cur on unthrottling")

[Pratik Rajesh Sampat]

The patch avoids allocating cpufreq_policy on stack hence fixing frame
size overflow in 'powernv_cpufreq_work_fn'

Signed-off-by: Pratik Rajesh Sampat 
---
 drivers/cpufreq/powernv-cpufreq.c | 13 -
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/drivers/cpufreq/powernv-cpufreq.c 
b/drivers/cpufreq/powernv-cpufreq.c
index 56f4bc0d209e..20ee0661555a 100644
--- a/drivers/cpufreq/powernv-cpufreq.c
+++ b/drivers/cpufreq/powernv-cpufreq.c
@@ -902,6 +902,7 @@ static struct notifier_block powernv_cpufreq_reboot_nb = {
 void powernv_cpufreq_work_fn(struct work_struct *work)
 {
struct chip *chip = container_of(work, struct chip, throttle);
+   struct cpufreq_policy *policy;
unsigned int cpu;
cpumask_t mask;
 
@@ -916,12 +917,14 @@ void powernv_cpufreq_work_fn(struct work_struct *work)
chip->restore = false;
for_each_cpu(cpu, ) {
int index;
-   struct cpufreq_policy policy;
 
-   cpufreq_get_policy(, cpu);
-   index = cpufreq_table_find_index_c(, policy.cur);
-   powernv_cpufreq_target_index(, index);
-   cpumask_andnot(, , policy.cpus);
+   policy = cpufreq_cpu_get(cpu);
+   if (!policy)
+   continue;
+   index = cpufreq_table_find_index_c(policy, policy->cur);
+   powernv_cpufreq_target_index(policy, index);
+   cpumask_andnot(, , policy->cpus);
+   cpufreq_cpu_put(policy);
}
 out:
put_online_cpus();
-- 
2.17.1

[RFC 3/3] Introduce capability for firmware-enabled-stop

[Pratik Rajesh Sampat]

Design patch that introduces the capability for firmware to handle the
stop states instead. A bit is set based on the discovery of the feature
and correspondingly also the responsibility to handle the stop states.

The commit does not contain calling into the firmware to utilize
firmware enabled stop.

Signed-off-by: Pratik Rajesh Sampat 
---
 arch/powerpc/include/asm/processor.h | 1 +
 arch/powerpc/kernel/dt_cpu_ftrs.c| 9 +
 2 files changed, 10 insertions(+)

diff --git a/arch/powerpc/include/asm/processor.h 
b/arch/powerpc/include/asm/processor.h
index 277dbabafd02..978fab35d133 100644
--- a/arch/powerpc/include/asm/processor.h
+++ b/arch/powerpc/include/asm/processor.h
@@ -430,6 +430,7 @@ extern unsigned long cpuidle_disable;
 enum idle_boot_override {IDLE_NO_OVERRIDE = 0, IDLE_POWERSAVE_OFF};
 
 #define STOP_ENABLE0x0001
+#define FIRMWARE_STOP_ENABLE   0x0010
 
 #define STOP_VERSION_P9   0x1
 #define STOP_VERSION_P9_V10x2
diff --git a/arch/powerpc/kernel/dt_cpu_ftrs.c 
b/arch/powerpc/kernel/dt_cpu_ftrs.c
index 63e30aa49356..e00f8afabc46 100644
--- a/arch/powerpc/kernel/dt_cpu_ftrs.c
+++ b/arch/powerpc/kernel/dt_cpu_ftrs.c
@@ -313,6 +313,14 @@ static int __init feat_enable_idle_stop_quirk(struct 
dt_cpu_feature *f)
 
return 1;
 }
+
+static int __init feat_enable_firmware_stop(struct dt_cpu_feature *f)
+{
+   stop_dep.cpuidle_prop |= FIRMWARE_STOP_ENABLE;
+
+   return 1;
+}
+
 static int __init feat_enable_mmu_hash(struct dt_cpu_feature *f)
 {
u64 lpcr;
@@ -608,6 +616,7 @@ static struct dt_cpu_feature_match __initdata
{"alignment-interrupt-dsisr", feat_enable_align_dsisr, 0},
{"idle-stop", feat_enable_idle_stop, 0},
{"idle-stop-v1", feat_enable_idle_stop_quirk, 0},
+   {"firmware-stop-supported", feat_enable_firmware_stop, 0},
{"machine-check-power8", feat_enable_mce_power8, 0},
{"performance-monitor-power8", feat_enable_pmu_power8, 0},
{"data-stream-control-register", feat_enable_dscr, CPU_FTR_DSCR},
-- 
2.24.1

[RFC 2/3] Demonstration of handling an idle-stop quirk version

[Pratik Rajesh Sampat]

Concept patch demonstrating an idle-stop version discovery from the
device tree, along with population its support and versioning. It also
assigns the function pointer to handle any idle-stop specific quirks.

Signed-off-by: Pratik Rajesh Sampat 
---
 arch/powerpc/include/asm/processor.h  |  1 +
 arch/powerpc/kernel/dt_cpu_ftrs.c | 16 
 arch/powerpc/platforms/powernv/idle.c |  8 
 3 files changed, 25 insertions(+)

diff --git a/arch/powerpc/include/asm/processor.h 
b/arch/powerpc/include/asm/processor.h
index da59f01a5c09..277dbabafd02 100644
--- a/arch/powerpc/include/asm/processor.h
+++ b/arch/powerpc/include/asm/processor.h
@@ -432,6 +432,7 @@ enum idle_boot_override {IDLE_NO_OVERRIDE = 0, 
IDLE_POWERSAVE_OFF};
 #define STOP_ENABLE0x0001
 
 #define STOP_VERSION_P9   0x1
+#define STOP_VERSION_P9_V10x2
 
 /*
  * Classify the dependencies of the stop states
diff --git a/arch/powerpc/kernel/dt_cpu_ftrs.c 
b/arch/powerpc/kernel/dt_cpu_ftrs.c
index db1a525e090d..63e30aa49356 100644
--- a/arch/powerpc/kernel/dt_cpu_ftrs.c
+++ b/arch/powerpc/kernel/dt_cpu_ftrs.c
@@ -298,6 +298,21 @@ static int __init feat_enable_idle_stop(struct 
dt_cpu_feature *f)
return 1;
 }
 
+static int __init feat_enable_idle_stop_quirk(struct dt_cpu_feature *f)
+{
+   u64 lpcr;
+
+   /* Set PECE wakeup modes for ISAv3.0B */
+   lpcr = mfspr(SPRN_LPCR);
+   lpcr |=  LPCR_PECE0;
+   lpcr |=  LPCR_PECE1;
+   lpcr |=  LPCR_PECE2;
+   mtspr(SPRN_LPCR, lpcr);
+   stop_dep.cpuidle_prop |= STOP_ENABLE;
+   stop_dep.stop_version = STOP_VERSION_P9_V1;
+
+   return 1;
+}
 static int __init feat_enable_mmu_hash(struct dt_cpu_feature *f)
 {
u64 lpcr;
@@ -592,6 +607,7 @@ static struct dt_cpu_feature_match __initdata
{"idle-nap", feat_enable_idle_nap, 0},
{"alignment-interrupt-dsisr", feat_enable_align_dsisr, 0},
{"idle-stop", feat_enable_idle_stop, 0},
+   {"idle-stop-v1", feat_enable_idle_stop_quirk, 0},
{"machine-check-power8", feat_enable_mce_power8, 0},
{"performance-monitor-power8", feat_enable_pmu_power8, 0},
{"data-stream-control-register", feat_enable_dscr, CPU_FTR_DSCR},
diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index c32cdc37acf4..9f5b21da2fc5 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -805,6 +805,11 @@ static unsigned long power9_idle_stop(unsigned long psscr, 
bool mmu_on)
return srr1;
 }
 
+static unsigned long power9_idle_quirky_stop(unsigned long psscr, bool mmu_on)
+{
+   return power9_idle_stop(psscr, mmu_on);
+}
+
 #ifdef CONFIG_HOTPLUG_CPU
 static unsigned long power9_offline_stop(unsigned long psscr)
 {
@@ -1360,6 +1365,9 @@ static int __init pnv_init_idle_states(void)
case STOP_VERSION_P9:
stop_dep.idle_stop = power9_idle_stop;
break;
+   case STOP_VERSION_P9_V1:
+   stop_dep.idle_stop = power9_idle_quirky_stop;
+   break;
default:
stop_dep.idle_stop = NULL;
goto out;
-- 
2.24.1

[RFC 0/3] cpuidle/powernv: Interface to handle idle-stop versioning

[Pratik Rajesh Sampat]

A design patch series illuminates the idea of handling different
versions of idle-stop, the properties they support and the
quirks that need to be handled before entering or after exiting stop.

It also adds a functionality to identify firmware-enabled-stop and set
the according bits to encapsulate the support and corresponding handling

Corresponding RFC skiboot patch: 
https://lists.ozlabs.org/pipermail/skiboot/2020-March/016552.html

Pratik Rajesh Sampat (3):
  Interface for an idle-stop dependency structure
  Demonstration of handling an idle-stop quirk version
  Introduce capability for firmware-enabled-stop

 arch/powerpc/include/asm/processor.h  | 19 +
 arch/powerpc/kernel/dt_cpu_ftrs.c | 30 +++
 arch/powerpc/platforms/powernv/idle.c | 25 ++
 drivers/cpuidle/cpuidle-powernv.c |  3 ++-
 4 files changed, 72 insertions(+), 5 deletions(-)

-- 
2.24.1

[RFC 1/3] Interface for an idle-stop dependency structure

[Pratik Rajesh Sampat]

Design patch to introduce the idea of having a dependency structure for
idle-stop. The structure encapsulates the following:
1. Bitmask for version of idle-stop
2. Bitmask for propterties like ENABLE/DISABLE
3. Function pointer which helps handle how the stop must be invoked

The commit lays a foundation for other idle-stop versions to be added
and handled cleanly based on their specified requirments.
Currently it handles the existing "idle-stop" version by setting the
discovery bits and the function pointer.

Signed-off-by: Pratik Rajesh Sampat 
---
 arch/powerpc/include/asm/processor.h  | 17 +
 arch/powerpc/kernel/dt_cpu_ftrs.c |  5 +
 arch/powerpc/platforms/powernv/idle.c | 17 +
 drivers/cpuidle/cpuidle-powernv.c |  3 ++-
 4 files changed, 37 insertions(+), 5 deletions(-)

diff --git a/arch/powerpc/include/asm/processor.h 
b/arch/powerpc/include/asm/processor.h
index eedcbfb9a6ff..da59f01a5c09 100644
--- a/arch/powerpc/include/asm/processor.h
+++ b/arch/powerpc/include/asm/processor.h
@@ -429,6 +429,23 @@ extern void power4_idle_nap(void);
 extern unsigned long cpuidle_disable;
 enum idle_boot_override {IDLE_NO_OVERRIDE = 0, IDLE_POWERSAVE_OFF};
 
+#define STOP_ENABLE0x0001
+
+#define STOP_VERSION_P9   0x1
+
+/*
+ * Classify the dependencies of the stop states
+ * @idle_stop: function handler to handle the quirk stop version
+ * @cpuidle_prop: Signify support for stop states through kernel and/or 
firmware
+ * @stop_version: Classify quirk versions for stop states
+ */
+typedef struct {
+   unsigned long (*idle_stop)(unsigned long, bool);
+   uint8_t cpuidle_prop;
+   uint8_t stop_version;
+}stop_deps_t;
+extern stop_deps_t stop_dep;
+
 extern int powersave_nap;  /* set if nap mode can be used in idle loop */
 
 extern void power7_idle_type(unsigned long type);
diff --git a/arch/powerpc/kernel/dt_cpu_ftrs.c 
b/arch/powerpc/kernel/dt_cpu_ftrs.c
index 182b4047c1ef..db1a525e090d 100644
--- a/arch/powerpc/kernel/dt_cpu_ftrs.c
+++ b/arch/powerpc/kernel/dt_cpu_ftrs.c
@@ -292,6 +292,8 @@ static int __init feat_enable_idle_stop(struct 
dt_cpu_feature *f)
lpcr |=  LPCR_PECE1;
lpcr |=  LPCR_PECE2;
mtspr(SPRN_LPCR, lpcr);
+   stop_dep.cpuidle_prop |= STOP_ENABLE;
+   stop_dep.stop_version = STOP_VERSION_P9;
 
return 1;
 }
@@ -657,6 +659,9 @@ static void __init cpufeatures_setup_start(u32 isa)
}
 }
 
+stop_deps_t stop_dep = {NULL, 0x0, 0x0};
+EXPORT_SYMBOL(stop_dep);
+
 static bool __init cpufeatures_process_feature(struct dt_cpu_feature *f)
 {
const struct dt_cpu_feature_match *m;
diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 78599bca66c2..c32cdc37acf4 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -812,7 +812,7 @@ static unsigned long power9_offline_stop(unsigned long 
psscr)
 
 #ifndef CONFIG_KVM_BOOK3S_HV_POSSIBLE
__ppc64_runlatch_off();
-   srr1 = power9_idle_stop(psscr, true);
+   srr1 = stop_dep.idle_stop(psscr, true);
__ppc64_runlatch_on();
 #else
/*
@@ -828,7 +828,7 @@ static unsigned long power9_offline_stop(unsigned long 
psscr)
local_paca->kvm_hstate.hwthread_state = KVM_HWTHREAD_IN_IDLE;
 
__ppc64_runlatch_off();
-   srr1 = power9_idle_stop(psscr, false);
+   srr1 = stop_dep.idle_stop(psscr, true);
__ppc64_runlatch_on();
 
local_paca->kvm_hstate.hwthread_state = KVM_HWTHREAD_IN_KERNEL;
@@ -856,7 +856,7 @@ void power9_idle_type(unsigned long stop_psscr_val,
psscr = (psscr & ~stop_psscr_mask) | stop_psscr_val;
 
__ppc64_runlatch_off();
-   srr1 = power9_idle_stop(psscr, true);
+   srr1 = stop_dep.idle_stop(psscr, true);
__ppc64_runlatch_on();
 
fini_irq_for_idle_irqsoff();
@@ -1353,8 +1353,17 @@ static int __init pnv_init_idle_states(void)
nr_pnv_idle_states = 0;
supported_cpuidle_states = 0;
 
-   if (cpuidle_disable != IDLE_NO_OVERRIDE)
+   if (cpuidle_disable != IDLE_NO_OVERRIDE ||
+   !(stop_dep.cpuidle_prop & STOP_ENABLE))
goto out;
+   switch(stop_dep.stop_version) {
+   case STOP_VERSION_P9:
+   stop_dep.idle_stop = power9_idle_stop;
+   break;
+   default:
+   stop_dep.idle_stop = NULL;
+   goto out;
+   }
rc = pnv_parse_cpuidle_dt();
if (rc)
return rc;
diff --git a/drivers/cpuidle/cpuidle-powernv.c 
b/drivers/cpuidle/cpuidle-powernv.c
index 1b299e801f74..7430a8edf5c9 100644
--- a/drivers/cpuidle/cpuidle-powernv.c
+++ b/drivers/cpuidle/cpuidle-powernv.c
@@ -371,7 +371,8 @@ static int powernv_add_idle_states(void)
  */
 static int powernv_idle_probe(void)
 {
-   if (cpuidle_disable != IDLE_NO_OVERRIDE)
+   if (cpuidle_disable != IDLE_NO_OVERRIDE ||
+   !(stop_dep.cp

[RFC] Support stop state version quirk and firmware enabled stop

[Pratik Rajesh Sampat]

A concept patch in Skiboot to illustrate the case wherein handling of
stop states for different DD versions of a CPU can be achieved by a
simple modification in the list of cpu_features.
As an example idle-stop1 is defined which uses P9_CPU_DD1 to define the
cpu feature.

Along with that, an implementation is being worked upon the LE OPAL
series which helps OPAL handle the stop state entry and exit.

This patch advertises this capability of the firmware which can be
availed if the quirk-version-setting is not cognizable.

The firmware-enabled stop is being worked by Abhishek Goel
 building upon the LE OPAL series.

Signed-off-by: Pratik Rajesh Sampat 
---
 core/cpufeatures.c | 22 ++
 1 file changed, 22 insertions(+)

diff --git a/core/cpufeatures.c b/core/cpufeatures.c
index ec30c975..b9875e7b 100644
--- a/core/cpufeatures.c
+++ b/core/cpufeatures.c
@@ -510,6 +510,25 @@ static const struct cpu_feature cpu_features_table[] = {
-1, -1, -1,
NULL, },
 
+   /*
+* QUIRK for ISAv3.0B stop idle instructions and registers
+* Helps us determine if there are any quirks
+* XXX: Same of idle-stop
+*/
+   { "idle-stop-v1",
+   CPU_P9_DD1,
+   ISA_V3_0B, USABLE_HV|USABLE_OS,
+   HV_CUSTOM, OS_CUSTOM,
+   -1, -1, -1,
+   NULL, },
+
+   { "firmware-stop-supported",
+   CPU_P9,
+   ISA_V3_0B, USABLE_HV|USABLE_OS,
+   HV_CUSTOM, OS_CUSTOM,
+   -1, -1, -1,
+   NULL, },
+
/*
 * ISAv3.0B Hypervisor Virtualization Interrupt
 * Also associated system registers, LPCR EE, HEIC, HVICE,
@@ -883,6 +902,9 @@ static void add_cpufeatures(struct dt_node *cpus,
const struct cpu_feature *f = _features_table[i];
 
if (f->cpus_supported & cpu_feature_cpu) {
+   if (!strcmp(f->name, "firmware-stop-supported") &&
+   HAVE_BIG_ENDIAN)
+   continue;
DBG("  '%s'\n", f->name);
add_cpu_feature_nodeps(features, f);
}
-- 
2.24.1

[PATCH v4 1/3] powerpc/powernv: Interface to define support and preference for a SPR

[Pratik Rajesh Sampat]

Define a bitmask interface to determine support for the Self Restore,
Self Save or both.

Also define an interface to determine the preference of that SPR to
be strictly saved or restored or encapsulated with an order of preference.

The preference bitmask is shown as below:

|... | 2nd pref | 1st pref |

MSB   LSB

The preference from higher to lower is from LSB to MSB with a shift of 8
bits.
Example:
Prefer self save first, if not available then prefer self
restore
The preference mask for this scenario will be seen as below.
((SELF_RESTORE_STRICT << PREFERENCE_SHIFT) | SELF_SAVE_STRICT)
-
|... | Self restore | Self save |
-
MSB LSB

Finally, declare a list of preferred SPRs which encapsulate the bitmaks
for preferred and supported with defaults of both being set to support
legacy firmware.

This commit also implements using the above interface and retains the
legacy functionality of self restore.

Signed-off-by: Pratik Rajesh Sampat 
Reviewed-by: Ram Pai 
---
 arch/powerpc/platforms/powernv/idle.c | 316 +-
 1 file changed, 259 insertions(+), 57 deletions(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 78599bca66c2..03fe835aadd1 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -32,9 +32,112 @@
 #define P9_STOP_SPR_MSR 2000
 #define P9_STOP_SPR_PSSCR  855
 
+/* Interface for the stop state supported and preference */
+#define SELF_RESTORE_TYPE0
+#define SELF_SAVE_TYPE   1
+
+#define NR_PREFERENCES2
+#define PREFERENCE_SHIFT  4
+#define PREFERENCE_MASK   0xf
+
+#define UNSUPPORTED 0x0
+#define SELF_RESTORE_STRICT 0x1
+#define SELF_SAVE_STRICT0x2
+
+/*
+ * Bitmask defining the kind of preferences available.
+ * Note : The higher to lower preference is from LSB to MSB, with a shift of
+ * 4 bits.
+ * 
+ * || 2nd pref | 1st pref |
+ * 
+ * MSB   LSB
+ */
+/* Prefer Restore if available, otherwise unsupported */
+#define PREFER_SELF_RESTORE_ONLY   SELF_RESTORE_STRICT
+/* Prefer Save if available, otherwise unsupported */
+#define PREFER_SELF_SAVE_ONLY  SELF_SAVE_STRICT
+/* Prefer Restore when available, otherwise prefer Save */
+#define PREFER_RESTORE_SAVE((SELF_SAVE_STRICT << \
+ PREFERENCE_SHIFT)\
+ | SELF_RESTORE_STRICT)
+/* Prefer Save when available, otherwise prefer Restore*/
+#define PREFER_SAVE_RESTORE((SELF_RESTORE_STRICT <<\
+ PREFERENCE_SHIFT)\
+ | SELF_SAVE_STRICT)
 static u32 supported_cpuidle_states;
 struct pnv_idle_states_t *pnv_idle_states;
 int nr_pnv_idle_states;
+/* Caching the lpcr & ptcr support to use later */
+static bool is_lpcr_self_save;
+static bool is_ptcr_self_save;
+
+struct preferred_sprs {
+   u64 spr;
+   u32 preferred_mode;
+   u32 supported_mode;
+};
+
+/*
+ * Preferred mode: Order of precedence when both self-save and self-restore
+ *supported
+ * Supported mode: Default support. Can be overwritten during system
+ *initialization
+ */
+struct preferred_sprs preferred_sprs[] = {
+   {
+   .spr = SPRN_HSPRG0,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_LPCR,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_PTCR,
+   .preferred_mode = PREFER_SAVE_RESTORE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HMEER,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HID0,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = P9_STOP_SPR_MSR,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = P9_STOP_SPR_PSSCR,
+   .preferred_mode = PREFER_SAVE_RESTORE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HID1,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HID4,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode

[PATCH v4 2/3] powerpc/powernv: Introduce Self save support

[Pratik Rajesh Sampat]

This commit introduces and leverages the Self save API which OPAL now
supports.

Add the new Self Save OPAL API call in the list of OPAL calls.
Implement the self saving of the SPRs based on the support populated
while respecting it's preferences.

This implementation allows mixing of support for the SPRs, which
means that a SPR can be self restored while another SPR be self saved if
they support and prefer it to be so.

Signed-off-by: Pratik Rajesh Sampat 
Reviewed-by: Ram Pai 
---
 arch/powerpc/include/asm/opal-api.h|  3 ++-
 arch/powerpc/include/asm/opal.h|  1 +
 arch/powerpc/platforms/powernv/idle.c  | 22 ++
 arch/powerpc/platforms/powernv/opal-call.c |  1 +
 4 files changed, 26 insertions(+), 1 deletion(-)

diff --git a/arch/powerpc/include/asm/opal-api.h 
b/arch/powerpc/include/asm/opal-api.h
index c1f25a760eb1..1b6e1a68d431 100644
--- a/arch/powerpc/include/asm/opal-api.h
+++ b/arch/powerpc/include/asm/opal-api.h
@@ -214,7 +214,8 @@
 #define OPAL_SECVAR_GET176
 #define OPAL_SECVAR_GET_NEXT   177
 #define OPAL_SECVAR_ENQUEUE_UPDATE 178
-#define OPAL_LAST  178
+#define OPAL_SLW_SELF_SAVE_REG 181
+#define OPAL_LAST  181
 
 #define QUIESCE_HOLD   1 /* Spin all calls at entry */
 #define QUIESCE_REJECT 2 /* Fail all calls with OPAL_BUSY */
diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h
index 9986ac34b8e2..389a85b63805 100644
--- a/arch/powerpc/include/asm/opal.h
+++ b/arch/powerpc/include/asm/opal.h
@@ -203,6 +203,7 @@ int64_t opal_handle_hmi(void);
 int64_t opal_handle_hmi2(__be64 *out_flags);
 int64_t opal_register_dump_region(uint32_t id, uint64_t start, uint64_t end);
 int64_t opal_unregister_dump_region(uint32_t id);
+int64_t opal_slw_self_save_reg(uint64_t cpu_pir, uint64_t sprn);
 int64_t opal_slw_set_reg(uint64_t cpu_pir, uint64_t sprn, uint64_t val);
 int64_t opal_config_cpu_idle_state(uint64_t state, uint64_t flag);
 int64_t opal_pci_set_phb_cxl_mode(uint64_t phb_id, uint64_t mode, uint64_t 
pe_number);
diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 03fe835aadd1..97aeb45e897b 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -279,6 +279,26 @@ static int pnv_self_save_restore_sprs(void)
if (rc != 0)
return rc;
break;
+   } else if (preferred & curr_spr.supported_mode
+  & SELF_SAVE_STRICT) {
+   is_initialized = true;
+   if (curr_spr.spr == SPRN_HMEER &&
+   cpu_thread_in_core(cpu) != 0) {
+   continue;
+   }
+   rc = opal_slw_self_save_reg(pir,
+   curr_spr.spr);
+   if (rc != 0)
+   return rc;
+   switch (curr_spr.spr) {
+   case SPRN_LPCR:
+   is_lpcr_self_save = true;
+   break;
+   case SPRN_PTCR:
+   is_ptcr_self_save = true;
+   break;
+   }
+   break;
}
preferred_sprs[index].preferred_mode =
preferred_sprs[index].preferred_mode >>
@@ -1159,6 +1179,8 @@ void pnv_program_cpu_hotplug_lpcr(unsigned int cpu, u64 
lpcr_val)
if (!is_lpcr_self_save)
opal_slw_set_reg(pir, SPRN_LPCR,
 lpcr_val);
+   else
+   opal_slw_self_save_reg(pir, SPRN_LPCR);
}
 }
 
diff --git a/arch/powerpc/platforms/powernv/opal-call.c 
b/arch/powerpc/platforms/powernv/opal-call.c
index 5cd0f52d258f..11e0ceb90de0 100644
--- a/arch/powerpc/platforms/powernv/opal-call.c
+++ b/arch/powerpc/platforms/powernv/opal-call.c
@@ -223,6 +223,7 @@ OPAL_CALL(opal_handle_hmi,  
OPAL_HANDLE_HMI);
 OPAL_CALL(opal_handle_hmi2,OPAL_HANDLE_HMI2);
 OPAL_CALL(opal_config_cpu_idle_state,  OPAL_CONFIG_CPU_IDLE_STATE);
 OPAL_CALL(opal_slw_set_reg,OPAL_SLW_SET_REG);
+OPAL_CALL(

[PATCH v4 3/3] powerpc/powernv: Parse device tree, population of SPR support

[Pratik Rajesh Sampat]

Parse the device tree for nodes self-save, self-restore and populate
support for the preferred SPRs based what was advertised by the device
tree.

Signed-off-by: Pratik Rajesh Sampat 
Reviewed-by: Ram Pai 
---
 arch/powerpc/platforms/powernv/idle.c | 82 +++
 1 file changed, 82 insertions(+)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 97aeb45e897b..27dfadf609e8 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -1436,6 +1436,85 @@ static void __init pnv_probe_idle_states(void)
supported_cpuidle_states |= pnv_idle_states[i].flags;
 }
 
+/*
+ * Extracts and populates the self save or restore capabilities
+ * passed from the device tree node
+ */
+static int extract_save_restore_state_dt(struct device_node *np, int type)
+{
+   int nr_sprns = 0, i, bitmask_index;
+   int rc = 0;
+   u64 *temp_u64;
+   u64 bit_pos;
+
+   nr_sprns = of_property_count_u64_elems(np, "sprn-bitmask");
+   if (nr_sprns <= 0)
+   return rc;
+   temp_u64 = kcalloc(nr_sprns, sizeof(u64), GFP_KERNEL);
+   if (of_property_read_u64_array(np, "sprn-bitmask",
+  temp_u64, nr_sprns)) {
+   pr_warn("cpuidle-powernv: failed to find registers in DT\n");
+   kfree(temp_u64);
+   return -EINVAL;
+   }
+   /*
+* Populate acknowledgment of support for the sprs in the global vector
+* gotten by the registers supplied by the firmware.
+* The registers are in a bitmask, bit index within
+* that specifies the SPR
+*/
+   for (i = 0; i < nr_preferred_sprs; i++) {
+   bitmask_index = preferred_sprs[i].spr / 64;
+   bit_pos = preferred_sprs[i].spr % 64;
+   if ((temp_u64[bitmask_index] & (1UL << bit_pos)) == 0) {
+   if (type == SELF_RESTORE_TYPE)
+   preferred_sprs[i].supported_mode &=
+   ~SELF_RESTORE_STRICT;
+   else
+   preferred_sprs[i].supported_mode &=
+   ~SELF_SAVE_STRICT;
+   continue;
+   }
+   if (type == SELF_RESTORE_TYPE) {
+   preferred_sprs[i].supported_mode |=
+   SELF_RESTORE_STRICT;
+   } else {
+   preferred_sprs[i].supported_mode |=
+   SELF_SAVE_STRICT;
+   }
+   }
+
+   kfree(temp_u64);
+   return rc;
+}
+
+static int pnv_parse_deepstate_dt(void)
+{
+   struct device_node *sr_np, *ss_np;
+   int rc = 0, i;
+
+   /* Self restore register population */
+   sr_np = of_find_node_by_path("/ibm,opal/power-mgt/self-restore");
+   if (!sr_np) {
+   pr_warn("opal: self restore Node not found");
+   } else {
+   rc = extract_save_restore_state_dt(sr_np, SELF_RESTORE_TYPE);
+   if (rc != 0)
+   return rc;
+   }
+   /* Self save register population */
+   ss_np = of_find_node_by_path("/ibm,opal/power-mgt/self-save");
+   if (!ss_np) {
+   pr_warn("opal: self save Node not found");
+   pr_warn("Legacy firmware. Assuming default self-restore 
support");
+   for (i = 0; i < nr_preferred_sprs; i++)
+   preferred_sprs[i].supported_mode &= ~SELF_SAVE_STRICT;
+   } else {
+   rc = extract_save_restore_state_dt(ss_np, SELF_SAVE_TYPE);
+   }
+   return rc;
+}
+
 /*
  * This function parses device-tree and populates all the information
  * into pnv_idle_states structure. It also sets up nr_pnv_idle_states
@@ -1584,6 +1663,9 @@ static int __init pnv_init_idle_states(void)
return rc;
pnv_probe_idle_states();
 
+   rc = pnv_parse_deepstate_dt();
+   if (rc)
+   return rc;
if (!cpu_has_feature(CPU_FTR_ARCH_300)) {
if (!(supported_cpuidle_states & OPAL_PM_SLEEP_ENABLED_ER1)) {
power7_fastsleep_workaround_entry = false;
-- 
2.17.1

[PATCH v4 0/3] Introduce Self-Save API for deep stop states

[Pratik Rajesh Sampat]

Skiboot patches v4: 
https://lists.ozlabs.org/pipermail/skiboot/2020-February/016404.html
v3 patches: https://lkml.org/lkml/2020/1/13/333
Changelog
v3 --> v4
Device tree interface change: Cease to use the active property for
self-save and self-restore and use only presence/absence of the node to 
signify the status.

Currently the stop-API supports a mechanism called as self-restore
which allows us to restore the values of certain SPRs on wakeup from a
deep-stop state to a desired value. To use this, the Kernel makes an
OPAL call passing the PIR of the CPU, the SPR number and the value to
which the SPR should be restored when that CPU wakes up from a deep
stop state.

Recently, a new feature, named self-save has been enabled in the
stop-api, which is an alternative mechanism to do the same, except
that self-save will save the current content of the SPR before
entering a deep stop state and also restore the content back on
waking up from a deep stop state.

This patch series aims at introducing and leveraging the self-save feature in
the kernel.

Now, as the kernel has a choice to prefer one mode over the other and
there can be registers in both the save/restore SPR list which are sent
from the device tree, a new interface has been defined for the seamless
handing of the modes for each SPR.

A list of preferred SPRs are maintained in the kernel which contains two
properties:
1. supported_mode: Helps in identifying if it strictly supports self
   save or restore or both.
   Initialized using the information from device tree.
2. preferred_mode: Calls out what mode is preferred for each SPR. It
   could be strictly self save or restore, or it can also
   determine the preference of  mode over the other if both
   are present by encapsulating the other in bitmask from
   LSB to MSB.
   Initialized statically.

Below is a table to show the Scenario::Consequence when the self save and
self restore modes are available or disabled in different combinations as
perceived from the device tree thus giving complete backwards compatibly
regardless of an older firmware running a newer kernel or vise-versa.
Support for self save or self-restore is embedded in the device tree,
along with the set of registers it supports.

SR = Self restore; SS = Self save

.---..
| Scenario  |Consequence |
:---+:
| Legacy Firmware. No SS or SR node | Self restore is called for all |
|   | supported SPRs |
:---+:
| SR: !active SS: !active   | Deep stop states disabled  |
:---+:
| SR: active SS: !active| Self restore is called for all |
|   | supported SPRs |
:---+:
| SR: active SS: active | Goes through the preferences for each  |
|   | SPR and executes of the modes  |
|   | accordingly. Currently, Self restore is|
|   | called for all the SPRs except PSSCR   |
|   | which is self saved|
:---+:
| SR: active(only HID0) SS: active  | Self save called for all supported |
|   | registers expect HID0 (as HID0 cannot  |
|   | be self saved currently)   |
:---+:
| SR: !active SS: active| currently will disable deep states as  |
|   | HID0 is needed to be self restored and |
|   | cannot be self saved   |
'---''

Pratik Rajesh Sampat (3):
  powerpc/powernv: Interface to define support and preference for a SPR
  powerpc/powernv: Introduce Self save support
  powerpc/powernv: Parse device tree, population of SPR support

 arch/powerpc/include/asm/opal-api.h|   3 +-
 arch/powerpc/include/asm/opal.h|   1 +
 arch/powerpc/platforms/powernv/idle.c  | 420 ++---
 arch/powerpc/platforms/powernv/opal-call.c |   1 +
 4 files changed, 367 insertions(+), 58 deletions(-)

-- 
2.17.1

[PATCH v3 3/3] powerpc/powernv: Parse device tree, population of SPR support

[Pratik Rajesh Sampat]

Parse the device tree for nodes self-save, self-restore and populate
support for the preferred SPRs based what was advertised by the device
tree.

Signed-off-by: Pratik Rajesh Sampat 
Reviewed-by: Ram Pai 
---
 arch/powerpc/platforms/powernv/idle.c | 104 ++
 1 file changed, 104 insertions(+)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 97aeb45e897b..384980c743eb 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -1436,6 +1436,107 @@ static void __init pnv_probe_idle_states(void)
supported_cpuidle_states |= pnv_idle_states[i].flags;
 }
 
+/*
+ * Extracts and populates the self save or restore capabilities
+ * passed from the device tree node
+ */
+static int extract_save_restore_state_dt(struct device_node *np, int type)
+{
+   int nr_sprns = 0, i, bitmask_index;
+   int rc = 0;
+   u64 *temp_u64;
+   const char *state_prop;
+   u64 bit_pos;
+
+   state_prop = of_get_property(np, "status", NULL);
+   if (!state_prop) {
+   pr_warn("opal: failed to find the active value for self 
save/restore node");
+   return -EINVAL;
+   }
+   if (strncmp(state_prop, "disabled", 8) == 0) {
+   /*
+* if the feature is not active, strip the preferred_sprs from
+* that capability.
+*/
+   if (type == SELF_RESTORE_TYPE) {
+   for (i = 0; i < nr_preferred_sprs; i++) {
+   preferred_sprs[i].supported_mode &=
+   ~SELF_RESTORE_STRICT;
+   }
+   } else {
+   for (i = 0; i < nr_preferred_sprs; i++) {
+   preferred_sprs[i].supported_mode &=
+   ~SELF_SAVE_STRICT;
+   }
+   }
+   return 0;
+   }
+   nr_sprns = of_property_count_u64_elems(np, "sprn-bitmask");
+   if (nr_sprns <= 0)
+   return rc;
+   temp_u64 = kcalloc(nr_sprns, sizeof(u64), GFP_KERNEL);
+   if (of_property_read_u64_array(np, "sprn-bitmask",
+  temp_u64, nr_sprns)) {
+   pr_warn("cpuidle-powernv: failed to find registers in DT\n");
+   kfree(temp_u64);
+   return -EINVAL;
+   }
+   /*
+* Populate acknowledgment of support for the sprs in the global vector
+* gotten by the registers supplied by the firmware.
+* The registers are in a bitmask, bit index within
+* that specifies the SPR
+*/
+   for (i = 0; i < nr_preferred_sprs; i++) {
+   bitmask_index = preferred_sprs[i].spr / 64;
+   bit_pos = preferred_sprs[i].spr % 64;
+   if ((temp_u64[bitmask_index] & (1UL << bit_pos)) == 0) {
+   if (type == SELF_RESTORE_TYPE)
+   preferred_sprs[i].supported_mode &=
+   ~SELF_RESTORE_STRICT;
+   else
+   preferred_sprs[i].supported_mode &=
+   ~SELF_SAVE_STRICT;
+   continue;
+   }
+   if (type == SELF_RESTORE_TYPE) {
+   preferred_sprs[i].supported_mode |=
+   SELF_RESTORE_STRICT;
+   } else {
+   preferred_sprs[i].supported_mode |=
+   SELF_SAVE_STRICT;
+   }
+   }
+
+   kfree(temp_u64);
+   return rc;
+}
+
+static int pnv_parse_deepstate_dt(void)
+{
+   struct device_node *sr_np, *ss_np;
+   int rc = 0;
+
+   /* Self restore register population */
+   sr_np = of_find_node_by_path("/ibm,opal/power-mgt/self-restore");
+   if (!sr_np) {
+   pr_warn("opal: self restore Node not found");
+   } else {
+   rc = extract_save_restore_state_dt(sr_np, SELF_RESTORE_TYPE);
+   if (rc != 0)
+   return rc;
+   }
+   /* Self save register population */
+   ss_np = of_find_node_by_path("/ibm,opal/power-mgt/self-save");
+   if (!ss_np) {
+   pr_warn("opal: self save Node not found");
+   pr_warn("Legacy firmware. Assuming default self-restore 
support");
+   } else {
+   rc = extract_save_restore_state_dt(ss_np, SELF_SAVE_TYPE);
+   }
+   return rc;
+}
+
 /*
  * This function parses device-tree and populates all the information
  * into pnv_idle_states structure. It also sets up nr_pnv_idle_states
@@ -1584,6 +1685,9 @@ static int __init pnv_init_idle_states(

[PATCH v3 2/3] powerpc/powernv: Introduce Self save support

[Pratik Rajesh Sampat]

This commit introduces and leverages the Self save API which OPAL now
supports.

Add the new Self Save OPAL API call in the list of OPAL calls.
Implement the self saving of the SPRs based on the support populated
while respecting it's preferences.

This implementation allows mixing of support for the SPRs, which
means that a SPR can be self restored while another SPR be self saved if
they support and prefer it to be so.

Signed-off-by: Pratik Rajesh Sampat 
Reviewed-by: Ram Pai 
---
 arch/powerpc/include/asm/opal-api.h|  3 ++-
 arch/powerpc/include/asm/opal.h|  1 +
 arch/powerpc/platforms/powernv/idle.c  | 22 ++
 arch/powerpc/platforms/powernv/opal-call.c |  1 +
 4 files changed, 26 insertions(+), 1 deletion(-)

diff --git a/arch/powerpc/include/asm/opal-api.h 
b/arch/powerpc/include/asm/opal-api.h
index c1f25a760eb1..1b6e1a68d431 100644
--- a/arch/powerpc/include/asm/opal-api.h
+++ b/arch/powerpc/include/asm/opal-api.h
@@ -214,7 +214,8 @@
 #define OPAL_SECVAR_GET176
 #define OPAL_SECVAR_GET_NEXT   177
 #define OPAL_SECVAR_ENQUEUE_UPDATE 178
-#define OPAL_LAST  178
+#define OPAL_SLW_SELF_SAVE_REG 181
+#define OPAL_LAST  181
 
 #define QUIESCE_HOLD   1 /* Spin all calls at entry */
 #define QUIESCE_REJECT 2 /* Fail all calls with OPAL_BUSY */
diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h
index 9986ac34b8e2..389a85b63805 100644
--- a/arch/powerpc/include/asm/opal.h
+++ b/arch/powerpc/include/asm/opal.h
@@ -203,6 +203,7 @@ int64_t opal_handle_hmi(void);
 int64_t opal_handle_hmi2(__be64 *out_flags);
 int64_t opal_register_dump_region(uint32_t id, uint64_t start, uint64_t end);
 int64_t opal_unregister_dump_region(uint32_t id);
+int64_t opal_slw_self_save_reg(uint64_t cpu_pir, uint64_t sprn);
 int64_t opal_slw_set_reg(uint64_t cpu_pir, uint64_t sprn, uint64_t val);
 int64_t opal_config_cpu_idle_state(uint64_t state, uint64_t flag);
 int64_t opal_pci_set_phb_cxl_mode(uint64_t phb_id, uint64_t mode, uint64_t 
pe_number);
diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 03fe835aadd1..97aeb45e897b 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -279,6 +279,26 @@ static int pnv_self_save_restore_sprs(void)
if (rc != 0)
return rc;
break;
+   } else if (preferred & curr_spr.supported_mode
+  & SELF_SAVE_STRICT) {
+   is_initialized = true;
+   if (curr_spr.spr == SPRN_HMEER &&
+   cpu_thread_in_core(cpu) != 0) {
+   continue;
+   }
+   rc = opal_slw_self_save_reg(pir,
+   curr_spr.spr);
+   if (rc != 0)
+   return rc;
+   switch (curr_spr.spr) {
+   case SPRN_LPCR:
+   is_lpcr_self_save = true;
+   break;
+   case SPRN_PTCR:
+   is_ptcr_self_save = true;
+   break;
+   }
+   break;
}
preferred_sprs[index].preferred_mode =
preferred_sprs[index].preferred_mode >>
@@ -1159,6 +1179,8 @@ void pnv_program_cpu_hotplug_lpcr(unsigned int cpu, u64 
lpcr_val)
if (!is_lpcr_self_save)
opal_slw_set_reg(pir, SPRN_LPCR,
 lpcr_val);
+   else
+   opal_slw_self_save_reg(pir, SPRN_LPCR);
}
 }
 
diff --git a/arch/powerpc/platforms/powernv/opal-call.c 
b/arch/powerpc/platforms/powernv/opal-call.c
index 5cd0f52d258f..11e0ceb90de0 100644
--- a/arch/powerpc/platforms/powernv/opal-call.c
+++ b/arch/powerpc/platforms/powernv/opal-call.c
@@ -223,6 +223,7 @@ OPAL_CALL(opal_handle_hmi,  
OPAL_HANDLE_HMI);
 OPAL_CALL(opal_handle_hmi2,OPAL_HANDLE_HMI2);
 OPAL_CALL(opal_config_cpu_idle_state,  OPAL_CONFIG_CPU_IDLE_STATE);
 OPAL_CALL(opal_slw_set_reg,OPAL_SLW_SET_REG);
+OPAL_CALL(

[PATCH v3 1/3] powerpc/powernv: Interface to define support and preference for a SPR

[Pratik Rajesh Sampat]

Define a bitmask interface to determine support for the Self Restore,
Self Save or both.

Also define an interface to determine the preference of that SPR to
be strictly saved or restored or encapsulated with an order of preference.

The preference bitmask is shown as below:

|... | 2nd pref | 1st pref |

MSB   LSB

The preference from higher to lower is from LSB to MSB with a shift of 8
bits.
Example:
Prefer self save first, if not available then prefer self
restore
The preference mask for this scenario will be seen as below.
((SELF_RESTORE_STRICT << PREFERENCE_SHIFT) | SELF_SAVE_STRICT)
-
|... | Self restore | Self save |
-
MSB LSB

Finally, declare a list of preferred SPRs which encapsulate the bitmaks
for preferred and supported with defaults of both being set to support
legacy firmware.

This commit also implements using the above interface and retains the
legacy functionality of self restore.

Signed-off-by: Pratik Rajesh Sampat 
Reviewed-by: Ram Pai 
---
 arch/powerpc/platforms/powernv/idle.c | 316 +-
 1 file changed, 259 insertions(+), 57 deletions(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 78599bca66c2..03fe835aadd1 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -32,9 +32,112 @@
 #define P9_STOP_SPR_MSR 2000
 #define P9_STOP_SPR_PSSCR  855
 
+/* Interface for the stop state supported and preference */
+#define SELF_RESTORE_TYPE0
+#define SELF_SAVE_TYPE   1
+
+#define NR_PREFERENCES2
+#define PREFERENCE_SHIFT  4
+#define PREFERENCE_MASK   0xf
+
+#define UNSUPPORTED 0x0
+#define SELF_RESTORE_STRICT 0x1
+#define SELF_SAVE_STRICT0x2
+
+/*
+ * Bitmask defining the kind of preferences available.
+ * Note : The higher to lower preference is from LSB to MSB, with a shift of
+ * 4 bits.
+ * 
+ * || 2nd pref | 1st pref |
+ * 
+ * MSB   LSB
+ */
+/* Prefer Restore if available, otherwise unsupported */
+#define PREFER_SELF_RESTORE_ONLY   SELF_RESTORE_STRICT
+/* Prefer Save if available, otherwise unsupported */
+#define PREFER_SELF_SAVE_ONLY  SELF_SAVE_STRICT
+/* Prefer Restore when available, otherwise prefer Save */
+#define PREFER_RESTORE_SAVE((SELF_SAVE_STRICT << \
+ PREFERENCE_SHIFT)\
+ | SELF_RESTORE_STRICT)
+/* Prefer Save when available, otherwise prefer Restore*/
+#define PREFER_SAVE_RESTORE((SELF_RESTORE_STRICT <<\
+ PREFERENCE_SHIFT)\
+ | SELF_SAVE_STRICT)
 static u32 supported_cpuidle_states;
 struct pnv_idle_states_t *pnv_idle_states;
 int nr_pnv_idle_states;
+/* Caching the lpcr & ptcr support to use later */
+static bool is_lpcr_self_save;
+static bool is_ptcr_self_save;
+
+struct preferred_sprs {
+   u64 spr;
+   u32 preferred_mode;
+   u32 supported_mode;
+};
+
+/*
+ * Preferred mode: Order of precedence when both self-save and self-restore
+ *supported
+ * Supported mode: Default support. Can be overwritten during system
+ *initialization
+ */
+struct preferred_sprs preferred_sprs[] = {
+   {
+   .spr = SPRN_HSPRG0,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_LPCR,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_PTCR,
+   .preferred_mode = PREFER_SAVE_RESTORE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HMEER,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HID0,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = P9_STOP_SPR_MSR,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = P9_STOP_SPR_PSSCR,
+   .preferred_mode = PREFER_SAVE_RESTORE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HID1,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HID4,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode

[PATCH v3 0/3] Introduce Self-Save API for deep stop states

[Pratik Rajesh Sampat]

Skiboot patches: https://patchwork.ozlabs.org/patch/1221011/
v2 patches: https://lkml.org/lkml/2020/1/12/253
Changelog
Patch v2 --> v3
1. Addressed minor comments from Ram Pai

Currently the stop-API supports a mechanism called as self-restore
which allows us to restore the values of certain SPRs on wakeup from a
deep-stop state to a desired value. To use this, the Kernel makes an
OPAL call passing the PIR of the CPU, the SPR number and the value to
which the SPR should be restored when that CPU wakes up from a deep
stop state.

Recently, a new feature, named self-save has been enabled in the
stop-api, which is an alternative mechanism to do the same, except
that self-save will save the current content of the SPR before
entering a deep stop state and also restore the content back on
waking up from a deep stop state.

This patch series aims at introducing and leveraging the self-save feature in
the kernel.

Now, as the kernel has a choice to prefer one mode over the other and
there can be registers in both the save/restore SPR list which are sent
from the device tree, a new interface has been defined for the seamless
handing of the modes for each SPR.

A list of preferred SPRs are maintained in the kernel which contains two
properties:
1. supported_mode: Helps in identifying if it strictly supports self
   save or restore or both.
   Initialized using the information from device tree.
2. preferred_mode: Calls out what mode is preferred for each SPR. It
   could be strictly self save or restore, or it can also
   determine the preference of  mode over the other if both
   are present by encapsulating the other in bitmask from
   LSB to MSB.
   Initialized statically.

Below is a table to show the Scenario::Consequence when the self save and
self restore modes are available or disabled in different combinations as
perceived from the device tree thus giving complete backwards compatibly
regardless of an older firmware running a newer kernel or vise-versa.
Support for self save or self-restore is embedded in the device tree,
along with the set of registers it supports.

SR = Self restore; SS = Self save

.---..
| Scenario  |Consequence |
:---+:
| Legacy Firmware. No SS or SR node | Self restore is called for all |
|   | supported SPRs |
:---+:
| SR: !active SS: !active   | Deep stop states disabled  |
:---+:
| SR: active SS: !active| Self restore is called for all |
|   | supported SPRs |
:---+:
| SR: active SS: active | Goes through the preferences for each  |
|   | SPR and executes of the modes  |
|   | accordingly. Currently, Self restore is|
|   | called for all the SPRs except PSSCR   |
|   | which is self saved|
:---+:
| SR: active(only HID0) SS: active  | Self save called for all supported |
|   | registers expect HID0 (as HID0 cannot  |
|   | be self saved currently)   |
:---+:
| SR: !active SS: active| currently will disable deep states as  |
|   | HID0 is needed to be self restored and |
|   | cannot be self saved   |
'---''

Pratik Rajesh Sampat (3):
  powerpc/powernv: Interface to define support and preference for a SPR
  powerpc/powernv: Introduce Self save support
  powerpc/powernv: Parse device tree, population of SPR support

 arch/powerpc/include/asm/opal-api.h|   3 +-
 arch/powerpc/include/asm/opal.h|   1 +
 arch/powerpc/platforms/powernv/idle.c  | 442 ++---
 arch/powerpc/platforms/powernv/opal-call.c |   1 +
 4 files changed, 389 insertions(+), 58 deletions(-)

-- 
2.24.1

[RESEND PATCH v2 3/3] powerpc/powernv: Parse device tree, population of SPR support

[Pratik Rajesh Sampat]

Parse the device tree for nodes self-save, self-restore and populate
support for the preferred SPRs based what was advertised by the device
tree.

Signed-off-by: Pratik Rajesh Sampat 
---
 arch/powerpc/platforms/powernv/idle.c | 104 ++
 1 file changed, 104 insertions(+)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index d67d4d0b169b..e910ff40b7e6 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -1429,6 +1429,107 @@ static void __init pnv_probe_idle_states(void)
supported_cpuidle_states |= pnv_idle_states[i].flags;
 }
 
+/*
+ * Extracts and populates the self save or restore capabilities
+ * passed from the device tree node
+ */
+static int extract_save_restore_state_dt(struct device_node *np, int type)
+{
+   int nr_sprns = 0, i, bitmask_index;
+   int rc = 0;
+   u64 *temp_u64;
+   const char *state_prop;
+   u64 bit_pos;
+
+   state_prop = of_get_property(np, "status", NULL);
+   if (!state_prop) {
+   pr_warn("opal: failed to find the active value for self 
save/restore node");
+   return -EINVAL;
+   }
+   if (strncmp(state_prop, "disabled", 8) == 0) {
+   /*
+* if the feature is not active, strip the preferred_sprs from
+* that capability.
+*/
+   if (type == SELF_RESTORE_TYPE) {
+   for (i = 0; i < nr_preferred_sprs; i++) {
+   preferred_sprs[i].supported_mode &=
+   ~SELF_RESTORE_STRICT;
+   }
+   } else {
+   for (i = 0; i < nr_preferred_sprs; i++) {
+   preferred_sprs[i].supported_mode &=
+   ~SELF_SAVE_STRICT;
+   }
+   }
+   return 0;
+   }
+   nr_sprns = of_property_count_u64_elems(np, "sprn-bitmask");
+   if (nr_sprns <= 0)
+   return rc;
+   temp_u64 = kcalloc(nr_sprns, sizeof(u64), GFP_KERNEL);
+   if (of_property_read_u64_array(np, "sprn-bitmask",
+  temp_u64, nr_sprns)) {
+   pr_warn("cpuidle-powernv: failed to find registers in DT\n");
+   kfree(temp_u64);
+   return -EINVAL;
+   }
+   /*
+* Populate acknowledgment of support for the sprs in the global vector
+* gotten by the registers supplied by the firmware.
+* The registers are in a bitmask, bit index within
+* that specifies the SPR
+*/
+   for (i = 0; i < nr_preferred_sprs; i++) {
+   bitmask_index = preferred_sprs[i].spr / 64;
+   bit_pos = preferred_sprs[i].spr % 64;
+   if ((temp_u64[bitmask_index] & (1UL << bit_pos)) == 0) {
+   if (type == SELF_RESTORE_TYPE)
+   preferred_sprs[i].supported_mode &=
+   ~SELF_RESTORE_STRICT;
+   else
+   preferred_sprs[i].supported_mode &=
+   ~SELF_SAVE_STRICT;
+   continue;
+   }
+   if (type == SELF_RESTORE_TYPE) {
+   preferred_sprs[i].supported_mode |=
+   SELF_RESTORE_STRICT;
+   } else {
+   preferred_sprs[i].supported_mode |=
+   SELF_SAVE_STRICT;
+   }
+   }
+
+   kfree(temp_u64);
+   return rc;
+}
+
+static int pnv_parse_deepstate_dt(void)
+{
+   struct device_node *np, *np1;
+   int rc = 0;
+
+   /* Self restore register population */
+   np = of_find_node_by_path("/ibm,opal/power-mgt/self-restore");
+   if (!np) {
+   pr_warn("opal: self restore Node not found");
+   } else {
+   rc = extract_save_restore_state_dt(np, SELF_RESTORE_TYPE);
+   if (rc != 0)
+   return rc;
+   }
+   /* Self save register population */
+   np1 = of_find_node_by_path("/ibm,opal/power-mgt/self-save");
+   if (!np1) {
+   pr_warn("opal: self save Node not found");
+   pr_warn("Legacy firmware. Assuming default self-restore 
support");
+   } else {
+   rc = extract_save_restore_state_dt(np1, SELF_SAVE_TYPE);
+   }
+   return rc;
+}
+
 /*
  * This function parses device-tree and populates all the information
  * into pnv_idle_states structure. It also sets up nr_pnv_idle_states
@@ -1577,6 +1678,9 @@ static int __init pnv_init_idle_states(void)
return rc;
pnv_pr

[RESEND PATCH v2 2/3] powerpc/powernv: Introduce Self save support

[Pratik Rajesh Sampat]

This commit introduces and leverages the Self save API which OPAL now
supports.

Add the new Self Save OPAL API call in the list of OPAL calls.
Implement the self saving of the SPRs based on the support populated
while respecting it's preferences.

This implementation allows mixing of support for the SPRs, which
means that a SPR can be self restored while another SPR be self saved if
they support and prefer it to be so.

Signed-off-by: Pratik Rajesh Sampat 
---
 arch/powerpc/include/asm/opal-api.h| 3 ++-
 arch/powerpc/include/asm/opal.h| 1 +
 arch/powerpc/platforms/powernv/idle.c  | 2 ++
 arch/powerpc/platforms/powernv/opal-call.c | 1 +
 4 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/arch/powerpc/include/asm/opal-api.h 
b/arch/powerpc/include/asm/opal-api.h
index c1f25a760eb1..1b6e1a68d431 100644
--- a/arch/powerpc/include/asm/opal-api.h
+++ b/arch/powerpc/include/asm/opal-api.h
@@ -214,7 +214,8 @@
 #define OPAL_SECVAR_GET176
 #define OPAL_SECVAR_GET_NEXT   177
 #define OPAL_SECVAR_ENQUEUE_UPDATE 178
-#define OPAL_LAST  178
+#define OPAL_SLW_SELF_SAVE_REG 181
+#define OPAL_LAST  181
 
 #define QUIESCE_HOLD   1 /* Spin all calls at entry */
 #define QUIESCE_REJECT 2 /* Fail all calls with OPAL_BUSY */
diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h
index 9986ac34b8e2..389a85b63805 100644
--- a/arch/powerpc/include/asm/opal.h
+++ b/arch/powerpc/include/asm/opal.h
@@ -203,6 +203,7 @@ int64_t opal_handle_hmi(void);
 int64_t opal_handle_hmi2(__be64 *out_flags);
 int64_t opal_register_dump_region(uint32_t id, uint64_t start, uint64_t end);
 int64_t opal_unregister_dump_region(uint32_t id);
+int64_t opal_slw_self_save_reg(uint64_t cpu_pir, uint64_t sprn);
 int64_t opal_slw_set_reg(uint64_t cpu_pir, uint64_t sprn, uint64_t val);
 int64_t opal_config_cpu_idle_state(uint64_t state, uint64_t flag);
 int64_t opal_pci_set_phb_cxl_mode(uint64_t phb_id, uint64_t mode, uint64_t 
pe_number);
diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 2f328403b0dc..d67d4d0b169b 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -1172,6 +1172,8 @@ void pnv_program_cpu_hotplug_lpcr(unsigned int cpu, u64 
lpcr_val)
if (!is_lpcr_self_save)
opal_slw_set_reg(pir, SPRN_LPCR,
 lpcr_val);
+   else
+   opal_slw_self_save_reg(pir, SPRN_LPCR);
}
 }
 
diff --git a/arch/powerpc/platforms/powernv/opal-call.c 
b/arch/powerpc/platforms/powernv/opal-call.c
index 5cd0f52d258f..11e0ceb90de0 100644
--- a/arch/powerpc/platforms/powernv/opal-call.c
+++ b/arch/powerpc/platforms/powernv/opal-call.c
@@ -223,6 +223,7 @@ OPAL_CALL(opal_handle_hmi,  
OPAL_HANDLE_HMI);
 OPAL_CALL(opal_handle_hmi2,OPAL_HANDLE_HMI2);
 OPAL_CALL(opal_config_cpu_idle_state,  OPAL_CONFIG_CPU_IDLE_STATE);
 OPAL_CALL(opal_slw_set_reg,OPAL_SLW_SET_REG);
+OPAL_CALL(opal_slw_self_save_reg,  OPAL_SLW_SELF_SAVE_REG);
 OPAL_CALL(opal_register_dump_region,   OPAL_REGISTER_DUMP_REGION);
 OPAL_CALL(opal_unregister_dump_region, OPAL_UNREGISTER_DUMP_REGION);
 OPAL_CALL(opal_pci_set_phb_cxl_mode,   OPAL_PCI_SET_PHB_CAPI_MODE);
-- 
2.24.1

[RESEND PATCH v2 1/3] powerpc/powernv: Interface to define support and preference for a SPR

[Pratik Rajesh Sampat]

Define a bitmask interface to determine support for the Self Restore,
Self Save or both.

Also define an interface to determine the preference of that SPR to
be strictly saved or restored or encapsulated with an order of preference.

The preference bitmask is shown as below:

|... | 2nd pref | 1st pref |

MSB   LSB

The preference from higher to lower is from LSB to MSB with a shift of 8
bits.
Example:
Prefer self save first, if not available then prefer self
restore
The preference mask for this scenario will be seen as below.
((SELF_RESTORE_STRICT << PREFERENCE_SHIFT) | SELF_SAVE_STRICT)
-
|... | Self restore | Self save |
-
MSB LSB

Finally, declare a list of preferred SPRs which encapsulate the bitmaks
for preferred and supported with defaults of both being set to support
legacy firmware.

This commit also implements using the above interface and retains the
legacy functionality of self restore.

Signed-off-by: Pratik Rajesh Sampat 
---
 arch/powerpc/platforms/powernv/idle.c | 327 +-
 1 file changed, 271 insertions(+), 56 deletions(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 78599bca66c2..2f328403b0dc 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -32,9 +32,106 @@
 #define P9_STOP_SPR_MSR 2000
 #define P9_STOP_SPR_PSSCR  855
 
+/* Interface for the stop state supported and preference */
+#define SELF_RESTORE_TYPE0
+#define SELF_SAVE_TYPE   1
+
+#define NR_PREFERENCES2
+#define PREFERENCE_SHIFT  4
+#define PREFERENCE_MASK   0xf
+
+#define UNSUPPORTED 0x0
+#define SELF_RESTORE_STRICT 0x1
+#define SELF_SAVE_STRICT0x2
+
+/*
+ * Bitmask defining the kind of preferences available.
+ * Note : The higher to lower preference is from LSB to MSB, with a shift of
+ * 4 bits.
+ * 
+ * || 2nd pref | 1st pref |
+ * 
+ * MSB   LSB
+ */
+/* Prefer Restore if available, otherwise unsupported */
+#define PREFER_SELF_RESTORE_ONLY   SELF_RESTORE_STRICT
+/* Prefer Save if available, otherwise unsupported */
+#define PREFER_SELF_SAVE_ONLY  SELF_SAVE_STRICT
+/* Prefer Restore when available, otherwise prefer Save */
+#define PREFER_RESTORE_SAVE((SELF_SAVE_STRICT << \
+ PREFERENCE_SHIFT)\
+ | SELF_RESTORE_STRICT)
+/* Prefer Save when available, otherwise prefer Restore*/
+#define PREFER_SAVE_RESTORE((SELF_RESTORE_STRICT <<\
+ PREFERENCE_SHIFT)\
+ | SELF_SAVE_STRICT)
 static u32 supported_cpuidle_states;
 struct pnv_idle_states_t *pnv_idle_states;
 int nr_pnv_idle_states;
+/* Caching the lpcr & ptcr support to use later */
+static bool is_lpcr_self_save;
+static bool is_ptcr_self_save;
+
+struct preferred_sprs {
+   u64 spr;
+   u32 preferred_mode;
+   u32 supported_mode;
+};
+
+struct preferred_sprs preferred_sprs[] = {
+   {
+   .spr = SPRN_HSPRG0,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_LPCR,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_PTCR,
+   .preferred_mode = PREFER_SAVE_RESTORE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HMEER,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HID0,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = P9_STOP_SPR_MSR,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = P9_STOP_SPR_PSSCR,
+   .preferred_mode = PREFER_SAVE_RESTORE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HID1,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HID4,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HID5,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   }
+};
+
+const int nr_preferred_sprs = ARR

[RESEND PATCH v2 0/3] Introduce Self-Save API for deep stop states

[Pratik Rajesh Sampat]

RESEND Changes:
1. Updated OPAL API numbering for consistency

Skiboot patches: https://patchwork.ozlabs.org/patch/1221011/
RFC v1 patches: https://lkml.org/lkml/2019/12/4/193
Changelog
RFC v1 --> v2
1. Optimized preference bitmask
2. Addressed comments from Ram Pai

Currently the stop-API supports a mechanism called as self-restore
which allows us to restore the values of certain SPRs on wakeup from a
deep-stop state to a desired value. To use this, the Kernel makes an
OPAL call passing the PIR of the CPU, the SPR number and the value to
which the SPR should be restored when that CPU wakes up from a deep
stop state.

Recently, a new feature, named self-save has been enabled in the
stop-api, which is an alternative mechanism to do the same, except
that self-save will save the current content of the SPR before
entering a deep stop state and also restore the content back on
waking up from a deep stop state.

This patch series aims at introducing and leveraging the self-save feature in
the kernel.

Now, as the kernel has a choice to prefer one mode over the other and
there can be registers in both the save/restore SPR list which are sent
from the device tree, a new interface has been defined for the seamless
handing of the modes for each SPR.

A list of preferred SPRs are maintained in the kernel which contains two
properties:
1. supported_mode: Helps in identifying if it strictly supports self
   save or restore or both.
   Initialized using the information from device tree.
2. preferred_mode: Calls out what mode is preferred for each SPR. It
   could be strictly self save or restore, or it can also
   determine the preference of  mode over the other if both
   are present by encapsulating the other in bitmask from
   LSB to MSB.
   Initialized statically.

Below is a table to show the Scenario::Consequence when the self save and
self restore modes are available or disabled in different combinations as
perceived from the device tree thus giving complete backwards compatibly
regardless of an older firmware running a newer kernel or vise-versa.
Support for self save or self-restore is embedded in the device tree,
along with the set of registers it supports.

SR = Self restore; SS = Self save

.---..
| Scenario  |Consequence |
:---+:
| Legacy Firmware. No SS or SR node | Self restore is called for all |
|   | supported SPRs |
:---+:
| SR: !active SS: !active   | Deep stop states disabled  |
:---+:
| SR: active SS: !active| Self restore is called for all |
|   | supported SPRs |
:---+:
| SR: active SS: active | Goes through the preferences for each  |
|   | SPR and executes of the modes  |
|   | accordingly. Currently, Self restore is|
|   | called for all the SPRs except PSSCR   |
|   | which is self saved|
:---+:
| SR: active(only HID0) SS: active  | Self save called for all supported |
|   | registers expect HID0 (as HID0 cannot  |
|   | be self saved currently)   |
:---+:
| SR: !active SS: active| currently will disable deep states as  |
|   | HID0 is needed to be self restored and |
|   | cannot be self saved   |
'---''

Pratik Rajesh Sampat (3):
  powerpc/powernv: Interface to define support and preference for a SPR
  powerpc/powernv: Introduce Self save support
  powerpc/powernv: Parse device tree, population of SPR support

 arch/powerpc/include/asm/opal-api.h|   3 +-
 arch/powerpc/include/asm/opal.h|   1 +
 arch/powerpc/platforms/powernv/idle.c  | 433 ++---
 arch/powerpc/platforms/powernv/opal-call.c |   1 +
 4 files changed, 381 insertions(+), 57 deletions(-)

-- 
2.24.1

[PATCH v2 3/3] powerpc/powernv: Parse device tree, population of SPR support

[Pratik Rajesh Sampat]

Parse the device tree for nodes self-save, self-restore and populate
support for the preferred SPRs based what was advertised by the device
tree.

Signed-off-by: Pratik Rajesh Sampat 
---
 arch/powerpc/platforms/powernv/idle.c | 104 ++
 1 file changed, 104 insertions(+)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index d67d4d0b169b..e910ff40b7e6 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -1429,6 +1429,107 @@ static void __init pnv_probe_idle_states(void)
supported_cpuidle_states |= pnv_idle_states[i].flags;
 }
 
+/*
+ * Extracts and populates the self save or restore capabilities
+ * passed from the device tree node
+ */
+static int extract_save_restore_state_dt(struct device_node *np, int type)
+{
+   int nr_sprns = 0, i, bitmask_index;
+   int rc = 0;
+   u64 *temp_u64;
+   const char *state_prop;
+   u64 bit_pos;
+
+   state_prop = of_get_property(np, "status", NULL);
+   if (!state_prop) {
+   pr_warn("opal: failed to find the active value for self 
save/restore node");
+   return -EINVAL;
+   }
+   if (strncmp(state_prop, "disabled", 8) == 0) {
+   /*
+* if the feature is not active, strip the preferred_sprs from
+* that capability.
+*/
+   if (type == SELF_RESTORE_TYPE) {
+   for (i = 0; i < nr_preferred_sprs; i++) {
+   preferred_sprs[i].supported_mode &=
+   ~SELF_RESTORE_STRICT;
+   }
+   } else {
+   for (i = 0; i < nr_preferred_sprs; i++) {
+   preferred_sprs[i].supported_mode &=
+   ~SELF_SAVE_STRICT;
+   }
+   }
+   return 0;
+   }
+   nr_sprns = of_property_count_u64_elems(np, "sprn-bitmask");
+   if (nr_sprns <= 0)
+   return rc;
+   temp_u64 = kcalloc(nr_sprns, sizeof(u64), GFP_KERNEL);
+   if (of_property_read_u64_array(np, "sprn-bitmask",
+  temp_u64, nr_sprns)) {
+   pr_warn("cpuidle-powernv: failed to find registers in DT\n");
+   kfree(temp_u64);
+   return -EINVAL;
+   }
+   /*
+* Populate acknowledgment of support for the sprs in the global vector
+* gotten by the registers supplied by the firmware.
+* The registers are in a bitmask, bit index within
+* that specifies the SPR
+*/
+   for (i = 0; i < nr_preferred_sprs; i++) {
+   bitmask_index = preferred_sprs[i].spr / 64;
+   bit_pos = preferred_sprs[i].spr % 64;
+   if ((temp_u64[bitmask_index] & (1UL << bit_pos)) == 0) {
+   if (type == SELF_RESTORE_TYPE)
+   preferred_sprs[i].supported_mode &=
+   ~SELF_RESTORE_STRICT;
+   else
+   preferred_sprs[i].supported_mode &=
+   ~SELF_SAVE_STRICT;
+   continue;
+   }
+   if (type == SELF_RESTORE_TYPE) {
+   preferred_sprs[i].supported_mode |=
+   SELF_RESTORE_STRICT;
+   } else {
+   preferred_sprs[i].supported_mode |=
+   SELF_SAVE_STRICT;
+   }
+   }
+
+   kfree(temp_u64);
+   return rc;
+}
+
+static int pnv_parse_deepstate_dt(void)
+{
+   struct device_node *np, *np1;
+   int rc = 0;
+
+   /* Self restore register population */
+   np = of_find_node_by_path("/ibm,opal/power-mgt/self-restore");
+   if (!np) {
+   pr_warn("opal: self restore Node not found");
+   } else {
+   rc = extract_save_restore_state_dt(np, SELF_RESTORE_TYPE);
+   if (rc != 0)
+   return rc;
+   }
+   /* Self save register population */
+   np1 = of_find_node_by_path("/ibm,opal/power-mgt/self-save");
+   if (!np1) {
+   pr_warn("opal: self save Node not found");
+   pr_warn("Legacy firmware. Assuming default self-restore 
support");
+   } else {
+   rc = extract_save_restore_state_dt(np1, SELF_SAVE_TYPE);
+   }
+   return rc;
+}
+
 /*
  * This function parses device-tree and populates all the information
  * into pnv_idle_states structure. It also sets up nr_pnv_idle_states
@@ -1577,6 +1678,9 @@ static int __init pnv_init_idle_states(void)
return rc;
pnv_pr

[PATCH v2 2/3] powerpc/powernv: Introduce Self save support

[Pratik Rajesh Sampat]

This commit introduces and leverages the Self save API which OPAL now
supports.

Add the new Self Save OPAL API call in the list of OPAL calls.
Implement the self saving of the SPRs based on the support populated
while respecting it's preferences.

This implementation allows mixing of support for the SPRs, which
means that a SPR can be self restored while another SPR be self saved if
they support and prefer it to be so.

Signed-off-by: Pratik Rajesh Sampat 
---
 arch/powerpc/include/asm/opal-api.h| 3 ++-
 arch/powerpc/include/asm/opal.h| 1 +
 arch/powerpc/platforms/powernv/idle.c  | 2 ++
 arch/powerpc/platforms/powernv/opal-call.c | 1 +
 4 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/arch/powerpc/include/asm/opal-api.h 
b/arch/powerpc/include/asm/opal-api.h
index c1f25a760eb1..89b7c44124e6 100644
--- a/arch/powerpc/include/asm/opal-api.h
+++ b/arch/powerpc/include/asm/opal-api.h
@@ -214,7 +214,8 @@
 #define OPAL_SECVAR_GET176
 #define OPAL_SECVAR_GET_NEXT   177
 #define OPAL_SECVAR_ENQUEUE_UPDATE 178
-#define OPAL_LAST  178
+#define OPAL_SLW_SELF_SAVE_REG 179
+#define OPAL_LAST  179
 
 #define QUIESCE_HOLD   1 /* Spin all calls at entry */
 #define QUIESCE_REJECT 2 /* Fail all calls with OPAL_BUSY */
diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h
index 9986ac34b8e2..389a85b63805 100644
--- a/arch/powerpc/include/asm/opal.h
+++ b/arch/powerpc/include/asm/opal.h
@@ -203,6 +203,7 @@ int64_t opal_handle_hmi(void);
 int64_t opal_handle_hmi2(__be64 *out_flags);
 int64_t opal_register_dump_region(uint32_t id, uint64_t start, uint64_t end);
 int64_t opal_unregister_dump_region(uint32_t id);
+int64_t opal_slw_self_save_reg(uint64_t cpu_pir, uint64_t sprn);
 int64_t opal_slw_set_reg(uint64_t cpu_pir, uint64_t sprn, uint64_t val);
 int64_t opal_config_cpu_idle_state(uint64_t state, uint64_t flag);
 int64_t opal_pci_set_phb_cxl_mode(uint64_t phb_id, uint64_t mode, uint64_t 
pe_number);
diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 2f328403b0dc..d67d4d0b169b 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -1172,6 +1172,8 @@ void pnv_program_cpu_hotplug_lpcr(unsigned int cpu, u64 
lpcr_val)
if (!is_lpcr_self_save)
opal_slw_set_reg(pir, SPRN_LPCR,
 lpcr_val);
+   else
+   opal_slw_self_save_reg(pir, SPRN_LPCR);
}
 }
 
diff --git a/arch/powerpc/platforms/powernv/opal-call.c 
b/arch/powerpc/platforms/powernv/opal-call.c
index 5cd0f52d258f..11e0ceb90de0 100644
--- a/arch/powerpc/platforms/powernv/opal-call.c
+++ b/arch/powerpc/platforms/powernv/opal-call.c
@@ -223,6 +223,7 @@ OPAL_CALL(opal_handle_hmi,  
OPAL_HANDLE_HMI);
 OPAL_CALL(opal_handle_hmi2,OPAL_HANDLE_HMI2);
 OPAL_CALL(opal_config_cpu_idle_state,  OPAL_CONFIG_CPU_IDLE_STATE);
 OPAL_CALL(opal_slw_set_reg,OPAL_SLW_SET_REG);
+OPAL_CALL(opal_slw_self_save_reg,  OPAL_SLW_SELF_SAVE_REG);
 OPAL_CALL(opal_register_dump_region,   OPAL_REGISTER_DUMP_REGION);
 OPAL_CALL(opal_unregister_dump_region, OPAL_UNREGISTER_DUMP_REGION);
 OPAL_CALL(opal_pci_set_phb_cxl_mode,   OPAL_PCI_SET_PHB_CAPI_MODE);
-- 
2.24.1

[PATCH v2 1/3] powerpc/powernv: Interface to define support and preference for a SPR

[Pratik Rajesh Sampat]

Define a bitmask interface to determine support for the Self Restore,
Self Save or both.

Also define an interface to determine the preference of that SPR to
be strictly saved or restored or encapsulated with an order of preference.

The preference bitmask is shown as below:

|... | 2nd pref | 1st pref |

MSB   LSB

The preference from higher to lower is from LSB to MSB with a shift of 8
bits.
Example:
Prefer self save first, if not available then prefer self
restore
The preference mask for this scenario will be seen as below.
((SELF_RESTORE_STRICT << PREFERENCE_SHIFT) | SELF_SAVE_STRICT)
-
|... | Self restore | Self save |
-
MSB LSB

Finally, declare a list of preferred SPRs which encapsulate the bitmaks
for preferred and supported with defaults of both being set to support
legacy firmware.

This commit also implements using the above interface and retains the
legacy functionality of self restore.

Signed-off-by: Pratik Rajesh Sampat 
---
 arch/powerpc/platforms/powernv/idle.c | 327 +-
 1 file changed, 271 insertions(+), 56 deletions(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 78599bca66c2..2f328403b0dc 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -32,9 +32,106 @@
 #define P9_STOP_SPR_MSR 2000
 #define P9_STOP_SPR_PSSCR  855
 
+/* Interface for the stop state supported and preference */
+#define SELF_RESTORE_TYPE0
+#define SELF_SAVE_TYPE   1
+
+#define NR_PREFERENCES2
+#define PREFERENCE_SHIFT  4
+#define PREFERENCE_MASK   0xf
+
+#define UNSUPPORTED 0x0
+#define SELF_RESTORE_STRICT 0x1
+#define SELF_SAVE_STRICT0x2
+
+/*
+ * Bitmask defining the kind of preferences available.
+ * Note : The higher to lower preference is from LSB to MSB, with a shift of
+ * 4 bits.
+ * 
+ * || 2nd pref | 1st pref |
+ * 
+ * MSB   LSB
+ */
+/* Prefer Restore if available, otherwise unsupported */
+#define PREFER_SELF_RESTORE_ONLY   SELF_RESTORE_STRICT
+/* Prefer Save if available, otherwise unsupported */
+#define PREFER_SELF_SAVE_ONLY  SELF_SAVE_STRICT
+/* Prefer Restore when available, otherwise prefer Save */
+#define PREFER_RESTORE_SAVE((SELF_SAVE_STRICT << \
+ PREFERENCE_SHIFT)\
+ | SELF_RESTORE_STRICT)
+/* Prefer Save when available, otherwise prefer Restore*/
+#define PREFER_SAVE_RESTORE((SELF_RESTORE_STRICT <<\
+ PREFERENCE_SHIFT)\
+ | SELF_SAVE_STRICT)
 static u32 supported_cpuidle_states;
 struct pnv_idle_states_t *pnv_idle_states;
 int nr_pnv_idle_states;
+/* Caching the lpcr & ptcr support to use later */
+static bool is_lpcr_self_save;
+static bool is_ptcr_self_save;
+
+struct preferred_sprs {
+   u64 spr;
+   u32 preferred_mode;
+   u32 supported_mode;
+};
+
+struct preferred_sprs preferred_sprs[] = {
+   {
+   .spr = SPRN_HSPRG0,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_LPCR,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_PTCR,
+   .preferred_mode = PREFER_SAVE_RESTORE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HMEER,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HID0,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = P9_STOP_SPR_MSR,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = P9_STOP_SPR_PSSCR,
+   .preferred_mode = PREFER_SAVE_RESTORE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HID1,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HID4,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HID5,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   }
+};
+
+const int nr_preferred_sprs = ARR

[PATCH v2 0/3] Introduce Self-Save API for deep stop states

[Pratik Rajesh Sampat]

RFC v1 patches: https://lkml.org/lkml/2019/12/4/193
Changelog
RFC v1 --> v2
1. Optimized preference bitmask
2. Addressed comments from Ram Pai

Currently the stop-API supports a mechanism called as self-restore
which allows us to restore the values of certain SPRs on wakeup from a
deep-stop state to a desired value. To use this, the Kernel makes an
OPAL call passing the PIR of the CPU, the SPR number and the value to
which the SPR should be restored when that CPU wakes up from a deep
stop state.

Recently, a new feature, named self-save has been enabled in the
stop-api, which is an alternative mechanism to do the same, except
that self-save will save the current content of the SPR before
entering a deep stop state and also restore the content back on
waking up from a deep stop state.

This patch series aims at introducing and leveraging the self-save feature in
the kernel.

Now, as the kernel has a choice to prefer one mode over the other and
there can be registers in both the save/restore SPR list which are sent
from the device tree, a new interface has been defined for the seamless
handing of the modes for each SPR.

A list of preferred SPRs are maintained in the kernel which contains two
properties:
1. supported_mode: Helps in identifying if it strictly supports self
   save or restore or both.
   Initialized using the information from device tree.
2. preferred_mode: Calls out what mode is preferred for each SPR. It
   could be strictly self save or restore, or it can also
   determine the preference of  mode over the other if both
   are present by encapsulating the other in bitmask from
   LSB to MSB.
   Initialized statically.

Below is a table to show the Scenario::Consequence when the self save and
self restore modes are available or disabled in different combinations as
perceived from the device tree thus giving complete backwards compatibly
regardless of an older firmware running a newer kernel or vise-versa.
Support for self save or self-restore is embedded in the device tree,
along with the set of registers it supports.

SR = Self restore; SS = Self save

.---..
| Scenario  |Consequence |
:---+:
| Legacy Firmware. No SS or SR node | Self restore is called for all |
|   | supported SPRs |
:---+:
| SR: !active SS: !active   | Deep stop states disabled  |
:---+:
| SR: active SS: !active| Self restore is called for all |
|   | supported SPRs |
:---+:
| SR: active SS: active | Goes through the preferences for each  |
|   | SPR and executes of the modes  |
|   | accordingly. Currently, Self restore is|
|   | called for all the SPRs except PSSCR   |
|   | which is self saved|
:---+:
| SR: active(only HID0) SS: active  | Self save called for all supported |
|   | registers expect HID0 (as HID0 cannot  |
|   | be self saved currently)   |
:---+:
| SR: !active SS: active| currently will disable deep states as  |
|   | HID0 is needed to be self restored and |
|   | cannot be self saved   |
'---''

Pratik Rajesh Sampat (3):
  powerpc/powernv: Interface to define support and preference for a SPR
  powerpc/powernv: Introduce Self save support
  powerpc/powernv: Parse device tree, population of SPR support

 arch/powerpc/include/asm/opal-api.h|   3 +-
 arch/powerpc/include/asm/opal.h|   1 +
 arch/powerpc/platforms/powernv/idle.c  | 433 ++---
 arch/powerpc/platforms/powernv/opal-call.c |   1 +
 4 files changed, 381 insertions(+), 57 deletions(-)

-- 
2.24.1

[RFC 3/3] powerpc/powernv: Parse device tree, population of SPR support

[Pratik Rajesh Sampat]

Parse the device tree for nodes self-save, self-restore and populate
support for the preferred SPRs based what was advertised by the device
tree.

Signed-off-by: Pratik Rajesh Sampat 
---
 arch/powerpc/platforms/powernv/idle.c | 104 ++
 1 file changed, 104 insertions(+)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index e33bb3fd88ac..b86d5da4561d 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -1427,6 +1427,107 @@ static void __init pnv_probe_idle_states(void)
supported_cpuidle_states |= pnv_idle_states[i].flags;
 }
 
+/*
+ * Extracts and populates the self save or restore capabilities
+ * passed from the device tree node
+ */
+static int extract_save_restore_state_dt(struct device_node *np, int type)
+{
+   int nr_sprns = 0, i, bitmask_index;
+   int rc = 0;
+   u64 *temp_u64;
+   const char *state_prop;
+   u64 bit_pos;
+
+   state_prop = of_get_property(np, "status", NULL);
+   if (!state_prop) {
+   pr_warn("opal: failed to find the active value for self 
save/restore node");
+   return -EINVAL;
+   }
+   if (strncmp(state_prop, "disabled", 8) == 0) {
+   /*
+* if the feature is not active, strip the preferred_sprs from
+* that capability.
+*/
+   if (type == SELF_RESTORE_TYPE) {
+   for (i = 0; i < nr_preferred_sprs; i++) {
+   preferred_sprs[i].supported_mode &=
+   ~SELF_RESTORE_STRICT;
+   }
+   } else {
+   for (i = 0; i < nr_preferred_sprs; i++) {
+   preferred_sprs[i].supported_mode &=
+   ~SELF_SAVE_STRICT;
+   }
+   }
+   return 0;
+   }
+   nr_sprns = of_property_count_u64_elems(np, "sprn-bitmask");
+   if (nr_sprns <= 0)
+   return rc;
+   temp_u64 = kcalloc(nr_sprns, sizeof(u64), GFP_KERNEL);
+   if (of_property_read_u64_array(np, "sprn-bitmask",
+  temp_u64, nr_sprns)) {
+   pr_warn("cpuidle-powernv: failed to find registers in DT\n");
+   kfree(temp_u64);
+   return -EINVAL;
+   }
+   /*
+* Populate acknowledgment of support for the sprs in the global vector
+* gotten by the registers supplied by the firmware.
+* The registers are in a bitmask, bit index within
+* that specifies the SPR
+*/
+   for (i = 0; i < nr_preferred_sprs; i++) {
+   bitmask_index = preferred_sprs[i].spr / 64;
+   bit_pos = preferred_sprs[i].spr % 64;
+   if ((temp_u64[bitmask_index] & (1UL << bit_pos)) == 0) {
+   if (type == SELF_RESTORE_TYPE)
+   preferred_sprs[i].supported_mode &=
+   ~SELF_RESTORE_STRICT;
+   else
+   preferred_sprs[i].supported_mode &=
+   ~SELF_SAVE_STRICT;
+   continue;
+   }
+   if (type == SELF_RESTORE_TYPE) {
+   preferred_sprs[i].supported_mode |=
+   SELF_RESTORE_STRICT;
+   } else {
+   preferred_sprs[i].supported_mode |=
+   SELF_SAVE_STRICT;
+   }
+   }
+
+   kfree(temp_u64);
+   return rc;
+}
+
+static int pnv_parse_deepstate_dt(void)
+{
+   struct device_node *np, *np1;
+   int rc = 0;
+
+   /* Self restore register population */
+   np = of_find_node_by_path("/ibm,opal/power-mgt/self-restore");
+   if (!np) {
+   pr_warn("opal: self restore Node not found");
+   } else {
+   rc = extract_save_restore_state_dt(np, SELF_RESTORE_TYPE);
+   if (rc != 0)
+   return rc;
+   }
+   /* Self save register population */
+   np1 = of_find_node_by_path("/ibm,opal/power-mgt/self-save");
+   if (!np1) {
+   pr_warn("opal: self save Node not found");
+   pr_warn("Legacy firmware. Assuming default self-restore 
support");
+   } else {
+   rc = extract_save_restore_state_dt(np1, SELF_SAVE_TYPE);
+   }
+   return rc;
+}
+
 /*
  * This function parses device-tree and populates all the information
  * into pnv_idle_states structure. It also sets up nr_pnv_idle_states
@@ -1575,6 +1676,9 @@ static int __init pnv_init_idle_states(void)
return rc;
pnv_pr

[RFC 0/3] Integrate Support for self-save and determine

[Pratik Rajesh Sampat]

Currently the stop-api supports a mechanism called as self-restore
which allows us to restore the values of certain SPRs on wakeup from a
deep-stop state to a desired value. To use this, the Kernel makes an
OPAL call passing the PIR of the CPU, the SPR number and the value to
which the SPR should be restored when that CPU wakes up from a deep
stop state.

Recently, a new feature, named self-save has been enabled in the
stop-api, which is an alternative mechanism to do the same, except
that self-save will save the current content of the SPR before
entering a deep stop state and also restore the content back on
waking up from a deep stop state.

This patch series aims at introducing and leveraging the self-save feature in
the kernel.

Now, as the kernel has a choice to prefer one mode over the other and
there can be registers in both the save/restore SPR list which are sent
from the device tree, a new interface has been defined for the seamless
handing of the modes for each SPR.

A list of preferred SPRs are maintained in the kernel which contains two
properties:
1. supported_mode: Helps in identifying if it strictly supports self
   save or restore or both.
2. preferred_mode: Calls out what mode is preferred for each SPR. It
   could be strictly self save or restore, or it can also
   determine the preference of  mode over the other if both
   are present by encapsulating the other in bitmask from
   LSB to MSB.
Below is a table to show the Scenario::Consequence when the self save and
self restore modes are available or disabled in different combinations as
perceived from the device tree thus giving complete backwards compatibly
regardless of an older firmware running a newer kernel or vise-versa.

SR = Self restore; SS = Self save

.---..
| Scenario  |Consequence |
:---+:
| Legacy Firmware. No SS or SR node | Self restore is called for all |
|   | supported SPRs |
:---+:
| SR: !active SS: !active   | Deep stop states disabled  |
:---+:
| SR: active SS: !active| Self restore is called for all |
|   | supported SPRs |
:---+:
| SR: active SS: active | Goes through the preferences for each  |
|   | SPR and executes of the modes  |
|   | accordingly. Currently, Self restore is|
|   | called for all the SPRs except PSSCR   |
|   | which is self saved|
:---+:
| SR: active(only HID0) SS: active  | Self save called for all supported |
|   | registers expect HID0 (as HID0 cannot  |
|   | be self saved currently)   |
:---+:
| SR: !active SS: active| currently will disable deep states as  |
|   | HID0 is needed to be self restored and |
|   | cannot be self saved   |
'---''

Pratik Rajesh Sampat (3):
  powerpc/powernv: Interface to define support and preference for a SPR
  powerpc/powernv: Introduce Self save support
  powerpc/powernv: Parse device tree, population of SPR support

 arch/powerpc/include/asm/opal-api.h|   3 +-
 arch/powerpc/include/asm/opal.h|   1 +
 arch/powerpc/platforms/powernv/idle.c  | 431 ++---
 arch/powerpc/platforms/powernv/opal-call.c |   1 +
 4 files changed, 379 insertions(+), 57 deletions(-)

-- 
2.21.0

[RFC 2/3] powerpc/powernv: Introduce Self save support

[Pratik Rajesh Sampat]

This commit introduces and leverages the Self save API which OPAL now
supports.

Add the new Self Save OPAL API call in the list of OPAL calls.
Implement the self saving of the SPRs based on the support populated
while respecting it's preferences.

This implementation allows mixing of support for the SPRs, which
means that a SPR can be self restored while another SPR be self saved if
they support and prefer it to be so.

Signed-off-by: Pratik Rajesh Sampat 
---
 arch/powerpc/include/asm/opal-api.h| 3 ++-
 arch/powerpc/include/asm/opal.h| 1 +
 arch/powerpc/platforms/powernv/idle.c  | 2 ++
 arch/powerpc/platforms/powernv/opal-call.c | 1 +
 4 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/arch/powerpc/include/asm/opal-api.h 
b/arch/powerpc/include/asm/opal-api.h
index c1f25a760eb1..89b7c44124e6 100644
--- a/arch/powerpc/include/asm/opal-api.h
+++ b/arch/powerpc/include/asm/opal-api.h
@@ -214,7 +214,8 @@
 #define OPAL_SECVAR_GET176
 #define OPAL_SECVAR_GET_NEXT   177
 #define OPAL_SECVAR_ENQUEUE_UPDATE 178
-#define OPAL_LAST  178
+#define OPAL_SLW_SELF_SAVE_REG 179
+#define OPAL_LAST  179
 
 #define QUIESCE_HOLD   1 /* Spin all calls at entry */
 #define QUIESCE_REJECT 2 /* Fail all calls with OPAL_BUSY */
diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h
index 9986ac34b8e2..389a85b63805 100644
--- a/arch/powerpc/include/asm/opal.h
+++ b/arch/powerpc/include/asm/opal.h
@@ -203,6 +203,7 @@ int64_t opal_handle_hmi(void);
 int64_t opal_handle_hmi2(__be64 *out_flags);
 int64_t opal_register_dump_region(uint32_t id, uint64_t start, uint64_t end);
 int64_t opal_unregister_dump_region(uint32_t id);
+int64_t opal_slw_self_save_reg(uint64_t cpu_pir, uint64_t sprn);
 int64_t opal_slw_set_reg(uint64_t cpu_pir, uint64_t sprn, uint64_t val);
 int64_t opal_config_cpu_idle_state(uint64_t state, uint64_t flag);
 int64_t opal_pci_set_phb_cxl_mode(uint64_t phb_id, uint64_t mode, uint64_t 
pe_number);
diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index d38b8b6dcbce..e33bb3fd88ac 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -1170,6 +1170,8 @@ void pnv_program_cpu_hotplug_lpcr(unsigned int cpu, u64 
lpcr_val)
if (!is_lpcr_self_save)
opal_slw_set_reg(pir, SPRN_LPCR,
 lpcr_val);
+   else
+   opal_slw_self_save_reg(pir, SPRN_LPCR);
}
 }
 
diff --git a/arch/powerpc/platforms/powernv/opal-call.c 
b/arch/powerpc/platforms/powernv/opal-call.c
index 5cd0f52d258f..11e0ceb90de0 100644
--- a/arch/powerpc/platforms/powernv/opal-call.c
+++ b/arch/powerpc/platforms/powernv/opal-call.c
@@ -223,6 +223,7 @@ OPAL_CALL(opal_handle_hmi,  
OPAL_HANDLE_HMI);
 OPAL_CALL(opal_handle_hmi2,OPAL_HANDLE_HMI2);
 OPAL_CALL(opal_config_cpu_idle_state,  OPAL_CONFIG_CPU_IDLE_STATE);
 OPAL_CALL(opal_slw_set_reg,OPAL_SLW_SET_REG);
+OPAL_CALL(opal_slw_self_save_reg,  OPAL_SLW_SELF_SAVE_REG);
 OPAL_CALL(opal_register_dump_region,   OPAL_REGISTER_DUMP_REGION);
 OPAL_CALL(opal_unregister_dump_region, OPAL_UNREGISTER_DUMP_REGION);
 OPAL_CALL(opal_pci_set_phb_cxl_mode,   OPAL_PCI_SET_PHB_CAPI_MODE);
-- 
2.21.0

[RFC 1/3] powerpc/powernv: Interface to define support and preference for a SPR

[Pratik Rajesh Sampat]

Define a bitmask interface to determine support for the Self Restore,
Self Save or both.

Also define an interface to determine the preference of that SPR to
be strictly saved or restored or encapsulated with an order of preference.

The preference bitmask is shown as below:

|... | 2nd pref | 1st pref |

MSB   LSB

The preference from higher to lower is from LSB to MSB with a shift of 8
bits.
Example:
Prefer self save first, if not available then prefer self
restore
The preference mask for this scenario will be seen as below.
((SELF_RESTORE_STRICT << PREFERENCE_SHIFT) | SELF_SAVE_STRICT)
-
|... | Self restore | Self save |
-
MSB LSB

Finally, declare a list of preferred SPRs which encapsulate the bitmaks
for preferred and supported with defaults of both being set to support
legacy firmware.

This commit also implements using the above interface and retains the
legacy functionality of self restore.

Signed-off-by: Pratik Rajesh Sampat 
---
 arch/powerpc/platforms/powernv/idle.c | 325 +-
 1 file changed, 269 insertions(+), 56 deletions(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 78599bca66c2..d38b8b6dcbce 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -32,9 +32,106 @@
 #define P9_STOP_SPR_MSR 2000
 #define P9_STOP_SPR_PSSCR  855
 
+/* Interface for the stop state supported and preference */
+#define SELF_RESTORE_TYPE0
+#define SELF_SAVE_TYPE   1
+
+#define NR_PREFERENCES2
+#define PREFERENCE_SHIFT  8
+#define PREFERENCE_MASK   0xff
+
+#define UNSUPPORTED 0x0
+#define SELF_RESTORE_STRICT 0x01
+#define SELF_SAVE_STRICT0x10
+
+/*
+ * Bitmask defining the kind of preferences available.
+ * Note : The higher to lower preference is from LSB to MSB, with a shift of
+ * 8 bits.
+ * 
+ * || 2nd pref | 1st pref |
+ * 
+ * MSB   LSB
+ */
+/* Prefer Restore if available, otherwise unsupported */
+#define PREFER_SELF_RESTORE_ONLY   SELF_RESTORE_STRICT
+/* Prefer Save if available, otherwise unsupported */
+#define PREFER_SELF_SAVE_ONLY  SELF_SAVE_STRICT
+/* Prefer Restore when available, otherwise prefer Save */
+#define PREFER_RESTORE_SAVE((SELF_SAVE_STRICT << \
+ PREFERENCE_SHIFT)\
+ | SELF_RESTORE_STRICT)
+/* Prefer Save when available, otherwise prefer Restore*/
+#define PREFER_SAVE_RESTORE((SELF_RESTORE_STRICT <<\
+ PREFERENCE_SHIFT)\
+ | SELF_SAVE_STRICT)
 static u32 supported_cpuidle_states;
 struct pnv_idle_states_t *pnv_idle_states;
 int nr_pnv_idle_states;
+/* Caching the lpcr & ptcr support to use later */
+static bool is_lpcr_self_save;
+static bool is_ptcr_self_save;
+
+struct preferred_sprs {
+   u64 spr;
+   u32 preferred_mode;
+   u32 supported_mode;
+};
+
+struct preferred_sprs preferred_sprs[] = {
+   {
+   .spr = SPRN_HSPRG0,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_LPCR,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_PTCR,
+   .preferred_mode = PREFER_SAVE_RESTORE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HMEER,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HID0,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = P9_STOP_SPR_MSR,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = P9_STOP_SPR_PSSCR,
+   .preferred_mode = PREFER_SAVE_RESTORE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HID1,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HID4,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HID5,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   }
+};
+
+const int nr_preferred_sprs = ARR

[PATCH v1 3/3] powerpc/powernv: Parse device tree, population of SPR support

[Pratik Rajesh Sampat]

Parse the device tree for nodes self-save, self-restore and populate
support for the preferred SPRs based what was advertised by the device
tree.

Signed-off-by: Pratik Rajesh Sampat 
---
 arch/powerpc/platforms/powernv/idle.c | 104 ++
 1 file changed, 104 insertions(+)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index f0dd377820c7..43ad0e543b84 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -1408,6 +1408,107 @@ static void __init pnv_probe_idle_states(void)
supported_cpuidle_states |= pnv_idle_states[i].flags;
 }
 
+/*
+ * Extracts and populates the self save or restore capabilities
+ * passed from the device tree node
+ */
+static int extract_save_restore_state_dt(struct device_node *np, int type)
+{
+   int nr_sprns = 0, i, bitmask_index;
+   int rc = 0;
+   u32 active = 0;
+   u64 *temp_u64;
+   u64 bit_pos;
+
+   if (of_property_read_u32(np, "active", )) {
+   pr_warn("opal: failed to find the active value for self 
save/restore node");
+   return -EINVAL;
+   }
+
+   if (!active) {
+   /*
+* if the feature is not active, strip the preferred_sprs from
+* that capability.
+*/
+   if (type == SELF_RESTORE_TYPE) {
+   for (i = 0; i < nr_preferred_sprs; i++) {
+   preferred_sprs[i].supported_mode &=
+   ~SELF_RESTORE_STRICT;
+   }
+   } else {
+   for (i = 0; i < nr_preferred_sprs; i++) {
+   preferred_sprs[i].supported_mode &=
+   ~SELF_SAVE_STRICT;
+   }
+   }
+   return 0;
+   }
+   nr_sprns = of_property_count_u64_elems(np, "sprn-bitmask");
+   if (nr_sprns <= 0)
+   return rc;
+   temp_u64 = kcalloc(nr_sprns, sizeof(u64), GFP_KERNEL);
+   if (of_property_read_u64_array(np, "sprn-bitmask",
+  temp_u64, nr_sprns)) {
+   pr_warn("cpuidle-powernv: failed to find registers in DT\n");
+   kfree(temp_u64);
+   return -EINVAL;
+   }
+   /*
+* Populate acknowledgment of support for the sprs in the global vector
+* gotten by the registers supplied by the firmware.
+* The registers are in a bitmask, bit index within
+* that specifies the SPR
+*/
+   for (i = 0; i < nr_preferred_sprs; i++) {
+   bitmask_index = preferred_sprs[i].spr / 64;
+   bit_pos = preferred_sprs[i].spr % 64;
+   if ((temp_u64[bitmask_index] & (1UL << bit_pos)) == 0) {
+   if (type == SELF_RESTORE_TYPE)
+   preferred_sprs[i].supported_mode &=
+   ~SELF_RESTORE_STRICT;
+   else
+   preferred_sprs[i].supported_mode &=
+   ~SELF_SAVE_STRICT;
+   continue;
+   }
+   if (type == SELF_RESTORE_TYPE) {
+   preferred_sprs[i].supported_mode |=
+   SELF_RESTORE_STRICT;
+   } else {
+   preferred_sprs[i].supported_mode |=
+   SELF_SAVE_STRICT;
+   }
+   }
+
+   kfree(temp_u64);
+   return rc;
+}
+
+static int pnv_parse_deepstate_dt(void)
+{
+   struct device_node *np, *np1;
+   int rc = 0;
+
+   /* Self restore register population */
+   np = of_find_node_by_path("/ibm,opal/power-mgt/self-restore");
+   if (!np) {
+   pr_warn("opal: self restore Node not found");
+   } else {
+   rc = extract_save_restore_state_dt(np, SELF_RESTORE_TYPE);
+   if (rc != 0)
+   return rc;
+   }
+   /* Self save register population */
+   np1 = of_find_node_by_path("/ibm,opal/power-mgt/self-save");
+   if (!np1) {
+   pr_warn("opal: self save Node not found");
+   pr_warn("Legacy firmware. Assuming default self-restore 
support");
+   } else {
+   rc = extract_save_restore_state_dt(np1, SELF_SAVE_TYPE);
+   }
+   return rc;
+}
+
 /*
  * This function parses device-tree and populates all the information
  * into pnv_idle_states structure. It also sets up nr_pnv_idle_states
@@ -1556,6 +1657,9 @@ static int __init pnv_init_idle_states(void)
return rc;
pnv_probe_idle_states();
 
+   rc = pnv_parse_deepstate_dt();
+   if (rc)
+

[PATCH v1 2/3] powerpc/powernv: Introduce Self save support

[Pratik Rajesh Sampat]

This commit introduces and leverages the Self save API which OPAL now
supports.

Add the new Self Save OPAL API call in the list of OPAL calls.
Implement the self saving of the SPRs based on the support populated
while respecting it's preferences.

This implementation allows mixing of support for the SPRs, which
means that a SPR can be self restored while another SPR be self saved if
they support and prefer it to be so.

Signed-off-by: Pratik Rajesh Sampat 
---
 arch/powerpc/include/asm/opal-api.h|  3 ++-
 arch/powerpc/include/asm/opal.h|  1 +
 arch/powerpc/platforms/powernv/idle.c  | 16 
 arch/powerpc/platforms/powernv/opal-call.c |  1 +
 4 files changed, 20 insertions(+), 1 deletion(-)

diff --git a/arch/powerpc/include/asm/opal-api.h 
b/arch/powerpc/include/asm/opal-api.h
index 383242eb0dea..4ee1641c60a4 100644
--- a/arch/powerpc/include/asm/opal-api.h
+++ b/arch/powerpc/include/asm/opal-api.h
@@ -208,7 +208,8 @@
 #define OPAL_HANDLE_HMI2   166
 #defineOPAL_NX_COPROC_INIT 167
 #define OPAL_XIVE_GET_VP_STATE 170
-#define OPAL_LAST  170
+#define OPAL_SLW_SELF_SAVE_REG 173
+#define OPAL_LAST  173
 
 #define QUIESCE_HOLD   1 /* Spin all calls at entry */
 #define QUIESCE_REJECT 2 /* Fail all calls with OPAL_BUSY */
diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h
index 57bd029c715e..832eff21d189 100644
--- a/arch/powerpc/include/asm/opal.h
+++ b/arch/powerpc/include/asm/opal.h
@@ -202,6 +202,7 @@ int64_t opal_handle_hmi(void);
 int64_t opal_handle_hmi2(__be64 *out_flags);
 int64_t opal_register_dump_region(uint32_t id, uint64_t start, uint64_t end);
 int64_t opal_unregister_dump_region(uint32_t id);
+int64_t opal_slw_self_save_reg(uint64_t cpu_pir, uint64_t sprn);
 int64_t opal_slw_set_reg(uint64_t cpu_pir, uint64_t sprn, uint64_t val);
 int64_t opal_config_cpu_idle_state(uint64_t state, uint64_t flag);
 int64_t opal_pci_set_phb_cxl_mode(uint64_t phb_id, uint64_t mode, uint64_t 
pe_number);
diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 00b3fd58fafc..f0dd377820c7 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -264,6 +264,20 @@ static int pnv_self_save_restore_sprs(void)
if (rc != 0)
return rc;
break;
+   } else if (preferred & curr_spr.supported_mode
+  & SELF_SAVE_STRICT) {
+   is_initialized = true;
+   if (curr_spr.spr == SPRN_HMEER &&
+   cpu_thread_in_core(cpu) != 0) {
+   continue;
+   }
+   if (curr_spr.spr == SPRN_LPCR)
+   is_lpcr_self_save = true;
+   rc = opal_slw_self_save_reg(pir,
+   curr_spr.spr);
+   if (rc != 0)
+   return rc;
+   break;
}
preferred_sprs[index].preferred_mode =
preferred_sprs[index].preferred_mode >>
@@ -1137,6 +1151,8 @@ void pnv_program_cpu_hotplug_lpcr(unsigned int cpu, u64 
lpcr_val)
if (!is_lpcr_self_save)
opal_slw_set_reg(pir, SPRN_LPCR,
 lpcr_val);
+   else
+   opal_slw_self_save_reg(pir, SPRN_LPCR);
}
 }
 
diff --git a/arch/powerpc/platforms/powernv/opal-call.c 
b/arch/powerpc/platforms/powernv/opal-call.c
index 29ca523c1c79..716f2118939b 100644
--- a/arch/powerpc/platforms/powernv/opal-call.c
+++ b/arch/powerpc/platforms/powernv/opal-call.c
@@ -223,6 +223,7 @@ OPAL_CALL(opal_handle_hmi,  
OPAL_HANDLE_HMI);
 OPAL_CALL(opal_handle_hmi2,OPAL_HANDLE_HMI2);
 OPAL_CALL(opal_config_cpu_idle_state,  OPAL_CONFIG_CPU_IDLE_STATE);
 OPAL_CALL(opal_slw_set_reg,OPAL_SLW_SET_REG);
+OPAL_CALL(opal_slw_self_save_reg,  OPAL_SLW_SELF_SAVE_REG);
 OPAL_CALL(opal_register_dump_region,   OPAL_REGISTER_DUMP_REGION);
 OPAL_CALL(opal_unregister_dump_region, OPAL_UNREGISTER_DUMP_REGION);
 OPAL_CALL(opal_pci_set_phb_cxl_mode,   OPAL_PCI_SET_PHB_CAPI_MODE);
-- 
2.21.0

[PATCH v1 1/3] powerpc/powernv: Interface to define support and preference for a SPR

[Pratik Rajesh Sampat]

Define a bitmask interface to determine support for the Self Restore,
Self Save or both.

Also define an interface to determine the preference of that SPR to
be strictly saved or restored or encapsulated with an order of preference.

The preference bitmask is shown as below:

|... | 2nd pref | 1st pref |

MSB   LSB

The preference from higher to lower is from LSB to MSB with a shift of 8
bits.
Example:
Prefer self save first, if not available then prefer self
restore
The preference mask for this scenario will be seen as below.
((SELF_RESTORE_STRICT << PREFERENCE_SHIFT) | SELF_SAVE_STRICT)
-
|... | Self restore | Self save |
-
MSB LSB

Finally, declare a list of preferred SPRs which encapsulate the bitmaks
for preferred and supported with defaults of both being set to support
legacy firmware.

This commit also implements using the above interface and retains the
legacy functionality of self restore.

Signed-off-by: Pratik Rajesh Sampat 
---
 arch/powerpc/platforms/powernv/idle.c | 284 +-
 1 file changed, 233 insertions(+), 51 deletions(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c 
b/arch/powerpc/platforms/powernv/idle.c
index 09f49eed7fb8..00b3fd58fafc 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -32,9 +32,100 @@
 #define P9_STOP_SPR_MSR 2000
 #define P9_STOP_SPR_PSSCR  855
 
+/* Interface for the stop state supported and preference */
+#define SELF_RESTORE_TYPE0
+#define SELF_SAVE_TYPE   1
+
+#define NR_PREFERENCES2
+#define PREFERENCE_SHIFT  8
+#define PREFERENCE_MASK   0xff
+
+#define UNSUPPORTED 0x0
+#define SELF_RESTORE_STRICT 0x01
+#define SELF_SAVE_STRICT0x10
+
+/*
+ * Bitmask defining the kind of preferences available.
+ * Note : The higher to lower preference is from LSB to MSB, with a shift of
+ * 8 bits.
+ * 
+ * || 2nd pref | 1st pref |
+ * 
+ * MSB   LSB
+ */
+/* Prefer Restore if available, otherwise unsupported */
+#define PREFER_SELF_RESTORE_ONLY   SELF_RESTORE_STRICT
+/* Prefer Save if available, otherwise unsupported */
+#define PREFER_SELF_SAVE_ONLY  SELF_SAVE_STRICT
+/* Prefer Restore when available, otherwise prefer Save */
+#define PREFER_RESTORE_SAVE((SELF_SAVE_STRICT << \
+ PREFERENCE_SHIFT)\
+ | SELF_RESTORE_STRICT)
+/* Prefer Save when available, otherwise prefer Restore*/
+#define PREFER_SAVE_RESTORE((SELF_RESTORE_STRICT <<\
+ PREFERENCE_SHIFT)\
+ | SELF_SAVE_STRICT)
 static u32 supported_cpuidle_states;
 struct pnv_idle_states_t *pnv_idle_states;
 int nr_pnv_idle_states;
+/* Caching the lpcr support to use later */
+static bool is_lpcr_self_save;
+
+struct preferred_sprs {
+   u64 spr;
+   u32 preferred_mode;
+   u32 supported_mode;
+};
+
+struct preferred_sprs preferred_sprs[] = {
+   {
+   .spr = SPRN_HSPRG0,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_LPCR,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HMEER,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HID0,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = P9_STOP_SPR_MSR,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = P9_STOP_SPR_PSSCR,
+   .preferred_mode = PREFER_SAVE_RESTORE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HID1,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HID4,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   },
+   {
+   .spr = SPRN_HID5,
+   .preferred_mode = PREFER_RESTORE_SAVE,
+   .supported_mode = SELF_RESTORE_STRICT,
+   }
+};
+
+const int nr_preferred_sprs = ARRAY_SIZE(preferred_sprs);
 
 /*
  * The default stop state that will be used by ppc_md.power_save
@@ -61,78 +152,166 @@ static bool deepest_stop_found;
 
 static unsigned long power7_offline_t

[PATCH v1 0/3] Integrate support for Self save, determine support and preference for self save or restore

[Pratik Rajesh Sampat]

Currently the stop-api supports a mechanism called as self-restore
which allows us to restore the values of certain SPRs on wakeup from a
deep-stop state to a desired value. To use this, the Kernel makes an
OPAL call passing the PIR of the CPU, the SPR number and the value to
which the SPR should be restored when that CPU wakes up from a deep
stop state.

Recently, a new feature, named self-save has been enabled in the
stop-api, which is an alternative mechanism to do the same, except
that self-save will save the current content of the SPR before
entering a deep stop state and also restore the content back on
waking up from a deep stop state.

This patch series aims at introducing and leveraging the self-save feature in
the kernel.

Now, as the kernel has a choice to prefer one mode over the other and
there can be registers in both the save/restore SPR list which are sent
from the device tree, a new interface has been defined for the seamless
handing of the modes for each SPR.

A list of preferred SPRs are maintained in the kernel which contains two
properties:
1. supported_mode: Helps in identifying if it strictly supports self
   save or restore or both.
2. preferred_mode: Calls out what mode is preferred for each SPR. It
   could be strictly self save or restore, or it can also
   determine the preference of  mode over the other if both
   are present by encapsulating the other in bitmask from
   LSB to MSB.
Below is a table to show the Scenario::Consequence when the self save and
self restore modes are available or disabled in different combinations as
perceived from the device tree.

SR = Self restore; SS = Self save

.---..
| Scenario  |Consequence |
:---+:
| Legacy Firmware. No SS or SR node | Self restore is called for all |
|   | supported SPRs |
:---+:
| SR: !active SS: !active   | Deep stop states disabled  |
:---+:
| SR: active SS: !active| Self restore is called for all |
|   | supported SPRs |
:---+:
| SR: active SS: active | Goes through the preferences for each  |
|   | SPR and executes of the modes  |
|   | accordingly. Currently, Self restore is|
|   | called for all the SPRs except PSSCR   |
|   | which is self saved|
:---+:
| SR: active(only HID0) SS: active  | Self save called for all supported |
|   | registers expect HID0 (as HID0 cannot  |
|   | be self saved currently)   |
:---+:
| SR: !active SS: active| currently will disable deep states as  |
|   | HID0 is needed to be self restored and |
|   | cannot be self saved   |
'---''

Pratik Rajesh Sampat (3):
  powerpc/powernv: Interface to define support and preference for a SPR
  powerpc/powernv: Introduce Self save support
  powerpc/powernv: Parse device tree, population of SPR support

 arch/powerpc/include/asm/opal-api.h|   3 +-
 arch/powerpc/include/asm/opal.h|   1 +
 arch/powerpc/platforms/powernv/idle.c  | 404 ++---
 arch/powerpc/platforms/powernv/opal-call.c |   1 +
 4 files changed, 357 insertions(+), 52 deletions(-)

-- 
2.21.0