Re: [PATCH 02/10] powerpc/xive: guest exploitation of the XIVE interrupt controller

[David Gibson]

On Thu, Aug 10, 2017 at 09:36:04PM +1000, Benjamin Herrenschmidt wrote:
> On Thu, 2017-08-10 at 09:19 +0200, Cédric Le Goater wrote:
> > > > > > +  /* Perform the acknowledge hypervisor to register cycle */
> > > > > > +  ack = be16_to_cpu(__raw_readw(xive_tima + TM_SPC_ACK_OS_REG));
> > > > > 
> > > > > Why do you need the raw_readw() + be16_to_cpu + mb, rather than one of
> > > > > the higher level IO helpers?
> > > > 
> > > > This is one of the many ways to do MMIOs on the TIMA. This memory 
> > > > region defines a set of offsets and sizes for which loads and 
> > > > stores have different effects. 
> > > > 
> > > > See the arch/powerpc/include/asm/xive-regs.h file and 
> > > > arch/powerpc/kvm/book3s_xive_template.c for some more usage.
> > > 
> > > Sure, much like any IO region.  My point is, why do you want this kind
> > > of complex combo, rather than say an in_be16() or readw_be().
> > > 
> > 
> > The code is inherited from the native backend. 
> > 
> > I think this is because we know what we are doing and we skip 
> > the synchronization routines of the higher level IO helpers. 
> > That might not be necessary for sPAPR. Ben ? 
> 
> It's a performance optimisation, we know we don't need the
> sync,twi,isync crap of the higher level accessor here.

Ok.  A comment mentioning this would be good - unless you're very
familiar with the code and hardware it's not going to be obvious if
the nonstandard IO accessors are for legitimate performance reasons,
or just cargo-culting.


-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [PATCH 02/10] powerpc/xive: guest exploitation of the XIVE interrupt controller

[Benjamin Herrenschmidt]

On Thu, 2017-08-10 at 09:19 +0200, Cédric Le Goater wrote:
> > > > > +  /* Perform the acknowledge hypervisor to register cycle */
> > > > > +  ack = be16_to_cpu(__raw_readw(xive_tima + TM_SPC_ACK_OS_REG));
> > > > 
> > > > Why do you need the raw_readw() + be16_to_cpu + mb, rather than one of
> > > > the higher level IO helpers?
> > > 
> > > This is one of the many ways to do MMIOs on the TIMA. This memory 
> > > region defines a set of offsets and sizes for which loads and 
> > > stores have different effects. 
> > > 
> > > See the arch/powerpc/include/asm/xive-regs.h file and 
> > > arch/powerpc/kvm/book3s_xive_template.c for some more usage.
> > 
> > Sure, much like any IO region.  My point is, why do you want this kind
> > of complex combo, rather than say an in_be16() or readw_be().
> > 
> 
> The code is inherited from the native backend. 
> 
> I think this is because we know what we are doing and we skip 
> the synchronization routines of the higher level IO helpers. 
> That might not be necessary for sPAPR. Ben ? 

It's a performance optimisation, we know we don't need the
sync,twi,isync crap of the higher level accessor here.

Cheers,
Ben.

Re: [PATCH 02/10] powerpc/xive: guest exploitation of the XIVE interrupt controller

[Benjamin Herrenschmidt]

On Thu, 2017-08-10 at 08:45 +0200, Cédric Le Goater wrote:
> > The problem with doorbells on POWER9 guests is that they may have
> > to trap and be emulated by the hypervisor, since the guest threads
> > on P9 don't have to match the HW threads of the core.
> 
> Well, the pseries cause_ipi() handler does :
> 
> if (doorbell_try_core_ipi(cpu))
> return;
> 
> to limit the doorbells to the same core. So we should be fine ?

No. It's theorically possible to create a guest that think it has 4
threads on P9 but those threads run on different cores of the host.

The doorbells are useful if KVM uses a "P8 style" whole-core dispatch
model or with PowerVM. We should probably invent some kind of DT
property to tell the guest I suppoes.

>  If not
> I suppose we should check CPU_FTR_ARCH_300 and use IPIs only for XIVE.
> 
> > Thus it's quite possible that using XIVE for IPIs is actually faster
> > than doorbells in that case.
> 
> How can we measure that ? ebizzy may be.

Or a simple socket ping pong with processes pinned to different
threads.

However the current KVM for P9 doesn't do threads yet afaik.

Cheers,
Ben.

Re: [PATCH 02/10] powerpc/xive: guest exploitation of the XIVE interrupt controller

[Cédric Le Goater]

On 08/10/2017 07:54 AM, David Gibson wrote:
> On Thu, Aug 10, 2017 at 02:46:00PM +1000, Benjamin Herrenschmidt wrote:
>> On Thu, 2017-08-10 at 14:28 +1000, David Gibson wrote:
>>>
>>> Also, will POWER9 always have doorbells?  In which case you could
>>> reduce it to 3 options.
>>
>> The problem with doorbells on POWER9 guests is that they may have
>> to trap and be emulated by the hypervisor, since the guest threads
>> on P9 don't have to match the HW threads of the core.
>>
>> Thus it's quite possible that using XIVE for IPIs is actually faster
>> than doorbells in that case.
> 
> Ok, well unless there's a compelling reason to use doorbells you can
> instead make it 3 cases with POWER9 never using doorbells.

yes I suppose we could improve things for XIVE. For XICS, there are 
another 3 backends behind icp_ops->cause_ipi. I need to take a look.

Thanks,

C. 

Re: [PATCH 02/10] powerpc/xive: guest exploitation of the XIVE interrupt controller

[Cédric Le Goater]

 +static void xive_spapr_update_pending(struct xive_cpu *xc)
 +{
 +  u8 nsr, cppr;
 +  u16 ack;
 +
 +  /* Perform the acknowledge hypervisor to register cycle */
 +  ack = be16_to_cpu(__raw_readw(xive_tima + TM_SPC_ACK_OS_REG));
>>>
>>> Why do you need the raw_readw() + be16_to_cpu + mb, rather than one of
>>> the higher level IO helpers?
>>
>> This is one of the many ways to do MMIOs on the TIMA. This memory 
>> region defines a set of offsets and sizes for which loads and 
>> stores have different effects. 
>>
>> See the arch/powerpc/include/asm/xive-regs.h file and 
>> arch/powerpc/kvm/book3s_xive_template.c for some more usage.
> 
> Sure, much like any IO region.  My point is, why do you want this kind
> of complex combo, rather than say an in_be16() or readw_be().
> 

The code is inherited from the native backend. 

I think this is because we know what we are doing and we skip 
the synchronization routines of the higher level IO helpers. 
That might not be necessary for sPAPR. Ben ? 

Thanks,

C.   

Re: [PATCH 02/10] powerpc/xive: guest exploitation of the XIVE interrupt controller

[Cédric Le Goater]

On 08/10/2017 06:46 AM, Benjamin Herrenschmidt wrote:
> On Thu, 2017-08-10 at 14:28 +1000, David Gibson wrote:
>>
>> Also, will POWER9 always have doorbells?  In which case you could
>> reduce it to 3 options.
> 
> The problem with doorbells on POWER9 guests is that they may have
> to trap and be emulated by the hypervisor, since the guest threads
> on P9 don't have to match the HW threads of the core.

Well, the pseries cause_ipi() handler does :

if (doorbell_try_core_ipi(cpu))
return;

to limit the doorbells to the same core. So we should be fine ? If not
I suppose we should check CPU_FTR_ARCH_300 and use IPIs only for XIVE.

> Thus it's quite possible that using XIVE for IPIs is actually faster
> than doorbells in that case.

How can we measure that ? ebizzy may be.

C.

Re: [PATCH 02/10] powerpc/xive: guest exploitation of the XIVE interrupt controller

[David Gibson]

On Thu, Aug 10, 2017 at 02:46:00PM +1000, Benjamin Herrenschmidt wrote:
> On Thu, 2017-08-10 at 14:28 +1000, David Gibson wrote:
> > 
> > Also, will POWER9 always have doorbells?  In which case you could
> > reduce it to 3 options.
> 
> The problem with doorbells on POWER9 guests is that they may have
> to trap and be emulated by the hypervisor, since the guest threads
> on P9 don't have to match the HW threads of the core.
> 
> Thus it's quite possible that using XIVE for IPIs is actually faster
> than doorbells in that case.

Ok, well unless there's a compelling reason to use doorbells you can
instead make it 3 cases with POWER9 never using doorbells.

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [PATCH 02/10] powerpc/xive: guest exploitation of the XIVE interrupt controller

[Benjamin Herrenschmidt]

On Thu, 2017-08-10 at 14:28 +1000, David Gibson wrote:
> 
> Also, will POWER9 always have doorbells?  In which case you could
> reduce it to 3 options.

The problem with doorbells on POWER9 guests is that they may have
to trap and be emulated by the hypervisor, since the guest threads
on P9 don't have to match the HW threads of the core.

Thus it's quite possible that using XIVE for IPIs is actually faster
than doorbells in that case.

Cheers,
Ben.

Re: [PATCH 02/10] powerpc/xive: guest exploitation of the XIVE interrupt controller

[David Gibson]

On Wed, Aug 09, 2017 at 10:48:48AM +0200, Cédric Le Goater wrote:
> On 08/09/2017 05:53 AM, David Gibson wrote:
> > On Tue, Aug 08, 2017 at 10:56:12AM +0200, Cédric Le Goater wrote:
> >> This is the framework for using XIVE in a PowerVM guest. The support
> >> is very similar to the native one in a much simpler form.
> >>
> >> Instead of OPAL calls, a set of Hypervisors call are used to configure
> >> the interrupt sources and the event/notification queues of the guest:
> >>
> >>  - H_INT_GET_SOURCE_INFO
> >>
> >>used to obtain the address of the MMIO page of the Event State
> >>Buffer (PQ bits) entry associated with the source.
> >>
> >>  - H_INT_SET_SOURCE_CONFIG
> >>
> >>assigns a source to a "target".
> >>
> >>  - H_INT_GET_SOURCE_CONFIG
> >>
> >>determines to which "target" and "priority" is assigned to a source
> >>
> >>  - H_INT_GET_QUEUE_INFO
> >>
> >>returns the address of the notification management page associated
> >>with the specified "target" and "priority".
> >>
> >>  - H_INT_SET_QUEUE_CONFIG
> >>
> >>sets or resets the event queue for a given "target" and "priority".
> >>It is also used to set the notification config associated with the
> >>queue, only unconditional notification for the moment.  Reset is
> >>performed with a queue size of 0 and queueing is disabled in that
> >>case.
> >>
> >>  - H_INT_GET_QUEUE_CONFIG
> >>
> >>returns the queue settings for a given "target" and "priority".
> >>
> >>  - H_INT_RESET
> >>
> >>resets all of the partition's interrupt exploitation structures to
> >>their initial state, losing all configuration set via the hcalls
> >>H_INT_SET_SOURCE_CONFIG and H_INT_SET_QUEUE_CONFIG.
> >>
> >>  - H_INT_SYNC
> >>
> >>issue a synchronisation on a source to make sure sure all
> >>notifications have reached their queue.
> >>
> >> As for XICS, the XIVE interface for the guest is described in the
> >> device tree under the interrupt controller node. A couple of new
> >> properties are specific to XIVE :
> >>
> >>  - "reg"
> >>
> >>contains the base address and size of the thread interrupt
> >>managnement areas (TIMA) for the user level for the OS level. Only
> >>the OS level is taken into account.
> >>
> >>  - "ibm,xive-eq-sizes"
> >>
> >>the size of the event queues.
> >>
> >>  - "ibm,xive-lisn-ranges"
> >>
> >>the interrupt numbers ranges assigned to the guest. These are
> >>allocated using a simple bitmap.
> >>
> >> Tested with a QEMU XIVE model for pseries and with the Power
> >> hypervisor
> >>
> >> Signed-off-by: Cédric Le Goater 
> > 
> > I don't know XIVE well enough to review in detail, but I've made some
> > comments based on general knowledge.
> 
> Thanks for taking a look.

np

[snip]
> >> +/* Cause IPI as setup by the interrupt controller (xics or xive) */
> >> +static void (*ic_cause_ipi)(int cpu);
> >> +
> >>  static void smp_pseries_cause_ipi(int cpu)
> >>  {
> >> -  /* POWER9 should not use this handler */
> >>if (doorbell_try_core_ipi(cpu))
> >>return;
> >>  
> >> -  icp_ops->cause_ipi(cpu);
> >> +  ic_cause_ipi(cpu);
> > 
> > Wouldn't it make more sense to change smp_ops->cause_ipi, rather than
> > having a double indirection through smp_ops, then the ic_cause_ipi
> > global?
> 
> we need to retain the original setting of smp_ops->cause_ipi 
> somewhere as it can be set from xive_smp_probe() to : 
> 
>   icp_ops->cause_ipi 
> 
> or from xics_smp_probe() to :
> 
>   xive_cause_ipi()
> 
> I am not sure we can do any better ? 

I don't see why not.  You basically have two bits of options xics vs
xive, and doorbell vs no doorbells.  At worst that gives you 4
different versions of ->cause_ipi, and you can work out the right one
in smp_probe().  If the number of combinations got too much larger you
might indeed need some more indirection.  But with 4 there's a little
code duplication, but small enough that I think it's preferable to an
extra global and an extra indirection.

Also, will POWER9 always have doorbells?  In which case you could
reduce it to 3 options.

[snip]
> >> +static void xive_spapr_update_pending(struct xive_cpu *xc)
> >> +{
> >> +  u8 nsr, cppr;
> >> +  u16 ack;
> >> +
> >> +  /* Perform the acknowledge hypervisor to register cycle */
> >> +  ack = be16_to_cpu(__raw_readw(xive_tima + TM_SPC_ACK_OS_REG));
> > 
> > Why do you need the raw_readw() + be16_to_cpu + mb, rather than one of
> > the higher level IO helpers?
> 
> This is one of the many ways to do MMIOs on the TIMA. This memory 
> region defines a set of offsets and sizes for which loads and 
> stores have different effects. 
> 
> See the arch/powerpc/include/asm/xive-regs.h file and 
> arch/powerpc/kvm/book3s_xive_template.c for some more usage.

Sure, much like any IO region.  My point is, why do you want this kind
of complex combo, rather than say an in_be16() or readw_be().

-- 
David Gibson| I'll have my music baroque, and my 

Re: [PATCH 02/10] powerpc/xive: guest exploitation of the XIVE interrupt controller

[Cédric Le Goater]

On 08/09/2017 05:53 AM, David Gibson wrote:
> On Tue, Aug 08, 2017 at 10:56:12AM +0200, Cédric Le Goater wrote:
>> This is the framework for using XIVE in a PowerVM guest. The support
>> is very similar to the native one in a much simpler form.
>>
>> Instead of OPAL calls, a set of Hypervisors call are used to configure
>> the interrupt sources and the event/notification queues of the guest:
>>
>>  - H_INT_GET_SOURCE_INFO
>>
>>used to obtain the address of the MMIO page of the Event State
>>Buffer (PQ bits) entry associated with the source.
>>
>>  - H_INT_SET_SOURCE_CONFIG
>>
>>assigns a source to a "target".
>>
>>  - H_INT_GET_SOURCE_CONFIG
>>
>>determines to which "target" and "priority" is assigned to a source
>>
>>  - H_INT_GET_QUEUE_INFO
>>
>>returns the address of the notification management page associated
>>with the specified "target" and "priority".
>>
>>  - H_INT_SET_QUEUE_CONFIG
>>
>>sets or resets the event queue for a given "target" and "priority".
>>It is also used to set the notification config associated with the
>>queue, only unconditional notification for the moment.  Reset is
>>performed with a queue size of 0 and queueing is disabled in that
>>case.
>>
>>  - H_INT_GET_QUEUE_CONFIG
>>
>>returns the queue settings for a given "target" and "priority".
>>
>>  - H_INT_RESET
>>
>>resets all of the partition's interrupt exploitation structures to
>>their initial state, losing all configuration set via the hcalls
>>H_INT_SET_SOURCE_CONFIG and H_INT_SET_QUEUE_CONFIG.
>>
>>  - H_INT_SYNC
>>
>>issue a synchronisation on a source to make sure sure all
>>notifications have reached their queue.
>>
>> As for XICS, the XIVE interface for the guest is described in the
>> device tree under the interrupt controller node. A couple of new
>> properties are specific to XIVE :
>>
>>  - "reg"
>>
>>contains the base address and size of the thread interrupt
>>managnement areas (TIMA) for the user level for the OS level. Only
>>the OS level is taken into account.
>>
>>  - "ibm,xive-eq-sizes"
>>
>>the size of the event queues.
>>
>>  - "ibm,xive-lisn-ranges"
>>
>>the interrupt numbers ranges assigned to the guest. These are
>>allocated using a simple bitmap.
>>
>> Tested with a QEMU XIVE model for pseries and with the Power
>> hypervisor
>>
>> Signed-off-by: Cédric Le Goater 
> 
> I don't know XIVE well enough to review in detail, but I've made some
> comments based on general knowledge.

Thanks for taking a look. 
 
> 
>> ---
>>  arch/powerpc/include/asm/hvcall.h|  13 +-
>>  arch/powerpc/include/asm/xive.h  |   2 +
>>  arch/powerpc/platforms/pseries/Kconfig   |   1 +
>>  arch/powerpc/platforms/pseries/hotplug-cpu.c |  10 +-
>>  arch/powerpc/platforms/pseries/kexec.c   |   6 +-
>>  arch/powerpc/platforms/pseries/setup.c   |   8 +-
>>  arch/powerpc/platforms/pseries/smp.c |  32 +-
>>  arch/powerpc/sysdev/xive/Kconfig |   5 +
>>  arch/powerpc/sysdev/xive/Makefile|   1 +
>>  arch/powerpc/sysdev/xive/spapr.c | 554 
>> +++
>>  10 files changed, 619 insertions(+), 13 deletions(-)
>>  create mode 100644 arch/powerpc/sysdev/xive/spapr.c
>>
>> diff --git a/arch/powerpc/include/asm/hvcall.h 
>> b/arch/powerpc/include/asm/hvcall.h
>> index 57d38b504ff7..3d34dc0869f6 100644
>> --- a/arch/powerpc/include/asm/hvcall.h
>> +++ b/arch/powerpc/include/asm/hvcall.h
>> @@ -280,7 +280,18 @@
>>  #define H_RESIZE_HPT_COMMIT 0x370
>>  #define H_REGISTER_PROC_TBL 0x37C
>>  #define H_SIGNAL_SYS_RESET  0x380
>> -#define MAX_HCALL_OPCODEH_SIGNAL_SYS_RESET
>> +#define H_INT_GET_SOURCE_INFO   0x3A8
>> +#define H_INT_SET_SOURCE_CONFIG 0x3AC
>> +#define H_INT_GET_SOURCE_CONFIG 0x3B0
>> +#define H_INT_GET_QUEUE_INFO0x3B4
>> +#define H_INT_SET_QUEUE_CONFIG  0x3B8
>> +#define H_INT_GET_QUEUE_CONFIG  0x3BC
>> +#define H_INT_SET_OS_REPORTING_LINE 0x3C0
>> +#define H_INT_GET_OS_REPORTING_LINE 0x3C4
>> +#define H_INT_ESB   0x3C8
>> +#define H_INT_SYNC  0x3CC
>> +#define H_INT_RESET 0x3D0
>> +#define MAX_HCALL_OPCODEH_INT_RESET
>>  
>>  /* H_VIOCTL functions */
>>  #define H_GET_VIOA_DUMP_SIZE0x01
>> diff --git a/arch/powerpc/include/asm/xive.h 
>> b/arch/powerpc/include/asm/xive.h
>> index c23ff4389ca2..1deb10032d61 100644
>> --- a/arch/powerpc/include/asm/xive.h
>> +++ b/arch/powerpc/include/asm/xive.h
>> @@ -110,6 +110,7 @@ extern bool __xive_enabled;
>>  
>>  static inline bool xive_enabled(void) { return __xive_enabled; }
>>  
>> +extern bool xive_spapr_init(void);
>>  extern bool xive_native_init(void);
>>  extern void xive_smp_probe(void);
>>  extern int  xive_smp_prepare_cpu(unsigned int cpu);
>> @@ -147,6 +148,7 @@ extern int xive_native_get_vp_info(u32 vp_id, u32 
>> *out_cam_id, u32 *out_chip_id)
>>  
>>  static inline bool xive_enabled(void) { return false; }
>>  
>> +static 

Re: [PATCH 02/10] powerpc/xive: guest exploitation of the XIVE interrupt controller

[David Gibson]

On Tue, Aug 08, 2017 at 10:56:12AM +0200, Cédric Le Goater wrote:
> This is the framework for using XIVE in a PowerVM guest. The support
> is very similar to the native one in a much simpler form.
> 
> Instead of OPAL calls, a set of Hypervisors call are used to configure
> the interrupt sources and the event/notification queues of the guest:
> 
>  - H_INT_GET_SOURCE_INFO
> 
>used to obtain the address of the MMIO page of the Event State
>Buffer (PQ bits) entry associated with the source.
> 
>  - H_INT_SET_SOURCE_CONFIG
> 
>assigns a source to a "target".
> 
>  - H_INT_GET_SOURCE_CONFIG
> 
>determines to which "target" and "priority" is assigned to a source
> 
>  - H_INT_GET_QUEUE_INFO
> 
>returns the address of the notification management page associated
>with the specified "target" and "priority".
> 
>  - H_INT_SET_QUEUE_CONFIG
> 
>sets or resets the event queue for a given "target" and "priority".
>It is also used to set the notification config associated with the
>queue, only unconditional notification for the moment.  Reset is
>performed with a queue size of 0 and queueing is disabled in that
>case.
> 
>  - H_INT_GET_QUEUE_CONFIG
> 
>returns the queue settings for a given "target" and "priority".
> 
>  - H_INT_RESET
> 
>resets all of the partition's interrupt exploitation structures to
>their initial state, losing all configuration set via the hcalls
>H_INT_SET_SOURCE_CONFIG and H_INT_SET_QUEUE_CONFIG.
> 
>  - H_INT_SYNC
> 
>issue a synchronisation on a source to make sure sure all
>notifications have reached their queue.
> 
> As for XICS, the XIVE interface for the guest is described in the
> device tree under the interrupt controller node. A couple of new
> properties are specific to XIVE :
> 
>  - "reg"
> 
>contains the base address and size of the thread interrupt
>managnement areas (TIMA) for the user level for the OS level. Only
>the OS level is taken into account.
> 
>  - "ibm,xive-eq-sizes"
> 
>the size of the event queues.
> 
>  - "ibm,xive-lisn-ranges"
> 
>the interrupt numbers ranges assigned to the guest. These are
>allocated using a simple bitmap.
> 
> Tested with a QEMU XIVE model for pseries and with the Power
> hypervisor
> 
> Signed-off-by: Cédric Le Goater 

I don't know XIVE well enough to review in detail, but I've made some
comments based on general knowledge.

> ---
>  arch/powerpc/include/asm/hvcall.h|  13 +-
>  arch/powerpc/include/asm/xive.h  |   2 +
>  arch/powerpc/platforms/pseries/Kconfig   |   1 +
>  arch/powerpc/platforms/pseries/hotplug-cpu.c |  10 +-
>  arch/powerpc/platforms/pseries/kexec.c   |   6 +-
>  arch/powerpc/platforms/pseries/setup.c   |   8 +-
>  arch/powerpc/platforms/pseries/smp.c |  32 +-
>  arch/powerpc/sysdev/xive/Kconfig |   5 +
>  arch/powerpc/sysdev/xive/Makefile|   1 +
>  arch/powerpc/sysdev/xive/spapr.c | 554 
> +++
>  10 files changed, 619 insertions(+), 13 deletions(-)
>  create mode 100644 arch/powerpc/sysdev/xive/spapr.c
> 
> diff --git a/arch/powerpc/include/asm/hvcall.h 
> b/arch/powerpc/include/asm/hvcall.h
> index 57d38b504ff7..3d34dc0869f6 100644
> --- a/arch/powerpc/include/asm/hvcall.h
> +++ b/arch/powerpc/include/asm/hvcall.h
> @@ -280,7 +280,18 @@
>  #define H_RESIZE_HPT_COMMIT  0x370
>  #define H_REGISTER_PROC_TBL  0x37C
>  #define H_SIGNAL_SYS_RESET   0x380
> -#define MAX_HCALL_OPCODE H_SIGNAL_SYS_RESET
> +#define H_INT_GET_SOURCE_INFO   0x3A8
> +#define H_INT_SET_SOURCE_CONFIG 0x3AC
> +#define H_INT_GET_SOURCE_CONFIG 0x3B0
> +#define H_INT_GET_QUEUE_INFO0x3B4
> +#define H_INT_SET_QUEUE_CONFIG  0x3B8
> +#define H_INT_GET_QUEUE_CONFIG  0x3BC
> +#define H_INT_SET_OS_REPORTING_LINE 0x3C0
> +#define H_INT_GET_OS_REPORTING_LINE 0x3C4
> +#define H_INT_ESB   0x3C8
> +#define H_INT_SYNC  0x3CC
> +#define H_INT_RESET 0x3D0
> +#define MAX_HCALL_OPCODE H_INT_RESET
>  
>  /* H_VIOCTL functions */
>  #define H_GET_VIOA_DUMP_SIZE 0x01
> diff --git a/arch/powerpc/include/asm/xive.h b/arch/powerpc/include/asm/xive.h
> index c23ff4389ca2..1deb10032d61 100644
> --- a/arch/powerpc/include/asm/xive.h
> +++ b/arch/powerpc/include/asm/xive.h
> @@ -110,6 +110,7 @@ extern bool __xive_enabled;
>  
>  static inline bool xive_enabled(void) { return __xive_enabled; }
>  
> +extern bool xive_spapr_init(void);
>  extern bool xive_native_init(void);
>  extern void xive_smp_probe(void);
>  extern int  xive_smp_prepare_cpu(unsigned int cpu);
> @@ -147,6 +148,7 @@ extern int xive_native_get_vp_info(u32 vp_id, u32 
> *out_cam_id, u32 *out_chip_id)
>  
>  static inline bool xive_enabled(void) { return false; }
>  
> +static inline bool xive_spapr_init(void) { return false; }
>  static inline bool xive_native_init(void) { return false; }
>  static inline void xive_smp_probe(void) { }
>  extern inline int