Re: [PATCH v6 01/17] powerpc/vas: Define macros, register fields and structures

[Sukadev Bhattiprolu]

Michael Ellerman [m...@ellerman.id.au] wrote:
> Sukadev Bhattiprolu  writes:
> 
> > Nicholas Piggin [npig...@gmail.com] wrote:
> >> On Mon, 14 Aug 2017 15:21:48 +1000
> >> Michael Ellerman  wrote:
> >> 
> >> > Sukadev Bhattiprolu  writes:
> >> 
> >> > >  arch/powerpc/include/asm/vas.h   |  35 
> >> > >  arch/powerpc/include/uapi/asm/vas.h  |  25 +++  
> >> > 
> >> > I thought we weren't exposing VAS to userspace yet?
> >> > 
> >> > If we are then we need to get things straight WRT copy/paste abort.
> ...
> >
> > In the FTW case, there is no data transfer from user space to the hardware.

Sorry, that was focussed on the paste side.

> > i.e the copy/paste submit a NULL CRB and hardware will be configured (see
> > ->fifo_disable setting in winctx) to ignore any data they specify in the 
> > CRB.
> 
> I thought the copy did copy a cacheline, but then the paste to the VAS
> window just ignores the contents, and doesn't allow userspace to get the
> content in any way?

Yes, you are right. The copy instruction does read the CRB into its copy-
buffer but for the FTW, VAS ignores the copy-buffer contents on paste.
So, the CRB may be zeroed, but must be a valid buffer.

> 
> Which means we have two thirds of a covert channel, ie. something can be
> copied into the copy buffer by one process, and then a second process
> can paste it, but because it can only paste to foreign memory, and the
> only foreign memory it can get is a VAS FTW window, it can't actually
> see the content of the copy buffer.
> 
> > Would we be able to allow copy/paste from user space in that case?
> 
> Yeah I think so, but it is all a bit fragile.
> 
> cheers

Re: [PATCH v6 01/17] powerpc/vas: Define macros, register fields and structures

[Sukadev Bhattiprolu]

Michael Ellerman [m...@ellerman.id.au] wrote:
> Sukadev Bhattiprolu  writes:
> 
> > Nicholas Piggin [npig...@gmail.com] wrote:
> >> On Mon, 14 Aug 2017 15:21:48 +1000
> >> Michael Ellerman  wrote:
> >> 
> >> > Sukadev Bhattiprolu  writes:
> >> 
> >> > >  arch/powerpc/include/asm/vas.h   |  35 
> >> > >  arch/powerpc/include/uapi/asm/vas.h  |  25 +++  
> >> > 
> >> > I thought we weren't exposing VAS to userspace yet?
> >> > 
> >> > If we are then we need to get things straight WRT copy/paste abort.
> ...
> >
> > In the FTW case, there is no data transfer from user space to the hardware.

Sorry, that was focussed on the paste side.

> > i.e the copy/paste submit a NULL CRB and hardware will be configured (see
> > ->fifo_disable setting in winctx) to ignore any data they specify in the 
> > CRB.
> 
> I thought the copy did copy a cacheline, but then the paste to the VAS
> window just ignores the contents, and doesn't allow userspace to get the
> content in any way?

Yes, you are right. The copy instruction does read the CRB into its copy-
buffer but for the FTW, VAS ignores the copy-buffer contents on paste.
So, the CRB may be zeroed, but must be a valid buffer.

> 
> Which means we have two thirds of a covert channel, ie. something can be
> copied into the copy buffer by one process, and then a second process
> can paste it, but because it can only paste to foreign memory, and the
> only foreign memory it can get is a VAS FTW window, it can't actually
> see the content of the copy buffer.
> 
> > Would we be able to allow copy/paste from user space in that case?
> 
> Yeah I think so, but it is all a bit fragile.
> 
> cheers

Re: [PATCH v6 01/17] powerpc/vas: Define macros, register fields and structures

[Michael Ellerman]

Sukadev Bhattiprolu  writes:

> Nicholas Piggin [npig...@gmail.com] wrote:
>> On Mon, 14 Aug 2017 15:21:48 +1000
>> Michael Ellerman  wrote:
>> 
>> > Sukadev Bhattiprolu  writes:
>> 
>> > >  arch/powerpc/include/asm/vas.h   |  35 
>> > >  arch/powerpc/include/uapi/asm/vas.h  |  25 +++  
>> > 
>> > I thought we weren't exposing VAS to userspace yet?
>> > 
>> > If we are then we need to get things straight WRT copy/paste abort.
...
>
> In the FTW case, there is no data transfer from user space to the hardware.
> i.e the copy/paste submit a NULL CRB and hardware will be configured (see
> ->fifo_disable setting in winctx) to ignore any data they specify in the CRB.

I thought the copy did copy a cacheline, but then the paste to the VAS
window just ignores the contents, and doesn't allow userspace to get the
content in any way?

Which means we have two thirds of a covert channel, ie. something can be
copied into the copy buffer by one process, and then a second process
can paste it, but because it can only paste to foreign memory, and the
only foreign memory it can get is a VAS FTW window, it can't actually
see the content of the copy buffer.

> Would we be able to allow copy/paste from user space in that case?

Yeah I think so, but it is all a bit fragile.

cheers

Re: [PATCH v6 01/17] powerpc/vas: Define macros, register fields and structures

[Michael Ellerman]

Sukadev Bhattiprolu  writes:

> Nicholas Piggin [npig...@gmail.com] wrote:
>> On Mon, 14 Aug 2017 15:21:48 +1000
>> Michael Ellerman  wrote:
>> 
>> > Sukadev Bhattiprolu  writes:
>> 
>> > >  arch/powerpc/include/asm/vas.h   |  35 
>> > >  arch/powerpc/include/uapi/asm/vas.h  |  25 +++  
>> > 
>> > I thought we weren't exposing VAS to userspace yet?
>> > 
>> > If we are then we need to get things straight WRT copy/paste abort.
...
>
> In the FTW case, there is no data transfer from user space to the hardware.
> i.e the copy/paste submit a NULL CRB and hardware will be configured (see
> ->fifo_disable setting in winctx) to ignore any data they specify in the CRB.

I thought the copy did copy a cacheline, but then the paste to the VAS
window just ignores the contents, and doesn't allow userspace to get the
content in any way?

Which means we have two thirds of a covert channel, ie. something can be
copied into the copy buffer by one process, and then a second process
can paste it, but because it can only paste to foreign memory, and the
only foreign memory it can get is a VAS FTW window, it can't actually
see the content of the copy buffer.

> Would we be able to allow copy/paste from user space in that case?

Yeah I think so, but it is all a bit fragile.

cheers

Re: [PATCH v6 01/17] powerpc/vas: Define macros, register fields and structures

[Sukadev Bhattiprolu]

Nicholas Piggin [npig...@gmail.com] wrote:
> On Mon, 14 Aug 2017 15:21:48 +1000
> Michael Ellerman  wrote:
> 
> > Sukadev Bhattiprolu  writes:
> 
> > >  arch/powerpc/include/asm/vas.h   |  35 
> > >  arch/powerpc/include/uapi/asm/vas.h  |  25 +++  
> > 
> > I thought we weren't exposing VAS to userspace yet?
> > 
> > If we are then we need to get things straight WRT copy/paste abort.
> 

> No we should not be. This might be just a leftover hunk that should
> be moved to a future series.

Yes, I should have posted patches 14..17 separately as an RFC that goes
on top of the VAS kernel patches 1..13.

> 
> At the moment (as far as I understand) it should be limited to
> preempt-disabled, process context, kernel users which avoids any
> concern for switch_to.
> 

In the FTW case, there is no data transfer from user space to the hardware.
i.e the copy/paste submit a NULL CRB and hardware will be configured (see
->fifo_disable setting in winctx) to ignore any data they specify in the CRB.

Would we be able to allow copy/paste from user space in that case?

Sukadev

Re: [PATCH v6 01/17] powerpc/vas: Define macros, register fields and structures

[Sukadev Bhattiprolu]

Nicholas Piggin [npig...@gmail.com] wrote:
> On Mon, 14 Aug 2017 15:21:48 +1000
> Michael Ellerman  wrote:
> 
> > Sukadev Bhattiprolu  writes:
> 
> > >  arch/powerpc/include/asm/vas.h   |  35 
> > >  arch/powerpc/include/uapi/asm/vas.h  |  25 +++  
> > 
> > I thought we weren't exposing VAS to userspace yet?
> > 
> > If we are then we need to get things straight WRT copy/paste abort.
> 

> No we should not be. This might be just a leftover hunk that should
> be moved to a future series.

Yes, I should have posted patches 14..17 separately as an RFC that goes
on top of the VAS kernel patches 1..13.

> 
> At the moment (as far as I understand) it should be limited to
> preempt-disabled, process context, kernel users which avoids any
> concern for switch_to.
> 

In the FTW case, there is no data transfer from user space to the hardware.
i.e the copy/paste submit a NULL CRB and hardware will be configured (see
->fifo_disable setting in winctx) to ignore any data they specify in the CRB.

Would we be able to allow copy/paste from user space in that case?

Sukadev

Re: [PATCH v6 01/17] powerpc/vas: Define macros, register fields and structures

[Michael Ellerman]

Nicholas Piggin  writes:

> On Mon, 14 Aug 2017 15:21:48 +1000
> Michael Ellerman  wrote:
>
>> Sukadev Bhattiprolu  writes:
>
>> >  arch/powerpc/include/asm/vas.h   |  35 
>> >  arch/powerpc/include/uapi/asm/vas.h  |  25 +++  
>> 
>> I thought we weren't exposing VAS to userspace yet?
>> 
>> If we are then we need to get things straight WRT copy/paste abort.
>
> No we should not be. This might be just a leftover hunk that should
> be moved to a future series.
>
> At the moment (as far as I understand) it should be limited to
> preempt-disabled, process context, kernel users which avoids any
> concern for switch_to.

I think that comment applied to a previous version, see patch 16.

cheers

Re: [PATCH v6 01/17] powerpc/vas: Define macros, register fields and structures

[Michael Ellerman]

Nicholas Piggin  writes:

> On Mon, 14 Aug 2017 15:21:48 +1000
> Michael Ellerman  wrote:
>
>> Sukadev Bhattiprolu  writes:
>
>> >  arch/powerpc/include/asm/vas.h   |  35 
>> >  arch/powerpc/include/uapi/asm/vas.h  |  25 +++  
>> 
>> I thought we weren't exposing VAS to userspace yet?
>> 
>> If we are then we need to get things straight WRT copy/paste abort.
>
> No we should not be. This might be just a leftover hunk that should
> be moved to a future series.
>
> At the moment (as far as I understand) it should be limited to
> preempt-disabled, process context, kernel users which avoids any
> concern for switch_to.

I think that comment applied to a previous version, see patch 16.

cheers

Re: [PATCH v6 01/17] powerpc/vas: Define macros, register fields and structures

[Nicholas Piggin]

On Mon, 14 Aug 2017 15:21:48 +1000
Michael Ellerman  wrote:

> Sukadev Bhattiprolu  writes:

> >  arch/powerpc/include/asm/vas.h   |  35 
> >  arch/powerpc/include/uapi/asm/vas.h  |  25 +++  
> 
> I thought we weren't exposing VAS to userspace yet?
> 
> If we are then we need to get things straight WRT copy/paste abort.

No we should not be. This might be just a leftover hunk that should
be moved to a future series.

At the moment (as far as I understand) it should be limited to
preempt-disabled, process context, kernel users which avoids any
concern for switch_to.

Thanks,
Nick

Re: [PATCH v6 01/17] powerpc/vas: Define macros, register fields and structures

[Nicholas Piggin]

On Mon, 14 Aug 2017 15:21:48 +1000
Michael Ellerman  wrote:

> Sukadev Bhattiprolu  writes:

> >  arch/powerpc/include/asm/vas.h   |  35 
> >  arch/powerpc/include/uapi/asm/vas.h  |  25 +++  
> 
> I thought we weren't exposing VAS to userspace yet?
> 
> If we are then we need to get things straight WRT copy/paste abort.

No we should not be. This might be just a leftover hunk that should
be moved to a future series.

At the moment (as far as I understand) it should be limited to
preempt-disabled, process context, kernel users which avoids any
concern for switch_to.

Thanks,
Nick

[PATCH v6 01/17] powerpc/vas: Define macros, register fields and structures

[Sukadev Bhattiprolu]

Define macros for the VAS hardware registers and bit-fields as well
as couple of data structures needed by the VAS driver.

Signed-off-by: Sukadev Bhattiprolu 
---
Changelog[v6]
- Add some fields for FTW windows

Changelog[v4]
- [Michael Neuling] Move VAS code to arch/powerpc; Reorg vas.h and
  vas-internal.h to kernel and uapi versions; rather than creating
  separate properties for window context/address entries in device
  tree, combine them into "reg" properties; drop ->hwirq and irq_port
  fields from vas_window as they are only needed with user space
  windows.
- Drop the error check for CONFIG_PPC_4K_PAGES. Instead in a
  follow-on patch add a "depends on CONFIG_PPC_64K_PAGES".

Changelog[v3]
- Rename winctx->pid to winctx->pidr to reflect that its a value
  from the PID register (SPRN_PID), not the linux process id.
- Make it easier to split header into kernel/user parts
- To keep user interface simple, use macros rather than enum for
  the threshold-control modes.
- Add a pid field to struct vas_window - needed for user space
  send windows.

Changelog[v2]
- Add an overview of VAS in vas-internal.h
- Get window context parameters from device tree and drop
  unnecessary macros.
---
 arch/powerpc/include/asm/vas.h   |  35 
 arch/powerpc/include/uapi/asm/vas.h  |  25 +++
 arch/powerpc/platforms/powernv/vas.h | 382 +++
 3 files changed, 442 insertions(+)
 create mode 100644 arch/powerpc/include/asm/vas.h
 create mode 100644 arch/powerpc/include/uapi/asm/vas.h
 create mode 100644 arch/powerpc/platforms/powernv/vas.h

diff --git a/arch/powerpc/include/asm/vas.h b/arch/powerpc/include/asm/vas.h
new file mode 100644
index 000..2c8558a
--- /dev/null
+++ b/arch/powerpc/include/asm/vas.h
@@ -0,0 +1,35 @@
+/*
+ * Copyright 2016 IBM Corp.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#ifndef _MISC_VAS_H
+#define _MISC_VAS_H
+
+#include 
+
+/*
+ * Min and max FIFO sizes are based on Version 1.05 Section 3.1.4.25
+ * (Local FIFO Size Register) of the VAS workbook.
+ */
+#define VAS_RX_FIFO_SIZE_MIN   (1 << 10)   /* 1KB */
+#define VAS_RX_FIFO_SIZE_MAX   (8 << 20)   /* 8MB */
+
+/*
+ * Co-processor Engine type.
+ */
+enum vas_cop_type {
+   VAS_COP_TYPE_FAULT,
+   VAS_COP_TYPE_842,
+   VAS_COP_TYPE_842_HIPRI,
+   VAS_COP_TYPE_GZIP,
+   VAS_COP_TYPE_GZIP_HIPRI,
+   VAS_COP_TYPE_FTW,
+   VAS_COP_TYPE_MAX,
+};
+
+#endif /* _MISC_VAS_H */
diff --git a/arch/powerpc/include/uapi/asm/vas.h 
b/arch/powerpc/include/uapi/asm/vas.h
new file mode 100644
index 000..ddfe046
--- /dev/null
+++ b/arch/powerpc/include/uapi/asm/vas.h
@@ -0,0 +1,25 @@
+/*
+ * Copyright 2016 IBM Corp.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#ifndef _UAPI_MISC_VAS_H
+#define _UAPI_MISC_VAS_H
+
+/*
+ * Threshold Control Mode: Have paste operation fail if the number of
+ * requests in receive FIFO exceeds a threshold.
+ *
+ * NOTE: No special error code yet if paste is rejected because of these
+ *  limits. So users can't distinguish between this and other errors.
+ */
+#define VAS_THRESH_DISABLED0
+#define VAS_THRESH_FIFO_GT_HALF_FULL   1
+#define VAS_THRESH_FIFO_GT_QTR_FULL2
+#define VAS_THRESH_FIFO_GT_EIGHTH_FULL 3
+
+#endif /* _UAPI_MISC_VAS_H */
diff --git a/arch/powerpc/platforms/powernv/vas.h 
b/arch/powerpc/platforms/powernv/vas.h
new file mode 100644
index 000..312a378
--- /dev/null
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -0,0 +1,382 @@
+/*
+ * Copyright 2016 IBM Corp.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#ifndef _VAS_H
+#define _VAS_H
+#include 
+#include 
+#include 
+
+/*
+ * Overview of Virtual Accelerator Switchboard (VAS).
+ *
+ * VAS is a hardware "switchboard" that allows senders and receivers to
+ * exchange messages with _minimal_ kernel involvment. The receivers are
+ * typically NX coprocessor engines that perform compression or encryption
+ * in hardware, but receivers can also be other software threads.
+ *
+ * Senders are user/kernel threads that submit compression/encryption or
+ * other requests to the receivers. Senders must format their messages as
+ * Coprocessor 

[PATCH v6 01/17] powerpc/vas: Define macros, register fields and structures

[Sukadev Bhattiprolu]

Define macros for the VAS hardware registers and bit-fields as well
as couple of data structures needed by the VAS driver.

Signed-off-by: Sukadev Bhattiprolu 
---
Changelog[v6]
- Add some fields for FTW windows

Changelog[v4]
- [Michael Neuling] Move VAS code to arch/powerpc; Reorg vas.h and
  vas-internal.h to kernel and uapi versions; rather than creating
  separate properties for window context/address entries in device
  tree, combine them into "reg" properties; drop ->hwirq and irq_port
  fields from vas_window as they are only needed with user space
  windows.
- Drop the error check for CONFIG_PPC_4K_PAGES. Instead in a
  follow-on patch add a "depends on CONFIG_PPC_64K_PAGES".

Changelog[v3]
- Rename winctx->pid to winctx->pidr to reflect that its a value
  from the PID register (SPRN_PID), not the linux process id.
- Make it easier to split header into kernel/user parts
- To keep user interface simple, use macros rather than enum for
  the threshold-control modes.
- Add a pid field to struct vas_window - needed for user space
  send windows.

Changelog[v2]
- Add an overview of VAS in vas-internal.h
- Get window context parameters from device tree and drop
  unnecessary macros.
---
 arch/powerpc/include/asm/vas.h   |  35 
 arch/powerpc/include/uapi/asm/vas.h  |  25 +++
 arch/powerpc/platforms/powernv/vas.h | 382 +++
 3 files changed, 442 insertions(+)
 create mode 100644 arch/powerpc/include/asm/vas.h
 create mode 100644 arch/powerpc/include/uapi/asm/vas.h
 create mode 100644 arch/powerpc/platforms/powernv/vas.h

diff --git a/arch/powerpc/include/asm/vas.h b/arch/powerpc/include/asm/vas.h
new file mode 100644
index 000..2c8558a
--- /dev/null
+++ b/arch/powerpc/include/asm/vas.h
@@ -0,0 +1,35 @@
+/*
+ * Copyright 2016 IBM Corp.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#ifndef _MISC_VAS_H
+#define _MISC_VAS_H
+
+#include 
+
+/*
+ * Min and max FIFO sizes are based on Version 1.05 Section 3.1.4.25
+ * (Local FIFO Size Register) of the VAS workbook.
+ */
+#define VAS_RX_FIFO_SIZE_MIN   (1 << 10)   /* 1KB */
+#define VAS_RX_FIFO_SIZE_MAX   (8 << 20)   /* 8MB */
+
+/*
+ * Co-processor Engine type.
+ */
+enum vas_cop_type {
+   VAS_COP_TYPE_FAULT,
+   VAS_COP_TYPE_842,
+   VAS_COP_TYPE_842_HIPRI,
+   VAS_COP_TYPE_GZIP,
+   VAS_COP_TYPE_GZIP_HIPRI,
+   VAS_COP_TYPE_FTW,
+   VAS_COP_TYPE_MAX,
+};
+
+#endif /* _MISC_VAS_H */
diff --git a/arch/powerpc/include/uapi/asm/vas.h 
b/arch/powerpc/include/uapi/asm/vas.h
new file mode 100644
index 000..ddfe046
--- /dev/null
+++ b/arch/powerpc/include/uapi/asm/vas.h
@@ -0,0 +1,25 @@
+/*
+ * Copyright 2016 IBM Corp.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#ifndef _UAPI_MISC_VAS_H
+#define _UAPI_MISC_VAS_H
+
+/*
+ * Threshold Control Mode: Have paste operation fail if the number of
+ * requests in receive FIFO exceeds a threshold.
+ *
+ * NOTE: No special error code yet if paste is rejected because of these
+ *  limits. So users can't distinguish between this and other errors.
+ */
+#define VAS_THRESH_DISABLED0
+#define VAS_THRESH_FIFO_GT_HALF_FULL   1
+#define VAS_THRESH_FIFO_GT_QTR_FULL2
+#define VAS_THRESH_FIFO_GT_EIGHTH_FULL 3
+
+#endif /* _UAPI_MISC_VAS_H */
diff --git a/arch/powerpc/platforms/powernv/vas.h 
b/arch/powerpc/platforms/powernv/vas.h
new file mode 100644
index 000..312a378
--- /dev/null
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -0,0 +1,382 @@
+/*
+ * Copyright 2016 IBM Corp.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#ifndef _VAS_H
+#define _VAS_H
+#include 
+#include 
+#include 
+
+/*
+ * Overview of Virtual Accelerator Switchboard (VAS).
+ *
+ * VAS is a hardware "switchboard" that allows senders and receivers to
+ * exchange messages with _minimal_ kernel involvment. The receivers are
+ * typically NX coprocessor engines that perform compression or encryption
+ * in hardware, but receivers can also be other software threads.
+ *
+ * Senders are user/kernel threads that submit compression/encryption or
+ * other requests to the receivers. Senders must format their messages as
+ * Coprocessor Request Blocks (CRB)s and submit