Re: [PATCH 2/9] misc: Add Xilinx AI engine device driver
On Fri, Dec 11, 2020 at 06:27:05PM -0800, Jiaying Liang wrote: > > On 12/9/20 4:47 AM, Daniel Vetter wrote: > > On Tue, Dec 08, 2020 at 11:54:57AM -0800, Jiaying Liang wrote: > > > On 12/8/20 9:12 AM, Nicolas Dufresne wrote: > > > > Le mercredi 18 novembre 2020 à 00:06 -0800, Wendy Liang a écrit : > > > > > Create AI engine device/partition hierarchical structure. > > > > > > > > > > Each AI engine device can have multiple logical partitions(groups of > > > > > AI > > > > > engine tiles). Each partition is column based and has its own node ID > > > > > in the system. AI engine device driver manages its partitions. > > > > > > > > > > Applications can access AI engine partition through the AI engine > > > > > partition driver instance. AI engine registers write is moved to > > > > > kernel > > > > > as there are registers in the AI engine array needs privilege > > > > > permission. > > > > Hi there, it's nice to see an effort to upstream an AI driver. I'm a > > > > little > > > > worried this driver is not obvious to use from it's source code itself. > > > > So you > > > > have reference to some Open Source code that demonstrate it's usage ? > > > We have AI engine library which provides a cross platforms APIs for other > > > > > > libraries/application to use the hardware. Here is the source code: > > > > > > https://github.com/Xilinx/embeddedsw/tree/master/XilinxProcessorIPLib/drivers/aienginev2/src > > > > > > The cross platforms AI engine library runs in LInux userspace it defines > > > how > > > to > > > > > > configure, and the kernel driver controls if what can be access and manage > > > errors from device. > > So I kinda ignored this driver submission because in the past all these AI > > drivers had at best incomplete open source (usually the compiler is > > closed, often also large parts of the runtime). I think yours would be the > > first that breaks this trend, is that really the case? I.e. I could make > > full use of this hw without any closed source bits to run DNN workloads > > and things like that? > AI engine can be used for signaling processing or high performance computing > > the kernel driver works on the AI engine software library which I mentioned > above, > > which will be used by Xilinx runtime: > https://xilinx.github.io/XRT/2020.2/html/index.html > > Xilinx runtime is a layer for acceleration libraries or applications to use > Xilinx accelerators. Hm maybe I'm mixing things up, but it feels like there was a discussion about the xilinx runtime in the past with a drm driver even. The hangup was that the runtime is all open, but the offline compiler isn't. Is that still the case, or am I mixing things up here a bit? > e.g. it has OpenCL implementation > > If that's the case then I think there's nothing stopping us from doing the > > right thing and merging this driver into the right subsystem: The > > subsystem for accelerators which their own memory and who want dma-buf > > integration is drivers/gpu, not drivers/misc. > > The AI engine kernel driver is used for device runtime configuration update, > > and runtime monitoring, such as async errors detection. The buffer > management > > is out of the AI engine driver, but it is covered by Xilinx runtime: > > https://github.com/Xilinx/XRT/tree/master/src/runtime_src/core/edge/drm/zocl > > AI engine driver imports the DMA buf. > > > The AI engine device is quite different to the GPU devices. The AI engine > > operations are still needs driver specific ioctls. > > We have more than 100 cores tiles, each tiles functions can be defined at > compilation > > time, at runtime, we load the configuration (application defined I/O > commands) to > > configure each tiles registers to set up routing, set up DMAs, configure > local memories, > > and enable the tiles. Setting up DMA does sound like (buffer) memory management to me. Note that the memory management that the kernel does does not necessarily translate directly to client api buffers like opencl has. At least for gpu drivers you have memory management in both the kernel and userspace. > As the AI engine device hardware is different to the GPUs, > > we are not able to make use of functions abstracted for GPUs, and we don't > manage the > > buffers in this driver. I am not sure if it is ok to add the driver to > drivers/gpu but not > > using abstractions from the GPU abstraction. > > > There is another reply to the patch series to ask for clarification on the > overview of the > > driver, and I had some discussions with other team members. I will reply to > that email > > to provide more details on overall how this driver is used. Yeah I think that should help with understanding what's the best option here. -Daniel > > Any suggestions on how to fit the driver in the drivers/gpu or other drivers > framework > > will be much appreciated. > > > Thanks, > > Wendy > > > Apologies that I'm jumping with the really big arch review when v3 is > >
Re: [PATCH 2/9] misc: Add Xilinx AI engine device driver
On 12/9/20 4:47 AM, Daniel Vetter wrote: On Tue, Dec 08, 2020 at 11:54:57AM -0800, Jiaying Liang wrote: On 12/8/20 9:12 AM, Nicolas Dufresne wrote: Le mercredi 18 novembre 2020 à 00:06 -0800, Wendy Liang a écrit : Create AI engine device/partition hierarchical structure. Each AI engine device can have multiple logical partitions(groups of AI engine tiles). Each partition is column based and has its own node ID in the system. AI engine device driver manages its partitions. Applications can access AI engine partition through the AI engine partition driver instance. AI engine registers write is moved to kernel as there are registers in the AI engine array needs privilege permission. Hi there, it's nice to see an effort to upstream an AI driver. I'm a little worried this driver is not obvious to use from it's source code itself. So you have reference to some Open Source code that demonstrate it's usage ? We have AI engine library which provides a cross platforms APIs for other libraries/application to use the hardware. Here is the source code: https://github.com/Xilinx/embeddedsw/tree/master/XilinxProcessorIPLib/drivers/aienginev2/src The cross platforms AI engine library runs in LInux userspace it defines how to configure, and the kernel driver controls if what can be access and manage errors from device. So I kinda ignored this driver submission because in the past all these AI drivers had at best incomplete open source (usually the compiler is closed, often also large parts of the runtime). I think yours would be the first that breaks this trend, is that really the case? I.e. I could make full use of this hw without any closed source bits to run DNN workloads and things like that? AI engine can be used for signaling processing or high performance computing the kernel driver works on the AI engine software library which I mentioned above, which will be used by Xilinx runtime: https://xilinx.github.io/XRT/2020.2/html/index.html Xilinx runtime is a layer for acceleration libraries or applications to use Xilinx accelerators. e.g. it has OpenCL implementation If that's the case then I think there's nothing stopping us from doing the right thing and merging this driver into the right subsystem: The subsystem for accelerators which their own memory and who want dma-buf integration is drivers/gpu, not drivers/misc. The AI engine kernel driver is used for device runtime configuration update, and runtime monitoring, such as async errors detection. The buffer management is out of the AI engine driver, but it is covered by Xilinx runtime: https://github.com/Xilinx/XRT/tree/master/src/runtime_src/core/edge/drm/zocl AI engine driver imports the DMA buf. The AI engine device is quite different to the GPU devices. The AI engine operations are still needs driver specific ioctls. We have more than 100 cores tiles, each tiles functions can be defined at compilation time, at runtime, we load the configuration (application defined I/O commands) to configure each tiles registers to set up routing, set up DMAs, configure local memories, and enable the tiles. As the AI engine device hardware is different to the GPUs, we are not able to make use of functions abstracted for GPUs, and we don't manage the buffers in this driver. I am not sure if it is ok to add the driver to drivers/gpu but not using abstractions from the GPU abstraction. There is another reply to the patch series to ask for clarification on the overview of the driver, and I had some discussions with other team members. I will reply to that email to provide more details on overall how this driver is used. Any suggestions on how to fit the driver in the drivers/gpu or other drivers framework will be much appreciated. Thanks, Wendy Apologies that I'm jumping with the really big arch review when v3 is already on the list. But last few times merging AI drivers to drivers/misc was really just a way to avoid the merge criteria for drivers/gpu acceleration drivers. I'd love to land the first real open AI driver in upstream, properly. Cheers, Daniel Best Regards, Wendy Signed-off-by: Wendy Liang Signed-off-by: Hyun Kwon --- MAINTAINERS | 8 + drivers/misc/Kconfig | 12 + drivers/misc/Makefile | 1 + drivers/misc/xilinx-ai-engine/Makefile | 11 + drivers/misc/xilinx-ai-engine/ai-engine-aie.c | 115 + drivers/misc/xilinx-ai-engine/ai-engine-dev.c | 448 ++ drivers/misc/xilinx-ai-engine/ai-engine-internal.h | 226 ++ drivers/misc/xilinx-ai-engine/ai-engine-part.c | 498 + drivers/misc/xilinx-ai-engine/ai-engine-res.c | 114 + include/uapi/linux/xlnx-ai-engine.h | 107 + 10 files changed, 1540 insertions(+) create mode 100644 drivers/misc/xilinx-ai-engine/Makefile
Re: [PATCH 2/9] misc: Add Xilinx AI engine device driver
On Tue, Dec 08, 2020 at 11:54:57AM -0800, Jiaying Liang wrote: > > On 12/8/20 9:12 AM, Nicolas Dufresne wrote: > > Le mercredi 18 novembre 2020 à 00:06 -0800, Wendy Liang a écrit : > > > Create AI engine device/partition hierarchical structure. > > > > > > Each AI engine device can have multiple logical partitions(groups of AI > > > engine tiles). Each partition is column based and has its own node ID > > > in the system. AI engine device driver manages its partitions. > > > > > > Applications can access AI engine partition through the AI engine > > > partition driver instance. AI engine registers write is moved to kernel > > > as there are registers in the AI engine array needs privilege > > > permission. > > Hi there, it's nice to see an effort to upstream an AI driver. I'm a little > > worried this driver is not obvious to use from it's source code itself. So > > you > > have reference to some Open Source code that demonstrate it's usage ? > > We have AI engine library which provides a cross platforms APIs for other > > libraries/application to use the hardware. Here is the source code: > > https://github.com/Xilinx/embeddedsw/tree/master/XilinxProcessorIPLib/drivers/aienginev2/src > > The cross platforms AI engine library runs in LInux userspace it defines how > to > > configure, and the kernel driver controls if what can be access and manage > errors from device. So I kinda ignored this driver submission because in the past all these AI drivers had at best incomplete open source (usually the compiler is closed, often also large parts of the runtime). I think yours would be the first that breaks this trend, is that really the case? I.e. I could make full use of this hw without any closed source bits to run DNN workloads and things like that? If that's the case then I think there's nothing stopping us from doing the right thing and merging this driver into the right subsystem: The subsystem for accelerators which their own memory and who want dma-buf integration is drivers/gpu, not drivers/misc. Apologies that I'm jumping with the really big arch review when v3 is already on the list. But last few times merging AI drivers to drivers/misc was really just a way to avoid the merge criteria for drivers/gpu acceleration drivers. I'd love to land the first real open AI driver in upstream, properly. Cheers, Daniel > > > Best Regards, > > Wendy > > > > > > > Signed-off-by: Wendy Liang > > > Signed-off-by: Hyun Kwon > > > --- > > > MAINTAINERS | 8 + > > > drivers/misc/Kconfig | 12 + > > > drivers/misc/Makefile | 1 + > > > drivers/misc/xilinx-ai-engine/Makefile | 11 + > > > drivers/misc/xilinx-ai-engine/ai-engine-aie.c | 115 + > > > drivers/misc/xilinx-ai-engine/ai-engine-dev.c | 448 > > > ++ > > > drivers/misc/xilinx-ai-engine/ai-engine-internal.h | 226 ++ > > > drivers/misc/xilinx-ai-engine/ai-engine-part.c | 498 > > > + > > > drivers/misc/xilinx-ai-engine/ai-engine-res.c | 114 + > > > include/uapi/linux/xlnx-ai-engine.h | 107 + > > > 10 files changed, 1540 insertions(+) > > > create mode 100644 drivers/misc/xilinx-ai-engine/Makefile > > > create mode 100644 drivers/misc/xilinx-ai-engine/ai-engine-aie.c > > > create mode 100644 drivers/misc/xilinx-ai-engine/ai-engine-dev.c > > > create mode 100644 drivers/misc/xilinx-ai-engine/ai-engine-internal.h > > > create mode 100644 drivers/misc/xilinx-ai-engine/ai-engine-part.c > > > create mode 100644 drivers/misc/xilinx-ai-engine/ai-engine-res.c > > > create mode 100644 include/uapi/linux/xlnx-ai-engine.h > > > > > > diff --git a/MAINTAINERS b/MAINTAINERS > > > index 5cc595a..40e3351 100644 > > > --- a/MAINTAINERS > > > +++ b/MAINTAINERS > > > @@ -19283,6 +19283,14 @@ T: git > > > https://github.com/Xilinx/linux-xlnx.git > > > F: Documentation/devicetree/bindings/phy/xlnx,zynqmp-psgtr.yaml > > > F: drivers/phy/xilinx/phy-zynqmp.c > > > +XILINX AI ENGINE DRIVER > > > +M: Wendy Liang > > > +S: Supported > > > +F: Documentation/devicetree/bindings/soc/xilinx/xlnx,ai-engine.yaml > > > +F: drivers/misc/xilinx-ai-engine/ > > > +F: include/linux/xlnx-ai-engine.h > > > +F: include/uapi/linux/xlnx-ai-engine.h > > > + > > > XILLYBUS DRIVER > > > M: Eli Billauer > > > L: linux-ker...@vger.kernel.org > > > diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig > > > index fafa8b0..0b8ce4d 100644 > > > --- a/drivers/misc/Kconfig > > > +++ b/drivers/misc/Kconfig > > > @@ -444,6 +444,18 @@ config XILINX_SDFEC > > > If unsure, say N. > > > +config XILINX_AIE > > > + tristate "Xilinx AI engine" > > > + depends on ARM64 || COMPILE_TEST > > > + help > > > + This option enables support for the Xilinx AI engine driver. > > > +
Re: [PATCH 2/9] misc: Add Xilinx AI engine device driver
Le mercredi 18 novembre 2020 à 00:06 -0800, Wendy Liang a écrit : > Create AI engine device/partition hierarchical structure. > > Each AI engine device can have multiple logical partitions(groups of AI > engine tiles). Each partition is column based and has its own node ID > in the system. AI engine device driver manages its partitions. > > Applications can access AI engine partition through the AI engine > partition driver instance. AI engine registers write is moved to kernel > as there are registers in the AI engine array needs privilege > permission. Hi there, it's nice to see an effort to upstream an AI driver. I'm a little worried this driver is not obvious to use from it's source code itself. So you have reference to some Open Source code that demonstrate it's usage ? > > Signed-off-by: Wendy Liang > Signed-off-by: Hyun Kwon > --- > MAINTAINERS | 8 + > drivers/misc/Kconfig | 12 + > drivers/misc/Makefile | 1 + > drivers/misc/xilinx-ai-engine/Makefile | 11 + > drivers/misc/xilinx-ai-engine/ai-engine-aie.c | 115 + > drivers/misc/xilinx-ai-engine/ai-engine-dev.c | 448 ++ > drivers/misc/xilinx-ai-engine/ai-engine-internal.h | 226 ++ > drivers/misc/xilinx-ai-engine/ai-engine-part.c | 498 > + > drivers/misc/xilinx-ai-engine/ai-engine-res.c | 114 + > include/uapi/linux/xlnx-ai-engine.h | 107 + > 10 files changed, 1540 insertions(+) > create mode 100644 drivers/misc/xilinx-ai-engine/Makefile > create mode 100644 drivers/misc/xilinx-ai-engine/ai-engine-aie.c > create mode 100644 drivers/misc/xilinx-ai-engine/ai-engine-dev.c > create mode 100644 drivers/misc/xilinx-ai-engine/ai-engine-internal.h > create mode 100644 drivers/misc/xilinx-ai-engine/ai-engine-part.c > create mode 100644 drivers/misc/xilinx-ai-engine/ai-engine-res.c > create mode 100644 include/uapi/linux/xlnx-ai-engine.h > > diff --git a/MAINTAINERS b/MAINTAINERS > index 5cc595a..40e3351 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -19283,6 +19283,14 @@ T: git https://github.com/Xilinx/linux-xlnx.git > F: Documentation/devicetree/bindings/phy/xlnx,zynqmp-psgtr.yaml > F: drivers/phy/xilinx/phy-zynqmp.c > > +XILINX AI ENGINE DRIVER > +M: Wendy Liang > +S: Supported > +F: Documentation/devicetree/bindings/soc/xilinx/xlnx,ai-engine.yaml > +F: drivers/misc/xilinx-ai-engine/ > +F: include/linux/xlnx-ai-engine.h > +F: include/uapi/linux/xlnx-ai-engine.h > + > XILLYBUS DRIVER > M: Eli Billauer > L: linux-ker...@vger.kernel.org > diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig > index fafa8b0..0b8ce4d 100644 > --- a/drivers/misc/Kconfig > +++ b/drivers/misc/Kconfig > @@ -444,6 +444,18 @@ config XILINX_SDFEC > > If unsure, say N. > > +config XILINX_AIE > + tristate "Xilinx AI engine" > + depends on ARM64 || COMPILE_TEST > + help > + This option enables support for the Xilinx AI engine driver. > + One Xilinx AI engine device can have multiple partitions (groups of > + AI engine tiles). Xilinx AI engine device driver instance manages > + AI engine partitions. User application access its partitions through > + AI engine partition instance file operations. > + > + If unsure, say N > + > config MISC_RTSX > tristate > default MISC_RTSX_PCI || MISC_RTSX_USB > diff --git a/drivers/misc/Makefile b/drivers/misc/Makefile > index d23231e..2176b18 100644 > --- a/drivers/misc/Makefile > +++ b/drivers/misc/Makefile > @@ -57,3 +57,4 @@ obj-$(CONFIG_HABANA_AI) += habanalabs/ > obj-$(CONFIG_UACCE)+= uacce/ > obj-$(CONFIG_XILINX_SDFEC) += xilinx_sdfec.o > obj-$(CONFIG_HISI_HIKEY_USB) += hisi_hikey_usb.o > +obj-$(CONFIG_XILINX_AIE) += xilinx-ai-engine/ > diff --git a/drivers/misc/xilinx-ai-engine/Makefile b/drivers/misc/xilinx-ai- > engine/Makefile > new file mode 100644 > index 000..7827a0a > --- /dev/null > +++ b/drivers/misc/xilinx-ai-engine/Makefile > @@ -0,0 +1,11 @@ > +# SPDX-License-Identifier: GPL-2.0-only > +# > +# Makefile for Xilinx AI engine device driver > +# > + > +obj-$(CONFIG_XILINX_AIE) += xilinx-aie.o > + > +xilinx-aie-$(CONFIG_XILINX_AIE) := ai-engine-aie.o \ > + ai-engine-dev.o \ > + ai-engine-part.o \ > + ai-engine-res.o > diff --git a/drivers/misc/xilinx-ai-engine/ai-engine-aie.c > b/drivers/misc/xilinx-ai-engine/ai-engine-aie.c > new file mode 100644 > index 000..319260f > --- /dev/null > +++ b/drivers/misc/xilinx-ai-engine/ai-engine-aie.c > @@ -0,0 +1,115 @@ > +// SPDX-License-Identifier: GPL-2.0 > +/* > + * Xilinx AI Engine driver AIE device specific implementation > + * > + * Copyright (C) 2020 Xilinx, Inc. >
Re: [PATCH 2/9] misc: Add Xilinx AI engine device driver
On 12/8/20 9:12 AM, Nicolas Dufresne wrote: Le mercredi 18 novembre 2020 à 00:06 -0800, Wendy Liang a écrit : Create AI engine device/partition hierarchical structure. Each AI engine device can have multiple logical partitions(groups of AI engine tiles). Each partition is column based and has its own node ID in the system. AI engine device driver manages its partitions. Applications can access AI engine partition through the AI engine partition driver instance. AI engine registers write is moved to kernel as there are registers in the AI engine array needs privilege permission. Hi there, it's nice to see an effort to upstream an AI driver. I'm a little worried this driver is not obvious to use from it's source code itself. So you have reference to some Open Source code that demonstrate it's usage ? We have AI engine library which provides a cross platforms APIs for other libraries/application to use the hardware. Here is the source code: https://github.com/Xilinx/embeddedsw/tree/master/XilinxProcessorIPLib/drivers/aienginev2/src The cross platforms AI engine library runs in LInux userspace it defines how to configure, and the kernel driver controls if what can be access and manage errors from device. Best Regards, Wendy Signed-off-by: Wendy Liang Signed-off-by: Hyun Kwon --- MAINTAINERS | 8 + drivers/misc/Kconfig | 12 + drivers/misc/Makefile | 1 + drivers/misc/xilinx-ai-engine/Makefile | 11 + drivers/misc/xilinx-ai-engine/ai-engine-aie.c | 115 + drivers/misc/xilinx-ai-engine/ai-engine-dev.c | 448 ++ drivers/misc/xilinx-ai-engine/ai-engine-internal.h | 226 ++ drivers/misc/xilinx-ai-engine/ai-engine-part.c | 498 + drivers/misc/xilinx-ai-engine/ai-engine-res.c | 114 + include/uapi/linux/xlnx-ai-engine.h | 107 + 10 files changed, 1540 insertions(+) create mode 100644 drivers/misc/xilinx-ai-engine/Makefile create mode 100644 drivers/misc/xilinx-ai-engine/ai-engine-aie.c create mode 100644 drivers/misc/xilinx-ai-engine/ai-engine-dev.c create mode 100644 drivers/misc/xilinx-ai-engine/ai-engine-internal.h create mode 100644 drivers/misc/xilinx-ai-engine/ai-engine-part.c create mode 100644 drivers/misc/xilinx-ai-engine/ai-engine-res.c create mode 100644 include/uapi/linux/xlnx-ai-engine.h diff --git a/MAINTAINERS b/MAINTAINERS index 5cc595a..40e3351 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -19283,6 +19283,14 @@ T: git https://github.com/Xilinx/linux-xlnx.git F: Documentation/devicetree/bindings/phy/xlnx,zynqmp-psgtr.yaml F: drivers/phy/xilinx/phy-zynqmp.c +XILINX AI ENGINE DRIVER +M: Wendy Liang +S: Supported +F: Documentation/devicetree/bindings/soc/xilinx/xlnx,ai-engine.yaml +F: drivers/misc/xilinx-ai-engine/ +F: include/linux/xlnx-ai-engine.h +F: include/uapi/linux/xlnx-ai-engine.h + XILLYBUS DRIVER M: Eli Billauer L: linux-ker...@vger.kernel.org diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig index fafa8b0..0b8ce4d 100644 --- a/drivers/misc/Kconfig +++ b/drivers/misc/Kconfig @@ -444,6 +444,18 @@ config XILINX_SDFEC If unsure, say N. +config XILINX_AIE + tristate "Xilinx AI engine" + depends on ARM64 || COMPILE_TEST + help + This option enables support for the Xilinx AI engine driver. + One Xilinx AI engine device can have multiple partitions (groups of + AI engine tiles). Xilinx AI engine device driver instance manages + AI engine partitions. User application access its partitions through + AI engine partition instance file operations. + + If unsure, say N + config MISC_RTSX tristate default MISC_RTSX_PCI || MISC_RTSX_USB diff --git a/drivers/misc/Makefile b/drivers/misc/Makefile index d23231e..2176b18 100644 --- a/drivers/misc/Makefile +++ b/drivers/misc/Makefile @@ -57,3 +57,4 @@ obj-$(CONFIG_HABANA_AI) += habanalabs/ obj-$(CONFIG_UACCE)+= uacce/ obj-$(CONFIG_XILINX_SDFEC) += xilinx_sdfec.o obj-$(CONFIG_HISI_HIKEY_USB) += hisi_hikey_usb.o +obj-$(CONFIG_XILINX_AIE) += xilinx-ai-engine/ diff --git a/drivers/misc/xilinx-ai-engine/Makefile b/drivers/misc/xilinx-ai- engine/Makefile new file mode 100644 index 000..7827a0a --- /dev/null +++ b/drivers/misc/xilinx-ai-engine/Makefile @@ -0,0 +1,11 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# Makefile for Xilinx AI engine device driver +# + +obj-$(CONFIG_XILINX_AIE) += xilinx-aie.o + +xilinx-aie-$(CONFIG_XILINX_AIE) := ai-engine-aie.o \ + ai-engine-dev.o \ + ai-engine-part.o \ + ai-engine-res.o diff --git a/drivers/misc/xilinx-ai-engine/ai-engine-aie.c
Re: [PATCH 2/9] misc: Add Xilinx AI engine device driver
On 12/8/20 9:12 AM, Nicolas Dufresne wrote: Le mercredi 18 novembre 2020 à 00:06 -0800, Wendy Liang a écrit : Create AI engine device/partition hierarchical structure. Each AI engine device can have multiple logical partitions(groups of AI engine tiles). Each partition is column based and has its own node ID in the system. AI engine device driver manages its partitions. Applications can access AI engine partition through the AI engine partition driver instance. AI engine registers write is moved to kernel as there are registers in the AI engine array needs privilege permission. Hi there, it's nice to see an effort to upstream an AI driver. I'm a little worried this driver is not obvious to use from it's source code itself. So you have reference to some Open Source code that demonstrate it's usage ? We have AI engine library which provides a cross platforms APIs for other libraries/application to use the hardware. Here is the source code: https://github.com/Xilinx/embeddedsw/tree/master/XilinxProcessorIPLib/drivers/aienginev2/src The cross platforms AI engine library runs in LInux userspace it defines how to configure, and the kernel driver controls if what can be access and manage errors from device. Best Regards, Wendy Signed-off-by: Wendy Liang Signed-off-by: Hyun Kwon --- MAINTAINERS | 8 + drivers/misc/Kconfig | 12 + drivers/misc/Makefile | 1 + drivers/misc/xilinx-ai-engine/Makefile | 11 + drivers/misc/xilinx-ai-engine/ai-engine-aie.c | 115 + drivers/misc/xilinx-ai-engine/ai-engine-dev.c | 448 ++ drivers/misc/xilinx-ai-engine/ai-engine-internal.h | 226 ++ drivers/misc/xilinx-ai-engine/ai-engine-part.c | 498 + drivers/misc/xilinx-ai-engine/ai-engine-res.c | 114 + include/uapi/linux/xlnx-ai-engine.h | 107 + 10 files changed, 1540 insertions(+) create mode 100644 drivers/misc/xilinx-ai-engine/Makefile create mode 100644 drivers/misc/xilinx-ai-engine/ai-engine-aie.c create mode 100644 drivers/misc/xilinx-ai-engine/ai-engine-dev.c create mode 100644 drivers/misc/xilinx-ai-engine/ai-engine-internal.h create mode 100644 drivers/misc/xilinx-ai-engine/ai-engine-part.c create mode 100644 drivers/misc/xilinx-ai-engine/ai-engine-res.c create mode 100644 include/uapi/linux/xlnx-ai-engine.h diff --git a/MAINTAINERS b/MAINTAINERS index 5cc595a..40e3351 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -19283,6 +19283,14 @@ T: git https://github.com/Xilinx/linux-xlnx.git F: Documentation/devicetree/bindings/phy/xlnx,zynqmp-psgtr.yaml F: drivers/phy/xilinx/phy-zynqmp.c +XILINX AI ENGINE DRIVER +M: Wendy Liang +S: Supported +F: Documentation/devicetree/bindings/soc/xilinx/xlnx,ai-engine.yaml +F: drivers/misc/xilinx-ai-engine/ +F: include/linux/xlnx-ai-engine.h +F: include/uapi/linux/xlnx-ai-engine.h + XILLYBUS DRIVER M: Eli Billauer L: linux-ker...@vger.kernel.org diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig index fafa8b0..0b8ce4d 100644 --- a/drivers/misc/Kconfig +++ b/drivers/misc/Kconfig @@ -444,6 +444,18 @@ config XILINX_SDFEC If unsure, say N. +config XILINX_AIE + tristate "Xilinx AI engine" + depends on ARM64 || COMPILE_TEST + help + This option enables support for the Xilinx AI engine driver. + One Xilinx AI engine device can have multiple partitions (groups of + AI engine tiles). Xilinx AI engine device driver instance manages + AI engine partitions. User application access its partitions through + AI engine partition instance file operations. + + If unsure, say N + config MISC_RTSX tristate default MISC_RTSX_PCI || MISC_RTSX_USB diff --git a/drivers/misc/Makefile b/drivers/misc/Makefile index d23231e..2176b18 100644 --- a/drivers/misc/Makefile +++ b/drivers/misc/Makefile @@ -57,3 +57,4 @@ obj-$(CONFIG_HABANA_AI) += habanalabs/ obj-$(CONFIG_UACCE)+= uacce/ obj-$(CONFIG_XILINX_SDFEC) += xilinx_sdfec.o obj-$(CONFIG_HISI_HIKEY_USB) += hisi_hikey_usb.o +obj-$(CONFIG_XILINX_AIE) += xilinx-ai-engine/ diff --git a/drivers/misc/xilinx-ai-engine/Makefile b/drivers/misc/xilinx-ai- engine/Makefile new file mode 100644 index 000..7827a0a --- /dev/null +++ b/drivers/misc/xilinx-ai-engine/Makefile @@ -0,0 +1,11 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# Makefile for Xilinx AI engine device driver +# + +obj-$(CONFIG_XILINX_AIE) += xilinx-aie.o + +xilinx-aie-$(CONFIG_XILINX_AIE) := ai-engine-aie.o \ + ai-engine-dev.o \ + ai-engine-part.o \ + ai-engine-res.o diff --git a/drivers/misc/xilinx-ai-engine/ai-engine-aie.c
[PATCH 2/9] misc: Add Xilinx AI engine device driver
Create AI engine device/partition hierarchical structure. Each AI engine device can have multiple logical partitions(groups of AI engine tiles). Each partition is column based and has its own node ID in the system. AI engine device driver manages its partitions. Applications can access AI engine partition through the AI engine partition driver instance. AI engine registers write is moved to kernel as there are registers in the AI engine array needs privilege permission. Signed-off-by: Wendy Liang Signed-off-by: Hyun Kwon --- MAINTAINERS| 8 + drivers/misc/Kconfig | 12 + drivers/misc/Makefile | 1 + drivers/misc/xilinx-ai-engine/Makefile | 11 + drivers/misc/xilinx-ai-engine/ai-engine-aie.c | 115 + drivers/misc/xilinx-ai-engine/ai-engine-dev.c | 448 ++ drivers/misc/xilinx-ai-engine/ai-engine-internal.h | 226 ++ drivers/misc/xilinx-ai-engine/ai-engine-part.c | 498 + drivers/misc/xilinx-ai-engine/ai-engine-res.c | 114 + include/uapi/linux/xlnx-ai-engine.h| 107 + 10 files changed, 1540 insertions(+) create mode 100644 drivers/misc/xilinx-ai-engine/Makefile create mode 100644 drivers/misc/xilinx-ai-engine/ai-engine-aie.c create mode 100644 drivers/misc/xilinx-ai-engine/ai-engine-dev.c create mode 100644 drivers/misc/xilinx-ai-engine/ai-engine-internal.h create mode 100644 drivers/misc/xilinx-ai-engine/ai-engine-part.c create mode 100644 drivers/misc/xilinx-ai-engine/ai-engine-res.c create mode 100644 include/uapi/linux/xlnx-ai-engine.h diff --git a/MAINTAINERS b/MAINTAINERS index 5cc595a..40e3351 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -19283,6 +19283,14 @@ T: git https://github.com/Xilinx/linux-xlnx.git F: Documentation/devicetree/bindings/phy/xlnx,zynqmp-psgtr.yaml F: drivers/phy/xilinx/phy-zynqmp.c +XILINX AI ENGINE DRIVER +M: Wendy Liang +S: Supported +F: Documentation/devicetree/bindings/soc/xilinx/xlnx,ai-engine.yaml +F: drivers/misc/xilinx-ai-engine/ +F: include/linux/xlnx-ai-engine.h +F: include/uapi/linux/xlnx-ai-engine.h + XILLYBUS DRIVER M: Eli Billauer L: linux-ker...@vger.kernel.org diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig index fafa8b0..0b8ce4d 100644 --- a/drivers/misc/Kconfig +++ b/drivers/misc/Kconfig @@ -444,6 +444,18 @@ config XILINX_SDFEC If unsure, say N. +config XILINX_AIE + tristate "Xilinx AI engine" + depends on ARM64 || COMPILE_TEST + help + This option enables support for the Xilinx AI engine driver. + One Xilinx AI engine device can have multiple partitions (groups of + AI engine tiles). Xilinx AI engine device driver instance manages + AI engine partitions. User application access its partitions through + AI engine partition instance file operations. + + If unsure, say N + config MISC_RTSX tristate default MISC_RTSX_PCI || MISC_RTSX_USB diff --git a/drivers/misc/Makefile b/drivers/misc/Makefile index d23231e..2176b18 100644 --- a/drivers/misc/Makefile +++ b/drivers/misc/Makefile @@ -57,3 +57,4 @@ obj-$(CONFIG_HABANA_AI) += habanalabs/ obj-$(CONFIG_UACCE)+= uacce/ obj-$(CONFIG_XILINX_SDFEC) += xilinx_sdfec.o obj-$(CONFIG_HISI_HIKEY_USB) += hisi_hikey_usb.o +obj-$(CONFIG_XILINX_AIE) += xilinx-ai-engine/ diff --git a/drivers/misc/xilinx-ai-engine/Makefile b/drivers/misc/xilinx-ai-engine/Makefile new file mode 100644 index 000..7827a0a --- /dev/null +++ b/drivers/misc/xilinx-ai-engine/Makefile @@ -0,0 +1,11 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# Makefile for Xilinx AI engine device driver +# + +obj-$(CONFIG_XILINX_AIE) += xilinx-aie.o + +xilinx-aie-$(CONFIG_XILINX_AIE) := ai-engine-aie.o \ + ai-engine-dev.o \ + ai-engine-part.o \ + ai-engine-res.o diff --git a/drivers/misc/xilinx-ai-engine/ai-engine-aie.c b/drivers/misc/xilinx-ai-engine/ai-engine-aie.c new file mode 100644 index 000..319260f --- /dev/null +++ b/drivers/misc/xilinx-ai-engine/ai-engine-aie.c @@ -0,0 +1,115 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Xilinx AI Engine driver AIE device specific implementation + * + * Copyright (C) 2020 Xilinx, Inc. + */ + +#include + +#include "ai-engine-internal.h" + +#define AIE_ARRAY_SHIFT30U +#define AIE_COL_SHIFT 23U +#define AIE_ROW_SHIFT 18U + +/* + * Registers offsets + */ +#define AIE_SHIMNOC_L2INTR_MASK_REGOFF 0x00015000U +#define AIE_SHIMNOC_L2INTR_INTR_REGOFF 0x00015010U +#define AIE_SHIMNOC_DMA_BD0_ADDRLOW_REGOFF 0x0001d000U +#define AIE_SHIMNOC_DMA_BD15_PACKET_REGOFF 0x0001d13cU +#define AIE_SHIMNOC_AXIMM_REGOFF 0x0001e020U +#define
