Re: Support for configurable PCIe endpoint
On 08/18/2016 07:25 AM, Kishon Vijay Abraham I wrote: > Hi, > > On Wednesday 17 August 2016 03:19 PM, Mingkai Hu wrote: >> >>> -Original Message- >>> From: Kishon Vijay Abraham I [mailto:kis...@ti.com] >>> Sent: Thursday, August 04, 2016 6:02 PM >>> To: Joao Pinto <joao.pi...@synopsys.com>; bhelg...@google.com; linux- >>> p...@vger.kernel.org; a...@arndb.de; Jingoo Han <jingooh...@gmail.com>; >>> Pratyush Anand <pratyush.an...@gmail.com> >>> Cc: Ley Foon Tan <lf...@altera.com>; Rob Herring <r...@kernel.org>; >>> Tanmay Inamdar <tinam...@apm.com>; Roy Zang >> fei.z...@freescale.com>; Mingkai Hu <mingkai...@freescale.com>; >>> Minghuan Lian <minghuan.l...@freescale.com>; Richard Zhu >>> <richard@freescale.com>; Lucas Stach <l.st...@pengutronix.de>; >>> Murali Karicheri <m-kariche...@ti.com>; Thomas Petazzoni >>> <thomas.petazz...@free-electrons.com>; Jason Cooper >>> <ja...@lakedaemon.net>; Thierry Reding <thierry.red...@gmail.com>; >>> Simon Horman <ho...@verge.net.au>; Zhou Wang >>> <wangzh...@hisilicon.com>; Gabriele Paoloni >>> <gabriele.paol...@huawei.com>; Stanimir Varbanov <svarbanov@mm- >>> sol.com>; David Daney <david.da...@cavium.com>; linux- >>> ker...@vger.kernel.org; linux-arm-ker...@lists.infradead.org; linux- >>> o...@vger.kernel.org; Carlos Palminha >>> <carlos.palmi...@synopsys.com> >>> Subject: Re: Support for configurable PCIe endpoint >>> >>> Hi, >>> >>> On Wednesday 03 August 2016 07:09 PM, Joao Pinto wrote: >>>> Hi Kishon, >>>> >>>> On 8/3/2016 7:03 AM, Kishon Vijay Abraham I wrote: >>>>> Hi, >>>>> >>>>> The PCIe controller present in TI's DRA7 SoC is capable of operating >>>>> either in Root Complex mode or Endpoint mode. (It uses Synopsys >>>>> Designware Core). I'd assume most of the PCIe controllers on other >>>>> platforms that use Designware core should also be capable to operate >>>>> in endpoint mode. But linux kernel right now supports only RC mode. >>>>> >>>>> PCIe endpoint support discussion came up briefly before [1] but it >>>>> was felt the practical use case will find firmware more suitable and >>>>> endpoint support in kernel can be used only for validation or demo. >>>>> >>>>> *) Modify platform driver to support EP mode (in my case pci-dra7xx.c). >>>>> >>>>> *) dt binding specific to EP mode should be created. >>>>> >>>>> Once I complete the implementation and start posting RFC patches, a >>>>> lot of these will become clear. But I want to check if this sounds >>>>> okay to you guys before starting the implementation. >>>>> >>>>> Let me know if you have some other ideas too. >>>>> >>>>> Cheers >>>>> Kishon >>>>> >>>>> [1] -> http://www.spinics.net/lists/linux-pci/msg26026.html >>>>> >>>> You are rising a topic that we are also addressing in Synopsys. >>>> >>>> For the PCIe RC hardware validation we are currently using the >>>> standard pcie-designware and pcie-designware-plat drivers. >>>> >>>> For the Endpoint we have to use an internal software package. Its main >>>> purpose is to initialize the IP registers, eDMA channels and make data >>>> transfer to prove that the everything is working properly. This is >>>> done in 2 levels, a custom driver built and loaded and an application >>>> that makes some ioctl to the driver executing some interesting >>>> functions to check the Endpoint status and make some data exchange. >>> hmm.. the platform I have doesn't have a DMA in PCIe IP >>> (http://www.ti.com/lit/ug/spruhz6g/spruhz6g.pdf). So in your testing does >>> the EP access RC memory? i.e the driver in the RC allocates memory from it's >>> DDR and gives it's DDR address to the EP. The EP then transfers data to this >>> address. (This is a typical use case with ethernet PCIe cards). IIUC that's >>> not >>> simple with configurable EPs. I'd like to know more about your testing >>> though. >>> >> Hi Kishon, >> >> This is a typical user case for EP to use DMA transfer data to/from RC >> memory. >> In our case, we imple
Re: Support for configurable PCIe endpoint
On 08/18/2016 07:25 AM, Kishon Vijay Abraham I wrote: > Hi, > > On Wednesday 17 August 2016 03:19 PM, Mingkai Hu wrote: >> >>> -Original Message- >>> From: Kishon Vijay Abraham I [mailto:kis...@ti.com] >>> Sent: Thursday, August 04, 2016 6:02 PM >>> To: Joao Pinto ; bhelg...@google.com; linux- >>> p...@vger.kernel.org; a...@arndb.de; Jingoo Han ; >>> Pratyush Anand >>> Cc: Ley Foon Tan ; Rob Herring ; >>> Tanmay Inamdar ; Roy Zang >> fei.z...@freescale.com>; Mingkai Hu ; >>> Minghuan Lian ; Richard Zhu >>> ; Lucas Stach ; >>> Murali Karicheri ; Thomas Petazzoni >>> ; Jason Cooper >>> ; Thierry Reding ; >>> Simon Horman ; Zhou Wang >>> ; Gabriele Paoloni >>> ; Stanimir Varbanov >> sol.com>; David Daney ; linux- >>> ker...@vger.kernel.org; linux-arm-ker...@lists.infradead.org; linux- >>> o...@vger.kernel.org; Carlos Palminha >>> >>> Subject: Re: Support for configurable PCIe endpoint >>> >>> Hi, >>> >>> On Wednesday 03 August 2016 07:09 PM, Joao Pinto wrote: >>>> Hi Kishon, >>>> >>>> On 8/3/2016 7:03 AM, Kishon Vijay Abraham I wrote: >>>>> Hi, >>>>> >>>>> The PCIe controller present in TI's DRA7 SoC is capable of operating >>>>> either in Root Complex mode or Endpoint mode. (It uses Synopsys >>>>> Designware Core). I'd assume most of the PCIe controllers on other >>>>> platforms that use Designware core should also be capable to operate >>>>> in endpoint mode. But linux kernel right now supports only RC mode. >>>>> >>>>> PCIe endpoint support discussion came up briefly before [1] but it >>>>> was felt the practical use case will find firmware more suitable and >>>>> endpoint support in kernel can be used only for validation or demo. >>>>> >>>>> *) Modify platform driver to support EP mode (in my case pci-dra7xx.c). >>>>> >>>>> *) dt binding specific to EP mode should be created. >>>>> >>>>> Once I complete the implementation and start posting RFC patches, a >>>>> lot of these will become clear. But I want to check if this sounds >>>>> okay to you guys before starting the implementation. >>>>> >>>>> Let me know if you have some other ideas too. >>>>> >>>>> Cheers >>>>> Kishon >>>>> >>>>> [1] -> http://www.spinics.net/lists/linux-pci/msg26026.html >>>>> >>>> You are rising a topic that we are also addressing in Synopsys. >>>> >>>> For the PCIe RC hardware validation we are currently using the >>>> standard pcie-designware and pcie-designware-plat drivers. >>>> >>>> For the Endpoint we have to use an internal software package. Its main >>>> purpose is to initialize the IP registers, eDMA channels and make data >>>> transfer to prove that the everything is working properly. This is >>>> done in 2 levels, a custom driver built and loaded and an application >>>> that makes some ioctl to the driver executing some interesting >>>> functions to check the Endpoint status and make some data exchange. >>> hmm.. the platform I have doesn't have a DMA in PCIe IP >>> (http://www.ti.com/lit/ug/spruhz6g/spruhz6g.pdf). So in your testing does >>> the EP access RC memory? i.e the driver in the RC allocates memory from it's >>> DDR and gives it's DDR address to the EP. The EP then transfers data to this >>> address. (This is a typical use case with ethernet PCIe cards). IIUC that's >>> not >>> simple with configurable EPs. I'd like to know more about your testing >>> though. >>> >> Hi Kishon, >> >> This is a typical user case for EP to use DMA transfer data to/from RC >> memory. >> In our case, we implement ring (like BD ring) or register in EP to >> communicate >> The address allocated in RC memory, then EP can transfer data to/from RC >> memory. > Initially I had some confusion w.r.t this because the address allocated in RC > memory can also be an address in EP system. For example let's assume we > connect > two similar systems one configured as RC and the other configured as EP. The > PCI driver in the RC allocates memory in it's DDR (say 0x8000) and > programs > this address in the EP. Since it's a similar system, 0x8000 will also be > an > address in the EPs DDR. This will result in EP transferring data to it's own > DDR (at 0x8000) instead of the same address in RC. > > But later realized instead of directly using the DDR address given by RC, this > address should only be used to program the outbound window. That way the > target > of the outbound window can be an address given by the RC and source should be > an address from the address space in the EP's system. > > Do you also use the RC memory address to program the outbound window? > When EP access RC memory, from EP perspective, there should be a offset added to 0x8 to match the pcie outbound access window. Thanks. Roy
Re: Support for configurable PCIe endpoint
Hi Arnd, On Thursday 25 August 2016 06:29 PM, Arnd Bergmann wrote: > On Thursday, August 18, 2016 6:44:09 PM CEST Kishon Vijay Abraham I wrote: >> Hi Arnd, >> >> On Thursday 04 August 2016 04:43 PM, Arnd Bergmann wrote: >>> On Thursday, August 4, 2016 3:32:01 PM CEST Kishon Vijay Abraham I wrote: On Wednesday 03 August 2016 07:09 PM, Joao Pinto wrote: > > You are rising a topic that we are also addressing in Synopsys. > > For the PCIe RC hardware validation we are currently using the standard > pcie-designware and pcie-designware-plat drivers. > > For the Endpoint we have to use an internal software package. Its main > purpose > is to initialize the IP registers, eDMA channels and make data transfer > to prove > that the everything is working properly. This is done in 2 levels, a > custom > driver built and loaded and an application that makes some ioctl to the > driver > executing some interesting functions to check the Endpoint status and > make some > data exchange. hmm.. the platform I have doesn't have a DMA in PCIe IP (http://www.ti.com/lit/ug/spruhz6g/spruhz6g.pdf). So in your testing does the EP access RC memory? i.e the driver in the RC allocates memory from it's DDR and gives it's DDR address to the EP. The EP then transfers data to this address. (This is a typical use case with ethernet PCIe cards). IIUC that's not simple with configurable EPs. I'd like to know more about your testing though. >>> >>> >>> What's the difference between using the EDMA on that chip or a DMA engine >>> that is part of the PCIe bridge? >> >> Do you mean the difference between using DMA on an EP (like ethernet card or >> sata card) and DMA on PCI RC system? or is it the difference between eDMA >> within the PCIe IP and system DMA? > > The latter. You write that there is no DMA in the PCIe IP, but from the > perspective of the RC, it should not matter whether the DMA engine is > part of the EP logic or behind it. right, from the RC perspective there is no difference. What I meant is DMA support in PCIe driver has to be added newly (i.e for designware) and the the platform I have doesn't have a DMA in PCIe IP. Anyways, we'll come back to this later after I post my RFC series, maybe by this week end. Thanks Kishon
Re: Support for configurable PCIe endpoint
Hi Arnd, On Thursday 25 August 2016 06:29 PM, Arnd Bergmann wrote: > On Thursday, August 18, 2016 6:44:09 PM CEST Kishon Vijay Abraham I wrote: >> Hi Arnd, >> >> On Thursday 04 August 2016 04:43 PM, Arnd Bergmann wrote: >>> On Thursday, August 4, 2016 3:32:01 PM CEST Kishon Vijay Abraham I wrote: On Wednesday 03 August 2016 07:09 PM, Joao Pinto wrote: > > You are rising a topic that we are also addressing in Synopsys. > > For the PCIe RC hardware validation we are currently using the standard > pcie-designware and pcie-designware-plat drivers. > > For the Endpoint we have to use an internal software package. Its main > purpose > is to initialize the IP registers, eDMA channels and make data transfer > to prove > that the everything is working properly. This is done in 2 levels, a > custom > driver built and loaded and an application that makes some ioctl to the > driver > executing some interesting functions to check the Endpoint status and > make some > data exchange. hmm.. the platform I have doesn't have a DMA in PCIe IP (http://www.ti.com/lit/ug/spruhz6g/spruhz6g.pdf). So in your testing does the EP access RC memory? i.e the driver in the RC allocates memory from it's DDR and gives it's DDR address to the EP. The EP then transfers data to this address. (This is a typical use case with ethernet PCIe cards). IIUC that's not simple with configurable EPs. I'd like to know more about your testing though. >>> >>> >>> What's the difference between using the EDMA on that chip or a DMA engine >>> that is part of the PCIe bridge? >> >> Do you mean the difference between using DMA on an EP (like ethernet card or >> sata card) and DMA on PCI RC system? or is it the difference between eDMA >> within the PCIe IP and system DMA? > > The latter. You write that there is no DMA in the PCIe IP, but from the > perspective of the RC, it should not matter whether the DMA engine is > part of the EP logic or behind it. right, from the RC perspective there is no difference. What I meant is DMA support in PCIe driver has to be added newly (i.e for designware) and the the platform I have doesn't have a DMA in PCIe IP. Anyways, we'll come back to this later after I post my RFC series, maybe by this week end. Thanks Kishon
Re: Support for configurable PCIe endpoint
On Thursday, August 18, 2016 6:44:09 PM CEST Kishon Vijay Abraham I wrote: > Hi Arnd, > > On Thursday 04 August 2016 04:43 PM, Arnd Bergmann wrote: > > On Thursday, August 4, 2016 3:32:01 PM CEST Kishon Vijay Abraham I wrote: > >> On Wednesday 03 August 2016 07:09 PM, Joao Pinto wrote: > >>> > >>> You are rising a topic that we are also addressing in Synopsys. > >>> > >>> For the PCIe RC hardware validation we are currently using the standard > >>> pcie-designware and pcie-designware-plat drivers. > >>> > >>> For the Endpoint we have to use an internal software package. Its main > >>> purpose > >>> is to initialize the IP registers, eDMA channels and make data transfer > >>> to prove > >>> that the everything is working properly. This is done in 2 levels, a > >>> custom > >>> driver built and loaded and an application that makes some ioctl to the > >>> driver > >>> executing some interesting functions to check the Endpoint status and > >>> make some > >>> data exchange. > >> > >> hmm.. the platform I have doesn't have a DMA in PCIe IP > >> (http://www.ti.com/lit/ug/spruhz6g/spruhz6g.pdf). So in your testing does > >> the > >> EP access RC memory? i.e the driver in the RC allocates memory from it's > >> DDR > >> and gives it's DDR address to the EP. The EP then transfers data to this > >> address. (This is a typical use case with ethernet PCIe cards). IIUC > >> that's not > >> simple with configurable EPs. I'd like to know more about your testing > >> though. > > > > > > What's the difference between using the EDMA on that chip or a DMA engine > > that is part of the PCIe bridge? > > Do you mean the difference between using DMA on an EP (like ethernet card or > sata card) and DMA on PCI RC system? or is it the difference between eDMA > within the PCIe IP and system DMA? The latter. You write that there is no DMA in the PCIe IP, but from the perspective of the RC, it should not matter whether the DMA engine is part of the EP logic or behind it. Arnd
Re: Support for configurable PCIe endpoint
On Thursday, August 18, 2016 6:44:09 PM CEST Kishon Vijay Abraham I wrote: > Hi Arnd, > > On Thursday 04 August 2016 04:43 PM, Arnd Bergmann wrote: > > On Thursday, August 4, 2016 3:32:01 PM CEST Kishon Vijay Abraham I wrote: > >> On Wednesday 03 August 2016 07:09 PM, Joao Pinto wrote: > >>> > >>> You are rising a topic that we are also addressing in Synopsys. > >>> > >>> For the PCIe RC hardware validation we are currently using the standard > >>> pcie-designware and pcie-designware-plat drivers. > >>> > >>> For the Endpoint we have to use an internal software package. Its main > >>> purpose > >>> is to initialize the IP registers, eDMA channels and make data transfer > >>> to prove > >>> that the everything is working properly. This is done in 2 levels, a > >>> custom > >>> driver built and loaded and an application that makes some ioctl to the > >>> driver > >>> executing some interesting functions to check the Endpoint status and > >>> make some > >>> data exchange. > >> > >> hmm.. the platform I have doesn't have a DMA in PCIe IP > >> (http://www.ti.com/lit/ug/spruhz6g/spruhz6g.pdf). So in your testing does > >> the > >> EP access RC memory? i.e the driver in the RC allocates memory from it's > >> DDR > >> and gives it's DDR address to the EP. The EP then transfers data to this > >> address. (This is a typical use case with ethernet PCIe cards). IIUC > >> that's not > >> simple with configurable EPs. I'd like to know more about your testing > >> though. > > > > > > What's the difference between using the EDMA on that chip or a DMA engine > > that is part of the PCIe bridge? > > Do you mean the difference between using DMA on an EP (like ethernet card or > sata card) and DMA on PCI RC system? or is it the difference between eDMA > within the PCIe IP and system DMA? The latter. You write that there is no DMA in the PCIe IP, but from the perspective of the RC, it should not matter whether the DMA engine is part of the EP logic or behind it. Arnd
Re: Support for configurable PCIe endpoint
Hi Arnd, On Thursday 04 August 2016 04:43 PM, Arnd Bergmann wrote: > On Thursday, August 4, 2016 3:32:01 PM CEST Kishon Vijay Abraham I wrote: >> On Wednesday 03 August 2016 07:09 PM, Joao Pinto wrote: >>> >>> You are rising a topic that we are also addressing in Synopsys. >>> >>> For the PCIe RC hardware validation we are currently using the standard >>> pcie-designware and pcie-designware-plat drivers. >>> >>> For the Endpoint we have to use an internal software package. Its main >>> purpose >>> is to initialize the IP registers, eDMA channels and make data transfer to >>> prove >>> that the everything is working properly. This is done in 2 levels, a custom >>> driver built and loaded and an application that makes some ioctl to the >>> driver >>> executing some interesting functions to check the Endpoint status and make >>> some >>> data exchange. >> >> hmm.. the platform I have doesn't have a DMA in PCIe IP >> (http://www.ti.com/lit/ug/spruhz6g/spruhz6g.pdf). So in your testing does the >> EP access RC memory? i.e the driver in the RC allocates memory from it's DDR >> and gives it's DDR address to the EP. The EP then transfers data to this >> address. (This is a typical use case with ethernet PCIe cards). IIUC that's >> not >> simple with configurable EPs. I'd like to know more about your testing >> though. > > > What's the difference between using the EDMA on that chip or a DMA engine > that is part of the PCIe bridge? Do you mean the difference between using DMA on an EP (like ethernet card or sata card) and DMA on PCI RC system? or is it the difference between eDMA within the PCIe IP and system DMA? Thanks Kishon
Re: Support for configurable PCIe endpoint
Hi Arnd, On Thursday 04 August 2016 04:43 PM, Arnd Bergmann wrote: > On Thursday, August 4, 2016 3:32:01 PM CEST Kishon Vijay Abraham I wrote: >> On Wednesday 03 August 2016 07:09 PM, Joao Pinto wrote: >>> >>> You are rising a topic that we are also addressing in Synopsys. >>> >>> For the PCIe RC hardware validation we are currently using the standard >>> pcie-designware and pcie-designware-plat drivers. >>> >>> For the Endpoint we have to use an internal software package. Its main >>> purpose >>> is to initialize the IP registers, eDMA channels and make data transfer to >>> prove >>> that the everything is working properly. This is done in 2 levels, a custom >>> driver built and loaded and an application that makes some ioctl to the >>> driver >>> executing some interesting functions to check the Endpoint status and make >>> some >>> data exchange. >> >> hmm.. the platform I have doesn't have a DMA in PCIe IP >> (http://www.ti.com/lit/ug/spruhz6g/spruhz6g.pdf). So in your testing does the >> EP access RC memory? i.e the driver in the RC allocates memory from it's DDR >> and gives it's DDR address to the EP. The EP then transfers data to this >> address. (This is a typical use case with ethernet PCIe cards). IIUC that's >> not >> simple with configurable EPs. I'd like to know more about your testing >> though. > > > What's the difference between using the EDMA on that chip or a DMA engine > that is part of the PCIe bridge? Do you mean the difference between using DMA on an EP (like ethernet card or sata card) and DMA on PCI RC system? or is it the difference between eDMA within the PCIe IP and system DMA? Thanks Kishon
Re: Support for configurable PCIe endpoint
Hi, On Wednesday 17 August 2016 03:19 PM, Mingkai Hu wrote: > > >> -Original Message- >> From: Kishon Vijay Abraham I [mailto:kis...@ti.com] >> Sent: Thursday, August 04, 2016 6:02 PM >> To: Joao Pinto <joao.pi...@synopsys.com>; bhelg...@google.com; linux- >> p...@vger.kernel.org; a...@arndb.de; Jingoo Han <jingooh...@gmail.com>; >> Pratyush Anand <pratyush.an...@gmail.com> >> Cc: Ley Foon Tan <lf...@altera.com>; Rob Herring <r...@kernel.org>; >> Tanmay Inamdar <tinam...@apm.com>; Roy Zang > fei.z...@freescale.com>; Mingkai Hu <mingkai...@freescale.com>; >> Minghuan Lian <minghuan.l...@freescale.com>; Richard Zhu >> <richard@freescale.com>; Lucas Stach <l.st...@pengutronix.de>; >> Murali Karicheri <m-kariche...@ti.com>; Thomas Petazzoni >> <thomas.petazz...@free-electrons.com>; Jason Cooper >> <ja...@lakedaemon.net>; Thierry Reding <thierry.red...@gmail.com>; >> Simon Horman <ho...@verge.net.au>; Zhou Wang >> <wangzh...@hisilicon.com>; Gabriele Paoloni >> <gabriele.paol...@huawei.com>; Stanimir Varbanov <svarbanov@mm- >> sol.com>; David Daney <david.da...@cavium.com>; linux- >> ker...@vger.kernel.org; linux-arm-ker...@lists.infradead.org; linux- >> o...@vger.kernel.org; Carlos Palminha >> <carlos.palmi...@synopsys.com> >> Subject: Re: Support for configurable PCIe endpoint >> >> Hi, >> >> On Wednesday 03 August 2016 07:09 PM, Joao Pinto wrote: >>> Hi Kishon, >>> >>> On 8/3/2016 7:03 AM, Kishon Vijay Abraham I wrote: >>>> Hi, >>>> >>>> The PCIe controller present in TI's DRA7 SoC is capable of operating >>>> either in Root Complex mode or Endpoint mode. (It uses Synopsys >>>> Designware Core). I'd assume most of the PCIe controllers on other >>>> platforms that use Designware core should also be capable to operate >>>> in endpoint mode. But linux kernel right now supports only RC mode. >>>> >>>> PCIe endpoint support discussion came up briefly before [1] but it >>>> was felt the practical use case will find firmware more suitable and >>>> endpoint support in kernel can be used only for validation or demo. >>>> >>>> *) Modify platform driver to support EP mode (in my case pci-dra7xx.c). >>>> >>>> *) dt binding specific to EP mode should be created. >>>> >>>> Once I complete the implementation and start posting RFC patches, a >>>> lot of these will become clear. But I want to check if this sounds >>>> okay to you guys before starting the implementation. >>>> >>>> Let me know if you have some other ideas too. >>>> >>>> Cheers >>>> Kishon >>>> >>>> [1] -> http://www.spinics.net/lists/linux-pci/msg26026.html >>>> >>> >>> You are rising a topic that we are also addressing in Synopsys. >>> >>> For the PCIe RC hardware validation we are currently using the >>> standard pcie-designware and pcie-designware-plat drivers. >>> >>> For the Endpoint we have to use an internal software package. Its main >>> purpose is to initialize the IP registers, eDMA channels and make data >>> transfer to prove that the everything is working properly. This is >>> done in 2 levels, a custom driver built and loaded and an application >>> that makes some ioctl to the driver executing some interesting >>> functions to check the Endpoint status and make some data exchange. >> >> hmm.. the platform I have doesn't have a DMA in PCIe IP >> (http://www.ti.com/lit/ug/spruhz6g/spruhz6g.pdf). So in your testing does >> the EP access RC memory? i.e the driver in the RC allocates memory from it's >> DDR and gives it's DDR address to the EP. The EP then transfers data to this >> address. (This is a typical use case with ethernet PCIe cards). IIUC that's >> not >> simple with configurable EPs. I'd like to know more about your testing >> though. >> > > Hi Kishon, > > This is a typical user case for EP to use DMA transfer data to/from RC memory. > In our case, we implement ring (like BD ring) or register in EP to communicate > The address allocated in RC memory, then EP can transfer data to/from RC > memory. Initially I had some confusion w.r.t this because the address allocated in RC memory can also be an address in EP system. For example let's assume we connect two similar systems one configured as RC and the other configured as EP. The PCI driver in the RC allocates memory in it's DDR (say 0x8000) and programs this address in the EP. Since it's a similar system, 0x8000 will also be an address in the EPs DDR. This will result in EP transferring data to it's own DDR (at 0x8000) instead of the same address in RC. But later realized instead of directly using the DDR address given by RC, this address should only be used to program the outbound window. That way the target of the outbound window can be an address given by the RC and source should be an address from the address space in the EP's system. Do you also use the RC memory address to program the outbound window? Thanks Kishon
Re: Support for configurable PCIe endpoint
Hi, On Wednesday 17 August 2016 03:19 PM, Mingkai Hu wrote: > > >> -Original Message- >> From: Kishon Vijay Abraham I [mailto:kis...@ti.com] >> Sent: Thursday, August 04, 2016 6:02 PM >> To: Joao Pinto ; bhelg...@google.com; linux- >> p...@vger.kernel.org; a...@arndb.de; Jingoo Han ; >> Pratyush Anand >> Cc: Ley Foon Tan ; Rob Herring ; >> Tanmay Inamdar ; Roy Zang > fei.z...@freescale.com>; Mingkai Hu ; >> Minghuan Lian ; Richard Zhu >> ; Lucas Stach ; >> Murali Karicheri ; Thomas Petazzoni >> ; Jason Cooper >> ; Thierry Reding ; >> Simon Horman ; Zhou Wang >> ; Gabriele Paoloni >> ; Stanimir Varbanov > sol.com>; David Daney ; linux- >> ker...@vger.kernel.org; linux-arm-ker...@lists.infradead.org; linux- >> o...@vger.kernel.org; Carlos Palminha >> >> Subject: Re: Support for configurable PCIe endpoint >> >> Hi, >> >> On Wednesday 03 August 2016 07:09 PM, Joao Pinto wrote: >>> Hi Kishon, >>> >>> On 8/3/2016 7:03 AM, Kishon Vijay Abraham I wrote: >>>> Hi, >>>> >>>> The PCIe controller present in TI's DRA7 SoC is capable of operating >>>> either in Root Complex mode or Endpoint mode. (It uses Synopsys >>>> Designware Core). I'd assume most of the PCIe controllers on other >>>> platforms that use Designware core should also be capable to operate >>>> in endpoint mode. But linux kernel right now supports only RC mode. >>>> >>>> PCIe endpoint support discussion came up briefly before [1] but it >>>> was felt the practical use case will find firmware more suitable and >>>> endpoint support in kernel can be used only for validation or demo. >>>> >>>> *) Modify platform driver to support EP mode (in my case pci-dra7xx.c). >>>> >>>> *) dt binding specific to EP mode should be created. >>>> >>>> Once I complete the implementation and start posting RFC patches, a >>>> lot of these will become clear. But I want to check if this sounds >>>> okay to you guys before starting the implementation. >>>> >>>> Let me know if you have some other ideas too. >>>> >>>> Cheers >>>> Kishon >>>> >>>> [1] -> http://www.spinics.net/lists/linux-pci/msg26026.html >>>> >>> >>> You are rising a topic that we are also addressing in Synopsys. >>> >>> For the PCIe RC hardware validation we are currently using the >>> standard pcie-designware and pcie-designware-plat drivers. >>> >>> For the Endpoint we have to use an internal software package. Its main >>> purpose is to initialize the IP registers, eDMA channels and make data >>> transfer to prove that the everything is working properly. This is >>> done in 2 levels, a custom driver built and loaded and an application >>> that makes some ioctl to the driver executing some interesting >>> functions to check the Endpoint status and make some data exchange. >> >> hmm.. the platform I have doesn't have a DMA in PCIe IP >> (http://www.ti.com/lit/ug/spruhz6g/spruhz6g.pdf). So in your testing does >> the EP access RC memory? i.e the driver in the RC allocates memory from it's >> DDR and gives it's DDR address to the EP. The EP then transfers data to this >> address. (This is a typical use case with ethernet PCIe cards). IIUC that's >> not >> simple with configurable EPs. I'd like to know more about your testing >> though. >> > > Hi Kishon, > > This is a typical user case for EP to use DMA transfer data to/from RC memory. > In our case, we implement ring (like BD ring) or register in EP to communicate > The address allocated in RC memory, then EP can transfer data to/from RC > memory. Initially I had some confusion w.r.t this because the address allocated in RC memory can also be an address in EP system. For example let's assume we connect two similar systems one configured as RC and the other configured as EP. The PCI driver in the RC allocates memory in it's DDR (say 0x8000) and programs this address in the EP. Since it's a similar system, 0x8000 will also be an address in the EPs DDR. This will result in EP transferring data to it's own DDR (at 0x8000) instead of the same address in RC. But later realized instead of directly using the DDR address given by RC, this address should only be used to program the outbound window. That way the target of the outbound window can be an address given by the RC and source should be an address from the address space in the EP's system. Do you also use the RC memory address to program the outbound window? Thanks Kishon
RE: Support for configurable PCIe endpoint
> -Original Message- > From: Kishon Vijay Abraham I [mailto:kis...@ti.com] > Sent: Thursday, August 04, 2016 6:02 PM > To: Joao Pinto <joao.pi...@synopsys.com>; bhelg...@google.com; linux- > p...@vger.kernel.org; a...@arndb.de; Jingoo Han <jingooh...@gmail.com>; > Pratyush Anand <pratyush.an...@gmail.com> > Cc: Ley Foon Tan <lf...@altera.com>; Rob Herring <r...@kernel.org>; > Tanmay Inamdar <tinam...@apm.com>; Roy Zang fei.z...@freescale.com>; Mingkai Hu <mingkai...@freescale.com>; > Minghuan Lian <minghuan.l...@freescale.com>; Richard Zhu > <richard@freescale.com>; Lucas Stach <l.st...@pengutronix.de>; > Murali Karicheri <m-kariche...@ti.com>; Thomas Petazzoni > <thomas.petazz...@free-electrons.com>; Jason Cooper > <ja...@lakedaemon.net>; Thierry Reding <thierry.red...@gmail.com>; > Simon Horman <ho...@verge.net.au>; Zhou Wang > <wangzh...@hisilicon.com>; Gabriele Paoloni > <gabriele.paol...@huawei.com>; Stanimir Varbanov <svarbanov@mm- > sol.com>; David Daney <david.da...@cavium.com>; linux- > ker...@vger.kernel.org; linux-arm-ker...@lists.infradead.org; linux- > o...@vger.kernel.org; Carlos Palminha > <carlos.palmi...@synopsys.com> > Subject: Re: Support for configurable PCIe endpoint > > Hi, > > On Wednesday 03 August 2016 07:09 PM, Joao Pinto wrote: > > Hi Kishon, > > > > On 8/3/2016 7:03 AM, Kishon Vijay Abraham I wrote: > >> Hi, > >> > >> The PCIe controller present in TI's DRA7 SoC is capable of operating > >> either in Root Complex mode or Endpoint mode. (It uses Synopsys > >> Designware Core). I'd assume most of the PCIe controllers on other > >> platforms that use Designware core should also be capable to operate > >> in endpoint mode. But linux kernel right now supports only RC mode. > >> > >> PCIe endpoint support discussion came up briefly before [1] but it > >> was felt the practical use case will find firmware more suitable and > >> endpoint support in kernel can be used only for validation or demo. > >> > >> *) Modify platform driver to support EP mode (in my case pci-dra7xx.c). > >> > >> *) dt binding specific to EP mode should be created. > >> > >> Once I complete the implementation and start posting RFC patches, a > >> lot of these will become clear. But I want to check if this sounds > >> okay to you guys before starting the implementation. > >> > >> Let me know if you have some other ideas too. > >> > >> Cheers > >> Kishon > >> > >> [1] -> http://www.spinics.net/lists/linux-pci/msg26026.html > >> > > > > You are rising a topic that we are also addressing in Synopsys. > > > > For the PCIe RC hardware validation we are currently using the > > standard pcie-designware and pcie-designware-plat drivers. > > > > For the Endpoint we have to use an internal software package. Its main > > purpose is to initialize the IP registers, eDMA channels and make data > > transfer to prove that the everything is working properly. This is > > done in 2 levels, a custom driver built and loaded and an application > > that makes some ioctl to the driver executing some interesting > > functions to check the Endpoint status and make some data exchange. > > hmm.. the platform I have doesn't have a DMA in PCIe IP > (http://www.ti.com/lit/ug/spruhz6g/spruhz6g.pdf). So in your testing does > the EP access RC memory? i.e the driver in the RC allocates memory from it's > DDR and gives it's DDR address to the EP. The EP then transfers data to this > address. (This is a typical use case with ethernet PCIe cards). IIUC that's > not > simple with configurable EPs. I'd like to know more about your testing though. > Hi Kishon, This is a typical user case for EP to use DMA transfer data to/from RC memory. In our case, we implement ring (like BD ring) or register in EP to communicate The address allocated in RC memory, then EP can transfer data to/from RC memory. Thanks, Mingkai
RE: Support for configurable PCIe endpoint
> -Original Message- > From: Kishon Vijay Abraham I [mailto:kis...@ti.com] > Sent: Thursday, August 04, 2016 6:02 PM > To: Joao Pinto ; bhelg...@google.com; linux- > p...@vger.kernel.org; a...@arndb.de; Jingoo Han ; > Pratyush Anand > Cc: Ley Foon Tan ; Rob Herring ; > Tanmay Inamdar ; Roy Zang fei.z...@freescale.com>; Mingkai Hu ; > Minghuan Lian ; Richard Zhu > ; Lucas Stach ; > Murali Karicheri ; Thomas Petazzoni > ; Jason Cooper > ; Thierry Reding ; > Simon Horman ; Zhou Wang > ; Gabriele Paoloni > ; Stanimir Varbanov sol.com>; David Daney ; linux- > ker...@vger.kernel.org; linux-arm-ker...@lists.infradead.org; linux- > o...@vger.kernel.org; Carlos Palminha > > Subject: Re: Support for configurable PCIe endpoint > > Hi, > > On Wednesday 03 August 2016 07:09 PM, Joao Pinto wrote: > > Hi Kishon, > > > > On 8/3/2016 7:03 AM, Kishon Vijay Abraham I wrote: > >> Hi, > >> > >> The PCIe controller present in TI's DRA7 SoC is capable of operating > >> either in Root Complex mode or Endpoint mode. (It uses Synopsys > >> Designware Core). I'd assume most of the PCIe controllers on other > >> platforms that use Designware core should also be capable to operate > >> in endpoint mode. But linux kernel right now supports only RC mode. > >> > >> PCIe endpoint support discussion came up briefly before [1] but it > >> was felt the practical use case will find firmware more suitable and > >> endpoint support in kernel can be used only for validation or demo. > >> > >> *) Modify platform driver to support EP mode (in my case pci-dra7xx.c). > >> > >> *) dt binding specific to EP mode should be created. > >> > >> Once I complete the implementation and start posting RFC patches, a > >> lot of these will become clear. But I want to check if this sounds > >> okay to you guys before starting the implementation. > >> > >> Let me know if you have some other ideas too. > >> > >> Cheers > >> Kishon > >> > >> [1] -> http://www.spinics.net/lists/linux-pci/msg26026.html > >> > > > > You are rising a topic that we are also addressing in Synopsys. > > > > For the PCIe RC hardware validation we are currently using the > > standard pcie-designware and pcie-designware-plat drivers. > > > > For the Endpoint we have to use an internal software package. Its main > > purpose is to initialize the IP registers, eDMA channels and make data > > transfer to prove that the everything is working properly. This is > > done in 2 levels, a custom driver built and loaded and an application > > that makes some ioctl to the driver executing some interesting > > functions to check the Endpoint status and make some data exchange. > > hmm.. the platform I have doesn't have a DMA in PCIe IP > (http://www.ti.com/lit/ug/spruhz6g/spruhz6g.pdf). So in your testing does > the EP access RC memory? i.e the driver in the RC allocates memory from it's > DDR and gives it's DDR address to the EP. The EP then transfers data to this > address. (This is a typical use case with ethernet PCIe cards). IIUC that's > not > simple with configurable EPs. I'd like to know more about your testing though. > Hi Kishon, This is a typical user case for EP to use DMA transfer data to/from RC memory. In our case, we implement ring (like BD ring) or register in EP to communicate The address allocated in RC memory, then EP can transfer data to/from RC memory. Thanks, Mingkai
Re: Support for configurable PCIe endpoint
On Thursday, August 4, 2016 3:32:01 PM CEST Kishon Vijay Abraham I wrote: > On Wednesday 03 August 2016 07:09 PM, Joao Pinto wrote: > > > > You are rising a topic that we are also addressing in Synopsys. > > > > For the PCIe RC hardware validation we are currently using the standard > > pcie-designware and pcie-designware-plat drivers. > > > > For the Endpoint we have to use an internal software package. Its main > > purpose > > is to initialize the IP registers, eDMA channels and make data transfer to > > prove > > that the everything is working properly. This is done in 2 levels, a custom > > driver built and loaded and an application that makes some ioctl to the > > driver > > executing some interesting functions to check the Endpoint status and make > > some > > data exchange. > > hmm.. the platform I have doesn't have a DMA in PCIe IP > (http://www.ti.com/lit/ug/spruhz6g/spruhz6g.pdf). So in your testing does the > EP access RC memory? i.e the driver in the RC allocates memory from it's DDR > and gives it's DDR address to the EP. The EP then transfers data to this > address. (This is a typical use case with ethernet PCIe cards). IIUC that's > not > simple with configurable EPs. I'd like to know more about your testing though. What's the difference between using the EDMA on that chip or a DMA engine that is part of the PCIe bridge? Arnd
Re: Support for configurable PCIe endpoint
On Thursday, August 4, 2016 3:32:01 PM CEST Kishon Vijay Abraham I wrote: > On Wednesday 03 August 2016 07:09 PM, Joao Pinto wrote: > > > > You are rising a topic that we are also addressing in Synopsys. > > > > For the PCIe RC hardware validation we are currently using the standard > > pcie-designware and pcie-designware-plat drivers. > > > > For the Endpoint we have to use an internal software package. Its main > > purpose > > is to initialize the IP registers, eDMA channels and make data transfer to > > prove > > that the everything is working properly. This is done in 2 levels, a custom > > driver built and loaded and an application that makes some ioctl to the > > driver > > executing some interesting functions to check the Endpoint status and make > > some > > data exchange. > > hmm.. the platform I have doesn't have a DMA in PCIe IP > (http://www.ti.com/lit/ug/spruhz6g/spruhz6g.pdf). So in your testing does the > EP access RC memory? i.e the driver in the RC allocates memory from it's DDR > and gives it's DDR address to the EP. The EP then transfers data to this > address. (This is a typical use case with ethernet PCIe cards). IIUC that's > not > simple with configurable EPs. I'd like to know more about your testing though. What's the difference between using the EDMA on that chip or a DMA engine that is part of the PCIe bridge? Arnd
Re: Support for configurable PCIe endpoint
Hi, On Wednesday 03 August 2016 07:09 PM, Joao Pinto wrote: > Hi Kishon, > > On 8/3/2016 7:03 AM, Kishon Vijay Abraham I wrote: >> Hi, >> >> The PCIe controller present in TI's DRA7 SoC is capable of operating either >> in >> Root Complex mode or Endpoint mode. (It uses Synopsys Designware Core). I'd >> assume most of the PCIe controllers on other platforms that use Designware >> core >> should also be capable to operate in endpoint mode. But linux kernel right >> now >> supports only RC mode. >> >> PCIe endpoint support discussion came up briefly before [1] but it was felt >> the >> practical use case will find firmware more suitable and endpoint support in >> kernel can be used only for validation or demo. >> >> *) Modify platform driver to support EP mode (in my case pci-dra7xx.c). >> >> *) dt binding specific to EP mode should be created. >> >> Once I complete the implementation and start posting RFC patches, a lot of >> these will become clear. But I want to check if this sounds okay to you guys >> before starting the implementation. >> >> Let me know if you have some other ideas too. >> >> Cheers >> Kishon >> >> [1] -> http://www.spinics.net/lists/linux-pci/msg26026.html >> > > You are rising a topic that we are also addressing in Synopsys. > > For the PCIe RC hardware validation we are currently using the standard > pcie-designware and pcie-designware-plat drivers. > > For the Endpoint we have to use an internal software package. Its main purpose > is to initialize the IP registers, eDMA channels and make data transfer to > prove > that the everything is working properly. This is done in 2 levels, a custom > driver built and loaded and an application that makes some ioctl to the driver > executing some interesting functions to check the Endpoint status and make > some > data exchange. hmm.. the platform I have doesn't have a DMA in PCIe IP (http://www.ti.com/lit/ug/spruhz6g/spruhz6g.pdf). So in your testing does the EP access RC memory? i.e the driver in the RC allocates memory from it's DDR and gives it's DDR address to the EP. The EP then transfers data to this address. (This is a typical use case with ethernet PCIe cards). IIUC that's not simple with configurable EPs. I'd like to know more about your testing though. Thanks Kishon
Re: Support for configurable PCIe endpoint
Hi, On Wednesday 03 August 2016 07:09 PM, Joao Pinto wrote: > Hi Kishon, > > On 8/3/2016 7:03 AM, Kishon Vijay Abraham I wrote: >> Hi, >> >> The PCIe controller present in TI's DRA7 SoC is capable of operating either >> in >> Root Complex mode or Endpoint mode. (It uses Synopsys Designware Core). I'd >> assume most of the PCIe controllers on other platforms that use Designware >> core >> should also be capable to operate in endpoint mode. But linux kernel right >> now >> supports only RC mode. >> >> PCIe endpoint support discussion came up briefly before [1] but it was felt >> the >> practical use case will find firmware more suitable and endpoint support in >> kernel can be used only for validation or demo. >> >> *) Modify platform driver to support EP mode (in my case pci-dra7xx.c). >> >> *) dt binding specific to EP mode should be created. >> >> Once I complete the implementation and start posting RFC patches, a lot of >> these will become clear. But I want to check if this sounds okay to you guys >> before starting the implementation. >> >> Let me know if you have some other ideas too. >> >> Cheers >> Kishon >> >> [1] -> http://www.spinics.net/lists/linux-pci/msg26026.html >> > > You are rising a topic that we are also addressing in Synopsys. > > For the PCIe RC hardware validation we are currently using the standard > pcie-designware and pcie-designware-plat drivers. > > For the Endpoint we have to use an internal software package. Its main purpose > is to initialize the IP registers, eDMA channels and make data transfer to > prove > that the everything is working properly. This is done in 2 levels, a custom > driver built and loaded and an application that makes some ioctl to the driver > executing some interesting functions to check the Endpoint status and make > some > data exchange. hmm.. the platform I have doesn't have a DMA in PCIe IP (http://www.ti.com/lit/ug/spruhz6g/spruhz6g.pdf). So in your testing does the EP access RC memory? i.e the driver in the RC allocates memory from it's DDR and gives it's DDR address to the EP. The EP then transfers data to this address. (This is a typical use case with ethernet PCIe cards). IIUC that's not simple with configurable EPs. I'd like to know more about your testing though. Thanks Kishon
Re: Support for configurable PCIe endpoint
Hi, On Wednesday 03 August 2016 03:17 PM, Christoph Hellwig wrote: > On Wed, Aug 03, 2016 at 11:33:19AM +0530, Kishon Vijay Abraham I wrote: >> Hi, >> >> The PCIe controller present in TI's DRA7 SoC is capable of operating either >> in >> Root Complex mode or Endpoint mode. (It uses Synopsys Designware Core).I'd >> assume most of the PCIe controllers on other platforms that use Designware >> core >> should also be capable to operate in endpoint mode. But linux kernel right >> now >> supports only RC mode. >> >> PCIe endpoint support discussion came up briefly before [1] but it was felt >> the >> practical use case will find firmware more suitable and endpoint support in >> kernel can be used only for validation or demo. > > I disagree. It's highly useful for rapid prototyping of hardware > interfaces, and I've been looking into PCIe EP drivers for exactly > that reason recently. Going a little offtopic: any good DRA7 eval > boards you'd recommend to try for this purpose? I think the only publicly available DRA7 based board with PCIe (mPCIe slot) is AM572x EVM (http://www.ti.com/tool/TMDSEVM572X). The board comes only with a female PCIe slot. So a special cable would be required to connect it to a PCIe host. However for my development I'm planning to use dra7-evm which has standard female PCIe connector and I'll use a cable like PE-FLEX1 male-to-male (in http://www.adexelec.com/pciexp.htm) to connect back-to-back boards. > > We already have a EP driver in the tree: > > drivers/misc/spear13xx_pcie_gadget.c > > but as far as I can tell it doesn't really work at the moment. Okay. I wasn't aware of that. I'll take a look at that one. > >> Validation or demo is itself a valid use case in my opinion (consider >> something >> similar to gadget zero for USB). There can be other use cases as well. The RC >> can use the SoC with EP mode support as an accelerator to accomplish specific >> task. Here RC gives data to the EP. The EP processes the data. The processing >> can be done either in ARM itself or it can use other hardware accelerators >> (like DSP, IVA-HD etc..) present in the EP system. If HW accelerator is used, >> the linux kernel running in ARM can be used to accomplish other tasks. Once >> EP >> mode support is added, I think more use cases will be added. > > That sounds useful as well. > >> >From the high level this should look _similar_ to the gadget framework of >> >USB. >> One difference from USB would be this should allow HW components (like DSP, >> PRU >> etc.. and maybe even some peripheral) in the EP system to be used by RC >> system. > > Indeed. > >> So these are the high-level steps that I thought would be needed to add EP >> support in linux. >> *) move pcie-designware.c out of drivers/pci/host (maybe create a >> drivers/pci/designware/ folder?). All users of pcie-designware.c should be >> moved here. >> This is in preparation for adding EP mode support to designware. > > I'd use a new drivers/pci/controller. Or maybe just skip the rename > for now and see how this evolves. Sure. That makes more sense. > > The rest of the plan sounds fine to me. cool. Thanks Kishon
Re: Support for configurable PCIe endpoint
Hi, On Wednesday 03 August 2016 03:17 PM, Christoph Hellwig wrote: > On Wed, Aug 03, 2016 at 11:33:19AM +0530, Kishon Vijay Abraham I wrote: >> Hi, >> >> The PCIe controller present in TI's DRA7 SoC is capable of operating either >> in >> Root Complex mode or Endpoint mode. (It uses Synopsys Designware Core).I'd >> assume most of the PCIe controllers on other platforms that use Designware >> core >> should also be capable to operate in endpoint mode. But linux kernel right >> now >> supports only RC mode. >> >> PCIe endpoint support discussion came up briefly before [1] but it was felt >> the >> practical use case will find firmware more suitable and endpoint support in >> kernel can be used only for validation or demo. > > I disagree. It's highly useful for rapid prototyping of hardware > interfaces, and I've been looking into PCIe EP drivers for exactly > that reason recently. Going a little offtopic: any good DRA7 eval > boards you'd recommend to try for this purpose? I think the only publicly available DRA7 based board with PCIe (mPCIe slot) is AM572x EVM (http://www.ti.com/tool/TMDSEVM572X). The board comes only with a female PCIe slot. So a special cable would be required to connect it to a PCIe host. However for my development I'm planning to use dra7-evm which has standard female PCIe connector and I'll use a cable like PE-FLEX1 male-to-male (in http://www.adexelec.com/pciexp.htm) to connect back-to-back boards. > > We already have a EP driver in the tree: > > drivers/misc/spear13xx_pcie_gadget.c > > but as far as I can tell it doesn't really work at the moment. Okay. I wasn't aware of that. I'll take a look at that one. > >> Validation or demo is itself a valid use case in my opinion (consider >> something >> similar to gadget zero for USB). There can be other use cases as well. The RC >> can use the SoC with EP mode support as an accelerator to accomplish specific >> task. Here RC gives data to the EP. The EP processes the data. The processing >> can be done either in ARM itself or it can use other hardware accelerators >> (like DSP, IVA-HD etc..) present in the EP system. If HW accelerator is used, >> the linux kernel running in ARM can be used to accomplish other tasks. Once >> EP >> mode support is added, I think more use cases will be added. > > That sounds useful as well. > >> >From the high level this should look _similar_ to the gadget framework of >> >USB. >> One difference from USB would be this should allow HW components (like DSP, >> PRU >> etc.. and maybe even some peripheral) in the EP system to be used by RC >> system. > > Indeed. > >> So these are the high-level steps that I thought would be needed to add EP >> support in linux. >> *) move pcie-designware.c out of drivers/pci/host (maybe create a >> drivers/pci/designware/ folder?). All users of pcie-designware.c should be >> moved here. >> This is in preparation for adding EP mode support to designware. > > I'd use a new drivers/pci/controller. Or maybe just skip the rename > for now and see how this evolves. Sure. That makes more sense. > > The rest of the plan sounds fine to me. cool. Thanks Kishon
Re: Support for configurable PCIe endpoint
Hi Arnd, On Wednesday 03 August 2016 01:12 PM, Arnd Bergmann wrote: > On Wednesday, August 3, 2016 11:33:19 AM CEST Kishon Vijay Abraham I wrote: >> Hi, >> >> The PCIe controller present in TI's DRA7 SoC is capable of operating either >> in >> Root Complex mode or Endpoint mode. (It uses Synopsys Designware Core).I'd >> assume most of the PCIe controllers on other platforms that use Designware >> core >> should also be capable to operate in endpoint mode. But linux kernel right >> now >> supports only RC mode. >> >> PCIe endpoint support discussion came up briefly before [1] but it was felt >> the >> practical use case will find firmware more suitable and endpoint support in >> kernel can be used only for validation or demo. >> >> Validation or demo is itself a valid use case in my opinion (consider >> something >> similar to gadget zero for USB). There can be other use cases as well. The RC >> can use the SoC with EP mode support as an accelerator to accomplish specific >> task. Here RC gives data to the EP. The EP processes the data. The processing >> can be done either in ARM itself or it can use other hardware accelerators >> (like DSP, IVA-HD etc..) present in the EP system. If HW accelerator is used, >> the linux kernel running in ARM can be used to accomplish other tasks. Once >> EP >> mode support is added, I think more use cases will be added. >> >> From the high level this should look _similar_ to the gadget framework of >> USB. >> One difference from USB would be this should allow HW components (like DSP, >> PRU >> etc.. and maybe even some peripheral) in the EP system to be used by RC >> system. > > (Adding Jon Mason) > > We have the drivers/ntb framework, which in theory should be able to handle > this, but in practice is only used for a very small number of bridge > implementations, and is also limited in the way it can be configured > (compared to USB gadget) > >> So these are the high-level steps that I thought would be needed to add EP >> support in linux. >> *) move pcie-designware.c out of drivers/pci/host (maybe create a >> drivers/pci/designware/ folder?). All users of pcie-designware.c should be >> moved here. >> This is in preparation for adding EP mode support to designware. > > A lot of the other drivers in drivers/pci/host support endpoint mode too, > I don't think moving them all elsewhere is helpful or necessary here. having drivers that support endpoint in *host* directory might be misleading IMO. > >> *) Add library functions in pcie-designware.c specific to EP mode like >> initializing general ecam registers, BAR registers etc.. These functions >> should >> be invoked from a *function* driver (function driver should determine the use >> of a particular EP). >> >> *) Add a sample *function* driver that can be used just for validation. This >> function driver will invoke the previously added functions in PCIe controller >> to initialize ecam, BAR etc.. This will invoke the PCIe controller functions >> through *ep-core* layer. That way the same function driver can be made to >> work >> with different PCIe controllers. (A PCIe driver corresponding to this >> function >> driver should also be implemented in RC side) >> >> *) Add *ep-core* layer to bind the function driver to the controller driver >> and >> using which the function driver will invoke controller driver callbacks (to >> initialize ecam, BAR registers etc..) > > I think we should first have a rough idea what the framework should look like, > and how it handles having multiple hardware implementation and multiple > protocol implementations, before we modify a particular driver. > > So this step here should come a bit earlier than the others. Okay. That makes sense to me as well. > >> *) Modify platform driver to support EP mode (in my case pci-dra7xx.c). >> >> *) dt binding specific to EP mode should be created. > > Right. Thanks Kishon
Re: Support for configurable PCIe endpoint
Hi Arnd, On Wednesday 03 August 2016 01:12 PM, Arnd Bergmann wrote: > On Wednesday, August 3, 2016 11:33:19 AM CEST Kishon Vijay Abraham I wrote: >> Hi, >> >> The PCIe controller present in TI's DRA7 SoC is capable of operating either >> in >> Root Complex mode or Endpoint mode. (It uses Synopsys Designware Core).I'd >> assume most of the PCIe controllers on other platforms that use Designware >> core >> should also be capable to operate in endpoint mode. But linux kernel right >> now >> supports only RC mode. >> >> PCIe endpoint support discussion came up briefly before [1] but it was felt >> the >> practical use case will find firmware more suitable and endpoint support in >> kernel can be used only for validation or demo. >> >> Validation or demo is itself a valid use case in my opinion (consider >> something >> similar to gadget zero for USB). There can be other use cases as well. The RC >> can use the SoC with EP mode support as an accelerator to accomplish specific >> task. Here RC gives data to the EP. The EP processes the data. The processing >> can be done either in ARM itself or it can use other hardware accelerators >> (like DSP, IVA-HD etc..) present in the EP system. If HW accelerator is used, >> the linux kernel running in ARM can be used to accomplish other tasks. Once >> EP >> mode support is added, I think more use cases will be added. >> >> From the high level this should look _similar_ to the gadget framework of >> USB. >> One difference from USB would be this should allow HW components (like DSP, >> PRU >> etc.. and maybe even some peripheral) in the EP system to be used by RC >> system. > > (Adding Jon Mason) > > We have the drivers/ntb framework, which in theory should be able to handle > this, but in practice is only used for a very small number of bridge > implementations, and is also limited in the way it can be configured > (compared to USB gadget) > >> So these are the high-level steps that I thought would be needed to add EP >> support in linux. >> *) move pcie-designware.c out of drivers/pci/host (maybe create a >> drivers/pci/designware/ folder?). All users of pcie-designware.c should be >> moved here. >> This is in preparation for adding EP mode support to designware. > > A lot of the other drivers in drivers/pci/host support endpoint mode too, > I don't think moving them all elsewhere is helpful or necessary here. having drivers that support endpoint in *host* directory might be misleading IMO. > >> *) Add library functions in pcie-designware.c specific to EP mode like >> initializing general ecam registers, BAR registers etc.. These functions >> should >> be invoked from a *function* driver (function driver should determine the use >> of a particular EP). >> >> *) Add a sample *function* driver that can be used just for validation. This >> function driver will invoke the previously added functions in PCIe controller >> to initialize ecam, BAR etc.. This will invoke the PCIe controller functions >> through *ep-core* layer. That way the same function driver can be made to >> work >> with different PCIe controllers. (A PCIe driver corresponding to this >> function >> driver should also be implemented in RC side) >> >> *) Add *ep-core* layer to bind the function driver to the controller driver >> and >> using which the function driver will invoke controller driver callbacks (to >> initialize ecam, BAR registers etc..) > > I think we should first have a rough idea what the framework should look like, > and how it handles having multiple hardware implementation and multiple > protocol implementations, before we modify a particular driver. > > So this step here should come a bit earlier than the others. Okay. That makes sense to me as well. > >> *) Modify platform driver to support EP mode (in my case pci-dra7xx.c). >> >> *) dt binding specific to EP mode should be created. > > Right. Thanks Kishon
Re: Support for configurable PCIe endpoint
On Wednesday, August 3, 2016 10:27:36 AM CEST Christoph Hellwig wrote: > On Wed, Aug 03, 2016 at 06:03:54PM +0200, Arnd Bergmann wrote: > > drivers/ntb seems like a reasonable start, while an alternative > > approach that we have discussed in the past would be based on top > > of virtio, so we could use the existing front-end drivers (net, block, > > v9fs, console, ...). > > I don't really think either is a good aproach for the lowest level > interface. To be useful the EP driver needs to be able to implement > any (reasonable) thing a PCIe device could do. Both NTB and virtio > can sit on top of that, though. Good point. NTB tries to be the low-level interface, but I guess you are right that it really isn't (I have not looked in a long time, maybe Jon can comment). While virtio transport over PCI would be great as the way to implement a lot of things on top of endpoint devices, it can't be the lowest level in the stack (see drivers/remoteproc) and building it on top of something that is useful for other things sounds like a good idea. Arnd
Re: Support for configurable PCIe endpoint
On Wednesday, August 3, 2016 10:27:36 AM CEST Christoph Hellwig wrote: > On Wed, Aug 03, 2016 at 06:03:54PM +0200, Arnd Bergmann wrote: > > drivers/ntb seems like a reasonable start, while an alternative > > approach that we have discussed in the past would be based on top > > of virtio, so we could use the existing front-end drivers (net, block, > > v9fs, console, ...). > > I don't really think either is a good aproach for the lowest level > interface. To be useful the EP driver needs to be able to implement > any (reasonable) thing a PCIe device could do. Both NTB and virtio > can sit on top of that, though. Good point. NTB tries to be the low-level interface, but I guess you are right that it really isn't (I have not looked in a long time, maybe Jon can comment). While virtio transport over PCI would be great as the way to implement a lot of things on top of endpoint devices, it can't be the lowest level in the stack (see drivers/remoteproc) and building it on top of something that is useful for other things sounds like a good idea. Arnd
Re: Support for configurable PCIe endpoint
On Wed, Aug 03, 2016 at 06:03:54PM +0200, Arnd Bergmann wrote: > drivers/ntb seems like a reasonable start, while an alternative > approach that we have discussed in the past would be based on top > of virtio, so we could use the existing front-end drivers (net, block, > v9fs, console, ...). I don't really think either is a good aproach for the lowest level interface. To be useful the EP driver needs to be able to implement any (reasonable) thing a PCIe device could do. Both NTB and virtio can sit on top of that, though.
Re: Support for configurable PCIe endpoint
On Wed, Aug 03, 2016 at 06:03:54PM +0200, Arnd Bergmann wrote: > drivers/ntb seems like a reasonable start, while an alternative > approach that we have discussed in the past would be based on top > of virtio, so we could use the existing front-end drivers (net, block, > v9fs, console, ...). I don't really think either is a good aproach for the lowest level interface. To be useful the EP driver needs to be able to implement any (reasonable) thing a PCIe device could do. Both NTB and virtio can sit on top of that, though.
Re: Support for configurable PCIe endpoint
On Wednesday, August 3, 2016 2:47:47 AM CEST Christoph Hellwig wrote: > On Wed, Aug 03, 2016 at 11:33:19AM +0530, Kishon Vijay Abraham I wrote: > > Hi, > > > > The PCIe controller present in TI's DRA7 SoC is capable of operating either > > in > > Root Complex mode or Endpoint mode. (It uses Synopsys Designware Core).I'd > > assume most of the PCIe controllers on other platforms that use Designware > > core > > should also be capable to operate in endpoint mode. But linux kernel right > > now > > supports only RC mode. > > > > PCIe endpoint support discussion came up briefly before [1] but it was felt > > the > > practical use case will find firmware more suitable and endpoint support in > > kernel can be used only for validation or demo. > > I disagree. It's highly useful for rapid prototyping of hardware > interfaces, and I've been looking into PCIe EP drivers for exactly > that reason recently. Going a little offtopic: any good DRA7 eval > boards you'd recommend to try for this purpose? > > We already have a EP driver in the tree: > > drivers/misc/spear13xx_pcie_gadget.c > > but as far as I can tell it doesn't really work at the moment. > This is indeed for the designware pcie hardware, but it never worked in a mainline kernel, and I marked it 'depends on BROKEN' after it had not correctly compiled for a long time. It's also lacking because it neither abstracts the hardware nor the protocol, and I think we want both. drivers/ntb seems like a reasonable start, while an alternative approach that we have discussed in the past would be based on top of virtio, so we could use the existing front-end drivers (net, block, v9fs, console, ...). Arnd
Re: Support for configurable PCIe endpoint
On Wednesday, August 3, 2016 2:47:47 AM CEST Christoph Hellwig wrote: > On Wed, Aug 03, 2016 at 11:33:19AM +0530, Kishon Vijay Abraham I wrote: > > Hi, > > > > The PCIe controller present in TI's DRA7 SoC is capable of operating either > > in > > Root Complex mode or Endpoint mode. (It uses Synopsys Designware Core).I'd > > assume most of the PCIe controllers on other platforms that use Designware > > core > > should also be capable to operate in endpoint mode. But linux kernel right > > now > > supports only RC mode. > > > > PCIe endpoint support discussion came up briefly before [1] but it was felt > > the > > practical use case will find firmware more suitable and endpoint support in > > kernel can be used only for validation or demo. > > I disagree. It's highly useful for rapid prototyping of hardware > interfaces, and I've been looking into PCIe EP drivers for exactly > that reason recently. Going a little offtopic: any good DRA7 eval > boards you'd recommend to try for this purpose? > > We already have a EP driver in the tree: > > drivers/misc/spear13xx_pcie_gadget.c > > but as far as I can tell it doesn't really work at the moment. > This is indeed for the designware pcie hardware, but it never worked in a mainline kernel, and I marked it 'depends on BROKEN' after it had not correctly compiled for a long time. It's also lacking because it neither abstracts the hardware nor the protocol, and I think we want both. drivers/ntb seems like a reasonable start, while an alternative approach that we have discussed in the past would be based on top of virtio, so we could use the existing front-end drivers (net, block, v9fs, console, ...). Arnd
Re: Support for configurable PCIe endpoint
Hi Kishon, On 8/3/2016 7:03 AM, Kishon Vijay Abraham I wrote: > Hi, > > The PCIe controller present in TI's DRA7 SoC is capable of operating either in > Root Complex mode or Endpoint mode. (It uses Synopsys Designware Core). I'd > assume most of the PCIe controllers on other platforms that use Designware > core > should also be capable to operate in endpoint mode. But linux kernel right now > supports only RC mode. > > PCIe endpoint support discussion came up briefly before [1] but it was felt > the > practical use case will find firmware more suitable and endpoint support in > kernel can be used only for validation or demo. > > *) Modify platform driver to support EP mode (in my case pci-dra7xx.c). > > *) dt binding specific to EP mode should be created. > > Once I complete the implementation and start posting RFC patches, a lot of > these will become clear. But I want to check if this sounds okay to you guys > before starting the implementation. > > Let me know if you have some other ideas too. > > Cheers > Kishon > > [1] -> http://www.spinics.net/lists/linux-pci/msg26026.html > You are rising a topic that we are also addressing in Synopsys. For the PCIe RC hardware validation we are currently using the standard pcie-designware and pcie-designware-plat drivers. For the Endpoint we have to use an internal software package. Its main purpose is to initialize the IP registers, eDMA channels and make data transfer to prove that the everything is working properly. This is done in 2 levels, a custom driver built and loaded and an application that makes some ioctl to the driver executing some interesting functions to check the Endpoint status and make some data exchange. We are very interest in the subject and we are available to participate in the development. I would suggest that also the IP eDMA initialization and manipulation be included in the framework and maybe produce a standard tool to test the endpoint. Thanks Joao
Re: Support for configurable PCIe endpoint
Hi Kishon, On 8/3/2016 7:03 AM, Kishon Vijay Abraham I wrote: > Hi, > > The PCIe controller present in TI's DRA7 SoC is capable of operating either in > Root Complex mode or Endpoint mode. (It uses Synopsys Designware Core). I'd > assume most of the PCIe controllers on other platforms that use Designware > core > should also be capable to operate in endpoint mode. But linux kernel right now > supports only RC mode. > > PCIe endpoint support discussion came up briefly before [1] but it was felt > the > practical use case will find firmware more suitable and endpoint support in > kernel can be used only for validation or demo. > > *) Modify platform driver to support EP mode (in my case pci-dra7xx.c). > > *) dt binding specific to EP mode should be created. > > Once I complete the implementation and start posting RFC patches, a lot of > these will become clear. But I want to check if this sounds okay to you guys > before starting the implementation. > > Let me know if you have some other ideas too. > > Cheers > Kishon > > [1] -> http://www.spinics.net/lists/linux-pci/msg26026.html > You are rising a topic that we are also addressing in Synopsys. For the PCIe RC hardware validation we are currently using the standard pcie-designware and pcie-designware-plat drivers. For the Endpoint we have to use an internal software package. Its main purpose is to initialize the IP registers, eDMA channels and make data transfer to prove that the everything is working properly. This is done in 2 levels, a custom driver built and loaded and an application that makes some ioctl to the driver executing some interesting functions to check the Endpoint status and make some data exchange. We are very interest in the subject and we are available to participate in the development. I would suggest that also the IP eDMA initialization and manipulation be included in the framework and maybe produce a standard tool to test the endpoint. Thanks Joao
Re: Support for configurable PCIe endpoint
On Wed, Aug 03, 2016 at 11:33:19AM +0530, Kishon Vijay Abraham I wrote: > Hi, > > The PCIe controller present in TI's DRA7 SoC is capable of operating either in > Root Complex mode or Endpoint mode. (It uses Synopsys Designware Core).I'd > assume most of the PCIe controllers on other platforms that use Designware > core > should also be capable to operate in endpoint mode. But linux kernel right now > supports only RC mode. > > PCIe endpoint support discussion came up briefly before [1] but it was felt > the > practical use case will find firmware more suitable and endpoint support in > kernel can be used only for validation or demo. I disagree. It's highly useful for rapid prototyping of hardware interfaces, and I've been looking into PCIe EP drivers for exactly that reason recently. Going a little offtopic: any good DRA7 eval boards you'd recommend to try for this purpose? We already have a EP driver in the tree: drivers/misc/spear13xx_pcie_gadget.c but as far as I can tell it doesn't really work at the moment. > Validation or demo is itself a valid use case in my opinion (consider > something > similar to gadget zero for USB). There can be other use cases as well. The RC > can use the SoC with EP mode support as an accelerator to accomplish specific > task. Here RC gives data to the EP. The EP processes the data. The processing > can be done either in ARM itself or it can use other hardware accelerators > (like DSP, IVA-HD etc..) present in the EP system. If HW accelerator is used, > the linux kernel running in ARM can be used to accomplish other tasks. Once EP > mode support is added, I think more use cases will be added. That sounds useful as well. > >From the high level this should look _similar_ to the gadget framework of > >USB. > One difference from USB would be this should allow HW components (like DSP, > PRU > etc.. and maybe even some peripheral) in the EP system to be used by RC > system. Indeed. > So these are the high-level steps that I thought would be needed to add EP > support in linux. > *) move pcie-designware.c out of drivers/pci/host (maybe create a > drivers/pci/designware/ folder?). All users of pcie-designware.c should be > moved here. > This is in preparation for adding EP mode support to designware. I'd use a new drivers/pci/controller. Or maybe just skip the rename for now and see how this evolves. The rest of the plan sounds fine to me.
Re: Support for configurable PCIe endpoint
On Wed, Aug 03, 2016 at 11:33:19AM +0530, Kishon Vijay Abraham I wrote: > Hi, > > The PCIe controller present in TI's DRA7 SoC is capable of operating either in > Root Complex mode or Endpoint mode. (It uses Synopsys Designware Core).I'd > assume most of the PCIe controllers on other platforms that use Designware > core > should also be capable to operate in endpoint mode. But linux kernel right now > supports only RC mode. > > PCIe endpoint support discussion came up briefly before [1] but it was felt > the > practical use case will find firmware more suitable and endpoint support in > kernel can be used only for validation or demo. I disagree. It's highly useful for rapid prototyping of hardware interfaces, and I've been looking into PCIe EP drivers for exactly that reason recently. Going a little offtopic: any good DRA7 eval boards you'd recommend to try for this purpose? We already have a EP driver in the tree: drivers/misc/spear13xx_pcie_gadget.c but as far as I can tell it doesn't really work at the moment. > Validation or demo is itself a valid use case in my opinion (consider > something > similar to gadget zero for USB). There can be other use cases as well. The RC > can use the SoC with EP mode support as an accelerator to accomplish specific > task. Here RC gives data to the EP. The EP processes the data. The processing > can be done either in ARM itself or it can use other hardware accelerators > (like DSP, IVA-HD etc..) present in the EP system. If HW accelerator is used, > the linux kernel running in ARM can be used to accomplish other tasks. Once EP > mode support is added, I think more use cases will be added. That sounds useful as well. > >From the high level this should look _similar_ to the gadget framework of > >USB. > One difference from USB would be this should allow HW components (like DSP, > PRU > etc.. and maybe even some peripheral) in the EP system to be used by RC > system. Indeed. > So these are the high-level steps that I thought would be needed to add EP > support in linux. > *) move pcie-designware.c out of drivers/pci/host (maybe create a > drivers/pci/designware/ folder?). All users of pcie-designware.c should be > moved here. > This is in preparation for adding EP mode support to designware. I'd use a new drivers/pci/controller. Or maybe just skip the rename for now and see how this evolves. The rest of the plan sounds fine to me.
Re: Support for configurable PCIe endpoint
On Wednesday, August 3, 2016 11:33:19 AM CEST Kishon Vijay Abraham I wrote: > Hi, > > The PCIe controller present in TI's DRA7 SoC is capable of operating either in > Root Complex mode or Endpoint mode. (It uses Synopsys Designware Core).I'd > assume most of the PCIe controllers on other platforms that use Designware > core > should also be capable to operate in endpoint mode. But linux kernel right now > supports only RC mode. > > PCIe endpoint support discussion came up briefly before [1] but it was felt > the > practical use case will find firmware more suitable and endpoint support in > kernel can be used only for validation or demo. > > Validation or demo is itself a valid use case in my opinion (consider > something > similar to gadget zero for USB). There can be other use cases as well. The RC > can use the SoC with EP mode support as an accelerator to accomplish specific > task. Here RC gives data to the EP. The EP processes the data. The processing > can be done either in ARM itself or it can use other hardware accelerators > (like DSP, IVA-HD etc..) present in the EP system. If HW accelerator is used, > the linux kernel running in ARM can be used to accomplish other tasks. Once EP > mode support is added, I think more use cases will be added. > > From the high level this should look _similar_ to the gadget framework of USB. > One difference from USB would be this should allow HW components (like DSP, > PRU > etc.. and maybe even some peripheral) in the EP system to be used by RC > system. (Adding Jon Mason) We have the drivers/ntb framework, which in theory should be able to handle this, but in practice is only used for a very small number of bridge implementations, and is also limited in the way it can be configured (compared to USB gadget) > So these are the high-level steps that I thought would be needed to add EP > support in linux. > *) move pcie-designware.c out of drivers/pci/host (maybe create a > drivers/pci/designware/ folder?). All users of pcie-designware.c should be > moved here. > This is in preparation for adding EP mode support to designware. A lot of the other drivers in drivers/pci/host support endpoint mode too, I don't think moving them all elsewhere is helpful or necessary here. > *) Add library functions in pcie-designware.c specific to EP mode like > initializing general ecam registers, BAR registers etc.. These functions > should > be invoked from a *function* driver (function driver should determine the use > of a particular EP). > > *) Add a sample *function* driver that can be used just for validation. This > function driver will invoke the previously added functions in PCIe controller > to initialize ecam, BAR etc.. This will invoke the PCIe controller functions > through *ep-core* layer. That way the same function driver can be made to work > with different PCIe controllers. (A PCIe driver corresponding to this function > driver should also be implemented in RC side) > > *) Add *ep-core* layer to bind the function driver to the controller driver > and > using which the function driver will invoke controller driver callbacks (to > initialize ecam, BAR registers etc..) I think we should first have a rough idea what the framework should look like, and how it handles having multiple hardware implementation and multiple protocol implementations, before we modify a particular driver. So this step here should come a bit earlier than the others. > *) Modify platform driver to support EP mode (in my case pci-dra7xx.c). > > *) dt binding specific to EP mode should be created. Right. Arnd
Re: Support for configurable PCIe endpoint
On Wednesday, August 3, 2016 11:33:19 AM CEST Kishon Vijay Abraham I wrote: > Hi, > > The PCIe controller present in TI's DRA7 SoC is capable of operating either in > Root Complex mode or Endpoint mode. (It uses Synopsys Designware Core).I'd > assume most of the PCIe controllers on other platforms that use Designware > core > should also be capable to operate in endpoint mode. But linux kernel right now > supports only RC mode. > > PCIe endpoint support discussion came up briefly before [1] but it was felt > the > practical use case will find firmware more suitable and endpoint support in > kernel can be used only for validation or demo. > > Validation or demo is itself a valid use case in my opinion (consider > something > similar to gadget zero for USB). There can be other use cases as well. The RC > can use the SoC with EP mode support as an accelerator to accomplish specific > task. Here RC gives data to the EP. The EP processes the data. The processing > can be done either in ARM itself or it can use other hardware accelerators > (like DSP, IVA-HD etc..) present in the EP system. If HW accelerator is used, > the linux kernel running in ARM can be used to accomplish other tasks. Once EP > mode support is added, I think more use cases will be added. > > From the high level this should look _similar_ to the gadget framework of USB. > One difference from USB would be this should allow HW components (like DSP, > PRU > etc.. and maybe even some peripheral) in the EP system to be used by RC > system. (Adding Jon Mason) We have the drivers/ntb framework, which in theory should be able to handle this, but in practice is only used for a very small number of bridge implementations, and is also limited in the way it can be configured (compared to USB gadget) > So these are the high-level steps that I thought would be needed to add EP > support in linux. > *) move pcie-designware.c out of drivers/pci/host (maybe create a > drivers/pci/designware/ folder?). All users of pcie-designware.c should be > moved here. > This is in preparation for adding EP mode support to designware. A lot of the other drivers in drivers/pci/host support endpoint mode too, I don't think moving them all elsewhere is helpful or necessary here. > *) Add library functions in pcie-designware.c specific to EP mode like > initializing general ecam registers, BAR registers etc.. These functions > should > be invoked from a *function* driver (function driver should determine the use > of a particular EP). > > *) Add a sample *function* driver that can be used just for validation. This > function driver will invoke the previously added functions in PCIe controller > to initialize ecam, BAR etc.. This will invoke the PCIe controller functions > through *ep-core* layer. That way the same function driver can be made to work > with different PCIe controllers. (A PCIe driver corresponding to this function > driver should also be implemented in RC side) > > *) Add *ep-core* layer to bind the function driver to the controller driver > and > using which the function driver will invoke controller driver callbacks (to > initialize ecam, BAR registers etc..) I think we should first have a rough idea what the framework should look like, and how it handles having multiple hardware implementation and multiple protocol implementations, before we modify a particular driver. So this step here should come a bit earlier than the others. > *) Modify platform driver to support EP mode (in my case pci-dra7xx.c). > > *) dt binding specific to EP mode should be created. Right. Arnd
