Re: [PATCH v11 8/8] PCI: endpoint: Remove "core_init_notifier" flag
On Wed, Mar 27, 2024 at 09:24:05AM +0100, Niklas Cassel wrote: > Hello Mani, > > On Wed, Mar 27, 2024 at 12:05:54PM +0530, Manivannan Sadhasivam wrote: > > "core_init_notifier" flag is set by the glue drivers requiring refclk from > > the host to complete the DWC core initialization. Also, those drivers will > > send a notification to the EPF drivers once the initialization is fully > > completed using the pci_epc_init_notify() API. Only then, the EPF drivers > > will start functioning. > > > > For the rest of the drivers generating refclk locally, EPF drivers will > > start functioning post binding with them. EPF drivers rely on the > > 'core_init_notifier' flag to differentiate between the drivers. > > Unfortunately, this creates two different flows for the EPF drivers. > > > > So to avoid that, let's get rid of the "core_init_notifier" flag and follow > > a single initialization flow for the EPF drivers. This is done by calling > > the dw_pcie_ep_init_notify() from all glue drivers after the completion of > > dw_pcie_ep_init_registers() API. This will allow all the glue drivers to > > send the notification to the EPF drivers once the initialization is fully > > completed. > > > > Only difference here is that, the drivers requiring refclk from host will > > send the notification once refclk is received, while others will send it > > during probe time itself. > > > > But this also requires the EPC core driver to deliver the notification > > after EPF driver bind. Because, the glue driver can send the notification > > before the EPF drivers bind() and in those cases the EPF drivers will miss > > the event. To accommodate this, EPC core is now caching the state of the > > EPC initialization in 'init_complete' flag and pci-ep-cfs driver sends the > > notification to EPF drivers based on that after each EPF driver bind. > > > > Tested-by: Niklas Cassel > > Signed-off-by: Manivannan Sadhasivam > > --- > > drivers/pci/controller/cadence/pcie-cadence-ep.c | 2 ++ > > drivers/pci/controller/dwc/pci-dra7xx.c | 2 ++ > > drivers/pci/controller/dwc/pci-imx6.c | 2 ++ > > drivers/pci/controller/dwc/pci-keystone.c | 2 ++ > > drivers/pci/controller/dwc/pci-layerscape-ep.c| 2 ++ > > drivers/pci/controller/dwc/pcie-artpec6.c | 2 ++ > > drivers/pci/controller/dwc/pcie-designware-ep.c | 1 + > > drivers/pci/controller/dwc/pcie-designware-plat.c | 2 ++ > > drivers/pci/controller/dwc/pcie-keembay.c | 2 ++ > > drivers/pci/controller/dwc/pcie-qcom-ep.c | 1 - > > drivers/pci/controller/dwc/pcie-rcar-gen4.c | 2 ++ > > drivers/pci/controller/dwc/pcie-tegra194.c| 1 - > > drivers/pci/controller/dwc/pcie-uniphier-ep.c | 2 ++ > > drivers/pci/controller/pcie-rcar-ep.c | 2 ++ > > drivers/pci/controller/pcie-rockchip-ep.c | 2 ++ > > drivers/pci/endpoint/functions/pci-epf-test.c | 18 +- > > drivers/pci/endpoint/pci-ep-cfs.c | 9 + > > drivers/pci/endpoint/pci-epc-core.c | 22 > > ++ > > include/linux/pci-epc.h | 7 --- > > 19 files changed, 65 insertions(+), 18 deletions(-) > > > > diff --git a/drivers/pci/controller/cadence/pcie-cadence-ep.c > > b/drivers/pci/controller/cadence/pcie-cadence-ep.c > > index 2d0a8d78bffb..da67a06ee790 100644 > > --- a/drivers/pci/controller/cadence/pcie-cadence-ep.c > > +++ b/drivers/pci/controller/cadence/pcie-cadence-ep.c > > @@ -734,6 +734,8 @@ int cdns_pcie_ep_setup(struct cdns_pcie_ep *ep) > > > > spin_lock_init(>lock); > > > > + dw_pcie_ep_init_notify(>ep); > > This looks wrong (and I think that you have not build tested this). > Ah, this is silly. Sorry, added the change in a rush :( > dw_* prefix indicates DWC, so it is a DWC specific function. > > I don't think that you can use this function for the 3 non-DWC EPC drivers. > I think that you need to use call pci_epc_init_notify() directly. > > > (Also perhaps rebase your series on v6.9-rc1, I got conflicts when trying > to apply it to v6.9-rc1, because it looks like the series is still based > on v6.8-rc1.) > I rebased the epf rework series and didn't get any conflict. But will rebase this one also and send next version. Thanks for noticing my idiocy. - Mani -- மணிவண்ணன் சதாசிவம்
Re: [PATCH v11 8/8] PCI: endpoint: Remove "core_init_notifier" flag
Hello Mani, On Wed, Mar 27, 2024 at 12:05:54PM +0530, Manivannan Sadhasivam wrote: > "core_init_notifier" flag is set by the glue drivers requiring refclk from > the host to complete the DWC core initialization. Also, those drivers will > send a notification to the EPF drivers once the initialization is fully > completed using the pci_epc_init_notify() API. Only then, the EPF drivers > will start functioning. > > For the rest of the drivers generating refclk locally, EPF drivers will > start functioning post binding with them. EPF drivers rely on the > 'core_init_notifier' flag to differentiate between the drivers. > Unfortunately, this creates two different flows for the EPF drivers. > > So to avoid that, let's get rid of the "core_init_notifier" flag and follow > a single initialization flow for the EPF drivers. This is done by calling > the dw_pcie_ep_init_notify() from all glue drivers after the completion of > dw_pcie_ep_init_registers() API. This will allow all the glue drivers to > send the notification to the EPF drivers once the initialization is fully > completed. > > Only difference here is that, the drivers requiring refclk from host will > send the notification once refclk is received, while others will send it > during probe time itself. > > But this also requires the EPC core driver to deliver the notification > after EPF driver bind. Because, the glue driver can send the notification > before the EPF drivers bind() and in those cases the EPF drivers will miss > the event. To accommodate this, EPC core is now caching the state of the > EPC initialization in 'init_complete' flag and pci-ep-cfs driver sends the > notification to EPF drivers based on that after each EPF driver bind. > > Tested-by: Niklas Cassel > Signed-off-by: Manivannan Sadhasivam > --- > drivers/pci/controller/cadence/pcie-cadence-ep.c | 2 ++ > drivers/pci/controller/dwc/pci-dra7xx.c | 2 ++ > drivers/pci/controller/dwc/pci-imx6.c | 2 ++ > drivers/pci/controller/dwc/pci-keystone.c | 2 ++ > drivers/pci/controller/dwc/pci-layerscape-ep.c| 2 ++ > drivers/pci/controller/dwc/pcie-artpec6.c | 2 ++ > drivers/pci/controller/dwc/pcie-designware-ep.c | 1 + > drivers/pci/controller/dwc/pcie-designware-plat.c | 2 ++ > drivers/pci/controller/dwc/pcie-keembay.c | 2 ++ > drivers/pci/controller/dwc/pcie-qcom-ep.c | 1 - > drivers/pci/controller/dwc/pcie-rcar-gen4.c | 2 ++ > drivers/pci/controller/dwc/pcie-tegra194.c| 1 - > drivers/pci/controller/dwc/pcie-uniphier-ep.c | 2 ++ > drivers/pci/controller/pcie-rcar-ep.c | 2 ++ > drivers/pci/controller/pcie-rockchip-ep.c | 2 ++ > drivers/pci/endpoint/functions/pci-epf-test.c | 18 +- > drivers/pci/endpoint/pci-ep-cfs.c | 9 + > drivers/pci/endpoint/pci-epc-core.c | 22 ++ > include/linux/pci-epc.h | 7 --- > 19 files changed, 65 insertions(+), 18 deletions(-) > > diff --git a/drivers/pci/controller/cadence/pcie-cadence-ep.c > b/drivers/pci/controller/cadence/pcie-cadence-ep.c > index 2d0a8d78bffb..da67a06ee790 100644 > --- a/drivers/pci/controller/cadence/pcie-cadence-ep.c > +++ b/drivers/pci/controller/cadence/pcie-cadence-ep.c > @@ -734,6 +734,8 @@ int cdns_pcie_ep_setup(struct cdns_pcie_ep *ep) > > spin_lock_init(>lock); > > + dw_pcie_ep_init_notify(>ep); This looks wrong (and I think that you have not build tested this). dw_* prefix indicates DWC, so it is a DWC specific function. I don't think that you can use this function for the 3 non-DWC EPC drivers. I think that you need to use call pci_epc_init_notify() directly. (Also perhaps rebase your series on v6.9-rc1, I got conflicts when trying to apply it to v6.9-rc1, because it looks like the series is still based on v6.8-rc1.) Kind regards, Niklas
[PATCH v11 8/8] PCI: endpoint: Remove "core_init_notifier" flag
"core_init_notifier" flag is set by the glue drivers requiring refclk from the host to complete the DWC core initialization. Also, those drivers will send a notification to the EPF drivers once the initialization is fully completed using the pci_epc_init_notify() API. Only then, the EPF drivers will start functioning. For the rest of the drivers generating refclk locally, EPF drivers will start functioning post binding with them. EPF drivers rely on the 'core_init_notifier' flag to differentiate between the drivers. Unfortunately, this creates two different flows for the EPF drivers. So to avoid that, let's get rid of the "core_init_notifier" flag and follow a single initialization flow for the EPF drivers. This is done by calling the dw_pcie_ep_init_notify() from all glue drivers after the completion of dw_pcie_ep_init_registers() API. This will allow all the glue drivers to send the notification to the EPF drivers once the initialization is fully completed. Only difference here is that, the drivers requiring refclk from host will send the notification once refclk is received, while others will send it during probe time itself. But this also requires the EPC core driver to deliver the notification after EPF driver bind. Because, the glue driver can send the notification before the EPF drivers bind() and in those cases the EPF drivers will miss the event. To accommodate this, EPC core is now caching the state of the EPC initialization in 'init_complete' flag and pci-ep-cfs driver sends the notification to EPF drivers based on that after each EPF driver bind. Tested-by: Niklas Cassel Signed-off-by: Manivannan Sadhasivam --- drivers/pci/controller/cadence/pcie-cadence-ep.c | 2 ++ drivers/pci/controller/dwc/pci-dra7xx.c | 2 ++ drivers/pci/controller/dwc/pci-imx6.c | 2 ++ drivers/pci/controller/dwc/pci-keystone.c | 2 ++ drivers/pci/controller/dwc/pci-layerscape-ep.c| 2 ++ drivers/pci/controller/dwc/pcie-artpec6.c | 2 ++ drivers/pci/controller/dwc/pcie-designware-ep.c | 1 + drivers/pci/controller/dwc/pcie-designware-plat.c | 2 ++ drivers/pci/controller/dwc/pcie-keembay.c | 2 ++ drivers/pci/controller/dwc/pcie-qcom-ep.c | 1 - drivers/pci/controller/dwc/pcie-rcar-gen4.c | 2 ++ drivers/pci/controller/dwc/pcie-tegra194.c| 1 - drivers/pci/controller/dwc/pcie-uniphier-ep.c | 2 ++ drivers/pci/controller/pcie-rcar-ep.c | 2 ++ drivers/pci/controller/pcie-rockchip-ep.c | 2 ++ drivers/pci/endpoint/functions/pci-epf-test.c | 18 +- drivers/pci/endpoint/pci-ep-cfs.c | 9 + drivers/pci/endpoint/pci-epc-core.c | 22 ++ include/linux/pci-epc.h | 7 --- 19 files changed, 65 insertions(+), 18 deletions(-) diff --git a/drivers/pci/controller/cadence/pcie-cadence-ep.c b/drivers/pci/controller/cadence/pcie-cadence-ep.c index 2d0a8d78bffb..da67a06ee790 100644 --- a/drivers/pci/controller/cadence/pcie-cadence-ep.c +++ b/drivers/pci/controller/cadence/pcie-cadence-ep.c @@ -734,6 +734,8 @@ int cdns_pcie_ep_setup(struct cdns_pcie_ep *ep) spin_lock_init(>lock); + dw_pcie_ep_init_notify(>ep); + return 0; free_epc_mem: diff --git a/drivers/pci/controller/dwc/pci-dra7xx.c b/drivers/pci/controller/dwc/pci-dra7xx.c index 395042b29ffc..d2d17d37d3e0 100644 --- a/drivers/pci/controller/dwc/pci-dra7xx.c +++ b/drivers/pci/controller/dwc/pci-dra7xx.c @@ -474,6 +474,8 @@ static int dra7xx_add_pcie_ep(struct dra7xx_pcie *dra7xx, return ret; } + dw_pcie_ep_init_notify(ep); + return 0; } diff --git a/drivers/pci/controller/dwc/pci-imx6.c b/drivers/pci/controller/dwc/pci-imx6.c index bfcafa440ddb..894b5de76e3a 100644 --- a/drivers/pci/controller/dwc/pci-imx6.c +++ b/drivers/pci/controller/dwc/pci-imx6.c @@ -1144,6 +1144,8 @@ static int imx6_add_pcie_ep(struct imx6_pcie *imx6_pcie, return ret; } + dw_pcie_ep_init_notify(ep); + /* Start LTSSM. */ imx6_pcie_ltssm_enable(dev); diff --git a/drivers/pci/controller/dwc/pci-keystone.c b/drivers/pci/controller/dwc/pci-keystone.c index 093dbb725e41..b7b30470b394 100644 --- a/drivers/pci/controller/dwc/pci-keystone.c +++ b/drivers/pci/controller/dwc/pci-keystone.c @@ -1293,6 +1293,8 @@ static int ks_pcie_probe(struct platform_device *pdev) goto err_ep_init; } + dw_pcie_ep_init_notify(>ep); + break; default: dev_err(dev, "INVALID device type %d\n", mode); diff --git a/drivers/pci/controller/dwc/pci-layerscape-ep.c b/drivers/pci/controller/dwc/pci-layerscape-ep.c index b712fdd06549..c513598a46d7 100644 --- a/drivers/pci/controller/dwc/pci-layerscape-ep.c +++ b/drivers/pci/controller/dwc/pci-layerscape-ep.c @@ -283,6 +283,8 @@ static int __init
