This series proposes support for mapping subranges within a PCIe endpoint
BAR and enables controllers to program inbound address translation for
those subranges.
Note: This series is a spin-off from a larger RFC series:
https://lore.kernel.org/all/[email protected]/
The first user will likely be epf-vntb for Remote eDMA-backed NTB
transport, demonstrated in that RFC series.
Motivation
==========
(This section is identical to my earlier explanation at:
https://lore.kernel.org/linux-pci/waapztvy6jyjqtfcoo3rbgvagi4z3p5afw6x2acgf5bxatcui6@nkodhtqqtetr)
The motivation for BAR subrange mapping is that some EP platforms
effectively have only two practically usable BARs, while needing multiple
logically independent inbound mapping.
For example, on Renesas R-Car Gen4 Spider, 64-bit BAR0 and BAR2 are the
only practically usable BARs, since BAR4 is only 256 bytes. epf-vntb
already needs two separate regions (config+spad and MW1 for the
data-plane), leaving no spare BAR. Adding ntb_msi requires yet another MW,
which simply does not fit unless an existing BAR is further divided.
In theory, some vNTB regions (e.g. config+spad and dynamically allocated
memory-backed MWs) could be tightly packed into a single physically
contiguous BAR region to barely make it work. However, it immediately makes
features mutually exclusive (e.g. ntb_msi / ntb_edma cannot coexist), and
the layout becomes extremely fragile.
Similarly, for remote eDMA-backed NTB transport, the host needs separate
inbound access to the eDMA register block and the LL regions, which are
distinct local address ranges. Without subrange mapping, the only choice is
unnatural layout (e.g. dedicating one BAR entirely to eDMA register block,
and another one to everything else that can be packed into a single locally
contiguous memory region), even when this is barely possible.
So while some cases might be made to work by aggressive packing, they are
already at the limit on platforms such as R-Car Spider. BAR subrange
mapping allows these features to be implemented in a straightforward,
loosely-coupled, and extensible way on platforms with severely constrained
BAR resources.
Patch layout
============
- Patch 1/5 introduces dynamic_inbound_mapping feature bit. This can be
used as a safeguard to check whether a BAR can really be reconfigured
without clearing/resetting it.
- Patch 2/5 introduces generic BAR subrange mapping support in the PCI
endpoint core.
- Patch 3/5 advertises dynamic inbound mapping support via
DWC_EPC_COMMON_FEATURES for all DWC-based glue drivers.
- Patch 4/5 adds an implementation for the DesignWare PCIe endpoint
controller using Address Match Mode IB iATU. It also advertises
subrange_mapping support via DWC_EPC_COMMON_FEATURES.
- Patch 5/5 updates a documentation for pci_epc_set_bar().
Kernel base
===========
- repo: git://git.kernel.org/pub/scm/linux/kernel/git/pci/pci.git
- branch: controller/dwc
- commit: 86291f774fe8 ("PCI: dwc: Remove duplicate
dw_pcie_ep_hide_ext_capability() function")
Changelog
=========
* v8->v9 changes:
- Dropped now meaningless source code comments, which should have been
removed in v8 (feedback from Frank).
- Added motivation text to the cover letter and to Patch 2/5 and Patch
4/5.
- Rebased onto the latest controller/dwc branch (see "Kernel base"
section above).
* v7->v8 changes:
- Drop the explicit submap offset and use_submap fields. The submap
array order now defines BAR offsets (gapless decomposition). Commit
messages and source code documentations are adjusted accordingly.
- Drop the no-longer-needed dw_pcie_ib_map and add ib_atu_indexes field
to track iatu indexes used for teardown.
- Move inbound mapping teardown into dw_pcie_ep_set_bar() to cover all
BAR transition cases.
- Centralize feature bit advertisement via DWC_EPC_COMMON_FEATURES.
- Added a (epf_bar->num_submap && !epf_bar->submap) check in
pci_epc_set_bar().
- Note that some Reviewed-by tags are dropped, as the changes need to be
re-reviewed.
* v6->v7 changes:
- Added missing check of dynamic_inbound_mapping feature bit in
pci_epc_set_bar() when use_submap is set true.
- Addressed the remaining review comments from Niklas (patch reordering,
splitting, and source code comment/documentation refinements).
* v5->v6 changes:
- Added a new feature bit dynamic_inbound_mapping and set it centrally
in dw_pcie_ep_get_features() for all DWC-based glue drivers.
- Updated documentation for pci_epc_set_bar().
- Dropped a needless and harmful dw_pcie_ep_clear_bar() call on the error
path.
- Fixed "Bar Match Mode" to "BAR Match Mode" in a source code comment.
* v4->v5 changes:
- Added subrange_mapping to struct pci_epc_features and enforced a
strict capability check in pci_epc_set_bar() (reject use_submap when
unsupported).
- Changed DWC-based glue drivers to return a mutable features pointer
and set subrange_mapping centrally at the DWC midlayer.
- Split the series into 3 patches accordingly.
* v3->v4 changes:
- Drop unused includes that should have been removed in v3
* v2->v3 changes:
- Remove submap copying and sorting from dw_pcie_ep_ib_atu_addr(), and
require callers to pass a sorted submap. The related source code
comments are updated accordingly.
- Refine source code comments and commit messages, including normalizing
"Address Match Mode" wording.
- Add const qualifiers where applicable.
* v1->v2 changes:
- Introduced stricter submap validation: no holes/overlaps and the
subranges must exactly cover the whole BAR. Added
dw_pcie_ep_validate_submap() to enforce alignment and full-coverage
constraints.
- Enforced one-shot (all-or-nothing) submap programming to avoid leaving
half-programmed BAR state:
* Dropped incremental/overwrite logic that is no longer needed with the
one-shot design.
* Added dw_pcie_ep_clear_ib_maps() and used it from multiple places to
tear down BAR match / address match inbound mappings without code
duplication.
- Updated kernel source code comments and commit messages, including a
small refinement made along the way.
- Changed num_submap type to unsigned int.
v8: https://lore.kernel.org/all/[email protected]/
v7: https://lore.kernel.org/all/[email protected]/
v6: https://lore.kernel.org/all/[email protected]/
v5: https://lore.kernel.org/all/[email protected]/
v4: https://lore.kernel.org/all/[email protected]/
v3: https://lore.kernel.org/all/[email protected]/
v2: https://lore.kernel.org/all/[email protected]/
v1: https://lore.kernel.org/all/[email protected]/
Thank you for reviewing,
Koichiro Den (5):
PCI: endpoint: Add dynamic_inbound_mapping EPC feature
PCI: endpoint: Add BAR subrange mapping support
PCI: dwc: Advertise dynamic inbound mapping support
PCI: dwc: ep: Support BAR subrange inbound mapping via Address Match
Mode iATU
Documentation: PCI: endpoint: Clarify pci_epc_set_bar() usage
Documentation/PCI/endpoint/pci-endpoint.rst | 24 +++
drivers/pci/controller/dwc/pci-dra7xx.c | 1 +
drivers/pci/controller/dwc/pci-imx6.c | 3 +
drivers/pci/controller/dwc/pci-keystone.c | 1 +
drivers/pci/controller/dwc/pcie-artpec6.c | 1 +
.../pci/controller/dwc/pcie-designware-ep.c | 203 +++++++++++++++++-
.../pci/controller/dwc/pcie-designware-plat.c | 1 +
drivers/pci/controller/dwc/pcie-designware.h | 8 +
drivers/pci/controller/dwc/pcie-dw-rockchip.c | 2 +
drivers/pci/controller/dwc/pcie-keembay.c | 1 +
drivers/pci/controller/dwc/pcie-qcom-ep.c | 1 +
drivers/pci/controller/dwc/pcie-rcar-gen4.c | 1 +
drivers/pci/controller/dwc/pcie-stm32-ep.c | 1 +
drivers/pci/controller/dwc/pcie-tegra194.c | 1 +
drivers/pci/controller/dwc/pcie-uniphier-ep.c | 2 +
drivers/pci/endpoint/pci-epc-core.c | 8 +
include/linux/pci-epc.h | 9 +
include/linux/pci-epf.h | 23 ++
18 files changed, 281 insertions(+), 10 deletions(-)
--
2.51.0
