Re: [PATCH v2 4/9] cxl/pci: Refactor cxl_pci_setup_regs

2021-09-28 Thread Dan Williams
On Thu, Sep 23, 2021 at 10:27 AM Ben Widawsky  wrote:
>
> In preparation for moving parts of register mapping to cxl_core, the
> cxl_pci driver is refactored to utilize a new helper to find register
> blocks by type.
>
> cxl_pci scanned through all register blocks and mapping the ones that
> the driver will use. This logic is inverted so that the driver
> specifically requests the register blocks from a new helper. Under the
> hood, the same implementation of scanning through all register locator
> DVSEC entries exists.
>
> There are 2 behavioral changes (#2 is arguable):
> 1. A dev_err is introduced if cxl_map_regs fails.
> 2. The previous logic would try to map component registers and device
>registers multiple times if there were present and keep the mapping
>of the last one found (furthest offset in the register locator).
>While this is disallowed in the spec, CXL 2.0 8.1.9: "Each register
>block identifier shall only occur once in the Register Locator DVSEC
>structure" it was how the driver would respond to the spec violation.
>The new logic will take the first found register block by type and
>move on.
>
> Signed-off-by: Ben Widawsky 
>
> ---
> Changes since v1:

No changes? Luckily git am strips this section...

Overall I think this refactor can be broken down further for
readability and cleanup the long standing problem that the driver maps
component registers for no reason. The main contributing factor to
readability is that cxl_setup_pci_regs() still exists after the
refactor, which also contributes to the component register problem. If
the register mapping is up leveled to the caller of
cxl_setup_pci_regs() (and drops mapping component registers) then a
follow-on patch to rename cxl_setup_pci_regs to find_register_block
becomes easier to read. Moving the cxl_register_map array out of
cxl_setup_pci_regs() also makes a later patch to pass component
register enumeration details to the endpoint-port that much cleaner.


[PATCH v2 4/9] cxl/pci: Refactor cxl_pci_setup_regs

2021-09-23 Thread Ben Widawsky
In preparation for moving parts of register mapping to cxl_core, the
cxl_pci driver is refactored to utilize a new helper to find register
blocks by type.

cxl_pci scanned through all register blocks and mapping the ones that
the driver will use. This logic is inverted so that the driver
specifically requests the register blocks from a new helper. Under the
hood, the same implementation of scanning through all register locator
DVSEC entries exists.

There are 2 behavioral changes (#2 is arguable):
1. A dev_err is introduced if cxl_map_regs fails.
2. The previous logic would try to map component registers and device
   registers multiple times if there were present and keep the mapping
   of the last one found (furthest offset in the register locator).
   While this is disallowed in the spec, CXL 2.0 8.1.9: "Each register
   block identifier shall only occur once in the Register Locator DVSEC
   structure" it was how the driver would respond to the spec violation.
   The new logic will take the first found register block by type and
   move on.

Signed-off-by: Ben Widawsky 

---
Changes since v1:
---
 drivers/cxl/pci.c | 126 +-
 1 file changed, 70 insertions(+), 56 deletions(-)

diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c
index 7256c236fdb3..bbbacbc94fbf 100644
--- a/drivers/cxl/pci.c
+++ b/drivers/cxl/pci.c
@@ -428,6 +428,45 @@ static void cxl_decode_register_block(u32 reg_lo, u32 
reg_hi,
*reg_type = FIELD_GET(CXL_REGLOC_RBI_MASK, reg_lo);
 }
 
+static int find_register_block(struct pci_dev *pdev, enum cxl_regloc_type type,
+  struct cxl_register_map *map)
+{
+   int regloc, i, rc = -ENODEV;
+   u32 regloc_size, regblocks;
+
+   regloc = cxl_pci_dvsec(pdev, PCI_DVSEC_ID_CXL_REGLOC_DVSEC_ID);
+   if (!regloc)
+   return -ENXIO;
+
+   pci_read_config_dword(pdev, regloc + PCI_DVSEC_HEADER1, _size);
+   regloc_size = FIELD_GET(PCI_DVSEC_HEADER1_LENGTH_MASK, regloc_size);
+
+   regloc += PCI_DVSEC_ID_CXL_REGLOC_BLOCK1_OFFSET;
+   regblocks = (regloc_size - PCI_DVSEC_ID_CXL_REGLOC_BLOCK1_OFFSET) / 8;
+
+   for (i = 0; i < regblocks; i++, regloc += 8) {
+   u32 reg_lo, reg_hi;
+   u8 reg_type, bar;
+   u64 offset;
+
+   pci_read_config_dword(pdev, regloc, _lo);
+   pci_read_config_dword(pdev, regloc + 4, _hi);
+
+   cxl_decode_register_block(reg_lo, reg_hi, , ,
+ _type);
+
+   if (reg_type == type) {
+   map->barno = bar;
+   map->block_offset = offset;
+   map->reg_type = reg_type;
+   rc = 0;
+   break;
+   }
+   }
+
+   return rc;
+}
+
 /**
  * cxl_pci_setup_regs() - Setup necessary MMIO.
  * @cxlm: The CXL memory device to communicate with.
@@ -440,69 +479,44 @@ static void cxl_decode_register_block(u32 reg_lo, u32 
reg_hi,
  */
 static int cxl_pci_setup_regs(struct cxl_mem *cxlm)
 {
-   void __iomem *base;
-   u32 regloc_size, regblocks;
-   int regloc, i, n_maps, ret = 0;
+   int rc, i;
struct device *dev = cxlm->dev;
struct pci_dev *pdev = to_pci_dev(dev);
-   struct cxl_register_map *map, maps[CXL_REGLOC_RBI_TYPES];
+   const enum cxl_regloc_type types[] = { CXL_REGLOC_RBI_MEMDEV,
+  CXL_REGLOC_RBI_COMPONENT };
 
-   regloc = cxl_pci_dvsec(pdev, PCI_DVSEC_ID_CXL_REGLOC_DVSEC_ID);
-   if (!regloc) {
-   dev_err(dev, "register location dvsec not found\n");
-   return -ENXIO;
-   }
+   for (i = 0; i < ARRAY_SIZE(types); i++) {
+   struct cxl_register_map map;
+   void __iomem *base;
 
-   /* Get the size of the Register Locator DVSEC */
-   pci_read_config_dword(pdev, regloc + PCI_DVSEC_HEADER1, _size);
-   regloc_size = FIELD_GET(PCI_DVSEC_HEADER1_LENGTH_MASK, regloc_size);
-
-   regloc += PCI_DVSEC_ID_CXL_REGLOC_BLOCK1_OFFSET;
-   regblocks = (regloc_size - PCI_DVSEC_ID_CXL_REGLOC_BLOCK1_OFFSET) / 8;
-
-   for (i = 0, n_maps = 0; i < regblocks; i++, regloc += 8) {
-   u32 reg_lo, reg_hi;
-   u8 reg_type;
-   u64 offset;
-   u8 bar;
-
-   pci_read_config_dword(pdev, regloc, _lo);
-   pci_read_config_dword(pdev, regloc + 4, _hi);
-
-   cxl_decode_register_block(reg_lo, reg_hi, , ,
- _type);
-
-   /* Ignore unknown register block types */
-   if (reg_type > CXL_REGLOC_RBI_MEMDEV)
-   continue;
-
-   base = cxl_pci_map_regblock(cxlm, bar, offset);
-   if (!base)
-   return -ENOMEM;
-
-   map = [n_maps];
-   map->barno = bar;
-