Re: [PATCH 01/14] cxl/mem: Introduce a driver for CXL-2.0-Type-3 endpoints
> +static int cxl_mem_dvsec(struct pci_dev *pdev, int dvsec)
> +{
> + int pos;
> +
> + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_DVSEC);
> + if (!pos)
> + return 0;
> +
> + while (pos) {
> + u16 vendor, id;
> +
> + pci_read_config_word(pdev, pos + PCI_DVSEC_VENDOR_ID_OFFSET,
> + );
> + pci_read_config_word(pdev, pos + PCI_DVSEC_ID_OFFSET, );
> + if (vendor == PCI_DVSEC_VENDOR_ID_CXL && dvsec == id)
> + return pos;
> +
> + pos = pci_find_next_ext_capability(pdev, pos,
> +PCI_EXT_CAP_ID_DVSEC);
> + }
> +
> + return 0;
> +}
There should be a patch with a generic version of this find a vendor
specific capability helper on linux-pci.
> +#define PCI_CLASS_MEMORY_CXL 0x050210
I think this should go into linux/pci_ids.h.
Re: [PATCH 01/14] cxl/mem: Introduce a driver for CXL-2.0-Type-3 endpoints
On Mon, Feb 01, 2021 at 12:34:11PM -0500, Konrad Rzeszutek Wilk wrote: > > +// SPDX-License-Identifier: GPL-2.0-only > > +/* Copyright(c) 2020 Intel Corporation. All rights reserved. */ > > Can those two comments have the same type? As in either > stay with // or do /*. No. // is only intended for the SPDX comments specifically so that they stand out.
Re: [PATCH 01/14] cxl/mem: Introduce a driver for CXL-2.0-Type-3 endpoints
> +cxl_mem-y := mem.o > diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c > new file mode 100644 > index ..f4ee9a507ac9 > --- /dev/null > +++ b/drivers/cxl/mem.c > @@ -0,0 +1,61 @@ > +// SPDX-License-Identifier: GPL-2.0-only > +/* Copyright(c) 2020 Intel Corporation. All rights reserved. */ Can those two comments have the same type? As in either stay with // or do /*. Also the year is incorrect.
Re: [PATCH 01/14] cxl/mem: Introduce a driver for CXL-2.0-Type-3 endpoints
On Fri, 29 Jan 2021 16:24:25 -0800 Ben Widawsky wrote: > From: Dan Williams > > The CXL.mem protocol allows a device to act as a provider of "System > RAM" and/or "Persistent Memory" that is fully coherent as if the memory > was attached to the typical CPU memory controller. > > With the CXL-2.0 specification a PCI endpoint can implement a "Type-3" > device interface and give the operating system control over "Host > Managed Device Memory". See section 2.3 Type 3 CXL Device. > > The memory range exported by the device may optionally be described by > the platform firmware memory map, or by infrastructure like LIBNVDIMM to > provision persistent memory capacity from one, or more, CXL.mem devices. > > A pre-requisite for Linux-managed memory-capacity provisioning is this > cxl_mem driver that can speak the mailbox protocol defined in section > 8.2.8.4 Mailbox Registers. > > For now just land the initial driver boiler-plate and Documentation/ > infrastructure. > > Link: https://www.computeexpresslink.org/download-the-specification > Cc: Jonathan Corbet > Signed-off-by: Dan Williams > Signed-off-by: Ben Widawsky Hi Ben, One thing below about using defs from generic PCI headers where they are not CXL specific. > diff --git a/drivers/cxl/pci.h b/drivers/cxl/pci.h > new file mode 100644 > index ..a8a9935fa90b > --- /dev/null > +++ b/drivers/cxl/pci.h > @@ -0,0 +1,20 @@ > +/* SPDX-License-Identifier: GPL-2.0-only */ > +/* Copyright(c) 2020 Intel Corporation. All rights reserved. */ > +#ifndef __CXL_PCI_H__ > +#define __CXL_PCI_H__ > + > +#define PCI_CLASS_MEMORY_CXL 0x050210 > + > +/* > + * See section 8.1 Configuration Space Registers in the CXL 2.0 > + * Specification > + */ > +#define PCI_EXT_CAP_ID_DVSEC 0x23 > +#define PCI_DVSEC_VENDOR_ID_CXL 0x1E98 > +#define PCI_DVSEC_VENDOR_ID_OFFSET 0x4 > +#define PCI_DVSEC_ID_CXL 0x0 > +#define PCI_DVSEC_ID_OFFSET 0x8 include/uapi/linux/pci-regs.h includes equivalents of generic parts of this already though PCI_DVSEC_HEADER1 isn't exactly informative naming. > + > +#define PCI_DVSEC_ID_CXL_REGLOC 0x8 > + > +#endif /* __CXL_PCI_H__ */
Re: [PATCH 01/14] cxl/mem: Introduce a driver for CXL-2.0-Type-3 endpoints
On Fri, 29 Jan 2021, Ben Widawsky wrote: > diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig > new file mode 100644 > index ..3b66b46af8a0 > --- /dev/null > +++ b/drivers/cxl/Kconfig > @@ -0,0 +1,35 @@ > +# SPDX-License-Identifier: GPL-2.0-only > +menuconfig CXL_BUS > + tristate "CXL (Compute Express Link) Devices Support" > + depends on PCI > + help > + CXL is a bus that is electrically compatible with PCI Express, but > + layers three protocols on that signalling (CXL.io, CXL.cache, and > + CXL.mem). The CXL.cache protocol allows devices to hold cachelines > + locally, the CXL.mem protocol allows devices to be fully coherent > + memory targets, the CXL.io protocol is equivalent to PCI Express. > + Say 'y' to enable support for the configuration and management of > + devices supporting these protocols. > + > +if CXL_BUS > + > +config CXL_MEM > + tristate "CXL.mem: Endpoint Support" Nit: "CXL.mem: Memory Devices" or "CXL Memory Devices: CXL.mem" might look better, but feel free to ignore. Acked-by: David Rientjes
[PATCH 01/14] cxl/mem: Introduce a driver for CXL-2.0-Type-3 endpoints
From: Dan Williams The CXL.mem protocol allows a device to act as a provider of "System RAM" and/or "Persistent Memory" that is fully coherent as if the memory was attached to the typical CPU memory controller. With the CXL-2.0 specification a PCI endpoint can implement a "Type-3" device interface and give the operating system control over "Host Managed Device Memory". See section 2.3 Type 3 CXL Device. The memory range exported by the device may optionally be described by the platform firmware memory map, or by infrastructure like LIBNVDIMM to provision persistent memory capacity from one, or more, CXL.mem devices. A pre-requisite for Linux-managed memory-capacity provisioning is this cxl_mem driver that can speak the mailbox protocol defined in section 8.2.8.4 Mailbox Registers. For now just land the initial driver boiler-plate and Documentation/ infrastructure. Link: https://www.computeexpresslink.org/download-the-specification Cc: Jonathan Corbet Signed-off-by: Dan Williams Signed-off-by: Ben Widawsky --- Documentation/driver-api/cxl/index.rst| 12 .../driver-api/cxl/memory-devices.rst | 29 + Documentation/driver-api/index.rst| 1 + drivers/Kconfig | 1 + drivers/Makefile | 1 + drivers/cxl/Kconfig | 35 +++ drivers/cxl/Makefile | 4 ++ drivers/cxl/mem.c | 61 +++ drivers/cxl/pci.h | 20 ++ 9 files changed, 164 insertions(+) create mode 100644 Documentation/driver-api/cxl/index.rst create mode 100644 Documentation/driver-api/cxl/memory-devices.rst create mode 100644 drivers/cxl/Kconfig create mode 100644 drivers/cxl/Makefile create mode 100644 drivers/cxl/mem.c create mode 100644 drivers/cxl/pci.h diff --git a/Documentation/driver-api/cxl/index.rst b/Documentation/driver-api/cxl/index.rst new file mode 100644 index ..036e49553542 --- /dev/null +++ b/Documentation/driver-api/cxl/index.rst @@ -0,0 +1,12 @@ +.. SPDX-License-Identifier: GPL-2.0 + + +Compute Express Link + + +.. toctree:: + :maxdepth: 1 + + memory-devices + +.. only:: subproject and html diff --git a/Documentation/driver-api/cxl/memory-devices.rst b/Documentation/driver-api/cxl/memory-devices.rst new file mode 100644 index ..43177e700d62 --- /dev/null +++ b/Documentation/driver-api/cxl/memory-devices.rst @@ -0,0 +1,29 @@ +.. SPDX-License-Identifier: GPL-2.0 +.. include:: + +=== +Compute Express Link Memory Devices +=== + +A Compute Express Link Memory Device is a CXL component that implements the +CXL.mem protocol. It contains some amount of volatile memory, persistent memory, +or both. It is enumerated as a PCI device for configuration and passing +messages over an MMIO mailbox. Its contribution to the System Physical +Address space is handled via HDM (Host Managed Device Memory) decoders +that optionally define a device's contribution to an interleaved address +range across multiple devices underneath a host-bridge or interleaved +across host-bridges. + +Driver Infrastructure += + +This section covers the driver infrastructure for a CXL memory device. + +CXL Memory Device +- + +.. kernel-doc:: drivers/cxl/mem.c + :doc: cxl mem + +.. kernel-doc:: drivers/cxl/mem.c + :internal: diff --git a/Documentation/driver-api/index.rst b/Documentation/driver-api/index.rst index 2456d0a97ed8..d246a18fd78f 100644 --- a/Documentation/driver-api/index.rst +++ b/Documentation/driver-api/index.rst @@ -35,6 +35,7 @@ available subsections can be seen below. usb/index firewire pci/index + cxl/index spi i2c ipmb diff --git a/drivers/Kconfig b/drivers/Kconfig index dcecc9f6e33f..62c753a73651 100644 --- a/drivers/Kconfig +++ b/drivers/Kconfig @@ -6,6 +6,7 @@ menu "Device Drivers" source "drivers/amba/Kconfig" source "drivers/eisa/Kconfig" source "drivers/pci/Kconfig" +source "drivers/cxl/Kconfig" source "drivers/pcmcia/Kconfig" source "drivers/rapidio/Kconfig" diff --git a/drivers/Makefile b/drivers/Makefile index fd11b9ac4cc3..678ea810410f 100644 --- a/drivers/Makefile +++ b/drivers/Makefile @@ -73,6 +73,7 @@ obj-$(CONFIG_NVM) += lightnvm/ obj-y += base/ block/ misc/ mfd/ nfc/ obj-$(CONFIG_LIBNVDIMM)+= nvdimm/ obj-$(CONFIG_DAX) += dax/ +obj-$(CONFIG_CXL_BUS) += cxl/ obj-$(CONFIG_DMA_SHARED_BUFFER) += dma-buf/ obj-$(CONFIG_NUBUS)+= nubus/ obj-y += macintosh/ diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig new file mode 100644 index ..3b66b46af8a0 --- /dev/null +++ b/drivers/cxl/Kconfig @@ -0,0 +1,35 @@ +# SPDX-License-Identifier: GPL-2.0-only +menuconfig CXL_BUS +
