Re: [PATCH v9 07/19] cxl/region: Add sparse DAX region support
On Sun, Apr 13, 2025 at 05:52:15PM -0500, Ira Weiny wrote:
> Dynamic Capacity CXL regions must allow memory to be added or removed
> dynamically. In addition to the quantity of memory available the
> location of the memory within a DC partition is dynamic based on the
> extents offered by a device. CXL DAX regions must accommodate the
> sparseness of this memory in the management of DAX regions and devices.
>
> Introduce the concept of a sparse DAX region. Introduce
> create_dynamic_ram_a_region() sysfs entry to create such regions.
> Special case dynamic capable regions to create a 0 sized seed DAX device
> to maintain compatibility which requires a default DAX device to hold a
> region reference.
>
> Indicate 0 byte available capacity until such time that capacity is
> added.
>
> Sparse regions complicate the range mapping of dax devices. There is no
> known use case for range mapping on sparse regions. Avoid the
> complication by preventing range mapping of dax devices on sparse
> regions.
>
> Interleaving is deferred for now. Add checks.
>
> Based on an original patch by Navneet Singh.
>
> Signed-off-by: Ira Weiny
LGTM, although I am not very familiar with dax.
Reviewed-by: Fan Ni
>
> ---
> Changes:
> [iweiny: adjust to new partition mode and new singular dynamic ram
> partition]
> ---
> Documentation/ABI/testing/sysfs-bus-cxl | 22 +--
> drivers/cxl/core/core.h | 11 ++
> drivers/cxl/core/port.c | 1 +
> drivers/cxl/core/region.c | 38
> +++--
> drivers/dax/bus.c | 10 +
> drivers/dax/bus.h | 1 +
> drivers/dax/cxl.c | 16 --
> 7 files changed, 84 insertions(+), 15 deletions(-)
>
> diff --git a/Documentation/ABI/testing/sysfs-bus-cxl
> b/Documentation/ABI/testing/sysfs-bus-cxl
> index b2754e6047ca..2e26d95ac66f 100644
> --- a/Documentation/ABI/testing/sysfs-bus-cxl
> +++ b/Documentation/ABI/testing/sysfs-bus-cxl
> @@ -434,20 +434,20 @@ Description:
> interleave_granularity).
>
>
> -What:/sys/bus/cxl/devices/decoderX.Y/create_{pmem,ram}_region
> -Date:May, 2022, January, 2023
> -KernelVersion: v6.0 (pmem), v6.3 (ram)
> +What:
> /sys/bus/cxl/devices/decoderX.Y/create_{pmem,ram,dynamic_ram_a}_region
> +Date:May, 2022, January, 2023, May 2025
> +KernelVersion: v6.0 (pmem), v6.3 (ram), v6.16 (dynamic_ram_a)
> Contact: linux-...@vger.kernel.org
> Description:
> (RW) Write a string in the form 'regionZ' to start the process
> - of defining a new persistent, or volatile memory region
> - (interleave-set) within the decode range bounded by root decoder
> - 'decoderX.Y'. The value written must match the current value
> - returned from reading this attribute. An atomic compare exchange
> - operation is done on write to assign the requested id to a
> - region and allocate the region-id for the next creation attempt.
> - EBUSY is returned if the region name written does not match the
> - current cached value.
> + of defining a new persistent, volatile, or dynamic RAM memory
> + region (interleave-set) within the decode range bounded by root
> + decoder 'decoderX.Y'. The value written must match the current
> + value returned from reading this attribute. An atomic compare
> + exchange operation is done on write to assign the requested id
> + to a region and allocate the region-id for the next creation
> + attempt. EBUSY is returned if the region name written does not
> + match the current cached value.
>
>
> What:/sys/bus/cxl/devices/decoderX.Y/delete_region
> diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h
> index 15699299dc11..08facbc2d270 100644
> --- a/drivers/cxl/core/core.h
> +++ b/drivers/cxl/core/core.h
> @@ -5,6 +5,7 @@
> #define __CXL_CORE_H__
>
> #include
> +#include
>
> extern const struct device_type cxl_nvdimm_bridge_type;
> extern const struct device_type cxl_nvdimm_type;
> @@ -12,9 +13,19 @@ extern const struct device_type cxl_pmu_type;
>
> extern struct attribute_group cxl_base_attribute_group;
>
> +static inline struct cxl_memdev_state *
> +cxled_to_mds(struct cxl_endpoint_decoder *cxled)
> +{
> + struct cxl_memdev *cxlmd = cxled_to_memdev(cxled);
> + struct cxl_dev_state *cxlds = cxlmd->cxlds;
> +
> + return container_of(cxlds, struct cxl_memdev_state, cxlds);
> +}
> +
> #ifdef CONFIG_CXL_REGION
> extern struct device_attribute dev_attr_create_pmem_region;
> extern struct device_attribute dev_attr_create_ram_region;
> +extern struct device_attribute dev_attr_create_dynamic_ram_a_region;
> extern struct device_attribu
Re: [PATCH v9 07/19] cxl/region: Add sparse DAX region support
On Sun, Apr 13, 2025 at 05:52:15PM -0500, Ira Weiny wrote:
> Dynamic Capacity CXL regions must allow memory to be added or removed
> dynamically. In addition to the quantity of memory available the
> location of the memory within a DC partition is dynamic based on the
> extents offered by a device. CXL DAX regions must accommodate the
> sparseness of this memory in the management of DAX regions and devices.
>
> Introduce the concept of a sparse DAX region. Introduce
> create_dynamic_ram_a_region() sysfs entry to create such regions.
> Special case dynamic capable regions to create a 0 sized seed DAX device
> to maintain compatibility which requires a default DAX device to hold a
> region reference.
>
> Indicate 0 byte available capacity until such time that capacity is
> added.
>
> Sparse regions complicate the range mapping of dax devices. There is no
> known use case for range mapping on sparse regions. Avoid the
> complication by preventing range mapping of dax devices on sparse
> regions.
>
> Interleaving is deferred for now. Add checks.
>
> Based on an original patch by Navneet Singh.
>
> Signed-off-by: Ira Weiny
>
> ---
> Changes:
> [iweiny: adjust to new partition mode and new singular dynamic ram
> partition]
> ---
> Documentation/ABI/testing/sysfs-bus-cxl | 22 +--
> drivers/cxl/core/core.h | 11 ++
> drivers/cxl/core/port.c | 1 +
> drivers/cxl/core/region.c | 38
> +++--
> drivers/dax/bus.c | 10 +
> drivers/dax/bus.h | 1 +
> drivers/dax/cxl.c | 16 --
> 7 files changed, 84 insertions(+), 15 deletions(-)
>
> diff --git a/Documentation/ABI/testing/sysfs-bus-cxl
> b/Documentation/ABI/testing/sysfs-bus-cxl
> index b2754e6047ca..2e26d95ac66f 100644
> --- a/Documentation/ABI/testing/sysfs-bus-cxl
> +++ b/Documentation/ABI/testing/sysfs-bus-cxl
> @@ -434,20 +434,20 @@ Description:
> interleave_granularity).
>
>
> -What:/sys/bus/cxl/devices/decoderX.Y/create_{pmem,ram}_region
> -Date:May, 2022, January, 2023
> -KernelVersion: v6.0 (pmem), v6.3 (ram)
> +What:
> /sys/bus/cxl/devices/decoderX.Y/create_{pmem,ram,dynamic_ram_a}_region
> +Date:May, 2022, January, 2023, May 2025
> +KernelVersion: v6.0 (pmem), v6.3 (ram), v6.16 (dynamic_ram_a)
> Contact: linux-...@vger.kernel.org
> Description:
> (RW) Write a string in the form 'regionZ' to start the process
> - of defining a new persistent, or volatile memory region
> - (interleave-set) within the decode range bounded by root decoder
> - 'decoderX.Y'. The value written must match the current value
> - returned from reading this attribute. An atomic compare exchange
> - operation is done on write to assign the requested id to a
> - region and allocate the region-id for the next creation attempt.
> - EBUSY is returned if the region name written does not match the
> - current cached value.
> + of defining a new persistent, volatile, or dynamic RAM memory
> + region (interleave-set) within the decode range bounded by root
> + decoder 'decoderX.Y'. The value written must match the current
> + value returned from reading this attribute. An atomic compare
> + exchange operation is done on write to assign the requested id
> + to a region and allocate the region-id for the next creation
> + attempt. EBUSY is returned if the region name written does not
> + match the current cached value.
>
>
> What:/sys/bus/cxl/devices/decoderX.Y/delete_region
> diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h
> index 15699299dc11..08facbc2d270 100644
> --- a/drivers/cxl/core/core.h
> +++ b/drivers/cxl/core/core.h
> @@ -5,6 +5,7 @@
> #define __CXL_CORE_H__
>
> #include
> +#include
>
> extern const struct device_type cxl_nvdimm_bridge_type;
> extern const struct device_type cxl_nvdimm_type;
> @@ -12,9 +13,19 @@ extern const struct device_type cxl_pmu_type;
>
> extern struct attribute_group cxl_base_attribute_group;
>
> +static inline struct cxl_memdev_state *
> +cxled_to_mds(struct cxl_endpoint_decoder *cxled)
> +{
> + struct cxl_memdev *cxlmd = cxled_to_memdev(cxled);
> + struct cxl_dev_state *cxlds = cxlmd->cxlds;
> +
> + return container_of(cxlds, struct cxl_memdev_state, cxlds);
> +}
> +
> #ifdef CONFIG_CXL_REGION
> extern struct device_attribute dev_attr_create_pmem_region;
> extern struct device_attribute dev_attr_create_ram_region;
> +extern struct device_attribute dev_attr_create_dynamic_ram_a_region;
> extern struct device_attribute dev_attr_delete_region;
> extern struct device_attribute dev_attr_r
Re: [PATCH v9 07/19] cxl/region: Add sparse DAX region support
On Sun, 13 Apr 2025 17:52:15 -0500 Ira Weiny wrote: > Dynamic Capacity CXL regions must allow memory to be added or removed > dynamically. In addition to the quantity of memory available the > location of the memory within a DC partition is dynamic based on the > extents offered by a device. CXL DAX regions must accommodate the > sparseness of this memory in the management of DAX regions and devices. > > Introduce the concept of a sparse DAX region. Introduce > create_dynamic_ram_a_region() sysfs entry to create such regions. > Special case dynamic capable regions to create a 0 sized seed DAX device > to maintain compatibility which requires a default DAX device to hold a > region reference. > > Indicate 0 byte available capacity until such time that capacity is > added. > > Sparse regions complicate the range mapping of dax devices. There is no > known use case for range mapping on sparse regions. Avoid the > complication by preventing range mapping of dax devices on sparse > regions. > > Interleaving is deferred for now. Add checks. > > Based on an original patch by Navneet Singh. > > Signed-off-by: Ira Weiny > I'm not that familiar with the DAX parts but looks fine to me. Reviewed-by: Jonathan Cameron
[PATCH v9 07/19] cxl/region: Add sparse DAX region support
Dynamic Capacity CXL regions must allow memory to be added or removed
dynamically. In addition to the quantity of memory available the
location of the memory within a DC partition is dynamic based on the
extents offered by a device. CXL DAX regions must accommodate the
sparseness of this memory in the management of DAX regions and devices.
Introduce the concept of a sparse DAX region. Introduce
create_dynamic_ram_a_region() sysfs entry to create such regions.
Special case dynamic capable regions to create a 0 sized seed DAX device
to maintain compatibility which requires a default DAX device to hold a
region reference.
Indicate 0 byte available capacity until such time that capacity is
added.
Sparse regions complicate the range mapping of dax devices. There is no
known use case for range mapping on sparse regions. Avoid the
complication by preventing range mapping of dax devices on sparse
regions.
Interleaving is deferred for now. Add checks.
Based on an original patch by Navneet Singh.
Signed-off-by: Ira Weiny
---
Changes:
[iweiny: adjust to new partition mode and new singular dynamic ram
partition]
---
Documentation/ABI/testing/sysfs-bus-cxl | 22 +--
drivers/cxl/core/core.h | 11 ++
drivers/cxl/core/port.c | 1 +
drivers/cxl/core/region.c | 38 +++--
drivers/dax/bus.c | 10 +
drivers/dax/bus.h | 1 +
drivers/dax/cxl.c | 16 --
7 files changed, 84 insertions(+), 15 deletions(-)
diff --git a/Documentation/ABI/testing/sysfs-bus-cxl
b/Documentation/ABI/testing/sysfs-bus-cxl
index b2754e6047ca..2e26d95ac66f 100644
--- a/Documentation/ABI/testing/sysfs-bus-cxl
+++ b/Documentation/ABI/testing/sysfs-bus-cxl
@@ -434,20 +434,20 @@ Description:
interleave_granularity).
-What: /sys/bus/cxl/devices/decoderX.Y/create_{pmem,ram}_region
-Date: May, 2022, January, 2023
-KernelVersion: v6.0 (pmem), v6.3 (ram)
+What:
/sys/bus/cxl/devices/decoderX.Y/create_{pmem,ram,dynamic_ram_a}_region
+Date: May, 2022, January, 2023, May 2025
+KernelVersion: v6.0 (pmem), v6.3 (ram), v6.16 (dynamic_ram_a)
Contact: linux-...@vger.kernel.org
Description:
(RW) Write a string in the form 'regionZ' to start the process
- of defining a new persistent, or volatile memory region
- (interleave-set) within the decode range bounded by root decoder
- 'decoderX.Y'. The value written must match the current value
- returned from reading this attribute. An atomic compare exchange
- operation is done on write to assign the requested id to a
- region and allocate the region-id for the next creation attempt.
- EBUSY is returned if the region name written does not match the
- current cached value.
+ of defining a new persistent, volatile, or dynamic RAM memory
+ region (interleave-set) within the decode range bounded by root
+ decoder 'decoderX.Y'. The value written must match the current
+ value returned from reading this attribute. An atomic compare
+ exchange operation is done on write to assign the requested id
+ to a region and allocate the region-id for the next creation
+ attempt. EBUSY is returned if the region name written does not
+ match the current cached value.
What: /sys/bus/cxl/devices/decoderX.Y/delete_region
diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h
index 15699299dc11..08facbc2d270 100644
--- a/drivers/cxl/core/core.h
+++ b/drivers/cxl/core/core.h
@@ -5,6 +5,7 @@
#define __CXL_CORE_H__
#include
+#include
extern const struct device_type cxl_nvdimm_bridge_type;
extern const struct device_type cxl_nvdimm_type;
@@ -12,9 +13,19 @@ extern const struct device_type cxl_pmu_type;
extern struct attribute_group cxl_base_attribute_group;
+static inline struct cxl_memdev_state *
+cxled_to_mds(struct cxl_endpoint_decoder *cxled)
+{
+ struct cxl_memdev *cxlmd = cxled_to_memdev(cxled);
+ struct cxl_dev_state *cxlds = cxlmd->cxlds;
+
+ return container_of(cxlds, struct cxl_memdev_state, cxlds);
+}
+
#ifdef CONFIG_CXL_REGION
extern struct device_attribute dev_attr_create_pmem_region;
extern struct device_attribute dev_attr_create_ram_region;
+extern struct device_attribute dev_attr_create_dynamic_ram_a_region;
extern struct device_attribute dev_attr_delete_region;
extern struct device_attribute dev_attr_region;
extern const struct device_type cxl_pmem_region_type;
diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c
index e98605bd39b4..b2bd24437484 100644
--- a/drivers/cxl/core/port.c
+++ b/drivers/cxl/core/port.c
@@ -334,6 +334,7 @@ static struct attribute *cxl_deco
