Private bug reported:
Compute Express Link (CXL) leverages the PCIe physical layer to enable
high-speed, low-latency communication between hosts and devices such as
memory expanders and accelerators. With the introduction of PCIe Gen6
(64 GT/s), CXL fabrics can scale further in bandwidth and device
density, making switch-based topologies essential for large-scale
deployments.
CXL switch topologies enable multiple hosts (multi-headed systems) to
connect to shared pools of memory and accelerators through one or more
levels of switches. These topologies include single-level (fan-out),
multi-level (cascaded), and fabric-based configurations supporting
memory pooling and sharing. PCIe Gen6 enhances these deployments with
higher throughput, but also introduces complexities such as PAM4
signaling, FLIT mode, and FEC-based reliability mechanisms.
In the Linux kernel, CXL support includes device enumeration, memory
region management, and fabric configuration through subsystems like
cxl_core, cxl_port, and cxl_mem. However, support for advanced
Gen6-based CXL switch topologies—especially multi-level switching,
dynamic resource allocation, and fabric-wide RAS—is still evolving.
Enhancing OS support is critical for enabling scalable, disaggregated
memory architectures.
Feature request:
Requested details to be enabled on OS:
Enable enumeration and management of CXL devices behind PCIe Gen6 switches.
Support multi-level (cascaded) CXL switch topologies.
Enhance CXL fabric management for memory pooling and sharing across hosts.
Integrate switch-aware resource discovery (ports, downstream devices, HDM
decoders).
Support dynamic capacity allocation and rebalancing across the CXL fabric.
Enhance RAS support across switches (error propagation, containment,
isolation).
Ensure compatibility with PCIe Gen6 features (FLIT mode, FEC/CRC reporting).
Provide sysfs/debugfs visibility into CXL switch hierarchy and topology.
Enable hotplug support for devices behind CXL switches.
Support virtualization and multi-tenant isolation in shared CXL fabrics.
Provide tooling for topology discovery, validation, and debugging.
Document supported topologies, limitations, and configuration workflows.
Business Justification:
Enables scalable memory pooling and composable infrastructure.
Supports high-bandwidth workloads leveraging PCIe Gen6 (AI/ML, HPC).
Improves resource utilization through shared memory and accelerators.
Enhances reliability and manageability in large-scale CXL deployments.
Positions platform for next-generation data center architectures.
Aligns with emerging industry adoption of CXL fabrics and disaggregated
systems.
References:
CXL 2.0 / 3.0 Specifications (Switching, Fabric Management)
PCI-SIG PCIe Gen6 Specification
Linux Kernel CXL Subsystem Documentation
CXL Consortium Fabric Architecture Whitepapers
** Affects: linux (Ubuntu)
Importance: Undecided
Status: New
** Information type changed from Public to Private
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2146659
Title:
Request for CXL over PCIe Gen6 Switch Topology Support
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2146659/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
