On Thu,  5 Nov 2026 08:02:29 +0900
Anisa Su <[email protected]> wrote:

> A Dynamic Capacity event can span multiple event records grouped
> together by the CXL_DCD_EVENT_MORE flag. Extents are staged in the
> pending list until the last event record ('More'=0) is received, at
> which point the chain is processed. If the device opens such a chain
> (More=1) but never sends the closing record, the staged list sits
> indefinitely.
> 
> Add a delayed-work watchdog that drops the chain on timeout: an add
> chain is refused with an empty Add Dynamic Capacity Response (4802h),
> a release chain is left for the device to retry.
> 
> The spec sets no limit; 20s is a generous upper bound that may be
> tightened later, and guards against a lost mailbox response or a
> crashed fabric manager.

My gut feeling on this is don't paper over it. 
I think there should be a standard recovery flow from crashed fabric
manager and today we may have a hole here that needs closing. That
might be as simple as a zero length dcd extent sent with the more
flag not set to terminate any sequences in flight.

Lost mailbox responses should only happen if something worse
went on so I'd expect some other ras indication.

Anyhow, I'd suggest asking your spec folk to raise this (or poking John
Groves)..

> 
> The deadline is fixed when the chain opens rather than extended per
> record, so a device cannot keep a chain alive, and the staged list
> growing, by streaming More=1 records.
> 
> Signed-off-by: Anisa Su <[email protected]>
> Tested-by: Wonjae Lee <[email protected]>
> Tested-by: Junhee Park <[email protected]>
> Tested-by: Heesoo Kim <[email protected]>
> ---
>  drivers/cxl/core/mbox.c | 84 +++++++++++++++++++++++++++++++++++++++++
>  drivers/cxl/cxlmem.h    | 15 ++++++++
>  2 files changed, 99 insertions(+)
> 
> diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c
> index 3acf5ace48db..fea910e6c510 100644
> --- a/drivers/cxl/core/mbox.c
> +++ b/drivers/cxl/core/mbox.c
> @@ -873,6 +873,8 @@ static const uuid_t log_uuid[] = {
>       [VENDOR_DEBUG_UUID] = DEFINE_CXL_VENDOR_DEBUG_UUID,
>  };
>  
> +static void cxl_cancel_dcd_chain_work(void *_mds);
> +
>  /**
>   * cxl_enumerate_cmds() - Enumerate commands for a device.
>   * @mds: The driver data for the operation
> @@ -891,6 +893,16 @@ int cxl_enumerate_cmds(struct cxl_memdev_state *mds)
>       struct cxl_mem_command *cmd;
>       int i, rc;
>  
> +     /*
> +      * The DC chain watchdog sends a mailbox command when it fires.  
> Register
> +      * its cancellation here, once the mailbox is usable, so devm runs it
> +      * before the mailbox goes away: in cxl_pci the device registers were
> +      * mapped before this point and are unmapped after the cancel.
> +      */
> +     rc = devm_add_action_or_reset(dev, cxl_cancel_dcd_chain_work, mds);
> +     if (rc)
> +             return rc;
> +
>       gsl = cxl_get_gsl(mds);
>       if (IS_ERR(gsl))
>               return PTR_ERR(gsl);
> @@ -1198,6 +1210,14 @@ static void clear_pending_extents(void *_mds)
>               delete_extent_node(pos);
>  }
>  
> +/*
> + * Bound on how long the host will wait for a device to finish a
> + * multi-record DC event chain (More=1 ... More=0) before refusing the
> + * chain.  The spec sets no limit; 20s is a generous upper bound, the
> + * maximum GPF timeout.
> + */
> +#define CXL_DC_CHAIN_TIMEOUT (20 * HZ)
> +
>  static const char *cxl_dcd_evt_type_str(u8 type)
>  {
>       switch (type) {
> @@ -1226,6 +1246,11 @@ static void cxl_dc_chain_abort(struct cxl_memdev_state 
> *mds)
>       struct cxl_dc_event_chain *chain = &mds->dc_chain;
>       struct device *dev = mds->cxlds.dev;
>  
> +     lockdep_assert_held(&chain->lock);
> +
> +     chain->armed = false;
> +     cancel_delayed_work(&chain->timeout_work);
> +
>       if (chain->event_type == DCD_ADD_CAPACITY &&
>           cxl_send_dc_response(mds, CXL_MBOX_OP_ADD_DC_RESPONSE,
>                                &chain->pending_extents, 0))
> @@ -1234,6 +1259,43 @@ static void cxl_dc_chain_abort(struct cxl_memdev_state 
> *mds)
>       clear_pending_extents(mds);
>  }
>  
> +static void cxl_dc_chain_timeout(struct work_struct *work)
> +{
> +     struct cxl_dc_event_chain *chain = container_of(to_delayed_work(work),
> +                                                     struct 
> cxl_dc_event_chain,
> +                                                     timeout_work);
> +     struct cxl_memdev_state *mds = container_of(chain,
> +                                                 struct cxl_memdev_state,
> +                                                 dc_chain);
> +     struct device *dev = mds->cxlds.dev;
> +
> +     guard(mutex)(&chain->lock);
> +
> +     /*
> +      * handle_dcd_chain_event() cancels this work non-synchronously (a sync
> +      * cancel would deadlock on @chain->lock, which the chain-close path
> +      * holds), so a callback that already started running can reach here
> +      * after its chain has moved on.  Abort only if a chain is still armed
> +      * AND the timer has not been re-armed since this expiry fired: a new
> +      * chain's queue_delayed_work() makes delayed_work_pending() true,
> +      * meaning this expiry belongs to a superseded deadline and must not
> +      * abort the current chain.
> +      */
> +     if (!chain->armed || delayed_work_pending(&chain->timeout_work))
> +             return;
> +
> +     dev_warn(dev, "DC %s chain timed out\n",
> +              cxl_dcd_evt_type_str(chain->event_type));
> +     cxl_dc_chain_abort(mds);
> +}
> +
> +static void cxl_cancel_dcd_chain_work(void *_mds)
> +{
> +     struct cxl_memdev_state *mds = _mds;
> +
> +     cancel_delayed_work_sync(&mds->dc_chain.timeout_work);
> +}
> +
>  static int stage_pending_extent(struct list_head *pending,
>                               struct cxl_extent *to_add)
>  {
> @@ -1260,6 +1322,8 @@ static int handle_dcd_chain_event(struct 
> cxl_memdev_state *mds,
>       struct device *dev = mds->cxlds.dev;
>       int rc;
>  
> +     guard(mutex)(&chain->lock);
> +
>       /* Chained records share one event type (CXL r4.0 Table 8-229) */
>       if (list_empty(&chain->pending_extents)) {
>               chain->event_type = event->event_type;
> @@ -1279,9 +1343,22 @@ static int handle_dcd_chain_event(struct 
> cxl_memdev_state *mds,
>  
>       if (event->flags & CXL_DCD_EVENT_MORE) {
>               dev_dbg(dev, "more bit set; delay processing of extent\n");
> +             /* The deadline is fixed when the chain opens, not per record */
> +             if (!chain->armed) {
> +                     queue_delayed_work(system_wq, &chain->timeout_work,
> +                                        CXL_DC_CHAIN_TIMEOUT);
> +                     chain->armed = true;
> +             }
>               return 0;
>       }
>  
> +     /*
> +      * Chain is closing.  Disarm before flushing so a pending watchdog
> +      * (queued but blocked on @chain->lock) sees !armed and bails out.
> +      */
> +     chain->armed = false;
> +     cancel_delayed_work(&chain->timeout_work);
> +
>       /* Nothing is tracked yet: refuse every add, acknowledge every release 
> */
>       if (event->event_type == DCD_ADD_CAPACITY)
>               rc = cxl_send_dc_response(mds, CXL_MBOX_OP_ADD_DC_RESPONSE,
> @@ -2021,7 +2098,14 @@ struct cxl_memdev_state 
> *cxl_memdev_state_create(struct device *dev, u64 serial,
>  
>       mutex_init(&mds->event.log_lock);
>       INIT_LIST_HEAD(&mds->dc_chain.pending_extents);
> +     mutex_init(&mds->dc_chain.lock);
> +     INIT_DELAYED_WORK(&mds->dc_chain.timeout_work, cxl_dc_chain_timeout);
>  
> +     /*
> +      * The watchdog cancel is registered later, by cxl_enumerate_cmds(), so
> +      * devm's reverse-order unwind cancels (and waits for) the watchdog
> +      * before this list cleanup runs.
> +      */
>       rc = devm_add_action_or_reset(dev, clear_pending_extents, mds);
>       if (rc)
>               return ERR_PTR(rc);
> diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h
> index c88de7cb65d1..52c936ec15a9 100644
> --- a/drivers/cxl/cxlmem.h
> +++ b/drivers/cxl/cxlmem.h
> @@ -8,6 +8,8 @@
>  #include <linux/uuid.h>
>  #include <linux/node.h>
>  #include <linux/list.h>
> +#include <linux/mutex.h>
> +#include <linux/workqueue.h>
>  #include <cxl/event.h>
>  #include <cxl/mailbox.h>
>  #include "cxl.h"
> @@ -431,6 +433,11 @@ static inline struct cxl_dev_state *mbox_to_cxlds(struct 
> cxl_mailbox *cxl_mbox)
>   * struct cxl_dc_event_chain - staged Dynamic Capacity event chain
>   * @pending_extents: extents received so far; flushed when the chain
>   *                closes (More=0)
> + * @timeout_work: watchdog that fires if a chain is opened with
> + *             CXL_DCD_EVENT_MORE but the closing record never arrives
> + * @lock: serialises updates to the chain state against the watchdog
> + * @armed: set when a More=1 chain opens; cleared when the chain closes
> + *      (More=0), is aborted by cxl_dc_chain_abort(), or times out.
>   * @event_type: DCD event type of the open chain (add or release)
>   *
>   * The records of one Dynamic Capacity operation can be chained by the
> @@ -439,9 +446,17 @@ static inline struct cxl_dev_state *mbox_to_cxlds(struct 
> cxl_mailbox *cxl_mbox)
>   * chain is open at a time.  Both Add Capacity and Release Capacity chains
>   * are staged here, @event_type says which.  Records are staged until
>   * More=0, then the chain is processed and responded to as a whole.
> + *
> + * If a chain is opened (More=1) but the device never sends the closing
> + * record, the staged list would otherwise sit indefinitely.  @timeout_work
> + * is a defensive watchdog that drops such a chain: an add is refused with
> + * an empty response, a release is held for the device to retry.
>   */
>  struct cxl_dc_event_chain {
>       struct list_head pending_extents;
> +     struct delayed_work timeout_work;
> +     struct mutex lock;      /* chain state, against the watchdog */
> +     bool armed;
>       u8 event_type;
>  };
>  


Reply via email to