On Saturday, August 29th, 2026 at 1:00 PM, [email protected] <[email protected]> wrote:
> On Sat, Aug 29, 2026 at 07:14:10AM +0000, [email protected] wrote: > > Hi all, > > > > This issue was discovered using a custom multi-threaded C fuzzer > > designed to stress-test MEI Virtual Tag (vtag) client lifecycles and > > multiplexing over /dev/mei0. By concurrently racing rapid vtag > > connections against file descriptor closures and streaming I/O, a > > race condition is triggered during client teardown. > > But that's not a normal use case, right? Yes, you are right. It is definitely not a normal, everyday workload. it is an extreme edge case that I caught unintentionally while stress-testing vtag multiplexing during a broader CSME research project. But because it results in an unrecoverable full system kernel and DRM/i915 display freeze when hit, I thought it was important to document and report it. > > In mei_release(), closing the last file descriptor holding a virtual tag > > invokes mei_cl_disconnect(). Inside __mei_cl_disconnect(), dev->device_lock > > is dropped while awaiting the firmware disconnect ACK on cl->wait. > > > > During this lock-drop window, a concurrent IOCTL_MEI_CONNECT_CLIENT_VTAG > > call on the same UUID (e.g. MKHI) scans dev->file_list, matches the tearing- > > down client 'pos' (in MEI_FILE_DISCONNECTING), repoints file->private_data > > to pos, and adds its new vtag to pos->vtag_map. > > > > When the disconnect ACK arrives, __mei_cl_disconnect() calls > > mei_cl_set_disconnected(cl), setting cl->me_cl = NULL and > > cl->state = MEI_FILE_DISCONNECTED. Because pos->vtag_map now contains the > > second thread's tag, mei_release() skips unlinking/freeing cl. The second > > thread then wakes up and attempts to reconnect via > > mei_ioctl_connect_client(). > > > > Additionally, shared clients lack tag based demuxing on cl->rd_pending in > > drivers/misc/mei/interrupt.c: > > cb = list_first_entry_or_null(&cl->rd_pending, struct mei_cl_cb, list); > > > > Incoming packets are matched to the head of the FIFO queue regardless of > > vtag, causing -EPROTO errors, dropped packets, and out-of-order HBM command > > sequences. This triggers continuous CSME hardware link resets: > > mei mei0: FW not ready: resetting: dev_state = 3 > > mei mei0: unexpected reset: dev_state = ENABLED fw status = ... > > > > During each reset, child client drivers (mei_hdcp, mei_pxp) unbind and > > rebind > > with i915 DRM. With resets looping at hundreds of cycles per second (>9,800 > > events in 27s), mei_cldev_enable() repeatedly fails with -EFAULT / -ENODEV, > > deadlocking i915 display worker mutexes in TASK_UNINTERRUPTIBLE and causing > > an unrecoverable full system freeze. > > > > Proposed Fix: > > - Prevent vtag reuse during teardown: in mei_ioctl_connect_vtag(), > > ignore existing clients on dev->file_list if they are in > > MEI_FILE_DISCONNECTING or MEI_FILE_DISCONNECTED states or undergoing > > teardown. > > - Implement proper reference counting / lifecycle synchronization on > > shared struct mei_cl instances. > > - In drivers/misc/mei/interrupt.c, demux cl->rd_pending by matching the > > incoming packet's vtag header to the corresponding callback rather > > than assuming FIFO order. > > - Add rate-limiting / backoff to MEI client reprobing during hardware > > link resets to prevent cascading bus storms into DRM / i915. > > Please send patches for this if you wish to see these issues addressed. > As you have a reproducer, it should be simple for you to do so. I appreciate the nudge. While the reproducer is straightforward, safely resolving the cross-subsystem interaction between MEI and i915 without introducing regressions seemed complex enough that I initially deferred to the Intel maintainers. However, since I've already mapped out the race in mei_ioctl_connect_vtag(), I am happy to take a shot at it. I will work on implementing the localized state checks to prevent vtag reuse during teardown, and I'll submit a patch for review in the next few days. > And how does this differ from the patch you have already sent but has > not yet been merged? The previous patch addressed a single-client teardown ordering issue where incoming IRQ completions populated cl->rd_completed after mei_cl_flush_queues() was called, triggering a WARN_ON() in mei_cl_unlink(). This issue is a separate, multi-client race specific to Virtual Tag (vtag) multiplexing in mei_ioctl_connect_vtag(). While mei_release() drops dev->device_lock to wait for firmware disconnect, a concurrent thread requesting a new vtag matches the tearing-down client on dev->file_list. When the disconnect finishes, cl->me_cl is cleared/NULL'd while the client is actively reused, resulting in CSME hardware reset loops and cascading DRM/i915 display freezes. Thanks, Nirbhay Kumar
