From b87c39149704a03872fd02adc59d56f16a9396f1 Mon Sep 17 00:00:00 2001 From: Peter Molfese <[email protected]> Date: Tue, 12 May 2026 15:05:34 -0400 Subject: [PATCH] glx/drisw: fix xshm_opcode never initialized, add XShmAttach fallback
fixes https://gitlab.freedesktop.org/mesa/mesa/-/work_items/15445 glx/drisw: fix xshm_opcode never initialized, add XShmAttach fallback xshm_opcode was declared and initialised to -1 but never assigned the actual MIT-SHM major opcode from the X server. This caused two bugs: 1. handle_xerror compared event->request_code against -1, which can never match a real opcode, so xshm_error was never set. Consequently XShmPutImage was called against a segment the X server had already rejected at XShmAttach time, producing: X Error of failed request: BadShmSeg (invalid shared segment parameter) Major opcode of failed request: 133 (MIT-SHM) Minor opcode of failed request: 3 (X_ShmPutImage) This is the failure mode seen on Mesa main with XQuartz on macOS. 2. Because xshm_opcode stayed at -1, the assert(xshm_opcode != -1) at the top of handle_xerror fired the moment any X error arrived while the handler was installed, aborting the process with: Assertion failed: (xshm_opcode != -1), function handle_xerror, file drisw_glx.c, line 62. Fix by: a) Calling XQueryExtension(\"MIT-SHM\") once per Display the first time XCreateDrawable attempts SHM, storing the returned major opcode in xshm_opcode. If the extension is absent, jump to the non-SHM path. b) Explicitly clearing xshm_error to 0 before installing the error handler, removing a latent dependency on the global being zero from a previous call. c) Adding a comment explaining why the XSync after XShmAttach is required: it forces the X server to process the attach before XShmPutImage is issued, which is the pattern required for servers (like XQuartz) that process the attach lazily. d) Replacing assert(xshm_opcode != -1) in handle_xerror with a conditional return, so a defensive path remains without aborting the process. e) Emitting a one-shot mesa_logw_once when the attach is rejected so the fallback to XPutImage is observable in the log without spam. f) Adding a no_shm label so the opcode-query failure path can jump directly to the non-SHM XCreateImage fallback. Tested on macOS with XQuartz using glxgears from mesa-demos 9.0.0. Previously crashed with the assertion; now runs correctly and emits: MESA: warning: MIT-SHM attach rejected by X server (error 10); falling back to XPutImage Error 10 is BadAccess, confirming XQuartz rejects the SHM attach due to macOS inter-process memory restrictions. The XPutImage fallback is transparent to the application. A demonstration Homebrew formula applying this patch is available at: https://github.com/pmolfese/homebrew-mesa-fix Cc: [email protected]" --- src/glx/drisw_glx.c | 53 +++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 51 insertions(+), 2 deletions(-) diff --git a/src/glx/drisw_glx.c b/src/glx/drisw_glx.c index 2a3b630fa20..712cfd9a724 100644 --- a/src/glx/drisw_glx.c +++ b/src/glx/drisw_glx.c @@ -59,7 +59,14 @@ handle_xerror(Display *dpy, XErrorEvent *event) { (void) dpy; - assert(xshm_opcode != -1); + /* + * xshm_opcode is populated by XCreateDrawable before this handler is + * installed. It should never be -1 here, but guard defensively rather + * than asserting so that a misbehaving X server cannot abort the process. + */ + if (xshm_opcode == -1) + return 0; + if (event->request_code != xshm_opcode) return 0; @@ -78,6 +85,33 @@ XCreateDrawable(struct drisw_drawable * pdp, int shmid, Display * dpy) } if (!xshm_error && shmid >= 0) { + /* + * Populate xshm_opcode before installing the error handler. + * Previously xshm_opcode was never assigned after being initialised to + * -1, so handle_xerror's request_code comparison never matched an MIT-SHM + * error and xshm_error was never set. This meant XShmPutImage was called + * with a segment that the X server had silently rejected at XShmAttach + * time, producing BadShmSeg (seen with XQuartz on macOS). + * + * XQueryExtension fills in the major opcode we need. If it fails the + * extension is not available and we fall through to plain XPutImage. + */ + if (xshm_opcode == -1) { + int opcode, ev, err; + /* + * XQueryExtension gives us the major opcode that the X server uses + * for MIT-SHM requests. We need it so handle_xerror can correctly + * identify MIT-SHM errors by request_code. Previously xshm_opcode + * was left at -1 forever, so the error handler's comparison never + * matched and xshm_error was never set, causing XShmPutImage to fire + * against a segment the server had already rejected (BadShmSeg on + * XQuartz/macOS). + */ + if (!XQueryExtension(dpy, "MIT-SHM", &opcode, &ev, &err)) + goto no_shm; + xshm_opcode = opcode; + } + pdp->shminfo.shmid = shmid; pdp->ximage = XShmCreateImage(dpy, NULL, @@ -92,13 +126,26 @@ XCreateDrawable(struct drisw_drawable * pdp, int shmid, Display * dpy) /* dispatch pending errors */ XSync(dpy, False); + xshm_error = 0; old_handler = XSetErrorHandler(handle_xerror); /* This may trigger the X protocol error we're ready to catch: */ XShmAttach(dpy, &pdp->shminfo); + /* + * Force a round-trip so the X server processes XShmAttach before we + * ever issue XShmPutImage. Without this sync, XQuartz (and other + * servers that process the attach lazily) reject the subsequent + * XShmPutImage with BadShmSeg because the segment is not yet + * registered server-side. + */ XSync(dpy, False); if (xshm_error) { - /* we are on a remote display, this error is normal, don't print it */ + /* X server rejected the SHM attach (e.g. XQuartz sandbox, + * remote display). Fall back to plain XPutImage silently; + * the caller treats ximage == NULL as the non-SHM path. */ + mesa_logw_once("MIT-SHM attach rejected by X server (error %d); " + "falling back to XPutImage", + xshm_error); XDestroyImage(pdp->ximage); pdp->ximage = NULL; } @@ -107,6 +154,8 @@ XCreateDrawable(struct drisw_drawable * pdp, int shmid, Display * dpy) } } +no_shm: + if (pdp->ximage == NULL) { pdp->shminfo.shmid = -1; pdp->ximage = XCreateImage(dpy, -- 2.50.1 (Apple Git-155)
