Hi Nicholas, Thank you for chiming in — I certainly didn't expect to draw the attention of tmux's author on the ncurses mailing list!
You're right, and I apologize for the noise. I should have read the 3.6 release notes more carefully before going down this rabbit hole. The note in https://github.com/tmux/tmux/issues/4699 describes exactly the symptom I was seeing. For what it's worth, the investigation did eventually converge on the same conclusion — the imsg library protocol change between 3.5a and 3.6b makes old servers unable to receive fds from new clients. I just took the scenic route getting there, and incorrectly blamed ncurses along the way. One thought: would it be feasible for the tmux client to detect a server version or protocol mismatch and print a more specific error message (e.g., "server was started by an older tmux version, please run 'tmux kill-server' and retry")? The current "open terminal failed: not a terminal" doesn't hint at the actual cause, which is probably why both I and the earlier 2026-02 reporter ended up looking in the wrong places. In any case, thank you for the clarification, and sorry again for the misdirected report. Best, Liam > 2026年5月27日 17:15，Nicholas Marriott <[email protected]> 写道： > > This was already mentioned in the tmux 3.6 release notes, see the second > paragraph here: > > https://github.com/tmux/tmux/issues/4699 > > > > > > On Wed, 27 May 2026, 08:27 Liam Huang, <[email protected] > <mailto:[email protected]>> wrote: >> Hi Thomas, >> >> Thank you for your reply and pointers. I'm writing back with a correction: >> after further investigation, I've determined that this is NOT an ncurses >> bug. I apologize for the noise. >> >> >> CORRECTION >> >> The "open terminal failed: not a terminal" error is caused by a tmux >> client/server version mismatch after a Homebrew upgrade, not by ncurses >> 6.6's setupterm(). ncurses is innocent here. >> >> >> WHAT WENT WRONG WITH MY ORIGINAL ANALYSIS >> >> My isolation test used LD_LIBRARY_PATH to swap ncurses versions at >> runtime. I believed this showed ncurses 6.6 as the regression boundary. >> However, the brew tmux binary has a hardcoded DT_RPATH (not DT_RUNPATH) >> pointing to its ncurses directory. DT_RPATH takes priority over >> LD_LIBRARY_PATH, so all three test groups were actually running with >> ncurses 6.6 — the test was invalid. >> >> The real variable I failed to control was the tmux server version. Before >> `brew upgrade`, an older tmux (3.5a) server was still running. After the >> upgrade, the new 3.6b client connected to the old server, triggering the >> failure. >> >> >> ACTUAL ROOT CAUSE >> >> tmux 3.6b upgraded its bundled OpenBSD imsg IPC library. The struct >> imsg_hdr layout changed in an ABI-incompatible way: >> >> Old (3.5a): type(u32) + len(u16) + flags(u16) + peerid(u32) + pid(u32) >> New (3.6b): type(u32) + len(u32) + peerid(u32) + pid(u32) >> >> The old library marks "message carries an fd" with flags & IMSGF_HASFD >> (bit 0 of the flags field). The new library uses len & 0x80000000 >> (bit 31 of the widened len field), with the flags field removed entirely. >> >> When the new client sends MSG_IDENTIFY_STDIN (carrying the terminal fd), >> the old server misparses the header: it reads len=0x0010, flags=0x8000, >> and since 0x8000 & IMSGF_HASFD(0x0001) == 0, it never dequeues the fd. >> The client's terminal fd is lost, isatty() fails on a stale fd, and tmux >> reports "not a terminal". >> >> I confirmed this with strace (ioctl(TCGETS) returns ENOTTY on the >> received fd) and ncurses trace (setupterm() is never called — the failure >> occurs before ncurses is involved). >> >> >> CLARIFICATIONS ON YOUR REPLY >> >> A few small points, respectfully: >> >> 1. "Homebrew isn't Ubuntu - it's MacOS" >> >> Homebrew has officially supported Linux since 2019 (the former >> Linuxbrew project merged into mainline). It is widely used on Linux >> servers and in CI environments. My setup — Homebrew on Ubuntu — is a >> supported configuration, not an exotic one. >> >> 2. "This isn't the Ubuntu bug-reporting system" >> >> Understood. I was not reporting an Ubuntu/Debian packaging issue. I was >> reporting what I believed to be a regression in ncurses 6.6 itself, >> tested with a from-source build using the same configure flags. As it >> turns out, the bug was not in ncurses at all, so this is moot. >> >> 3. The 2026-02 report you referenced (bug-ncurses/2026-02/msg00025) >> >> That earlier report may or may not be the same issue. If the reporter >> was also hitting a tmux version mismatch after an upgrade, it would >> explain why no reproducer was provided — the problem disappears once >> the old server is killed. It might be worth asking them whether they >> had recently upgraded tmux. >> >> >> SUMMARY >> >> - ncurses 6.6 setupterm() works correctly. I verified this with both a >> minimal C reproducer (SCM_RIGHTS fd passing + setupterm()) and ncurses >> trace output. >> - The bug is in tmux's cross-version compatibility (or lack thereof) >> after an imsg library ABI change. >> - I've updated the Homebrew issue (homebrew-core#284568) with the >> correction. >> >> ONE QUESTION >> >> Since the root cause is in tmux (the imsg library ABI change breaks >> cross-version client/server compatibility without bumping PROTOCOL_VERSION), >> would you recommend I report this to the tmux project? If so, could you >> suggest the appropriate channel — is it the GitHub issue tracker at >> https://github.com/tmux/tmux/issues, or the tmux-users mailing list? >> >> Again, I apologize for the incorrect report and for taking your time. >> Thank you for maintaining ncurses. >> >> Best regards, >> Liam >> >> >>> 2026年5月25日 22:42，Thomas Dickey <[email protected] >>> <mailto:[email protected]>> 写道： >>> >>> On Mon, May 25, 2026 at 10:15:16AM +0800, Liam Huang wrote: >>>> Hi, >>>> >>>> I've found a regression in ncurses 6.6 (patchdate 20251230) where >>>> setupterm() fails on valid Linux pseudo-terminals, causing tmux's >>>> tty_init() to fail and "open terminal failed: not a terminal" on attach. >>> >>> This sounds like a bug which was reported in tmux without providing a >>> way to reproduce it - >>> >>> https://lists.gnu.org/archive/html/bug-ncurses/2026-02/msg00025.html >>> >>>> The issue does NOT occur with ncurses 6.5 (patchdate 20240427) or >>>> the Ubuntu system ncurses 6.4. >>> >>> This isn't the Ubuntu bug-reporting system. Since Ubuntu only >>> mirrors ncurses updates from Debian, the place to report Ubuntu issues >>> is in Debian. >>> >>>> >>>> ENVIRONMENT >>>> >>>> OS: Ubuntu 24.04.4 LTS >>>> Kernel: 6.8.0-107-generic x86_64 >>>> tmux: 3.6b (Homebrew) >>> >>> Homebrew isn't Ubuntu - it's MacOS. >>> >>>> ncurses: 6.6.20251230 (Homebrew, wide-char build) >>>> 6.5.20240427 (built from source, same configure flags) >>>> 6.4+20240113 (Ubuntu system /lib/x86_64-linux-gnu) >>> >>> That's 3 versions of ncurses, one self-built, and (Homebrew isn't >>> far from that, either). If you're going to do all of that, you >>> should be able to configure ncurses for traces and provide that information. >>> >>>> REPRODUCTION >>>> >>>> # Using tmux 3.6b linked against ncurses 6.6: >>>> $ tmux new -d -s test >>>> $ tmux attach -t test >>>> open terminal failed: not a terminal >>>> >>>> This happens in: >>>> - Interactive SSH sessions (ssh user@host, then tmux a) >>>> - VS Code Remote integrated terminal >>>> - PTYs created by script(1) >>>> >>>> The ONLY working case is ssh -t host "tmux a -t test" (forced PTY >>>> with terminal size forwarding from the SSH client). >>>> >>>> >>>> ISOLATION >>>> >>>> Same tmux 3.6b binary, different ncurses library at runtime: >>>> >>>> | ncurses | method | result | >>>> |-------------------|-------------------------------------|---------| >>>> | 6.6.20251230 | default brew linkage | FAIL | >>>> | 6.5.20240427 | LD_LIBRARY_PATH=.../ncurses-6.5/lib | OK | >>>> | 6.4 (system) | LD_LIBRARY_PATH=/lib/x86_64-linux-gnu | OK | >>>> >>>> Commands to reproduce the isolation test: >>>> >>>> # FAIL with 6.6 >>>> script -qc 'tmux -S /tmp/t66 new -d -s x && \ >>>> tmux -S /tmp/t66 a -t x' /dev/null >>>> >>>> # OK with 6.5 >>>> script -qc 'LD_LIBRARY_PATH=/path/to/ncurses-6.5/lib \ >>>> tmux -S /tmp/t65 new -d -s x && \ >>>> LD_LIBRARY_PATH=/path/to/ncurses-6.5/lib \ >>>> tmux -S /tmp/t65 a -t x' /dev/null >>>> >>>> >>>> MECHANISM >>>> >>>> tmux's server receives the client's PTY fd via SCM_RIGHTS, then >>>> calls tty_init() -> setupterm(). With ncurses 6.6, setupterm() >>>> apparently rejects the terminal, causing tty_init() to return -1. >>> >>> "apparently" can be used with lots of supporting data (actual >>> system calls used, along with their results). >>> >>>> This leaves the CLIENT_TERMINAL flag unset, and server_client_open() >>>> returns "not a terminal". >>>> >>>> strace shows the PTY is valid — TCGETS succeeds, the fd resolves >>>> to /dev/pts/N, and permissions are correct (crw--w---- owner:tty). >>> >>> I don't see a trace (and in any case, this report deals with a large system, >>> in which things can be misconfigured) >>> >>>> The tmux source code for this path is identical between 3.4 and >>>> 3.6b, confirming the behavioral change is in ncurses, not tmux. >>>> >>>> >>>> BUILD FLAGS (both 6.5 and 6.6 built with): >>>> >>>> ./configure --prefix=... --with-shared --with-widec \ >>>> --enable-pc-files --enable-sigwinch --enable-ext-colors \ >>>> --without-debug --without-ada >>>> >>>> >>>> I haven't bisected the exact ncurses patch between 6.5 and 6.6 yet. >>>> Happy to test patches or provide additional traces if helpful. >>> >>> That's about 18 months of changes :-) >>> >>> -- >>> Thomas E. Dickey <[email protected] >>> <mailto:[email protected]>> >>> https://invisible-island.net <https://invisible-island.net/>
