Hi Thomas, Thank you for your reply and pointers. I'm writing back with a correction: after further investigation, I've determined that this is NOT an ncurses bug. I apologize for the noise.
CORRECTION The "open terminal failed: not a terminal" error is caused by a tmux client/server version mismatch after a Homebrew upgrade, not by ncurses 6.6's setupterm(). ncurses is innocent here. WHAT WENT WRONG WITH MY ORIGINAL ANALYSIS My isolation test used LD_LIBRARY_PATH to swap ncurses versions at runtime. I believed this showed ncurses 6.6 as the regression boundary. However, the brew tmux binary has a hardcoded DT_RPATH (not DT_RUNPATH) pointing to its ncurses directory. DT_RPATH takes priority over LD_LIBRARY_PATH, so all three test groups were actually running with ncurses 6.6 — the test was invalid. The real variable I failed to control was the tmux server version. Before `brew upgrade`, an older tmux (3.5a) server was still running. After the upgrade, the new 3.6b client connected to the old server, triggering the failure. ACTUAL ROOT CAUSE tmux 3.6b upgraded its bundled OpenBSD imsg IPC library. The struct imsg_hdr layout changed in an ABI-incompatible way: Old (3.5a): type(u32) + len(u16) + flags(u16) + peerid(u32) + pid(u32) New (3.6b): type(u32) + len(u32) + peerid(u32) + pid(u32) The old library marks "message carries an fd" with flags & IMSGF_HASFD (bit 0 of the flags field). The new library uses len & 0x80000000 (bit 31 of the widened len field), with the flags field removed entirely. When the new client sends MSG_IDENTIFY_STDIN (carrying the terminal fd), the old server misparses the header: it reads len=0x0010, flags=0x8000, and since 0x8000 & IMSGF_HASFD(0x0001) == 0, it never dequeues the fd. The client's terminal fd is lost, isatty() fails on a stale fd, and tmux reports "not a terminal". I confirmed this with strace (ioctl(TCGETS) returns ENOTTY on the received fd) and ncurses trace (setupterm() is never called — the failure occurs before ncurses is involved). CLARIFICATIONS ON YOUR REPLY A few small points, respectfully: 1. "Homebrew isn't Ubuntu - it's MacOS" Homebrew has officially supported Linux since 2019 (the former Linuxbrew project merged into mainline). It is widely used on Linux servers and in CI environments. My setup — Homebrew on Ubuntu — is a supported configuration, not an exotic one. 2. "This isn't the Ubuntu bug-reporting system" Understood. I was not reporting an Ubuntu/Debian packaging issue. I was reporting what I believed to be a regression in ncurses 6.6 itself, tested with a from-source build using the same configure flags. As it turns out, the bug was not in ncurses at all, so this is moot. 3. The 2026-02 report you referenced (bug-ncurses/2026-02/msg00025) That earlier report may or may not be the same issue. If the reporter was also hitting a tmux version mismatch after an upgrade, it would explain why no reproducer was provided — the problem disappears once the old server is killed. It might be worth asking them whether they had recently upgraded tmux. SUMMARY - ncurses 6.6 setupterm() works correctly. I verified this with both a minimal C reproducer (SCM_RIGHTS fd passing + setupterm()) and ncurses trace output. - The bug is in tmux's cross-version compatibility (or lack thereof) after an imsg library ABI change. - I've updated the Homebrew issue (homebrew-core#284568) with the correction. ONE QUESTION Since the root cause is in tmux (the imsg library ABI change breaks cross-version client/server compatibility without bumping PROTOCOL_VERSION), would you recommend I report this to the tmux project? If so, could you suggest the appropriate channel — is it the GitHub issue tracker at https://github.com/tmux/tmux/issues, or the tmux-users mailing list? Again, I apologize for the incorrect report and for taking your time. Thank you for maintaining ncurses. Best regards, Liam > 2026年5月25日 22:42，Thomas Dickey <[email protected]> 写道： > > On Mon, May 25, 2026 at 10:15:16AM +0800, Liam Huang wrote: >> Hi, >> >> I've found a regression in ncurses 6.6 (patchdate 20251230) where >> setupterm() fails on valid Linux pseudo-terminals, causing tmux's >> tty_init() to fail and "open terminal failed: not a terminal" on attach. > > This sounds like a bug which was reported in tmux without providing a > way to reproduce it - > > https://lists.gnu.org/archive/html/bug-ncurses/2026-02/msg00025.html > >> The issue does NOT occur with ncurses 6.5 (patchdate 20240427) or >> the Ubuntu system ncurses 6.4. > > This isn't the Ubuntu bug-reporting system. Since Ubuntu only > mirrors ncurses updates from Debian, the place to report Ubuntu issues > is in Debian. > >> >> ENVIRONMENT >> >> OS: Ubuntu 24.04.4 LTS >> Kernel: 6.8.0-107-generic x86_64 >> tmux: 3.6b (Homebrew) > > Homebrew isn't Ubuntu - it's MacOS. > >> ncurses: 6.6.20251230 (Homebrew, wide-char build) >> 6.5.20240427 (built from source, same configure flags) >> 6.4+20240113 (Ubuntu system /lib/x86_64-linux-gnu) > > That's 3 versions of ncurses, one self-built, and (Homebrew isn't > far from that, either). If you're going to do all of that, you > should be able to configure ncurses for traces and provide that information. > >> REPRODUCTION >> >> # Using tmux 3.6b linked against ncurses 6.6: >> $ tmux new -d -s test >> $ tmux attach -t test >> open terminal failed: not a terminal >> >> This happens in: >> - Interactive SSH sessions (ssh user@host, then tmux a) >> - VS Code Remote integrated terminal >> - PTYs created by script(1) >> >> The ONLY working case is ssh -t host "tmux a -t test" (forced PTY >> with terminal size forwarding from the SSH client). >> >> >> ISOLATION >> >> Same tmux 3.6b binary, different ncurses library at runtime: >> >> | ncurses | method | result | >> |-------------------|-------------------------------------|---------| >> | 6.6.20251230 | default brew linkage | FAIL | >> | 6.5.20240427 | LD_LIBRARY_PATH=.../ncurses-6.5/lib | OK | >> | 6.4 (system) | LD_LIBRARY_PATH=/lib/x86_64-linux-gnu | OK | >> >> Commands to reproduce the isolation test: >> >> # FAIL with 6.6 >> script -qc 'tmux -S /tmp/t66 new -d -s x && \ >> tmux -S /tmp/t66 a -t x' /dev/null >> >> # OK with 6.5 >> script -qc 'LD_LIBRARY_PATH=/path/to/ncurses-6.5/lib \ >> tmux -S /tmp/t65 new -d -s x && \ >> LD_LIBRARY_PATH=/path/to/ncurses-6.5/lib \ >> tmux -S /tmp/t65 a -t x' /dev/null >> >> >> MECHANISM >> >> tmux's server receives the client's PTY fd via SCM_RIGHTS, then >> calls tty_init() -> setupterm(). With ncurses 6.6, setupterm() >> apparently rejects the terminal, causing tty_init() to return -1. > > "apparently" can be used with lots of supporting data (actual > system calls used, along with their results). > >> This leaves the CLIENT_TERMINAL flag unset, and server_client_open() >> returns "not a terminal". >> >> strace shows the PTY is valid — TCGETS succeeds, the fd resolves >> to /dev/pts/N, and permissions are correct (crw--w---- owner:tty). > > I don't see a trace (and in any case, this report deals with a large system, > in which things can be misconfigured) > >> The tmux source code for this path is identical between 3.4 and >> 3.6b, confirming the behavioral change is in ncurses, not tmux. >> >> >> BUILD FLAGS (both 6.5 and 6.6 built with): >> >> ./configure --prefix=... --with-shared --with-widec \ >> --enable-pc-files --enable-sigwinch --enable-ext-colors \ >> --without-debug --without-ada >> >> >> I haven't bisected the exact ncurses patch between 6.5 and 6.6 yet. >> Happy to test patches or provide additional traces if helpful. > > That's about 18 months of changes :-) > > -- > Thomas E. Dickey <[email protected] > <mailto:[email protected]>> > https://invisible-island.net <https://invisible-island.net/>
