Hi, here I'll try to explain the struggles mentioned in some of my previous e-mails, this time from a programmer's perspective.
Let's say you want to make an ncurses app that simply supports all the keys on a standard (ANSI) full-size keyboard functional with their default meaning. But that's impossible now. Let's start with initialization. The app should call keypad(), which outputs smkx and rmkx to enable various application keypad modes, because terminfo entries expect those modes to be enabled. Otherwise, escape sequences will not match - especially the cursor keys on many terminals. But this also enables keypad application mode on many terminals, so the numeric keypad won't transmit numbers or cursor keys as usual, but instead special sequences. So the app has to handle the special sequences output by the numeric keypad itself. But there is no single way to do this. Some terminals have correct mappings, others have obscure undocumented mappings, and worst of all, these mappings overlap. For example, numeric keypad keys 1 and 3: On some terminals, they are mapped to capabilities kc1 and kc3. This is correct and documented. But on many other terminals, they are mapped to ka1 and ka3, which is incorrect. So the poor app can't know if a keypress of ka1, which it just received, means keypad key 7 or keypad key 1, because many terminals define it both ways. Why such a mess? It's the result of very, very ancient history, and the division between which terminals define it one way or another is essentially nothing but random in evolution of today's terminfo. Modern terminfo (distributed with ncurses) has its direct roots in Berkeley's BSD termcap, which didn't support DEC VT's application keypad mode for a long time. Paradoxically, this avoided keypad problems - the keypad just worked as expected. But in 1995, DEC VT entries were replaced and merged with others whose lineage traces which had very long journey to today's terminfo and which traces back to AT&T UNIX System V, developed in parallel for much of its history. In 1984, AT&T released UNIX System V Release 2, which introduced terminfo as a replacement for (even in that time!) obsolescent termcap. The R2 terminfo manual page described the capabilities precisely, including keypad chapter, which states that if the keypad can be toggled between transmit or not transmit, these codes should be set in smkx and rmkx, otherwise keypad is assumed as always transmit. And if the keypad has keys beyond the arrow keys in 3x3 array, they should be set as ka1 (upper-left key), ka3 (upper-right key), kb2 (center key), kc1 (lower-left key) and kc3 (lower-right key). As part of the system, there was already solid terminfo database, but no terminal had smkx and rmkx, or these keys defined - so on DEC VT terminals, the keypad worked just as expected (transmitting numbers or cursor keys). But just three years later, AT&T released UNIX System V Release 3, which added switching to cursor application mode and keypad application mode via smkx and rmkx, and defined sequences for numeric keypad for vt100 and others. However, there was a problem: DEC VT terminals have *two *keypads (cursor and numeric), generating too much sequences. But there was no room for all of them, simply because terminfo predefined capabilities didn't anticipate such complexity. Note that user-definable capabilities didn't exist until 13 years later. So the author of the vt100 entry came up with a dirty hack: assigning numeric keypad application mode sequences to *random *capabilities, regardless of their documented purpose. So keys 1, 2, 3 were mapped to caps belonging to 7, 5, and 9, keys 0 and dot were mapped to caps belonging to 1 and 3, key row 4, 5, 6, and comma was mapped to caps belonging to F5, F6, F7 and F8, and key row 7, 8, 9 was mapped to caps belonging to F9, F10 and F0. There is excerpt from SysV R3 terminfo: # Info: # This is how the keypad gets assigned. # PF1 PF2 PF3 PF4 kf1=\EOP, kf2=\EOQ, kf3=\EOR, kf4=\EOS, # 7 8 9 '-' does not send anything on my terminal. kf9=\EOw, kf10=\EOx, kf0=\EOy, # 4 5 6 , kf5=\EOt, kf6=\EOu, kf7=\EOv, kf8=\EOl, # 1 2 3 ka1=\EOq, kb2=\EOr, ka3=\EOs, # 0 . ENTER kc1=\EOp, kc3=\EOn, kent=\EOM, # That's it. No explaining commentary, nothing. Just this terminfo "rape". And of course smkx and rmkx were added to enable application modes. Surprisingly, this survived for years throughout SysV UNIX history. Later only VT220 added but without keypad app mode. In 1995, all this was imported into today's terminfo. Later that year, keypad app mode was added for VT320, which used the same app modes and the same escape sequences, but defined them correctly, in line with the documentation this time. But user-definable caps still didn't exist, so only ka1, ka3, kb2, kc1 and kc3 keys (7, 5, 9, 1, 3) were defined; other keypad cursor keys remained undefined because there is no standard cap for them. In 2002, Thomas Dickey created independent vt100+keypad and vt220+keypad building blocks, mapping VT100 with some other terminals to the first, and VT320 to the second. Note that VT220 itself still had no app mode enabled. Later many modern terminals inherited one or the other (vt100+keypad or vt220+keypad building block) basically randomly, so now there are many terminals based on either. For example, PuTTY, nsterm, mlterm and iTerm use vt100+keypad; xterm, rxvt use vt220+keypad. Both VT terminals actually share the same sequences and modes; the differences come only from terminfo editing history. Later in 2006, Thomas Dickey added the user-extendable caps ka2, kb1, kb3, and kc2, which were so long-missing from 80's, to vt220+keypad. So now both building blocks define (almost) the complete set of escape sequences (identical in VT100 and VT220), but both mapped inconsistently to various, overlapping caps. And terminals still randomly inherit one or the other. But back to the poor app developer: so they must use keypad app mode (otherwise the cursor keys won't work at all), even when only standard numeric keypad functionality is needed, and have to implement all the numeric keypad keys just to make the numeric keypad work. However, because of overlapping caps, the developer can't decide which key was pressed, because it might mean one key (if the terminal historically randomly happens to use vt100+keypad building block) or another (if the terminal historically randomly happens to use vt220+keypad building block). The app developer is hopeless. *** So what can be proposed as a solution? 1) Vaporize vt100+keypad Escape sequences mapped to invalid caps are really useless. No developer benefits from them. They only forces app developers to create their own built-in definitions, because they can't rely on a reliable terminfo database where a cap that should mean something actually does. And there's no compatibility issue either, since this broken behavior is something no app could rely on - neither in the past or now. Sure, there's a disadvantage: termcap users can't see user-definable caps (ka2, kb1, kb3, kc2). But that's a problem rooted in the lack of room for two distinct keypads in the original capability design. Solving it by abusing unrelated caps doesn't help anyone. 2) Don't enable keypad app mode as default Most ncurses applications don't want to handle the numeric keypad in a special way. They just want it to behave normally - i.e., numbers with Num Lock on, cursor keys with Num Lock off. But this is impossible now: every hap has to output smkx and rmkx, otherwise it can't rely on terminfo-defined escape sequences. In the original SysV R2 terminfo documentation (which has been preserved to the current version), though, the original purpose of smkx and rmkx was somewhat different: to enable transmit of the keypad at all. In ancient times, the keypad could also behave locally - i.e., it didn't transmit any sequences at all, but only moved the cursor on the terminal itself. The documentation says "to transmit or not transmit". It doesn't say "to transmit special sequences or to transmit normal sequences". So I think that mandatory activation of app mode through smkx and rmkx maybe has been an incorrect use of these capabilities from the start. So it might be a good idea to consider making the keypad app mode activation optional - for example, by moving it to *-appkeypad terminfo entries, or by introducing new capabilities which will specifically turn on keypad app mode - leaving it in normal mode for most use cases. Jakub
