Re: [dev] [st] Terminal abnormal key codes

[Greg Reagle]

On 09/02/2015 12:21 PM, Greg Reagle wrote:
> I've been reading this conversation with interest.  I tried different shells 
> with xterm versus st.
> 
> On 09/02/2015 11:29 AM, Fabian Homborg wrote:
>> If you launch fish in { konsole, xterm, gnome-terminal, linux in-kernel
>> VTs, iTerm2, ... } your keys work, without smkx.
>>
>> If you launch fish in st, your keys don't, until you do "tput smkx".
> 
> I confirm that this is true on my computer:  for the shells fish, bash, ksh, 
> and mksh, in xterm, the Delete key works; whereas in st it doesn't work 
> without tput smkx.
> 
> However, with zsh in st, the Delete key works immediately.

Sorry, correction.  I mis-reported on ksh.  Here are my results:

Delete key
|  | xterm | xterm | st| st|
|  | rmkx  | smkx  | rmkx  | smkx  |
|--+---+---+---+---|
| zsh  | works | works | works | works |
|--+---+---+---+---|
| bash | works | works | no-op | works |
| mksh | works | works | no-op | works |
| tcsh | works | works | no-op | works |
|--+---+---+---+---|
| fish | works | works | '[P'  | works |
| ksh  | '~'   | '~'   | '~'   | '~'   |


The anomaly of '[P' with fish might be fixed in a later version.  I am running 
fish, version 2.1.2-1256-g64af63b.

Re: [dev] [st] Terminal abnormal key codes

[Greg Reagle]

On 09/02/2015 11:29 AM, Fabian Homborg wrote:
> Roberto E. Vargas Caballero  writes:
> That's not what I'm talking about. Of course a tone of terminals have
> smkx defined, but fish currently doesn't send it and works on (as far as
> I know) anything but st.
> 
> In other words:
> 
> If you launch fish in { konsole, xterm, gnome-terminal, linux in-kernel
> VTs, iTerm2, ... } your keys work, without smkx.
> 
> If you launch fish in st, your keys don't, until you do "tput smkx".

Some friendly advice to Fabian and fish developers (I am a big fan of fish).  I 
am a suckless user not developer (although I've contributed some minor code) 
and I've been on this list a while.  The suckless developers thrive on being 
different and superior.  So any appeal to popularity ("all the other terminal 
emulators do it this way") will fall on deaf ears, and probably even backfire.  

You've got to make your case differently/better.  I do not have expertise on 
terminfo and terminal emulators myself, so I don't have an opinion about this 
issue, although I find it interesting.

You also have to have patience.  I suggested a correction to st based on a 
standard, and although it was vehemently opposed at first, it eventually was 
accepted. See 
http://git.suckless.org/st/commit/?id=28259f5750f0dc7f52bbaf8b746ec3dc576a58ee

Re: [dev] [st] Terminal abnormal key codes

[Fabian Homborg]

Roberto E. Vargas Caballero  writes:

> Hi,
>
>> Our question is probably better stated as the following:
>> 
>> st seems to e.g. send "[P" for delete instead of the '\E[3;2~' in
>> terminfo (which fish does use), unless we explicitly do the equivalent
>> of "tput smkx". This not only applies to delete, but a few other key
>> combinations as well, like insert or ctrl-up.
>> 
>> Of course this is stated in the FAQ, but the question is _why_ is it
>> this way and is it possible to change that?
>
> Because there are two ways of using the terminal, in ASCII mode and in
> application mode. Maybe you only use the terminal in application mode,
> but some of us (mainly me) use the terminal for other kind of things,
> connecting to non POSIX machines, over serial lines, and in these cases
> we use this mode. St is not only a Unix place where ppl can write unix
> commands (take a look to -l switch of st).
>
>> From our perspective st is the only terminal (we've found at least) that
>> behaves this way - every other term (like xterm, konsole or linux VTs)
>> seems to not need this and just always sends the sequences specified in
>> terminfo.
>
> You are wrong:
>
> $ curl http://www.catb.org/esr/terminfo/termtypes.ti.gz 2>/dev/null |
>  gunzip |
>  sed 's/#.*//g' |
>  grep -c smkx
> 111

That's not what I'm talking about. Of course a tone of terminals have
smkx defined, but fish currently doesn't send it and works on (as far as
I know) anything but st.

In other words:

If you launch fish in { konsole, xterm, gnome-terminal, linux in-kernel
VTs, iTerm2, ... } your keys work, without smkx.

If you launch fish in st, your keys don't, until you do "tput smkx".

>
> There are 111 definitions in the official terminfo definition which
> use smkx.  Maybe you don't use any of them, but some of use use
> some of them.  For example I have a physical vt500 that I use as
> console in several of my servers, and it has smkx sequence. Are
> you saying me that I am going to be able to use fish in this
> terminal? and only because you are too lazy to follow the terminfo
> standard?
>

Apparently, st is the only thing that does follow it.

>> There's two things we can do on our side - do "tput smkx" and "tput
>> rmkx" either only in st or always (which I don't know if it'll break
>> stuff elsewhere) or bind both kinds of sequences st sends, neither
>> solution being _great_.
>
> It should not break anything. If the terminal does not define smkx
> you will not print anything, and if the terminal define it is because
> it needs the sequence. All the application which use curses work
> without problems in st when they try to match any key. mainly because
> curses follows the terminfo specification perfectly.
>

I'll probably try that, thank you.

> The question here is, why do you want to write a shell knowing
> it has bugs and it will not be able of running in all the possible
> (current or future) terminals?

The only term I currently know of that has a problem with fish is st.

Regards,
Fabian Homborg


signature.asc
Description: PGP signature

Re: [dev] [st] Terminal abnormal key codes

[Greg Reagle]

On 09/02/2015 01:00 PM, Fabian Homborg wrote:
>> The anomaly of '[P' with fish might be fixed in a later version.  I am 
>> running fish, version 2.1.2-1256-g64af63b.
> 
> That's why we are discussing this.

I updated fish to 2.2.0-1~trusty amd64 (from the PPA) and the '[P' behavior 
still occurs.

> zsh doesn't work for me either, 
> Are you sure your distribution doesn't already add that?

Yes you're right.  My distribution adds code to /etc/zsh/zshrc to make it work.

Re: [dev] [st] Terminal abnormal key codes

[Fabian Homborg]

Greg Reagle  writes:

> On 09/02/2015 12:21 PM, Greg Reagle wrote:
>> I've been reading this conversation with interest.  I tried different shells 
>> with xterm versus st.
>> 
>> On 09/02/2015 11:29 AM, Fabian Homborg wrote:
>>> If you launch fish in { konsole, xterm, gnome-terminal, linux in-kernel
>>> VTs, iTerm2, ... } your keys work, without smkx.
>>>
>>> If you launch fish in st, your keys don't, until you do "tput smkx".
>> 
>> I confirm that this is true on my computer:  for the shells fish, bash, ksh, 
>> and mksh, in xterm, the Delete key works; whereas in st it doesn't work 
>> without tput smkx.
>> 
>> However, with zsh in st, the Delete key works immediately.
>
> Sorry, correction.  I mis-reported on ksh.  Here are my results:
>
> Delete key
> |  | xterm | xterm | st| st|
> |  | rmkx  | smkx  | rmkx  | smkx  |
> |--+---+---+---+---|
> | zsh  | works | works | works | works |
> |--+---+---+---+---|
> | bash | works | works | no-op | works |
> | mksh | works | works | no-op | works |
> | tcsh | works | works | no-op | works |
> |--+---+---+---+---|
> | fish | works | works | '[P'  | works |
> | ksh  | '~'   | '~'   | '~'   | '~'   |
>
>
> The anomaly of '[P' with fish might be fixed in a later version.  I am 
> running fish, version 2.1.2-1256-g64af63b.

That's why we are discussing this.

zsh doesn't work for me either, and they have an entry in their FAQ that
mentions adding configuration

From the zsh 5.1 FAQ:

> It should be noted that the `O' / `[' confusion can occur
> with other keys such as Home and End.  Some systems let you query
> the key sequences sent by these keys from the system's terminal
> database, terminfo.  Unfortunately, the key sequences given there
> typically apply to the mode that is not the one zsh uses by default (it's
> the "application" mode rather than the "raw" mode).  Explaining the use
> of terminfo is outside the scope of this FAQ, but if you wish to use the
> key sequences given there you can tell the line editor to turn on
> "application" mode when it starts and turn it off when it stops:
> 
>  function zle-line-init () { echoti smkx }
>  function zle-line-finish () { echoti rmkx }
>  zle -N zle-line-init
>  zle -N zle-line-finish

Are you sure your distribution doesn't already add that?


signature.asc
Description: PGP signature

Re: [dev] [st] Terminal abnormal key codes

[Pickfire]

On Tue, Sep 01, 2015 at 04:40:12PM +0200, Roberto E. Vargas Caballero wrote:

On Tue, Sep 01, 2015 at 10:22:52PM +0800, Pickfire wrote:

Hi, it seems that st does some terminal codes abnormally[1].

What is the reason for st's key codes to be different from the other
terminals? Is st following some standards?

[1]: https://github.com/fish-shell/fish-shell/issues/2309#issuecomment-136678774


Yes, st follows terminfo(5). Every terminal has different keys,
so, I don't know what you mean with st using different keys.
You can see the differences between xterm an linux vt:

$ infocmp xterm linux

comparing xterm to linux.
   comparing booleans.
ccc: F:T.
eo: F:T.
km: T:F.
mc5i: T:F.
npc: T:F.
xon: F:T.
   comparing numbers.
cols: 80, NULL.
lines: 24, NULL.
ncv: NULL, 18.
   comparing strings.
acsc: '``aaffggiijjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~', 
'+\020\,\021-\030.^Y0\333`\004a\261f\370g\361h\260i\316j\331k\277l\332m\300n\305o~p\304q\304r\304s_t\303u\264v\301w\302x\263y\363z\362{\343|\330}\234~\376'.
cbt: '\E[Z', NULL.
civis: '\E[?25l', '\E[?25l\E[?1c'.
clear: '\E[H\E[2J', '\E[H\E[J'.
cnorm: '\E[?12l\E[?25h', '\E[?25h\E[?0c'.
cvvis: '\E[?12;25h', '\E[?25h\E[?8c'.
...
smkx: '\E[?1h\E=', NULL.
smm: '\E[?1034h', NULL.
smpch: NULL, '\E[11m'.
u8: '\E[?1;2c', '\E[?6c'.


How do you know that smkx should be '\E[?1h\E='? Is there any
documentation about it?


If your program does not use terminfo(3)
(http://linux.die.net/man/3/setupterm), then is broken.
About the problems with DELETE, please take a look to
the FAQ (section Why doesn't the Del key work in some programs?)


I had read the FAQ again and again, but I just doesn't understand why some 
applications
doesn't work in st but works in other terminal, the FAQ just says that
the applications are buggy.

Good luck and have a nice day!

--
_
< Do what you like, like what you do. >
-
   \   ^__^
\  (oo)\___
   (__)\   )\/\
   ||w |
   || ||

Re: [dev] [st] Terminal abnormal key codes

[Fabian Homborg]

Pickfire  writes:

> On Tue, Sep 01, 2015 at 04:40:12PM +0200, Roberto E. Vargas Caballero wrote:
>>On Tue, Sep 01, 2015 at 10:22:52PM +0800, Pickfire wrote:
>>> Hi, it seems that st does some terminal codes abnormally[1].
>>>
>>> What is the reason for st's key codes to be different from the other
>>> terminals? Is st following some standards?
>>>
>>> [1]: 
>>> https://github.com/fish-shell/fish-shell/issues/2309#issuecomment-136678774
>>
>>Yes, st follows terminfo(5). Every terminal has different keys,
>>so, I don't know what you mean with st using different keys.

Hi, I'm faho from the fish team. @Pickfire: Let me take this over.

Our question is probably better stated as the following:

st seems to e.g. send "[P" for delete instead of the '\E[3;2~' in
terminfo (which fish does use), unless we explicitly do the equivalent
of "tput smkx". This not only applies to delete, but a few other key
combinations as well, like insert or ctrl-up.

Of course this is stated in the FAQ, but the question is _why_ is it
this way and is it possible to change that?

From our perspective st is the only terminal (we've found at least) that
behaves this way - every other term (like xterm, konsole or linux VTs)
seems to not need this and just always sends the sequences specified in
terminfo.

There's two things we can do on our side - do "tput smkx" and "tput
rmkx" either only in st or always (which I don't know if it'll break
stuff elsewhere) or bind both kinds of sequences st sends, neither
solution being _great_.

So: Is there a rationale for that decision and would you consider
changing it?

Thanks for your time,
Fabian Homborg
-- 



signature.asc
Description: PGP signature

Re: [dev] [st] Terminal abnormal key codes

[Roberto E. Vargas Caballero]

> How do you know that smkx should be '\E[?1h\E='? Is there any
> documentation about it?

http://www.vt100.net/docs/vt100-ug/chapter3.html#SM
http://www.vt100.net/docs/vt100-ug/chapter3.html#DECKPAM

There are two issues related to keyboard configuration in the terminal
world:

- configuration of cursor keys
- configuration of keypad

These keys have two ways of working, in ascii mode or in application mode.
In ascii mode they generate the ascii value expected by the terminal to
perform the desired functiona (for example in the case of DELETE it sends
ESC [ P which is the sequence to delete current character), and in application
mode they generate some sequence which allows one program to identify the key.
If a program needs to know if a key was pressed it has to activate the 
application
mode before of testing anything, or the terminal can generate the ASCII sequence
if the terminal was in ASCII mode.

> >If your program does not use terminfo(3)
> >(http://linux.die.net/man/3/setupterm), then is broken.
> >About the problems with DELETE, please take a look to
> >the FAQ (section Why doesn't the Del key work in some programs?)
> 
> I had read the FAQ again and again, but I just doesn't understand why some 
> applications
> doesn't work in st but works in other terminal, the FAQ just says that
> the applications are buggy.

Yes, because it means your program is not setting the correct mode,
and assume that DELETE is going to generate the same sequence in both
modes. Again, taken from terminfo(5):


If the terminal has a keypad that transmits codes when the keys
are pressed, this information can be given. Note that it is not
possible to handle terminals where the keypad only works in
local (this applies, for example, to the unshifted HP 2621 keys).
If the keypad can be set to transmit or not transmit, give these
codes as smkx and rmkx. Otherwise the keypad is assumed to
always transmit.

Maybe is difficult to understand it, but I will say again: IF THE KEYPAD
CAN BE SET TO TRANSMIT OR NOT TRANSMIT, GIVE THESE CODES AS SMKX AND RMKX.
OTHERWISE THE KEYPAD IS ASSUMED TO ALWAYS TRANSMIT.

St gives values for smkx and rmkx, so if an application wants to read
the keypad it must print these sequences before. You can take the values
of these sequences usging the terminfo database, 'tput smkx' in the shell,
or the variable smkx after calling setupterm() if you link your C program
with terminfo.

Regards,

Re: [dev] [st] Terminal abnormal key codes

[Roberto E. Vargas Caballero]

Hi,

> Our question is probably better stated as the following:
> 
> st seems to e.g. send "[P" for delete instead of the '\E[3;2~' in
> terminfo (which fish does use), unless we explicitly do the equivalent
> of "tput smkx". This not only applies to delete, but a few other key
> combinations as well, like insert or ctrl-up.
> 
> Of course this is stated in the FAQ, but the question is _why_ is it
> this way and is it possible to change that?

Because there are two ways of using the terminal, in ASCII mode and in
application mode. Maybe you only use the terminal in application mode,
but some of us (mainly me) use the terminal for other kind of things,
connecting to non POSIX machines, over serial lines, and in these cases
we use this mode. St is not only a Unix place where ppl can write unix
commands (take a look to -l switch of st).

> From our perspective st is the only terminal (we've found at least) that
> behaves this way - every other term (like xterm, konsole or linux VTs)
> seems to not need this and just always sends the sequences specified in
> terminfo.

You are wrong:

$ curl http://www.catb.org/esr/terminfo/termtypes.ti.gz 2>/dev/null |
 gunzip |
 sed 's/#.*//g' |
 grep -c smkx
111

There are 111 definitions in the official terminfo definition which
use smkx.  Maybe you don't use any of them, but some of use use
some of them.  For example I have a physical vt500 that I use as
console in several of my servers, and it has smkx sequence. Are
you saying me that I am going to be able to use fish in this
terminal? and only because you are too lazy to follow the terminfo
standard?

> There's two things we can do on our side - do "tput smkx" and "tput
> rmkx" either only in st or always (which I don't know if it'll break
> stuff elsewhere) or bind both kinds of sequences st sends, neither
> solution being _great_.

It should not break anything. If the terminal does not define smkx
you will not print anything, and if the terminal define it is because
it needs the sequence. All the application which use curses work
without problems in st when they try to match any key. mainly because
curses follows the terminfo specification perfectly.

> So: Is there a rationale for that decision and would you consider
> changing it?

Changing the key assignation is st is very simple, all the table
is in config.h file. In the past we have changed the key assignation
(we transformed backspace key in a delete key), and it is something
we can do again, although it is going to be a pain in the ass
because we already updated the definition of st in the official
terminfo definition.  I will admit a patch like this if all the
another suckless developer agree, which I don't think will happen
(I will continue with my custom key assignation), but even in this
case you must fix your shell, because if you don't follow the
terminfo rules it will not be able to run in a lot of different
terminals.

The question here is, why do you want to write a shell knowing
it has bugs and it will not be able of running in all the possible
(current or future) terminals?

Regards,

Re: [dev] [st] Terminal abnormal key codes

[Christoph Lohmann]

Greetings.

On Wed, 02 Sep 2015 16:22:30 +0200 "Roberto E. Vargas Caballero" 
 wrote:
> > So: Is there a rationale for that decision and would you consider
> > changing it?
> 
> terminfo definition.  I will admit a patch like this if all the
> another suckless developer agree, which I don't think will happen

Not approved.

St is right. Fish is wrong.


Sincerely,

Christoph Lohmann