Re: Supporting more key modifiers (was: Re: Dear Bram)
Ok, I am resurrecting a very old thread here because it keeps coming up time and time again on #vim in Freenode, and Paul still wants this to happen, as do many of us. On Fri, Feb 25, 2011 at 8:43 AM, Bram Moolenaar b...@moolenaar.net wrote: So the problem is that many users expect CTRL-M to have the same effect as Enter, just like people use CTRL-[ instead of Esc. And a few people would make the CTRL-M act different from Enter, and CTRL-[ different from Esc. First problem is to actually detect what key was pressed, in most terminal emulators this is not possible. In the GUI we can. Changing terminal emulators to support this and making this work with Vim is a separate issue, I'll not go into that here. Actually, that's exactly the issue that is being addressed. Vim shouldn't have to address it -- in theory termcap/terminfo were supposed to do this, but they didn't really do so well. Which is why we now have difficulty telling the difference between ESCo and A-o, as well as things like ESC vs C-[. Ultimately the proposal is that we let the raw keyboard input (for terminals) be handled by libtermkey, and it figures out if what was pressed was ESC or c-[ or C-[o vs ESCo vs A-o. It then passes a stream of information to vim, which vim interprets properly. And yes, by default there would be a mapping (either explicit or pseudo-builtin) that treats ESC and C-[ as the same, but that can then be changed by the user, just as they can now with any other mapping. Then we need a way to make the extra information available to be used in mappings, without breaking it for users relying on the current way. The only breakage I can foresee is if someone actually does a :map c-i fooinstead of :map Tab foo -- then, yes, you'd end up with different behavior. I think this is within the realm of acceptable change, particularly if it's clearly stated in the release notes (and yes, I think this is a Vim 8 thing). Some things that are no acceptable: - Have a setting to enable the new way. This will break existing stuff and make users pull their hair out because they don't know this setting exists. Forget it. I actually agree -- it should be just the way things work. And if it's done right then it won't break anything (beyond the above). - Change the input queue from a stream of bytes to some list of structs. This isn't adding any functionality and breaks all kinds of mapping and termcode handling, register execution, etc. Forget it. The termcode handling will be replaced by this, so that's not an issue. Mapping is also not an issue beyond the above. Registers are more of an issue. No doubt. Can we think on this to come up with a solution? Perhaps the register needs to become more like a :map, with special keys expressed specially? Otherwise I think we're stuck since you cannot differentiate between ESC and c-[, and there's no way to express things like c-space. What we can do is extend the existing modifier byte sequence. This is a bit tricky, but it should work. So we add a new byte sequence with the raw key encoded, plus modifiers. Thus for CTRL-[ you get the [ key with the CTRL modifier. Which is precisely what libtermkey emits, and how it can differentiate between ctrl-[ and alt-ctrl-[, etc. Really what happens is that the terminal input buffer starts to look a great deal more like events that the GUI versions get. Instead of ESCo you get o with alt depressed. To be very clear -- some of this will depend on a terminal with the proper support being used. But anything that's built on libvterm already has that, as well as xterm. But if you don't have those then moving to this new scheme should not break ANYTHING. Your term would still send ESCo and libtermkey would tell vim it saw o with alt. But if you do have a terminal with the right support, or you're using the GUI... well, to quote a bit from :help design-multi-platform, Support all the keys on the keyboard for mapping. It would be great to get closer to that. Tom Sorensen -- You received this message from the vim_dev maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php
Re: Supporting more key modifiers
On Fri, Mar 18, 2011 at 09:15:18PM +1100, Ben Schmidt wrote: A thought occurred to me a couple of days ago: would it be appropriate to actually use CSI as the byte-stream representation? Or something like it. It is a byte-stream representation, right? And it's extensible, right, so it is less likely to need a big overhaul in future? Does it allow for 'private use' stuff? Because Vim needs a way to represent a bunch of other events as keystrokes (mouse clicks, scroll wheel, its magic script-local meta-character thing, representing the CSI escape character itself, etc.). Also, is it easy to skip over CSI sequences if necessary with simple logic? Sure. CSI is intended for just this. A CSI sequence is matched by a regexp CSI [\x20-\x2f]? [\x30-\x3f]* [\x40-\x7e] Rougly possibly-one punctuation character, then a bunch of numbers, ; or ;, then finally a signle letter. The private use area in CSI is the character range \x70-\x7e. One of these, CSI u (\x75) is already in use by xterm and libtermkey to encode modified Unicode characters. To my knowledge, the remaining are free. Plus, vim could even use this encoding internally if it wanted. -- Paul LeoNerd Evans leon...@leonerd.org.uk ICQ# 4135350 | Registered Linux# 179460 http://www.leonerd.org.uk/ signature.asc Description: Digital signature
Re: Supporting more key modifiers (was: Re: Dear Bram)
As long as the two triplets of keypresses I suggested originally can all be represented uniquely, and without reference to timing information in the Escape vs Alt+ case, then I'm happy with whatever internal implementation makes it happen. The two triplets in question being Tab Ctrl-I Ctrl-Shift-I Escape C Alt+C é Yes I am aware that on current byte-driven terminals there is no way without using timing information to distinguish all these three cases in the latter triplet here. This should not stop GUI systems from distinguishing them however. Can someone (Bram?) tell us if Bram's proposed solution would allow us to map those triplets independently as wanted? I'm not familiar enough about vim's byte queue system to tell if the modifier byte really answers our needs here. Thanks. Philippe -- You received this message from the vim_dev maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php
Re: Supporting more key modifiers
On 18/03/11 8:38 PM, Philippe Vaucher wrote: As long as the two triplets of keypresses I suggested originally can all be represented uniquely, and without reference to timing information in the Escape vs Alt+ case, then I'm happy with whatever internal implementation makes it happen. The two triplets in question being Tab Ctrl-I Ctrl-Shift-I Escape C Alt+C é Yes I am aware that on current byte-driven terminals there is no way without using timing information to distinguish all these three cases in the latter triplet here. This should not stop GUI systems from distinguishing them however. Can someone (Bram?) tell us if Bram's proposed solution would allow us to map those triplets independently as wanted? Of course it does. Bram wouldn't have suggested it if it wouldn't work at all I'm not familiar enough about vim's byte queue system to tell if the modifier byte really answers our needs here. I don't think we need to keep the same byte-stream representation as we currently have, and I don't think that's what Bram meant. I think we can change it. As long as it remains a byte-stream representation. And I don't think it will be possible to do what we need without at least extending it. At the moment, keys are a fixed length of 3 bytes (one of which is the 'escape byte' 0x80, and the other two of which have restricted values). There simply isn't enough value-space to do what we need, and it's possibly not very future-proof even if we can squeeze it in. It's also a bit of a mess, IIRC. It would be better to give it an overhaul. And it probably needs to use variable-length sequences. A thought occurred to me a couple of days ago: would it be appropriate to actually use CSI as the byte-stream representation? Or something like it. It is a byte-stream representation, right? And it's extensible, right, so it is less likely to need a big overhaul in future? Does it allow for 'private use' stuff? Because Vim needs a way to represent a bunch of other events as keystrokes (mouse clicks, scroll wheel, its magic script-local meta-character thing, representing the CSI escape character itself, etc.). Also, is it easy to skip over CSI sequences if necessary with simple logic? I think there are some nice/helpful things that could be done design-wise though. For example, Vim is often used with enc=utf-8 these days. I think it would be wise to, if possible, ensure all key sequence representations are invalid UTF-8 in their entirety. Then they could be put in buffers and yanked back into registers without ambiguity (hopefully), i.e. there wouldn't be the ambiguity of whether the escape character was part of a key sequence that should be yanked as such, or part of the buffer text which should be escaped upon yanking. See, Vim already deals with the CSI character in its byte-stream code, and escapes it, so some aspects of this are already somewhat accounted for. But I wonder when precisely it is escaped, and whether at times it is escaped when it shouldn't be, or not when it should be. It would be good to design to avoid this problem. (I happened to be hacking on some Vim code recently where this came into play, so I noted it with interest.) Ben. -- You received this message from the vim_dev maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php
Re: Supporting more key modifiers (was: Re: Dear Bram)
On Mon, 7 Mar 2011, Paul LeoNerd Evans wrote: On Fri, Feb 25, 2011 at 02:43:23PM +0100, Bram Moolenaar wrote: First problem is to actually detect what key was pressed, in most terminal emulators this is not possible. In the GUI we can. Changing terminal emulators to support this and making this work with Vim is a separate issue, I'll not go into that here. Changing terminal emulators is a done task; xterm has supported this since 2008. In fact I asked Thomas Dickey to clarify this for me, and he informs me it's all present and working. Despite its popularity, xterm is not most terminal emulators. I think this is a great move on xterm's part (having a more-or-less consistent set of escapes for modifiers). But, it's still a chicken-and-egg problem. Once it's in xterm, it takes a while (if ever) to filter down to derivatives. And even under xterm the setting that allows this to work isn't the default. I almost didn't send this, since I don't think the lack of terminal support should be a show stopper (If any terminal supports it, Vim should use that support), but claiming that it's a done deal because xterm added a feature to support it a few years ago is naïve. -- Best, Ben -- You received this message from the vim_dev maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php
Re: Supporting more key modifiers (was: Re: Dear Bram)
On Mon, Mar 07, 2011 at 11:24:23AM -0500, Benjamin R. Haskell wrote: Changing terminal emulators is a done task; xterm has supported this since 2008. In fact I asked Thomas Dickey to clarify this for me, and he informs me it's all present and working. Despite its popularity, xterm is not most terminal emulators. I think this is a great move on xterm's part (having a more-or-less consistent set of escapes for modifiers). But, it's still a chicken-and-egg problem. Once it's in xterm, it takes a while (if ever) to filter down to derivatives. And even under xterm the setting that allows this to work isn't the default. I almost didn't send this, since I don't think the lack of terminal support should be a show stopper (If any terminal supports it, Vim should use that support), but claiming that it's a done deal because xterm added a feature to support it a few years ago is naïve. This is regrettable, yes. But not entirely true. Until xterm added the extended CSI notation using the 2nd parameter for modifiers in e.g. CSI 1;5C for Ctrl-Left, no terminal in the world did it. Now, apart from rxvt which uses its own scheme, prettymuch every terminal does it, and does it -that way-. These things aren't completely set in stone, but they do take a long long time to filter around. I'm very keen to try to shepherd people into one standard and consistently-designed way of doing things. That rxvt does modified keys differently is regrettable, and a consequence of splintered reinvention of wheels. I'm very keen to try not to let that happen any more. It's to this end I tried initially writing this spec for modified keypresses: http://www.leonerd.org.uk/hacks/fixterms/ It comes a little whiney and amateur at the moment, because I'm no good at the social engineering side of trying to motivate a world-full of 1970's-era engineers into trying to fix a big problem. :) But the technical details are there, awaiting someone with better writing skills than I to neaten it up a bit. -- Paul LeoNerd Evans leon...@leonerd.org.uk ICQ# 4135350 | Registered Linux# 179460 http://www.leonerd.org.uk/ signature.asc Description: Digital signature
Re: Supporting more key modifiers
Charles Campbell wrote: Bram Moolenaar wrote: [snip] Then we need a way to make the extra information available to be used in mappings, without breaking it for users relying on the current way. Some things that are no acceptable: - Have a setting to enable the new way. This will break existing stuff and make users pull their hair out because they don't know this setting exists. Forget it. - Change the input queue from a stream of bytes to some list of structs. This isn't adding any functionality and breaks all kinds of mapping and termcode handling, register execution, etc. Forget it. What we can do is extend the existing modifier byte sequence. This is a bit tricky, but it should work. So we add a new byte sequence with the raw key encoded, plus modifiers. Thus for CTRL-[ you get the [ key with the CTRL modifier. [snip] Another way to extend this might be to switch from a byte queue to a short integer queue. The lower 8-bits would be as it is now; the upper eight bits would encode modifier keys (shift, ctrl, alt, meta, whatever). The current special-key encoding is a bit clumsy. On the other hand, there are situations where the queue can get very long (recursive execution of mappings or 1000@@). Using 16 or 32 bits for every character is not attractive. One way to solve this is to use up to 128 bit integers, making sure that the higher bits are usually zero, and then use variable length encoding. ASCII characters should then still take only one byte. http://code.google.com/apis/protocolbuffers/docs/encoding.html Nevertheless, what gets encoded in registers must still be a byte sequence. And we need to be able to store double-byte and multi-byte characters. Thus I don't think this will make things simpler. -- This is the polymorph virus! Follow these instructions carefully: 1. Send this message to everybody you know. 2. Format your harddisk. Thank you for your cooperation in spreading the most powerful virus ever! /// Bram Moolenaar -- b...@moolenaar.net -- http://www.Moolenaar.net \\\ ///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\ \\\ an exciting new programming language -- http://www.Zimbu.org/// \\\help me help AIDS victims -- http://ICCF-Holland.org/// -- You received this message from the vim_dev maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php
Re: Supporting more key modifiers
Another way to extend this might be to switch from a byte queue to a short integer queue. The lower 8-bits would be as it is now; the upper eight bits would encode modifier keys (shift, ctrl, alt, meta, whatever). I think sticking to the byte queue is best. So much in Vim relies on text being represented in either a single byte encoding or UTF-8 (e.g. when you use UCS-2 or UCS-4, Vim will use UTF-8 internally, I believe). It will break less if we stick to a byte stream with escapes, as now, IMHO. Ben. -- You received this message from the vim_dev maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php
