Re: [PATCH] bracketed paste support
On 10/29/2014 08:49 PM, Chet Ramey wrote: On 10/27/14, 6:35 PM, Daniel Colascione wrote: This patch adds support for bracketed paste mode to readline. In this mode, readline instructs the terminal to wrap pasted strings in special control sequences so that programs can distinguish them from typed input. Thanks for the contribution. I'll look at the code; the approach seems sound. It went in very easily, though I changed some things around. This will be in the next release of bash and readline. The code appeared in the snapshot, so I was able to look at how you changed some things around. You removed the code that tries to determine whether a terminal actually supports the feature; instead, you blindly send the enable sequence to *any* terminal when enable-bracketed-paste is enabled, with potentially unknown consequences on those terminals. There's a comment in the code that indicates you expect users to enable the feature only on terminals where it's supported. I don't think it's reasonable to expect people to maintain a terminal database in their inputrc files. You also know very well that people are going to just set the feature to enabled in their configurations, test it, and see it work, all without considering whether they should enable this feature in a terminal-type-dependent conditional. Then they'll wonder why parts of their system silently break. Please restore the part of my patch, rl_bracketed_paste_probably_supported, that detects terminal support for this feature. signature.asc Description: OpenPGP digital signature
Re: [PATCH] bracketed paste support
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/5/14 9:02 PM, Daniel Colascione wrote: On 10/29/2014 08:49 PM, Chet Ramey wrote: On 10/27/14, 6:35 PM, Daniel Colascione wrote: This patch adds support for bracketed paste mode to readline. In this mode, readline instructs the terminal to wrap pasted strings in special control sequences so that programs can distinguish them from typed input. Thanks for the contribution. I'll look at the code; the approach seems sound. It went in very easily, though I changed some things around. This will be in the next release of bash and readline. The code appeared in the snapshot, so I was able to look at how you changed some things around. You removed the code that tries to determine whether a terminal actually supports the feature; instead, you blindly send the enable sequence to *any* terminal when enable-bracketed-paste is enabled, with potentially unknown consequences on those terminals. There's a comment in the code that indicates you expect users to enable the feature only on terminals where it's supported. I don't think it's reasonable to expect people to maintain a terminal database in their inputrc files. People rarely use more than one, maybe two, different terminals or terminal emulators. I don't doubt that people who enable this feature know enough to figure out whether or not it's going to work. Please restore the part of my patch, rl_bracketed_paste_probably_supported, that detects terminal support for this feature. It doesn't `detect terminal support' as such. It uses a heuristic to guess whether or not a terminal supports bracketed paste by checking for the presence of a string in another termcap sequence. Those kinds of heuristics always end up having an exception. - -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (Darwin) Comment: GPGTools - http://gpgtools.org iEYEARECAAYFAlRa30EACgkQu1hp8GTqdKsPvgCgnTgiPUAT27ADFyfZMErbr/F8 u+UAn1rYVN+4Ir/T+UEJEfLq2XO7fzvy =w6Rz -END PGP SIGNATURE-
Re: [PATCH] bracketed paste support
On 11/06/2014 02:38 AM, Chet Ramey wrote: On 11/5/14 9:02 PM, Daniel Colascione wrote: On 10/29/2014 08:49 PM, Chet Ramey wrote: On 10/27/14, 6:35 PM, Daniel Colascione wrote: This patch adds support for bracketed paste mode to readline. In this mode, readline instructs the terminal to wrap pasted strings in special control sequences so that programs can distinguish them from typed input. Thanks for the contribution. I'll look at the code; the approach seems sound. It went in very easily, though I changed some things around. This will be in the next release of bash and readline. The code appeared in the snapshot, so I was able to look at how you changed some things around. You removed the code that tries to determine whether a terminal actually supports the feature; instead, you blindly send the enable sequence to *any* terminal when enable-bracketed-paste is enabled, with potentially unknown consequences on those terminals. There's a comment in the code that indicates you expect users to enable the feature only on terminals where it's supported. I don't think it's reasonable to expect people to maintain a terminal database in their inputrc files. People rarely use more than one, maybe two, different terminals or terminal emulators. I don't doubt that people who enable this feature know enough to figure out whether or not it's going to work. You're assuming users are experts. In reality, users are going to just paste set enable-bracketed-paste on into their inputrc files; they'll then see an inscrutable control sequence appear in places they didn't expect and have no idea what's wrong with their system. Please optimize for what real users actually do, not what perfectly intelligent users with perfect understanding of their systems would ideally do. I'm already going to have to contact every readline and bash distributor on earth and ask them to enable this feature by default. I don't want to have to ask them to apply a code patch too. Please restore the part of my patch, rl_bracketed_paste_probably_supported, that detects terminal support for this feature. It doesn't `detect terminal support' as such. It uses a heuristic to guess whether or not a terminal supports bracketed paste by checking for the presence of a string in another termcap sequence. Those kinds of heuristics always end up having an exception. I don't think they will in this case, and at least it's a *conservative* heuristic. All terminals I've tested (urxvt, xterm, linux, and vte) either fail the heuristic, pass the heuristic and ignore the control sequence (which is harmless), or pass the heuristic and enable bracketed paste mode. Terminal emulation at this point is very mature. I don't expect significant development of new terminal types and new kinds of terminal emulators. The heuristic should work well for a long time. signature.asc Description: OpenPGP digital signature
Re: [PATCH] bracketed paste support
Daniel Colascione wrote: Bob Proulx wrote: I use paste into the shell with an embedded newline in order to immediately execute a command *a lot*. If that were removed I would be very unhappy. I strongly doubt that your use case is typical. It doesn't matter if it is typical or not. A feature need not have a 51% majority in order to be retained. Everyone does things differently. If we were to examine your operating modes carefully I am sure that we would find some practice that you use often that few other people are using and would be annoyed greatly if removed. Furthermore it is impossible to know how many people use it. This is a long standing feature that if removed would disenfranchise the users of it. That is all that we can say about it. I very often triple-click to select by line and then paste it into the shell for immediate execution. I less often triple-click to select multiple lines of a command set and paste them into the shell for immediate execution. I am sure there are many others not in this discussion who do the same. I've asked several of my colleagues; all have complained about accidentally pasting a large amount of text into the shell at one time or another. Nobody has complained about losing automatic execution of code after paste. Unfortunately that is not a useful survey. It is self-fullfilling! If I ask ten of my peers of whom I have each taught a feature if they use that feature then the answer will of course be yes. If on the other hand I have vilified a feature with my peers then it is likely that the reverse would be true. A survey asking colleagues around you is simply not a worthwhile survey. Sorry. Knowledge and use tends to group and cluster. Birds of a feather flock together. This is one of the reasons why things like conferences, newsgroups, and mailing lists are so useful. It allows people to meet others and learn about practices outside of their normal routine. People not only learn other things for themselves but they also learn that there is diversity in others too. In this case you learn that a feature that you despise is one that is liked by another. Speaking from personal experience, I've been using terminal emulators of various sorts for almost 20 years, and in that time, I've accidentally pasted documents into my terminal orders of magnitude more often than I've deliberately pasted multi-command sequences. I have used terminals for a very long time as well. I am using one now. I won't say that I haven't sometimes pasted in paragraphs of text into a terminal by mistake. I have. I also use kitchen knives to chop vegetables. I have sometimes cut myself doing so. That doesn't prevent me from avoiding cutting up vegetables with a kitchen knife. I have also accidentally removed a file with rm that I didn't intend to remove. In none of those cases do I want to remove rm or kitchen knives from the world to prevent anyone from doing either of those things again. As far as I'm concerned, automatic execution of code on paste isn't a feature: it's a bug and security hole. Users should have to opt into security holes. It is not a security hole. Simply declaring it to be so does not make it so. I get that it is a feature you hate. But labeling it with incorrect negative labels is simply a normal day of US political advertisements. It has no business on a technical list. We've only lived with the existing behavior so long because it's only recently become possible to distinguish pastes from other input. And I think it is great that you can now have it operate differently. Isn't it a wonderful world that we live in? Bob
Re: [PATCH] bracketed paste support
Chet Ramey wrote: * paste into the text editor invoked by the `edit-and-execute-command' key binding (C-xC-e in emacs mode), edit the command if desired, and have the shell automatically execute the contents of the editing buffer when the editor exits Cool! This was an idea that I hadn't thought about before. (I often would paste into my editor and clean it up. But doing it as part of the edit-and-execute-command is very nice!) Thanks for this hint! Bob
Re: [PATCH] bracketed paste support
On 10/30/2014 06:05 AM, Bob Proulx wrote: Daniel Colascione wrote: Well, I don't know whether Chet left the feature enabled by default. I hope he did, though, since preventing execution of pasted commands is one of the feature's key benefits. In bash, you should be able to execute a pasted command sequence by typing RET after the paste, but a paste by itself should never begin execution. I use paste into the shell with an embedded newline in order to immediately execute a command *a lot*. If that were removed I would be very unhappy. I strongly doubt that your use case is typical. I've asked several of my colleagues; all have complained about accidentally pasting a large amount of text into the shell at one time or another. Nobody has complained about losing automatic execution of code after paste. Speaking from personal experience, I've been using terminal emulators of various sorts for almost 20 years, and in that time, I've accidentally pasted documents into my terminal orders of magnitude more often than I've deliberately pasted multi-command sequences. As far as I'm concerned, automatic execution of code on paste isn't a feature: it's a bug and security hole. Users should have to opt into security holes. We've only lived with the existing behavior so long because it's only recently become possible to distinguish pastes from other input. However I read from Chet's replies that this is not the case. So I am not worried. Just voicing a counter opinion that this shouldn't be removed. Nobody is talking about removing the ability to select the present behavior. We're talking about the default setting. If you want it then you should enable it. This is like many other options available in the shell. They are optional. This feature significantly decreases the likelihood of user error, but if it's not on by default, most users won't enable it and won't see any benefits. I'd rather slightly power users like you with very specific use cases than continue to see accidental code execution. For better or worse, people copy and paste commands from websites all the time. Even if a bit of shell looks innocuous, a malicious bit of JavaScript could change the pasted text at the last second without the user being aware of the switch. Then the browser shouldn't copy it. The place to fix the problem is in the browser. Browser behavior is unlikely to change; even if it did, sequences of text that look safe can contain control characters like TAB that invoke various shell commands. Visually inspection is not a reliable way to inspect a piece of code pasted from an otherwise-untrusted source. Sure, you might argue that users should paste into a trusted intermediate location --- say a text editor --- inspect the code, and then paste into the shell. That would be the prudent thing to do, but users don't actually *do* that and never will. A problem with a browser should not break cutting and pasting in general. This change doesn't break pasting. It changes its behavior to one that's safer, more natural, and more consistent with that of other programs. (Tynt uses this technique to slightly less malicious ends.) If pasting in a terminal immediately begins execution, there's no opportunity to review pasted code. With bracketed paste support, the shell requires additional user interaction after a paste to begin execution, making this attack much less effective. I am very happy this is a feature you like. However I would hate that feature. It should definitely not default to on or it will make people like me very unhappy. You can disable the setting yourself. If you don't want to do that, all you need to do is type RET after pasting. That's a very small tradeoff for an increase in safety and predictability. signature.asc Description: OpenPGP digital signature
Re: [PATCH] bracketed paste support
Daniel Colascione wrote: Well, I don't know whether Chet left the feature enabled by default. I hope he did, though, since preventing execution of pasted commands is one of the feature's key benefits. In bash, you should be able to execute a pasted command sequence by typing RET after the paste, but a paste by itself should never begin execution. I use paste into the shell with an embedded newline in order to immediately execute a command *a lot*. If that were removed I would be very unhappy. There are a number of strategies you can employ if you're closer to Daniel than Bob on this and you want to avoid executing pasted shell commands with the current version of bash. Even if you don't want to change your selection habits to avoid selecting the entire line, there are a few things you can do: * configure your terminal emulator, if possible, to remove trailing newlines in pasted text * paste into a text editor running in a separate window, edit the command if desired, then re-select and re-paste into your terminal window * paste into the text editor invoked by the `edit-and-execute-command' key binding (C-xC-e in emacs mode), edit the command if desired, and have the shell automatically execute the contents of the editing buffer when the editor exits Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: [PATCH] bracketed paste support
On Thu, Oct 30, 2014 at 1:45 PM, Daniel Colascione dan...@dancol.org wrote: Sure, you might argue that users should paste into a trusted intermediate location --- say a text editor --- inspect the code, and then paste into the shell. That would be the prudent thing to do, but users don't actually *do* that and never will. And you have communities whose idea of an installer is wget | bash. I'm looking at you [redacted].
Re: [PATCH] bracketed paste support
On 10/30/2014 07:46 PM, Bob Proulx wrote: Chet Ramey wrote: * paste into the text editor invoked by the `edit-and-execute-command' key binding (C-xC-e in emacs mode), edit the command if desired, and have the shell automatically execute the contents of the editing buffer when the editor exits Cool! This was an idea that I hadn't thought about before. (I often would paste into my editor and clean it up. But doing it as part of the edit-and-execute-command is very nice!) Thanks for this hint! Be careful with this though as the editor may be more exploitable. I see vim for example interprets ESC even in paste mode. It probably shouldn't do that in paste mode, but you can see what happens if you paste from the following innocuous looking HTML page: printf 'echo \033:q!brls' t.html This just runs the q command, but could be anything of course. Pádraig.
Re: [PATCH] bracketed paste support
On 30 Oct 2014 18:45, Daniel Colascione wrote: +1 -mike signature.asc Description: Digital signature
Re: [PATCH] bracketed paste support
Daniel Colascione wrote: Well, I don't know whether Chet left the feature enabled by default. I hope he did, though, since preventing execution of pasted commands is one of the feature's key benefits. In bash, you should be able to execute a pasted command sequence by typing RET after the paste, but a paste by itself should never begin execution. I use paste into the shell with an embedded newline in order to immediately execute a command *a lot*. If that were removed I would be very unhappy. However I read from Chet's replies that this is not the case. So I am not worried. Just voicing a counter opinion that this shouldn't be removed. If you want it then you should enable it. This is like many other options available in the shell. They are optional. For better or worse, people copy and paste commands from websites all the time. Even if a bit of shell looks innocuous, a malicious bit of JavaScript could change the pasted text at the last second without the user being aware of the switch. Then the browser shouldn't copy it. The place to fix the problem is in the browser. A problem with a browser should not break cutting and pasting in general. (Tynt uses this technique to slightly less malicious ends.) If pasting in a terminal immediately begins execution, there's no opportunity to review pasted code. With bracketed paste support, the shell requires additional user interaction after a paste to begin execution, making this attack much less effective. I am very happy this is a feature you like. However I would hate that feature. It should definitely not default to on or it will make people like me very unhappy. Bob
Re: [PATCH] bracketed paste support
On 10/29/2014 09:35 PM, Pádraig Brady wrote: On 10/27/2014 10:35 PM, Daniel Colascione wrote: +@item enable-bracketed-paste +@vindex enable-bracketed-paste +If set to @samp{on} and the terminal supports bracketed paste mode, +configure it to insert each paste into the editing buffer as a string +instead of treating the characters pasted as normal input, preventing +inadvertent execution of pasted commands. The default is @samp{on}. I see this defaults on. Does this mean one can't paste command sequences to readline now? Well, I don't know whether Chet left the feature enabled by default. I hope he did, though, since preventing execution of pasted commands is one of the feature's key benefits. In bash, you should be able to execute a pasted command sequence by typing RET after the paste, but a paste by itself should never begin execution. For better or worse, people copy and paste commands from websites all the time. Even if a bit of shell looks innocuous, a malicious bit of JavaScript could change the pasted text at the last second without the user being aware of the switch. (Tynt uses this technique to slightly less malicious ends.) If pasting in a terminal immediately begins execution, there's no opportunity to review pasted code. With bracketed paste support, the shell requires additional user interaction after a paste to begin execution, making this attack much less effective. signature.asc Description: OpenPGP digital signature
Re: [PATCH] bracketed paste support
On 10/29/2014 09:42 PM, Daniel Colascione wrote: On 10/29/2014 09:35 PM, Pádraig Brady wrote: On 10/27/2014 10:35 PM, Daniel Colascione wrote: +@item enable-bracketed-paste +@vindex enable-bracketed-paste +If set to @samp{on} and the terminal supports bracketed paste mode, +configure it to insert each paste into the editing buffer as a string +instead of treating the characters pasted as normal input, preventing +inadvertent execution of pasted commands. The default is @samp{on}. I see this defaults on. Does this mean one can't paste command sequences to readline now? Well, I don't know whether Chet left the feature enabled by default. I hope he did, though, since preventing execution of pasted commands is one of the feature's key benefits. In bash, you should be able to execute a pasted command sequence by typing RET after the paste, but a paste by itself should never begin execution. For better or worse, people copy and paste commands from websites all the time. Even if a bit of shell looks innocuous, a malicious bit of JavaScript could change the pasted text at the last second without the user being aware of the switch. (Tynt uses this technique to slightly less malicious ends.) If pasting in a terminal immediately begins execution, there's no opportunity to review pasted code. With bracketed paste support, the shell requires additional user interaction after a paste to begin execution, making this attack much less effective. Requiring the extra RET after pasting shouldn't be too onerous. I found this to be a good summary: http://cirw.in/blog/bracketed-paste Thanks for the extra info! Pádraig.
Re: [PATCH] bracketed paste support
On 10/29/14 5:35 PM, Pádraig Brady wrote: On 10/27/2014 10:35 PM, Daniel Colascione wrote: +@item enable-bracketed-paste +@vindex enable-bracketed-paste +If set to @samp{on} and the terminal supports bracketed paste mode, +configure it to insert each paste into the editing buffer as a string +instead of treating the characters pasted as normal input, preventing +inadvertent execution of pasted commands. The default is @samp{on}. I see this defaults on. The version of the patch I put into readline defaults to off. Does this mean one can't paste command sequences to readline now? No. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
[PATCH] bracketed paste support
This patch adds support for bracketed paste mode to readline. In this mode, readline instructs the terminal to wrap pasted strings in special control sequences so that programs can distinguish them from typed input. This patch makes readline insert each pasted string as one big literal into the edit buffer; as there is no termcap capability for bracketed paste support, the patch detects capable terminals automatically by looking for support for similar escape sequences. diff --git a/lib/readline/bind.c b/lib/readline/bind.c index 8acf4ac..7173552 100644 --- a/lib/readline/bind.c +++ b/lib/readline/bind.c @@ -1486,6 +1486,7 @@ static const struct { { convert-meta,_rl_convert_meta_chars_to_ascii, 0 }, { disable-completion, rl_inhibit_completion, 0 }, { echo-control-characters, _rl_echo_control_chars,0 }, + { enable-bracketed-paste, _rl_enable_bracketed_paste,0 }, { enable-keypad, _rl_enable_keypad, 0 }, { enable-meta-key, _rl_enable_meta, 0 }, { expand-tilde,rl_complete_with_tilde_expansion, 0 }, diff --git a/lib/readline/doc/readline.3 b/lib/readline/doc/readline.3 index ca0a81a..808b4b7 100644 --- a/lib/readline/doc/readline.3 +++ b/lib/readline/doc/readline.3 @@ -578,6 +578,13 @@ following the cursor are not duplicated. If set to \fBOn\fP, a character denoting a file's type as reported by \fIstat\fP(2) is appended to the filename when listing possible completions. +.TP +.B enable\-bracketed\-paste (On) +If set to \fBOn\fP and the terminal supports bracketed +paste mode, configure it to insert each paste into the +editing buffer as a string instead of treating +the characters pasted as normal input, preventing inadvertent +execution of pasted commands. .PD .SS Conditional Constructs .PP diff --git a/lib/readline/doc/rluser.texi b/lib/readline/doc/rluser.texi index 577..0a26444 100644 --- a/lib/readline/doc/rluser.texi +++ b/lib/readline/doc/rluser.texi @@ -703,6 +703,13 @@ If set to @samp{on}, a character denoting a file's type is appended to the filename when listing possible completions. The default is @samp{off}. +@item enable-bracketed-paste +@vindex enable-bracketed-paste +If set to @samp{on} and the terminal supports bracketed paste mode, +configure it to insert each paste into the editing buffer as a string +instead of treating the characters pasted as normal input, preventing +inadvertent execution of pasted commands. The default is @samp{on}. + @end table @item Key Bindings diff --git a/lib/readline/funmap.c b/lib/readline/funmap.c index 363507b..29f089b 100644 --- a/lib/readline/funmap.c +++ b/lib/readline/funmap.c @@ -101,6 +101,7 @@ static const FUNMAP default_funmap[] = { { history-substring-search-backward, rl_history_substr_search_backward }, { history-substring-search-forward, rl_history_substr_search_forward }, { insert-comment, rl_insert_comment }, + { bracketed-paste-begin, rl_bracketed_paste_begin }, { insert-completions, rl_insert_completions }, { kill-whole-line, rl_kill_full_line }, { kill-line, rl_kill_line }, diff --git a/lib/readline/kill.c b/lib/readline/kill.c index 3d23745..79de79f 100644 --- a/lib/readline/kill.c +++ b/lib/readline/kill.c @@ -656,6 +656,61 @@ rl_yank_last_arg (count, key) return retval; } +/* We've recognized a terminal control sequence telling us to expected + * pasted content. Read characters from the terminal until we read a + * bracketed paste end sequence, treating the characters read as + * literal text to insert. */ +int +rl_bracketed_paste_begin (count, key) + int count, key; +{ + static const char endseq[] = \033[201~; + static const int endseqlen = sizeof (endseq) - 1; + + int rv = -1; + int c; + size_t len = 0; + size_t cap = 16; + char *buf = xmalloc (cap); + + RL_SETSTATE(RL_STATE_MOREINPUT); + while (0 = (c = rl_read_key ())) +{ + if (RL_ISSTATE (RL_STATE_MACRODEF)) +_rl_add_macro_char (c); + + if (c == '\r') +c = '\n'; + + if (len == cap) +buf = xrealloc (buf, (cap *= 2)); + + buf[len++] = c; + if (len = endseqlen + !strncmp (buf[len - endseqlen], +endseq, +endseqlen)) +{ + len -= endseqlen; + break; +} +} + RL_UNSETSTATE(RL_STATE_MOREINPUT); + + if (c = 0) +{ + if (len == cap) +buf = xrealloc (buf, (cap += 1)); + + buf[len++] = '\0'; + rl_insert_text (buf); + rv = 0; +} + + xfree (buf); + return rv; +} + /* A special paste command for users of Cygnus's cygwin32. */ #if defined (__CYGWIN__) #include windows.h diff --git a/lib/readline/readline.c b/lib/readline/readline.c index f8211e7..d6e9775 100644 --- a/lib/readline/readline.c +++ b/lib/readline/readline.c @@ -302,6 +302,11 @@ int _rl_revert_all_at_newline = 0; characters corresponding to keyboard-generated signals. */ int