[issue42819] readline 8.1 bracketed paste

[STINNER Victor]

STINNER Victor  added the comment:

https://lists.gnu.org/archive/html/bug-readline/2020-11/msg00010.html
says that bracketed mode is turned off if the terminal type is "dumb".

Setting TERM env var to dumb gives me a surprising behavior when I paste 
"1+1\n2+2" in Python REPL:
---
$ TERM=dumb python3
Python 3.9.1 (default, Jan 20 2021, 00:00:00) 
>>> ^J1+1^J2+2^J
---

If I press ENTER, I get:
---
>>> ^J1+1^J2+2^J
  File "", line 1

1+1
2+2

^
SyntaxError: multiple statements found while compiling a single statement
>>> 
---

Note: if I redirect Python output into a file ("python3 > output"), I get a 
different behavior since stdout is no longer a TTY.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42819] readline 8.1 bracketed paste

[STINNER Victor]

STINNER Victor  added the comment:

Related issue: test_pdb_interaction_doctest test of pytest fails on Fedora 
Rawhide because of a "\x1b[?2004h" string:
* https://github.com/pytest-dev/pytest/issues/8256
* https://bugzilla.redhat.com/show_bug.cgi?id=1892612

It fails with readline 8.1 which enables bracketed mode by default, but it 
fails with any readline version if ~/.inputrc contains "set 
enable-bracketed-paste on".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42819] readline 8.1 bracketed paste

[STINNER Victor]

STINNER Victor  added the comment:

Oh, I forgot to mention that I ran my tests in Gnome Terminal 3.38.1-2.fc33.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42819] readline 8.1 bracketed paste

[STINNER Victor]

STINNER Victor  added the comment:

On my Fedora 33 (readline-8.0-5.fc33.x86_64), I enabled the bracketed paste 
mode by adding "set enable-bracketed-paste" on to my ~/.inputrc config file.

When I test https://thejh.net/misc/website-terminal-copy-paste : the evil 
command is no long executed.

If I copy/paste "1+1" and "2+2" commands (two lines pasted at once) manually in 
the Python REPL, they are no longer executed immediately, Python waits for me 
pressing ENTER to execute, *as expected*:


$ ./python 
Python 3.10.0a5+ (heads/master:fcbe0cb04d, Feb 15 2021, 10:30:10) 
[GCC 10.2.1 20201125 (Red Hat 10.2.1-9)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 1+1
2+2


But using "./python -i < commands", each line is executed immediately and 
Python exits after displaying the results:

---
$ cat commands 
1+1
2+2

$ ./python -i < commands
Python 3.10.0a5+ (heads/master:fcbe0cb04d, Feb 15 2021, 10:30:10) 
>>> 2
>>> 4
>>> 
---

If I disable again bracketed mode (ex: remove ~/.inputrc file in my case), when 
I copy/paste manually the two commands at once, they are executed again, as 
expected:
---
$ ./python
>>> 1+1
2
>>> 2+2
4
---

The bracketed paste mode can be enabled explicitly in Python with this patch to 
simulate readline 8.1 default behavior:

diff --git a/Modules/readline.c b/Modules/readline.c
index c900e07954..ce8662c000 100644
--- a/Modules/readline.c
+++ b/Modules/readline.c
@@ -217,6 +217,7 @@ readline_read_init_file_impl(PyObject *module, PyObject 
*filename_obj)
 errno = rl_read_init_file(NULL);
 if (errno)
 return PyErr_SetFromErrno(PyExc_OSError);
+rl_variable_bind ("enable-bracketed-paste", "on");
 Py_RETURN_NONE;
 }
 


All of these behavior look consistent, correct and expected.

Note: When I redirect Python output into a file, I cannot see "\e[?2004h" 
(likely because stdout is not a TTY if it's redirected into a file), so I'm not 
sure how to check if this escape sequence is injected in Python stdout or not.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42819] readline 8.1 bracketed paste

[STINNER Victor]

STINNER Victor  added the comment:

I didn't know "bracketed paste mode". It seems like a protocol between a 
terminal and the user to add markers before/after a pasted text.
https://cirw.in/blog/bracketed-paste

"""
In summary:

1. Enable bracketed paste: printf "\e[?2004h"
2. Wait for paste to start: you’ll see \e[200~ on STDIN.
3. Wait for paste to stop: you’ll see \e[201~ on STDIN.
4. Disable bracketed paste: printf "\e[?2004l"
"""

I understand that this mode prevents to run arbitrary command when pasting 
blindly an untrusted command copied from a webpage, like this example:
https://thejh.net/misc/website-terminal-copy-paste

It can be enabled explicitly by adding "set enable-bracketed-paste" on to your 
~/.inputrc configuration file.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42819] readline 8.1 bracketed paste

[Miro Hrončok]

Miro Hrončok  added the comment:

https://lists.gnu.org/archive/html/bug-readline/2020-11/msg00010.html
https://lists.gnu.org/archive/html/bug-bash/2020-10/msg00048.html
https://lists.gnu.org/archive/html/bug-bash/2020-10/msg00087.html

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42819] readline 8.1 bracketed paste

[STINNER Victor]

STINNER Victor  added the comment:

> Readline 8.1 enables bracketed paste by default.

Can you try to find why it changed the default? Why should Python change the 
default readline default behavior?

--
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42819] readline 8.1 bracketed paste

[Miro Hrončok]

Miro Hrončok  added the comment:

This also affects Fedora 34+

--
nosy: +hroncok, petr.viktorin, vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42819] readline 8.1 bracketed paste

[Ned Deily]

Change by Ned Deily :


--
nosy: +twouters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42819] readline 8.1 bracketed paste

[Dustin Rodrigues]

Dustin Rodrigues  added the comment:

Bumping this issue because it's a bug affecting all users who build or use a 
Python built with the current version of readline, to include all macOS 
HomeBrew and Arch Linux users who use the python from the respective package 
managers.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42819] readline 8.1 bracketed paste

[Dustin Rodrigues]

Dustin Rodrigues  added the comment:

The linked PR disables bracketed paste regardless of if the user has it as a 
configuration option or if Python was configured with a version of readline 
which defaults to on for bracketed paste. Is this a viable approach?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42819] readline 8.1 bracketed paste

[Dustin Rodrigues]

Change by Dustin Rodrigues :


--
keywords: +patch
pull_requests: +22937
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/24108

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42819] readline 8.1 bracketed paste

[FX Coudert]

FX Coudert  added the comment:

This bug is present in all versions since the 3.6 branch at least, but was only 
triggered when users' readline settings explicitly enabled bracketed-paste. 
Previous reports: https://bugs.python.org/issue32019 and 
https://bugs.python.org/issue39820 Longer discussion (on a more extensive 
feature request) at https://bugs.python.org/issue38747

The new factor is that readline 8.1 now enables bracketed-paste by default. So 
the broken behaviour is seen for Python built with the latest readline, with no 
specific setting.

Python should probably in the long term be improved to handle bracketed-paste, 
which will make a nicer user experience. But in the short term, it should 
communication with readline to disable bracketed-paste at runtime, and restore 
the previous behaviour.

--
nosy: +fxcoudert

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42819] readline 8.1 bracketed paste

[Dustin Rodrigues]

New submission from Dustin Rodrigues :

Readline 8.1 enables bracketed paste by default. Package managers like Homebrew 
for macOS and distributions like Arch Linux which use the latest version of 
readline (released December 2020) now have new behavior when pasting multiline 
strings into the python REPL. Disabling bracketed paste on 8.1 reverts to the 
expected behavior, and enabling bracketed paste on 8.0 also reproduces the 
behavior. Further information in 
https://github.com/Homebrew/homebrew-core/issues/68193

Example with bracketed paste on:
$ cat in
1+2
3+4
5+6

$ pbcopy < in
$ /usr/local/Cellar/python\@3.9/3.9.1_3/bin/python3
Python 3.9.1 (default, Dec 28 2020, 11:22:14) 
[Clang 11.0.0 (clang-1100.0.33.17)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> 1+2
3+4
5+6

  File "", line 1
1+2
3+4
5+6


   ^
SyntaxError: multiple statements found while compiling a single statement
>>> 



Example with it off:
$ /usr/local/bin/python3
Python 3.9.1 (v3.9.1:1e5d33e9b9, Dec  7 2020, 12:10:52) 
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> 1+2
3
>>> 3+4
7
>>> 5+6
11
>>>

--
messages: 384297
nosy: dtrodrigues
priority: normal
severity: normal
status: open
title: readline 8.1 bracketed paste
type: behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com