[issue16182] readline: Wrong tab completion scope indices in Unicode terminals

Martin Panter added the comment: Think I got all the bugs fixed here. -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker

[issue16182] readline: Wrong tab completion scope indices in Unicode terminals

Roundup Robot added the comment: New changeset ef234a5c5817 by Martin Panter in branch '3.5': Issue #16182: One more check for set_pre_input_hook() https://hg.python.org/cpython/rev/ef234a5c5817 New changeset 241bae60cef8 by Martin Panter in branch 'default': Issue #16182: Merge test_readline

[issue16182] readline: Wrong tab completion scope indices in Unicode terminals

Roundup Robot added the comment: New changeset 005cab4f5629 by Martin Panter in branch '3.5': Issue #16182: set_pre_input_hook() may not exist; document, and update test https://hg.python.org/cpython/rev/005cab4f5629 New changeset c4dd384ee3fa by Martin Panter in branch 'default': Issue #16182:

[issue16182] readline: Wrong tab completion scope indices in Unicode terminals

Martin Panter added the comment: Also need a fix for missing set_pre_input_hook() on the AIX buildbot: == FAIL: test_nonascii (test.test_readline.TestReadline)

[issue16182] readline: Wrong tab completion scope indices in Unicode terminals

Roundup Robot added the comment: New changeset 0794bbfceec6 by Martin Panter in branch '3.5': Issue #16182: Attempted workarounds for Apple Editline https://hg.python.org/cpython/rev/0794bbfceec6 New changeset a1ca9c0ebc05 by Martin Panter in branch 'default': Issue #16182: Merge test_readline

[issue16182] readline: Wrong tab completion scope indices in Unicode terminals

Martin Panter added the comment: Failures from AMD64 Snow Leop buildbots: == FAIL: test_nonascii_history (test.test_readline.TestHistoryManipulation) --

[issue16182] readline: Wrong tab completion scope indices in Unicode terminals

Roundup Robot added the comment: New changeset 5122b3465a38 by Martin Panter in branch '3.5': Issue #16182: Fix readline begidx, endidx, and use locale encoding https://hg.python.org/cpython/rev/5122b3465a38 New changeset 2ae2657d87a6 by Martin Panter in branch 'default': Issue #16182: Merge

[issue16182] readline: Wrong tab completion scope indices in Unicode terminals

Martin Panter added the comment: I get two other test suite failures if I set PYTHONIOENCODING, so I am not going to bother addressing this in test_readline :) FAIL: test_forced_io_encoding (test.test_capi.EmbeddingTests) FAIL: test_7 (test.test_pkg.TestPkg) --

[issue16182] readline: Wrong tab completion scope indices in Unicode terminals

Serhiy Storchaka added the comment: LGTM. But the test fails with PYTHONIOENCODING=ascii. $ PYTHONIOENCODING=ascii ./python -m test test_readline Run tests sequentially 0:00:00 [1/1] test_readline test test_readline failed -- Traceback (most recent call last): File

[issue16182] readline: Wrong tab completion scope indices in Unicode terminals

Martin Panter added the comment: I updated the patch to fix the error handling and memory leak. it also now skips the test in case the locale cannot encode the test data. -- Added file: http://bugs.python.org/file43359/readline_locale.v4.patch ___

[issue16182] readline: Wrong tab completion scope indices in Unicode terminals

Serhiy Storchaka added the comment: Martin? -- ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue16182] readline: Wrong tab completion scope indices in Unicode terminals

Serhiy Storchaka added the comment: Thank you for your comments and updated patches, and especially for tests Martin. Added some comments on Rietveld. -- ___ Python tracker

[issue16182] readline: Wrong tab completion scope indices in Unicode terminals

Martin Panter added the comment: V3 finishes what I started in v2: * Changed unchecked PyBytes_AsString() → PyBytes_AS_STRING() * Testing more functions for non-ASCII characters I tried to test it with Editline on Linux (using my patch for Issue 13501). There seem to be many quirks with my

[issue16182] readline: Wrong tab completion scope indices in Unicode terminals

Changes by Martin Panter : Added file: http://bugs.python.org/file43068/readline_locale.v2.patch ___ Python tracker ___

[issue16182] readline: Wrong tab completion scope indices in Unicode terminals

Changes by Martin Panter : Removed file: http://bugs.python.org/file43067/readline_locale.v2.patch ___ Python tracker ___

[issue16182] readline: Wrong tab completion scope indices in Unicode terminals

Martin Panter added the comment: Attached patch adds a possible test, and fixes the truncation problem I mentioned above. I tried testing set_completer_delims() with a UTF-8 locale, but I suspect Gnu Readline does not support it. I called set_completer_delims("\xF6"), which encodes as C3 B6,

[issue16182] readline: Wrong tab completion scope indices in Unicode terminals

Changes by Mark Lawrence : -- nosy: -BreamoreBoy ___ Python tracker ___ ___

[issue16182] readline: Wrong tab completion scope indices in Unicode terminals

Martin Panter added the comment: I’m a bit worried about flex_complete() covering up errors. If I add calls to PyErr_WriteUnraisable(), I can see both the s1 and s2 decodes failing. Input the following line: >>> "©"; import x; and then move the cursor back one place, so it is directly after

[issue16182] readline: Wrong tab completion scope indices in Unicode terminals

Serhiy Storchaka added the comment: Yes, the readline module is broken in Python 3. Underlying C library operates C strings and use locale-depended C functions to split it on Unicode characters. The Python wrapper always uses the UTF-8 encoding for converting between Python strings and C

[issue16182] readline: Wrong tab completion scope indices in Unicode terminals

Mark Lawrence added the comment: @Kaarle please accept our apologies for the delay in getting back to you. Can one of our unicode gurus comment please. -- components: +Unicode nosy: +BreamoreBoy, ezio.melotti, lemburg, loewis versions: +Python 3.5 -Python 3.2, Python 3.3

[issue16182] readline: Wrong tab completion scope indices in Unicode terminals

New submission from Kaarle Ritvanen: Tab completion in the readline module does not seem to work well with Unicode terminals. The get_line_buffer function converts the line buffer to the str type (which are Unicode strings in Python 3), but the indices returned by get_begidx and get_endidx

[issue16182] readline: Wrong tab completion scope indices in Unicode terminals

Changes by Antoine Pitrou pit...@free.fr: -- nosy: +haypo stage: - patch review versions: +Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16182 ___