[issue21474] Idle: updata fixwordbreaks() for unicode identifiers

[miss-islington]

miss-islington  added the comment:


New changeset 3d11630ff401cfcdf094cf039cb575332ecaea20 by Miss Islington (bot) 
in branch '3.6':
bpo-21474: Update IDLE word/identifier definition from ascii to unicode. 
(GH-6643)
https://github.com/python/cpython/commit/3d11630ff401cfcdf094cf039cb575332ecaea20


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21474] Idle: updata fixwordbreaks() for unicode identifiers

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Thank you for the help.  My immediate goal for IDLE is to fix unicode problems, 
to the extent allowed by tk, before 3.7.0.

--
keywords:  -patch
nosy:  -miss-islington
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21474] Idle: updata fixwordbreaks() for unicode identifiers

[miss-islington]

miss-islington  added the comment:


New changeset 887b5f8fc622267e1fd48862ea9d0dfd4a0abdc6 by Miss Islington (bot) 
in branch '3.7':
bpo-21474: Update IDLE word/identifier definition from ascii to unicode. 
(GH-6643)
https://github.com/python/cpython/commit/887b5f8fc622267e1fd48862ea9d0dfd4a0abdc6


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21474] Idle: updata fixwordbreaks() for unicode identifiers

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

> Do we still need this line in fixwordbreaks?

Yes. Loading word.tcl sets tcl_wordchars and tcl_nonwordchars. We should ensure 
that word.tcl is loaded and these variables are set before we change them.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21474] Idle: updata fixwordbreaks() for unicode identifiers

[miss-islington]

Change by miss-islington :


--
pull_requests: +6345

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21474] Idle: updata fixwordbreaks() for unicode identifiers

[miss-islington]

Change by miss-islington :


--
keywords: +patch
pull_requests: +6344
stage: commit review -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21474] Idle: updata fixwordbreaks() for unicode identifiers

[Terry J. Reedy]

Terry J. Reedy  added the comment:


New changeset 5ff3a161c8a6b525c5e5b3e36e9c43f5a95bda60 by Terry Jan Reedy in 
branch 'master':
bpo-21474: Update IDLE word/identifier definition from ascii to unicode. 
(GH-6643)
https://github.com/python/cpython/commit/5ff3a161c8a6b525c5e5b3e36e9c43f5a95bda60


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21474] Idle: updata fixwordbreaks() for unicode identifiers

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Now-closed duplicate #33386 reported that μ, 0x3bc, is not selected as part of 
identifiers when double clicking.  This prompted some research.

The 'Windows' style imitates the behavior of Command Prompt, which I presume is 
a carryover from DOS days.  PowerShell stuck with it, but Notepad, Notepad++, 
Microsoft Word, Firefox, Thunderbird, and ??? have not. I think Tcl should have 
switched long ago.  In any case, I will go with whatever the tcl re engine 
defines as word chars, the 'Motif' style', rather than attempt to write a giant 
re, which would have to change as characters are added.

Do we still need this line in fixwordbreaks?
tk.call('tcl_wordBreakAfter', 'a b', 0) # make sure word.tcl is loaded
I will leave it until you say we don't.

After patching, 'abcμμμdef' ('0x3bc'*3) is selected as one word instead of 
word, nonword, word.  'abc+efg' is still selected in 3 pieces, instead of the 1 
word seen by Command Prompt.

--
keywords:  -patch
stage: patch review -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21474] Idle: updata fixwordbreaks() for unicode identifiers

[Terry J. Reedy]

Change by Terry J. Reedy :


--
keywords: +patch
pull_requests: +6341
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21474] Idle: updata fixwordbreaks() for unicode identifiers

[Terry J. Reedy]

Changes by Terry J. Reedy :


--
assignee:  -> terry.reedy
components: +IDLE
versions: +Python 3.6, Python 3.7 -Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21474] Idle: updata fixwordbreaks() for unicode identifiers

[Serhiy Storchaka]

Changes by Serhiy Storchaka storch...@gmail.com:


--
keywords: +easy
stage: test needed - needs patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21474
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21474] Idle: updata fixwordbreaks() for unicode identifiers

[Terry J. Reedy]

Terry J. Reedy added the comment:

Deleting the function and calls to it would be easy. But my memory, which could 
be off, is that I left fixwordbreaks(root) in the test function now called 
_editor_window (at the end of the file) because the test did not work right 
without it. I willhave to recheck.

Beyond that, my first experiments were aimed at discovering the functions 
affected and therefor what would have to be tested with any changes.

To properly test this requires simulating keystrokes, like control-backspace, 
as opposed to inserting characters. Are there any tk/tkinter tests that do 
this, that I could use as a model?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21474
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21474] Idle: updata fixwordbreaks() for unicode identifiers

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

word.tcl in Tcl library contains following lines:

if {$::tcl_platform(platform) eq windows} {
# Windows style - any but a unicode space char
set ::tcl_wordchars {\S}
set ::tcl_nonwordchars {\s}
} else {
# Motif style - any unicode word char (number, letter, or underscore)
set ::tcl_wordchars {\w}
set ::tcl_nonwordchars {\W}
}

So by default all works as expected in Motif style, but not in Windows style.

If you want to have same behavior in both styles, defines word chars as:

tk.call('set', 'tcl_wordchars', r'\w')
tk.call('set', 'tcl_nonwordchars', r'\W')

GUI tests are not needed, it is enough to test relevant Tcl commands: 
tcl_wordBreakAfter, tcl_wordBreakBefore, tcl_endOfWord, tcl_startOfNextWord, 
and tcl_startOfPreviousWord or TextSelectTo. It's interesting, there are no 
tests for these functions in Tcl test suite.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21474
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21474] Idle: updata fixwordbreaks() for unicode identifiers

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I think it is enough to get rid of this function.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21474
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21474] Idle: updata fixwordbreaks() for unicode identifiers

[Terry J. Reedy]

New submission from Terry J. Reedy:

EditorWindow.py has this function, applied to editor and shell windows, which 
is obsolete for Python3 and unicode identifiers.

def fixwordbreaks(root):
# Make sure that Tk's double-click and next/previous word
# operations use our definition of a word (i.e. an identifier)
tk = root.tk
tk.call('tcl_wordBreakAfter', 'a b', 0) # make sure word.tcl is loaded
tk.call('set', 'tcl_wordchars', '[a-zA-Z0-9_]')
tk.call('set', 'tcl_nonwordchars', '[^a-zA-Z0-9_]')

Double clicking selects a contiguous sequence of 'word' or 
'nonword' characters.
Control-backspace deletes word left, Control-DEL deletes word right.
Control-left/right Arrow moves by words in a strange but useful way.

It might be more useful if the REs were expanded.

--
messages: 218307
nosy: serhiy.storchaka, terry.reedy
priority: normal
severity: normal
stage: test needed
status: open
title: Idle: updata fixwordbreaks() for unicode identifiers
type: enhancement
versions: Python 3.4, Python 3.5

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21474
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com