Tal Einat added the comment:
I'm attaching a patch which really fixes this issue, along with additional
tests for idlelib.HyperParser.
I did indeed have to fix PyParse as well. I got it working with re.subn() as
Martin suggested, but the performance was much worse (between 100x and 1000x
slower according to my timings). So I came up with the clever solution of
passing a defaultdict to str.translate(), with the default value being
ord('x'). That works as expected and is much faster than the regexp.
Finally, since the defaultdict is kept around as long as IDLE is running, I
decided to avoid having it grow continually and consume memory unnecessarily.
So I wrote a simple Mapping class, which wraps a normal dict and uses a custom
default value instead of None, ord('x') in this case. Works like a charm :)
I haven't written tests for PyParse since it currently doesn't have any tests
at all. But from the HyperParser tests and from some manual testing, it seems
to be working as expected.
----------
Added file:
http://bugs.python.org/file35959/taleinat.20140716.IDLE_HyperParser_unicode_ids.patch
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue21765>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
