New submission from py.user:
https://docs.python.org/3/library/re.html#writing-a-tokenizer
There are redundant escapes in the regex:
('OP', r'[+*\/\-]'),# Arithmetic operators
Sequence -+*/ is sufficient.
It makes the loop to do all steps on every 4 spaces:
('SKIP',r'[ \t]'),
Changes by Raymond Hettinger raymond.hettin...@gmail.com:
--
assignee: docs@python - rhettinger
nosy: +rhettinger
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21977
___
Raymond Hettinger added the comment:
I will keep the \- because the - at the front of the character range is a
non-obvious special case. The other changes look reasonable.
--
priority: normal - low
stage: - commit review
type: enhancement - performance
versions: +Python 2.7, Python
Roundup Robot added the comment:
New changeset bb28542af060 by Raymond Hettinger in branch '3.4':
Issue 21977: Minor improvements to the regexes in the tokenizer example.
http://hg.python.org/cpython/rev/bb28542af060
--
nosy: +python-dev
___
Python
Changes by Raymond Hettinger raymond.hettin...@gmail.com:
--
resolution: - fixed
status: open - closed
versions: -Python 2.7
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21977
___
Changes by Berker Peksag berker.pek...@gmail.com:
--
stage: commit review - resolved
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21977
___
___
