[issue43676] Doctest ELLIPSIS explanation hard to follow when they're missing
New submission from Tim Hatch : The doctest docs try to explain directives like ELLIPSIS but those directives are absent from the rendered html. Where? Most of the code blocks in the Directives section, and https://docs.python.org/3/library/doctest.html#directives and the one introduced by "nice approach" subsequently are missing their directive comments. The docs today say they go with Python 3.9.2 generated by Sphinx 2.4.4. I haven't tried generating them manually, but it appears the `pycon` lexer for Pygments handles these fine, so I assume it's a problem in reST. -- assignee: docs@python components: Documentation messages: 389874 nosy: Tim.Hatch, docs@python priority: normal severity: normal status: open title: Doctest ELLIPSIS explanation hard to follow when they're missing type: enhancement versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue43676> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36541] Make lib2to3 grammar better match Python, support the := walrus
Change by Tim Hatch : -- pull_requests: +18682 pull_request: https://github.com/python/cpython/pull/19317 ___ Python tracker <https://bugs.python.org/issue36541> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37341] str.format and f-string divergence
Tim Hatch added the comment: ok, I suppose it's just documentation then. -- assignee: -> docs@python components: +Documentation -Library (Lib) nosy: +docs@python ___ Python tracker <https://bugs.python.org/issue37341> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37341] str.format and f-string divergence
New submission from Tim Hatch :
TL;DR
f"{x+2}" and f"{x=}" do something sensible.
"{x+2}".format(x=1) and "{x=}".format(x=1) raise KeyError.
f"{0.1}" and "{0.1}".format(...) are different.
Having had a feature request to be able to codemod f-strings[1] and just
learning about issue36817[2] last night, I went digging into how these are
parsed in the standard library to be able to reuse that.
str.format, string.Formatter, and the _string module can only parse literal
keys, not expressions, despite appearing to take the same syntax as f-strings.
I'm happy to contribute code to change this, but unsure if it's considered a
bug or feature (now that we're past feature freeze for 3.8). I would love to
see these converge to prevent confusion and let us document in just one place.
The {} and {1} style are still exceptional.
We already parse expressions and the "=" suffix acceptably,
>>> list(_string.formatter_parser("{1+1}"))
[('', '1+1', '', None)]
>>> list(_string.formatter_parser("{(x)}"))
[('', '(x)', '', None)]
>>> list(_string.formatter_parser("{x=!r:1}"))
[('', 'x=', '1', 'r')]
>>> list(_string.formatter_parser("{ x = }"))
[('', ' x = ', '', None)]
But the consumers would need to check for /=\s*$/ and call eval on the items in
formatter_field_name_split. (I lack a good heuristic, maybe we eval every time
unless it's strictly numeric?)
It would also break unusual uses like these in the name of unification
>>> "{1+1}".format(**{"1+1": "zzz"})
'zzz'
and
>>> class T:
... pass
...
>>> setattr(T, "0", "zero")
>>> f"{T.0}"
File "", line 1
(T.0)
^
SyntaxError: invalid syntax
>>> "{0.0}".format(T)
'zero'
[1] https://github.com/facebookincubator/Bowler/issues/87
[2] incorrectly listed in Misc/NEWS.d/3.8.0b1.rst as issue36774 btw
--
components: Library (Lib)
messages: 346065
nosy: barry, eric.smith, larry, lisroach, lukasz.langa, serhiy.storchaka, thatch
priority: normal
severity: normal
status: open
title: str.format and f-string divergence
type: enhancement
versions: Python 3.8, Python 3.9
___
Python tracker
<https://bugs.python.org/issue37341>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37166] inspect.findsource doesn't handle shortened files gracefully
Change by Tim Hatch : -- type: -> behavior ___ Python tracker <https://bugs.python.org/issue37166> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37166] inspect.findsource doesn't handle shortened files gracefully
New submission from Tim Hatch :
inspect.findsource() can trigger IndexError when co_firstlineno is larger than
len(linecache.getlines()).
If you have a situation where the file that linecache finds doesn't match the
imported module, then you're not guaranteed that co_firstlineno on the code
objects makes any sense. We hit this in a special kind of par file, but it
could be triggered by shortening the file, like doing a git checkout of an
older version while it's running.
a.py (3 lines):
# line 1
# line 2
def foo(): pass # co_firstlineno=3
a.py.v2 (1 line):
def foo(): pass
repro:
import a
# modification happens, cp a.py.v2 a.py
inspect.getsource(a.foo)
Although linecache.getline() does bounds checking, `inspect.findsource()`
(which is used by various other functions, including `inspect.stack()`) grabs
all the lines and loops through them. The bug is in this section of
`inspect.py`:
if iscode(object):
if not hasattr(object, 'co_firstlineno'):
raise OSError('could not find function definition')
lnum = object.co_firstlineno - 1
pat =
re.compile(r'^(\s*def\s)|(\s*async\s+def\s)|(.*(? 0:
if pat.match(lines[lnum]): break
lnum = lnum - 1
return lines, lnum
raise OSError('could not find code object')
Found through future.utils.raise_from which actually doesn't need to be using
`inspect.stack()`, it can switch to `sys._getframe(2)` or so. We should have a
PR ready shortly, but wondering if this can be backported to at least 3.6?
--
components: Library (Lib)
messages: 344761
nosy: lisroach, thatch, vstinner
priority: normal
severity: normal
status: open
title: inspect.findsource doesn't handle shortened files gracefully
versions: Python 3.6, Python 3.7, Python 3.8
___
Python tracker
<https://bugs.python.org/issue37166>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33348] lib2to3 doesn't parse f(*[] or [])
Tim Hatch added the comment: Also see discussion about divergence on issue36541 -- nosy: +thatch ___ Python tracker <https://bugs.python.org/issue33348> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36541] Make lib2to3 grammar more closely match Python
Tim Hatch added the comment: My strong preference would be getting the lib2to3 grammar to be the python grammar + additions, to make future changes easier to merge. The strongest argument against doing that is the backwards-incompatibility of patterns -- some won't compile, while others will compile but do something unexpected). It's good to hear (or at least infer) that parsing modern code is also a goal of lib2to3. -- ___ Python tracker <https://bugs.python.org/issue36541> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36607] asyncio.all_tasks() crashes if asyncio is used in multiple threads
Change by Tim Hatch : -- nosy: +thatch ___ Python tracker <https://bugs.python.org/issue36607> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36541] Make lib2to3 grammar more closely match Python
Tim Hatch added the comment: Here's approximately what it would look like to do the big change now: https://github.com/python/cpython/compare/master...thatch:lib2to3-update-grammar (one test failing, and some helpers may need more test coverage) -- ___ Python tracker <https://bugs.python.org/issue36541> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36541] Make lib2to3 grammar more closely match Python
Tim Hatch added the comment: jreese reminded me of pep570, which will make more grammar changes. I'm open to the idea of replacing the grammar with the live one, plus porting the 2isms forward like print, eval, except with comma. My sincere hope is that everyone that depends on this structure will have tests (mine and lib2to3 do); the only big user I'm aware of is probably libfuturize. Definitely worth a changelog entry if this is the way forward. -- ___ Python tracker <https://bugs.python.org/issue36541> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36541] Make lib2to3 grammar more closely match Python
Change by Tim Hatch : -- pull_requests: +12627 ___ Python tracker <https://bugs.python.org/issue36541> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36541] Make lib2to3 grammar more closely match Python
New submission from Tim Hatch : The grammar in lib2to3 is out of date and can't parse `:=` nor `f(**not x)` from running on real code. I've done a cursory `diff -uw Grammar/Grammar Lib/lib2to3/grammar.txt`, and would like to fix lib2to3 so we can merge into both fissix and blib2to3, to avoid further divergence of the forks. I'm unsure if I need a separate bug per pull request, but need at least one to get started. -- components: 2to3 (2.x to 3.x conversion tool) messages: 339522 nosy: georg.brandl, lukasz.langa, thatch priority: normal severity: normal status: open title: Make lib2to3 grammar more closely match Python type: behavior versions: Python 3.6, Python 3.7, Python 3.8 ___ Python tracker <https://bugs.python.org/issue36541> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22849] Double DECREF in TextIOWrapper
New submission from Tim Hatch:
There's a reproducible bug in textio.c that causes a double DECREF on codecs.
The conditions to trigger are probably rare in real life, so not remotely
exploitable (sandbox escape is the worst I can think of on its own, and I'm not
aware of any on 3.x):
* You need to create a TextIOWrapper wrapping a file-like object that only
partially supports the protocol. For example, supporting readable(),
writable(), and seekable() but not tell().
The crash I experience most of the time appears to be that the memory being
reused, such that the PyObject ob_type field is no longer a valid pointer.
Affected:
Source 3.5.0a0 (latest default branch yesterday, 524a004e93dd)
Archlinux: 3.3.5 and 3.4.2
Ubuntu: 3.4.0
Unaffected:
Centos: 3.3.2
All 2.7 branch (doesn't contain the faulty commit)
Here's where it's introduced --
https://hg.python.org/cpython/rev/f3ec00d2b75e/#l5.76
/* Modules/_io/textio.c line 1064 */
Py_DECREF(codec_info);
/* does not set codec_info = NULL; */
...
if(...) goto error;
...
error:
Py_XDECREF(codec_info);
The attached script is close to minimal -- I think at most you can reduce by
one TextIOWrapper instantiation. Sample stacktrace follows (which is after the
corruption occurs, on subsequent access to v-ob_type (which is invalid).
#0 0x004c8829 in PyObject_GetAttr (v=unknown at remote
0x77eb9688,
name='_is_text_encoding') at Objects/object.c:872
#1 0x004c871d in _PyObject_GetAttrId (v=unknown at remote
0x77eb9688,
name=0x945d50 PyId__is_text_encoding.10143) at Objects/object.c:835
#2 0x005c6674 in _PyCodec_LookupTextEncoding (
encoding=0x76f40220 utf-8, alternate_command=0x6c2fcd codecs.open())
at Python/codecs.c:541
#3 0x0064286e in textiowrapper_init (self=0x77f9ecb8,
args=(F at remote 0x76f40a18,), kwds={'encoding': 'utf-8'})
at ./Modules/_io/textio.c:965
--
components: Library (Lib)
files: segv.py
messages: 231054
nosy: thatch
priority: normal
severity: normal
status: open
title: Double DECREF in TextIOWrapper
type: crash
versions: Python 3.3, Python 3.4, Python 3.5
Added file: http://bugs.python.org/file37183/segv.py
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22849
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8016] Support for cp858
Tim Hatch t...@timhatch.com added the comment: I also noticed that libforensics supplies a codec for cp858, if that's helpful to double-check the implementation. http://code.google.com/p/libforensics/source/browse/code/lf/win/codepage/cp858.py -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8016 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8016] Support for cp858
Tim Hatch t...@timhatch.com added the comment: Uploading corrected diff -- the old one missed a couple of instances of DOTLESS I - EURO. -- versions: +Python 2.7 Added file: http://bugs.python.org/file17301/cp858.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8016 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6906] Tkinter sets an unicode environment variable on win32
Tim Hatch t...@timhatch.com added the comment: I'm running the exact same version as Gabriel (on Windows 7, 32 bit) from the python.org installer, and have the same behavior as Michał. Checking FixTk.py it appears that on Vista and above, it calls the Win32 API GetFinalPathNameByHandleW to expand symbolic links, but this is a -W function, not a -A so it deals with unicode. The blame shows this function was added in response to #3881 For my system, the paths are all lower ascii so changing the end of convert_path to return str(s) works. I don't know anything about filesystem encodings on Windows to do a more correct conversion. -- nosy: +thatch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6906 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2649] poss. patch for fnmatch.py to add {.htm, html} style globbing
Tim Hatch t...@timhatch.com added the comment: More discussion has gone on in issue #4573 on this topic. Can this bug be marked as a duplicate? -- nosy: +thatch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2649 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
