[issue45847] Port module setup to PY_STDLIB_MOD() macro and addext()

[Anthony Sottile]

Change by Anthony Sottile :


--
nosy:  -Anthony Sottile

___
Python tracker 
<https://bugs.python.org/issue45847>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45847] Port module setup to PY_STDLIB_MOD() macro and addext()

[Anthony Sottile]

Anthony Sottile  added the comment:

I could, but it's very unlikely to get fixed given I believe 18.04 is in 
security-only fixes and backporting a pkg-config file seems unlikely

this worked two days ago before this patch

--

___
Python tracker 
<https://bugs.python.org/issue45847>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45847] Port module setup to PY_STDLIB_MOD() macro and addext()

[Anthony Sottile]

Anthony Sottile  added the comment:

the `tk-dev` package on ubuntu bionic does not ship with a `pkg-config` file 
for tk so it does not build properly there: 

```
root@f0dd06a3e87c:/cpython# dpkg -L tk8.6-dev | grep pc
root@f0dd06a3e87c:/cpython# 
```

(a note: bionic reaches end of life in april 2023 so it is likely to still see 
significant use until then)

--

___
Python tracker 
<https://bugs.python.org/issue45847>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45847] Port module setup to PY_STDLIB_MOD() macro and addext()

[Anthony Sottile]

Anthony Sottile  added the comment:

this appears to break the tkinter extension for ubuntu bionic (18.04) -- I'm 
not entirely sure on the correct fix here but I get the following from trying 
to build there:

2022-04-02T15:52:08.0910452Z Python build finished successfully!
2022-04-02T15:52:08.0911924Z The necessary bits to build these optional modules 
were not found:
2022-04-02T15:52:08.0913469Z _tkinter


this is the tk I have available:

root@f0dd06a3e87c:/# dpkg -l | grep libtk
ii  libtk8.6:amd64   8.6.8-4 amd64  
  Tk toolkit for Tcl and X11 v8.6 - run-time files

--
nosy: +Anthony Sottile

___
Python tracker 
<https://bugs.python.org/issue45847>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue47152] Reorganize the re module sources

[Anthony Sottile]

Anthony Sottile  added the comment:

would it be possible to expose `parse_template` -- or at least some way to 
validate that a regex replacement string is correct prior to executing the 
replacement?

I'm currently using that for my text editor: 
https://github.com/asottile/babi/blob/d37d7d698d560aef7c6a0d1ec0668672e039bd9a/babi/screen.py#L501

--
nosy: +Anthony Sottile

___
Python tracker 
<https://bugs.python.org/issue47152>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46491] typing: allow Annotated in outermost scope

[Anthony Sottile]

Anthony Sottile  added the comment:

3.7.2 has another example where OrderedDict was added to typing

I don't have any personal investment in this particular change but I've had 
quite a few unhappy consumers of libraries due to instability in typing apis 
between patch versions in the past (heh and they're especially frustrated 
because they don't care about type checking and just want functional software)

it's also difficult and/or not cost effective to add every patch version to a 
CI matrix to catch these sorts of problems

--

___
Python tracker 
<https://bugs.python.org/issue46491>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46491] typing: allow Annotated in outermost scope

[Anthony Sottile]

Anthony Sottile  added the comment:

to me this is the same as the Union[Pattern] / Union[Match] "fixes" that landed 
in 3.5.3 -- and the pain caused by using that and having CI pass (because of 
modern 3.5.x) but having spurious bug reports from users stuck on 3.5.2

or in 3.6.1 when NamedTuple was "fixed" to allow methods, again having CI pass 
with modern pythons but having frustrated users running 3.6.0

I forsee the same class of problems here with Annotated where it works great in 
3.10.3 and 3.9.11 but anyone stuck on 3.10.2 or 3.9.10 or older will be broken

--

___
Python tracker 
<https://bugs.python.org/issue46491>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46491] typing: allow Annotated in outermost scope

[Anthony Sottile]

Anthony Sottile  added the comment:

should this behaviour change be backported?  it potentially creates an annoying 
edgecase where code seemingly works unless you use an older patch version

since this isn't a bugfix I wouldn't expect this to land in 3.9 and 3.10

--
nosy: +Anthony Sottile

___
Python tracker 
<https://bugs.python.org/issue46491>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44295] self.assertDictContainsSubset warning is unhelpful

[Anthony Sottile]

Anthony Sottile  added the comment:

looks like this was removed in 3.11 -- wondering if it's still worth correcting 
the deprecation warning location

--
versions: +Python 3.10 -Python 3.11

___
Python tracker 
<https://bugs.python.org/issue44295>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39214] Add curses.window.in_wch

[Anthony Sottile]

Anthony Sottile  added the comment:

rebased this onto 3.11

--
versions: +Python 3.11 -Python 3.9

___
Python tracker 
<https://bugs.python.org/issue39214>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45548] Update Modules/Setup

[Anthony Sottile]

Anthony Sottile  added the comment:

I'm seeing some weird breakage in the deadsnakes builds, presumably due to this 
change:

```
...
2021-10-27T08:55:21.9485959Z x86_64-linux-gnu-gcc -pthread -Wno-unused-result 
-Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -g -fdebug-prefix-map=/tmp/code=. 
-specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector -Wformat 
-Werror=format-security-std=c99 -Wextra -Wno-unused-result 
-Wno-unused-parameter -Wno-missing-field-initializers 
-Werror=implicit-function-declaration -fvisibility=hidden  
-I../Include/internal -IObjects -IInclude -IPython -I. -I../Include -Wdate-time 
-D_FORTIFY_SOURCE=2   -DPy_BUILD_CORE_BUILTIN  -c 
../Modules/_blake2/blake2module.c -o Modules/_blake2/blake2module.o
2021-10-27T08:55:22.0563233Z Assembler messages:
2021-10-27T08:55:22.0565823Z Fatal error: can't create 
Modules/_blake2/blake2module.o: No such file or directory
2021-10-27T08:55:22.0622105Z Makefile:2177: recipe for target 
'Modules/_blake2/blake2module.o' failed
...
```

https://github.com/deadsnakes/python3.11-nightly/actions/runs/1389342824

I believe this is due to 
https://github.com/python/cpython/pull/29225/files#diff-aac314cb71e95912e95795a4f4a278f2e75a5bc28e93f52e338185c8e7f7d7eaL225

--
nosy: +Anthony Sottile

___
Python tracker 
<https://bugs.python.org/issue45548>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45165] alighment format for nullable values

[Anthony Sottile]

Anthony Sottile  added the comment:

alignment was the only one I found which was applicable and didn't work

`!s` "works" but it's kind of a hack (of course I want string formatting, I'm 
making a string!) -- then I would want to write `!s` everywhere which is 
cumbersome and ugly

--

___
Python tracker 
<https://bugs.python.org/issue45165>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45165] alighment format for nullable values

[Anthony Sottile]

New submission from Anthony Sottile :

currently this works correctly:

```
>>> '%8s %8s' % (None, 1)
'None1'
```

but conversion to f-string fails:

```
>>> f'{None:>8} {1:>8}'
Traceback (most recent call last):
  File "", line 1, in 
TypeError: unsupported format string passed to NoneType.__format__
```

my proposal is to implement alignment `__format__` for `None` following the 
same as for `str` for alignment specifiers

--
components: Interpreter Core
messages: 401582
nosy: Anthony Sottile
priority: normal
severity: normal
status: open
title: alighment format for nullable values
type: behavior
versions: Python 3.10, Python 3.11

___
Python tracker 
<https://bugs.python.org/issue45165>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39562] Asynchronous comprehensions don't work in asyncio REPL

[Anthony Sottile]

Anthony Sottile  added the comment:

this maybe shouldn't have been backported to 3.8.x as the change in compiler 
flags appears to break pyc files in subtle ways: 
https://stackoverflow.com/q/68910329/812183

--
nosy: +Anthony Sottile

___
Python tracker 
<https://bugs.python.org/issue39562>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33671] Efficient zero-copy for shutil.copy* functions (Linux, OSX and Win)

[Anthony Sottile]

Change by Anthony Sottile :


--
nosy: +Anthony Sottile
nosy_count: 19.0 -> 20.0
pull_requests: +26032
pull_request: https://github.com/python/cpython/pull/27516

___
Python tracker 
<https://bugs.python.org/issue33671>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44467] profiling-compatible functools.wraps

[Anthony Sottile]

New submission from Anthony Sottile :

this is a small proposal to add a new function to the functools module which 
provides better profiling-compatible `@functools.wraps(...)`

the rationale comes from https://github.com/Yelp/named_decorator (which is dead 
/ abandoned)

the tl;dr from there is any time a decorator is involved in a profile it 
becomes very difficult to trace because everything becomes tangled around 
common decorators (because function names are used from the code object)

here is the proposal and an initial implementation:


def wraps_with_name(func, decorator, **kwargs):
def wraps_with_name_decorator(wrapped):
new_name = f'{func.__name__}@{decorator.__name__}'
new_code = wrapped.__code__.replace(co_name=new_name)
# would be nice if `types.FunctionType` had a `.replace(...)` too!
new_wrapped = types.FunctionType(
new_code,
wrapped.__globals__,
new_name,
wrapped.__defaults__,
wrapped.__closure__,
)
return functools.wraps(func, **kwargs)(new_wrapped)
return better_wraps_decorator

the usage would be similar to `functools.wraps`, here is an example:

```python
def my_decorator(func):
@functools.wraps_with_name(func, my_decorator)
def my_decorator_inner(*args, **kwargs):
return func(*args, **kwargs)
return my_decorator
```

--
components: Library (Lib)
messages: 396199
nosy: Anthony Sottile
priority: normal
severity: normal
status: open
title: profiling-compatible functools.wraps
type: enhancement
versions: Python 3.11

___
Python tracker 
<https://bugs.python.org/issue44467>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44297] Frame with -1 line number

[Anthony Sottile]

Anthony Sottile  added the comment:

here is a minimal reproduction:

```python
def iterboom():
raise AssertionError
yield 1

next(1 for x in iterboom())
```

python 3.9:


```
$ python3.9 t.py
Traceback (most recent call last):
  File "/tmp/rinohtype/t.py", line 5, in 
next(1 for x in iterboom())
  File "/tmp/rinohtype/t.py", line 5, in 
next(1 for x in iterboom())
  File "/tmp/rinohtype/t.py", line 2, in iterboom
raise AssertionError
AssertionError
```

```
$ python3.10 t.py
Traceback (most recent call last):
  File "/tmp/rinohtype/t.py", line 5, in 
next(1 for x in iterboom())
  File "/tmp/rinohtype/t.py", line -1, in 
  File "/tmp/rinohtype/t.py", line 2, in iterboom
raise AssertionError
AssertionError
```

--

___
Python tracker 
<https://bugs.python.org/issue44297>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44459] 3.10b2 -> 3.10b3 regression in importlib.metadata for rinoh

[Anthony Sottile]

New submission from Anthony Sottile :

installed from git:

```
$ git remote -v
origin  https://github.com/brechtm/rinohtype.git (fetch)
origin  https://github.com/brechtm/rinohtype.git (push)
$ git rev-parse HEAD
4054539bae53eaddd10291c8429a1a32aeeb4786
```

working in 3.10 b2:

```console
$ ./venv310/bin/python --version --version
Python 3.10.0b2 (default, Jun  2 2021, 00:22:18) [GCC 9.3.0]
$ ./venv310/bin/python -m rinoh
usage: rinoh [-h] [-f FORMAT] [-o OPTION=VALUE] [-t NAME or FILENAME] [-s NAME 
or FILENAME]
 [-O FILENAME or DIRECTORY] [-p PAPER] [-i] [--list-templates] 
[--list-stylesheets]
 [--list-fonts [FILENAME]] [--list-formats] [--list-options 
FRONTEND] [--version] [--docs]
 [input]

Render a structured document to PDF.

positional arguments:
  input the document to render

options:
  -h, --helpshow this help message and exit
  -f FORMAT, --format FORMAT
the format of the input file (default: autodetect)
  -o OPTION=VALUE, --option OPTION=VALUE
options to be passed to the input file reader
  -t NAME or FILENAME, --template NAME or FILENAME
the document template or template configuration file to 
use (default: article)
  -s NAME or FILENAME, --stylesheet NAME or FILENAME
the style sheet used to style the document elements 
(default: the template's default)
  -O FILENAME or DIRECTORY, --output FILENAME or DIRECTORY
write the PDF output to FILENAME or to an existing 
DIRECTORY with a filename derived
from the input filename (default: the current working 
directory)
  -p PAPER, --paper PAPER
the paper size to render to (default: the template's 
default)
  -i, --install-resources
automatically install missing resources (fonts, 
templates, style sheets) from PyPI
  --list-templates  list the installed document templates and exit
  --list-stylesheetslist the installed style sheets and exit
  --list-fonts [FILENAME]
list the installed fonts or, if FILENAME is given, 
write a PDF file displaying all
the fonts
  --list-formatslist the supported input formats and exit
  --list-options FRONTEND
list the options supported by the given frontend and 
exit
  --version show program's version number and exit
  --docsopen the online documentation in the default browser
```

broken in 3.10 b3:

```console
$ ./venv/bin/python --version --version
Python 3.10.0b3+ (heads/3.10:1b4addf3cb, Jun 18 2021, 17:21:48) [GCC 9.3.0]
$ ./venv/bin/python -m rinoh
Traceback (most recent call last):
  File "/home/asottile/workspace/cpython/prefix/lib/python3.10/runpy.py", line 
187, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/home/asottile/workspace/cpython/prefix/lib/python3.10/runpy.py", line 
146, in _get_module_details
return _get_module_details(pkg_main_name, error)
  File "/home/asottile/workspace/cpython/prefix/lib/python3.10/runpy.py", line 
110, in _get_module_details
__import__(pkg_name)
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/rinoh/__init__.py", 
line 41, in 
from . import resource
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/rinoh/resource.py", 
line 205, in 
from .template import DocumentTemplate
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/rinoh/template.py", 
line 42, in 
from .stylesheets import sphinx
  File 
"/tmp/rinohtype/venv/lib/python3.10/site-packages/rinoh/stylesheets/__init__.py",
 line 42, in 
.format(stylesheet.description, stylesheet))
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/rinoh/style.py", line 
670, in __str__
for name, entry_point in self.installed_resources:
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/rinoh/resource.py", 
line 54, in installed_resources
for entry_point in ilm.entry_points()[cls.entry_point_group]:
  File 
"/home/asottile/workspace/cpython/prefix/lib/python3.10/importlib/metadata/__init__.py",
 line 979, in entry_points
return SelectableGroups.load(eps).select(**params)
  File 
"/home/asottile/workspace/cpython/prefix/lib/python3.10/importlib/metadata/__init__.py",
 line 437, in load
ordered = sorted(eps, key=by_group)
  File 
"/home/asottile/workspace/cpython/prefix/lib/python3.10/importlib/metadata/__init__.py",
 line -1, in 
  File 
"/home/asottile/workspace/cpython/prefix/lib/python3.10/importlib/metadata/_itertools.py",
 line 16, in unique_everseen
k = key(element)
  File 
"/home/asottile/workspace/cpython/prefix/lib/python3.10/importlib/metadata/__init__.py",
 line 600, in _normalized_name
ret

[issue44297] Frame with -1 line number

[Anthony Sottile]

Anthony Sottile  added the comment:

here's the traceback pytest is trying to display and crashing:

```
Traceback (most recent call last):
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/_pytest/runner.py", 
line 311, in from_call
result: Optional[TResult] = func()
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/_pytest/runner.py", 
line 341, in 
call = CallInfo.from_call(lambda: list(collector.collect()), "collect")
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/_pytest/python.py", 
line 503, in collect
self._inject_setup_module_fixture()
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/_pytest/python.py", 
line 516, in _inject_setup_module_fixture
self.obj, ("setUpModule", "setup_module")
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/_pytest/python.py", 
line 291, in obj
self._obj = obj = self._getobj()
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/_pytest/python.py", 
line 500, in _getobj
return self._importtestmodule()
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/_pytest/python.py", 
line 578, in _importtestmodule
mod = import_path(self.fspath, mode=importmode)
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/_pytest/pathlib.py", 
line 524, in import_path
importlib.import_module(module_name)
  File 
"/home/asottile/workspace/cpython/prefix/lib/python3.10/importlib/__init__.py", 
line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
  File "", line 1050, in _gcd_import
  File "", line 1027, in _find_and_load
  File "", line 1006, in _find_and_load_unlocked
  File "", line 688, in _load_unlocked
  File 
"/tmp/rinohtype/venv/lib/python3.10/site-packages/_pytest/assertion/rewrite.py",
 line 170, in exec_module
exec(co, module.__dict__)
  File "/tmp/rinohtype/tests/test_attribute.py", line 11, in 
from rinoh.attribute import Attribute, Bool
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/rinoh/__init__.py", 
line 41, in 
from . import resource
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/rinoh/resource.py", 
line 205, in 
from .template import DocumentTemplate
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/rinoh/template.py", 
line 42, in 
from .stylesheets import sphinx
  File 
"/tmp/rinohtype/venv/lib/python3.10/site-packages/rinoh/stylesheets/__init__.py",
 line 42, in 
.format(stylesheet.description, stylesheet))
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/rinoh/style.py", line 
670, in __str__
for name, entry_point in self.installed_resources:
  File "/tmp/rinohtype/venv/lib/python3.10/site-packages/rinoh/resource.py", 
line 54, in installed_resources
for entry_point in ilm.entry_points()[cls.entry_point_group]:
  File 
"/home/asottile/workspace/cpython/prefix/lib/python3.10/importlib/metadata/__init__.py",
 line 979, in entry_points
return SelectableGroups.load(eps).select(**params)
  File 
"/home/asottile/workspace/cpython/prefix/lib/python3.10/importlib/metadata/__init__.py",
 line 437, in load
ordered = sorted(eps, key=by_group)
  File 
"/home/asottile/workspace/cpython/prefix/lib/python3.10/importlib/metadata/__init__.py",
 line -1, in 
  File 
"/home/asottile/workspace/cpython/prefix/lib/python3.10/importlib/metadata/_itertools.py",
 line 16, in unique_everseen
k = key(element)
  File 
"/home/asottile/workspace/cpython/prefix/lib/python3.10/importlib/metadata/__init__.py",
 line 600, in _normalized_name
return Prepared.normalize(self.name)
  File 
"/home/asottile/workspace/cpython/prefix/lib/python3.10/importlib/metadata/__init__.py",
 line 841, in normalize
return re.sub(r"[-_.]+", "-", name).lower().replace('-', '_')
  File "/home/asottile/workspace/cpython/prefix/lib/python3.10/re.py", line 
187, in sub
return _compile(pattern, flags).sub(repl, string, count)
TypeError: expected string or bytes-like object
```

--

___
Python tracker 
<https://bugs.python.org/issue44297>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44297] Frame with -1 line number

[Anthony Sottile]

Anthony Sottile  added the comment:

this appears to have regressed in 3.10 as well according to some reports on 
pytest: https://github.com/pytest-dev/pytest/pull/8227#issuecomment-864327090

--
nosy: +Anthony Sottile, pablogsal
versions: +Python 3.10

___
Python tracker 
<https://bugs.python.org/issue44297>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

[Anthony Sottile]

Anthony Sottile  added the comment:

they are directly coupled which is why I commented here

the api redesign forces O(N) lookups and O(N) constructions which directly 
impact performance causing the regression

--
resolution: fixed -> 
status: closed -> open

___
Python tracker 
<https://bugs.python.org/issue44246>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

[Anthony Sottile]

Anthony Sottile  added the comment:

here's the performance regressions, they affect any callers of 
`distributions()` and are even worse on callers of the new apis.

a call to distributions() is about 3x slower than in 3.9

here is the setup I am using:

virtualenv venv39 -ppython3.9
venv39/bin/pip install flake8 pytest twine pre-commit
virtualenv venv310 -ppython3.10
venv310/bin/pip install flake8 pytest twine pre-commit

to test just the `distributions()` call I'm using the following:

```console
$ venv39/bin/python -m timeit -n 20 -r 20 -s 'from importlib.metadata import 
entry_points' 'entry_points()'
20 loops, best of 20: 12.5 msec per loop
$ venv310/bin/python -m timeit -n 20 -r 20 -s 'from importlib.metadata import 
entry_points' 'entry_points()'
20 loops, best of 20: 36.7 msec per loop
```

this is a less-extreme example, many applications have more dependencies 
installed -- but even in this case this is adding ~24ms startup to any 
application using `entry_points()` -- and it gets worse

the return value of `entry_points()` alone isn't all that useful, next an 
application needs to retrieve its entry points.  let's start for the somewhat 
normal case of retrieving a single category of entry points:

```console
$ venv39/bin/python -m timeit -n 20 -r 20 -s 'from importlib.metadata import 
entry_points' 'entry_points()["flake8.extension"]'
20 loops, best of 20: 12.7 msec per loop
$ venv310/bin/python -m timeit -n 20 -r 20 -s 'from importlib.metadata import 
entry_points' 'entry_points(name="flake8.extension")'
20 loops, best of 20: 37.1 msec per loop
$ venv310/bin/python -m timeit -n 20 -r 20 -s 'from importlib.metadata import 
entry_points' 'entry_points().select(group="flake8.extension")'
20 loops, best of 20: 37.1 msec per loop
```

again, 3x slower and very real time to the end user (~24-25ms)

now let's show an example usage that something like flake8 uses where multiple 
groups are requested (this is common for apps and plugin systems which provide 
multiple distinct functionalities)

```console
$ venv39/bin/python -m timeit -n 20 -r 20 -s 'from importlib.metadata import 
entry_points' 'eps = entry_points(); eps["flake8.extension"]; 
eps["flake8.report"]'
 
20 loops, best of 20: 12.6 msec per loop
$ venv310/bin/python -m timeit -n 20 -r 20 -s 'from importlib.metadata import 
entry_points' 'eps = entry_points(); eps.select(group="flake8.extension"); 
eps.select(group="flake8.report")'
20 loops, best of 20: 38.2 msec per loop
```

also slower, but an additional ms per call to `.select(...)`


and it only gets worse with more and more packages installed

here's the versions I'm using to ensure they are up to date:

```console
$ venv39/bin/python --version --version
Python 3.9.5 (default, May 19 2021, 11:32:47) 
[GCC 9.3.0]
$ venv310/bin/python --version --version
Python 3.10.0b2 (default, Jun  2 2021, 00:22:18) [GCC 9.3.0]
```

--
resolution: fixed -> 
status: closed -> open

___
Python tracker 
<https://bugs.python.org/issue44246>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44295] self.assertDictContainsSubset warning is unhelpful

[Anthony Sottile]

Change by Anthony Sottile :


--
components: +Library (Lib)
type:  -> behavior

___
Python tracker 
<https://bugs.python.org/issue44295>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44307] date.today() is half as fast as datetime.now().date()

[Anthony Sottile]

Anthony Sottile  added the comment:

*terry.reedy oops typo!

--

___
Python tracker 
<https://bugs.python.org/issue44307>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44307] date.today() is half as fast as datetime.now().date()

[Anthony Sottile]

Anthony Sottile  added the comment:

@terry.reddy -- I believe your title change makes this more difficult to 
understand

--

___
Python tracker 
<https://bugs.python.org/issue44307>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44307] date.today() is 2x slower than datetime.now().date()

[Anthony Sottile]

New submission from Anthony Sottile :

```console
$ python3.10 -m timeit -s 'from datetime import datetime' 
'datetime.now().date()'
50 loops, best of 5: 708 nsec per loop
$ python3.10 -m timeit -s 'from datetime import date' 'date.today()'
20 loops, best of 5: 1.4 usec per loop
```

this surprised me so I dug into it -- it appears a fast path can be added to 
`date.today()` to make it faster than `datetime.date.now()` -- though I'm 
rather unfamiliar with the functions involved here

here is my ~sloppy patch attempting to add a fast path, I would need some 
guidance to improve it and get it accepted:

```diff
$ git diff -w
diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c
index 8ef2dad37a..7eaa5d1740 100644
--- a/Modules/_datetimemodule.c
+++ b/Modules/_datetimemodule.c
@@ -2875,6 +2875,17 @@ date_fromtimestamp(PyObject *cls, PyObject *obj)
 static PyObject *
 date_today(PyObject *cls, PyObject *dummy)
 {
+/* fast path, don't call fromtimestamp */
+if ((PyTypeObject *)cls == _DateType) {
+struct tm tm;
+time_t t;
+time();
+localtime_r(, );
+return new_date_ex(tm.tm_year + 1900,
+   tm.tm_mon + 1,
+   tm.tm_mday,
+   (PyTypeObject*)cls);
+} else {
 PyObject *time;
 PyObject *result;
 _Py_IDENTIFIER(fromtimestamp);
@@ -2893,6 +2904,7 @@ date_today(PyObject *cls, PyObject *dummy)
 Py_DECREF(time);
 return result;
 }
+}
 
 /*[clinic input]
 @classmethod
```

after this, `date.today()` is faster!

```console
$ ./python -m timeit -s 'from datetime import datetime' 'datetime.now().date()'
50 loops, best of 5: 764 nsec per loop
$ ./python -m timeit -s 'from datetime import date' 'date.today()'
50 loops, best of 5: 407 nsec per loop
```

\o/

--
components: Extension Modules
messages: 395061
nosy: Anthony Sottile
priority: normal
severity: normal
status: open
title: date.today() is 2x slower than datetime.now().date()
type: performance
versions: Python 3.11

___
Python tracker 
<https://bugs.python.org/issue44307>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44296] Should warnings.warn default to stacklevel=2?

[Anthony Sottile]

New submission from Anthony Sottile :

I have yet to come across a usecase where `stacklevel=1` makes sense -- usually 
it is more helpful to point at the calling code than the function which is 
itself warning

my proposal is to update the default for `stacklevel=` from `1` to `2`

an example bpo where this is relevant is bpo-44295

--
components: Library (Lib)
messages: 394960
nosy: Anthony Sottile
priority: normal
severity: normal
status: open
title: Should warnings.warn default to stacklevel=2?
type: behavior
versions: Python 3.11

___
Python tracker 
<https://bugs.python.org/issue44296>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44295] self.assertDictContainsSubset warning is unhelpful

[Anthony Sottile]

Change by Anthony Sottile :


--
keywords: +patch
pull_requests: +25093
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/26497

___
Python tracker 
<https://bugs.python.org/issue44295>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44295] self.assertDictContainsSubset warning is unhelpful

[Anthony Sottile]

New submission from Anthony Sottile :

it's missing stacklevel= -- mostly creating a bpo issue to link to

--
messages: 394953
nosy: Anthony Sottile
priority: normal
severity: normal
status: open
title: self.assertDictContainsSubset warning is unhelpful
versions: Python 3.11

___
Python tracker 
<https://bugs.python.org/issue44295>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44277] cpython forks are spammed with dependabot PRs

[Anthony Sottile]

New submission from Anthony Sottile :

for example: https://github.com/asottile/cpython/pull/1

--
messages: 394842
nosy: Anthony Sottile
priority: normal
severity: normal
status: open
title: cpython forks are spammed with dependabot PRs

___
Python tracker 
<https://bugs.python.org/issue44277>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

[Anthony Sottile]

Anthony Sottile  added the comment:

it does not, it restores apis but in a way which requires a huge performance 
hit to avoid deprecation warnings

it also still has the 2-500x performance regression I've stated above

--

___
Python tracker 
<https://bugs.python.org/issue44246>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

[Anthony Sottile]

Anthony Sottile  added the comment:

I have also shown that the performance is indeed not better in the nominal 
case, as demonstrated in the first case

--

___
Python tracker 
<https://bugs.python.org/issue44246>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

[Anthony Sottile]

Anthony Sottile  added the comment:

the toil is still present, the existing, good apis are deprecated and the new, 
bad apis are slow -- and the odd subclasses are still present

--

___
Python tracker 
<https://bugs.python.org/issue44246>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

[Anthony Sottile]

Anthony Sottile  added the comment:

oops, tiny typo in those code examples, they should say `group=` instead of 
`name=` -- though the performance is unchanged:

(first example)
```console
$ ./venv39/bin/python t.py
0.6641988754272461
$ ./venv310/bin/python t.py
1.3172023296356201
```

(second example)
```console
$ ./venv39/bin/python t.py
0.014233589172363281
$ ./venv310/bin/python t.py
8.910593271255493
```

--

___
Python tracker 
<https://bugs.python.org/issue44246>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

[Anthony Sottile]

Anthony Sottile  added the comment:

the `.select(...)` api is at least twice as slow as indexing as well:

setup:
```
virtualenv venv39 -p python3.9
venv39/bin/pip install flake8 pytest pytest-randomly
virtualenv venv39 -p python3.10
venv310/bin/pip install flake8 pytest pytest-randomly
```

```python
import importlib.metadata
import sys
import time


def f():
eps = importlib.metadata.entry_points()
if sys.version_info >= (3, 10):
eps.select(name='console_scripts')
else:
eps['console_scripts']


t0 = time.time()
for _ in range(100):
f()
t1 = time.time()
print(f'{t1-t0}')
```

```
$ ./venv39/bin/python t.py
0.687570333480835
$ ./venv310/bin/python t.py
1.3486714363098145
```

it is *way* worse when involving multiple entry points:

```python
import importlib.metadata
import sys
import time


# moved outside of the loop, already showed this component is slower
eps = importlib.metadata.entry_points()
def f():
# common for plugin systems to look up multiple entry points
for ep in ('console_scripts', 'flake8.extension', 'pytest11'):
if sys.version_info >= (3, 10):
eps.select(name=ep)
else:
eps[ep]


t0 = time.time()
for _ in range(1):
f()
t1 = time.time()
print(f'{t1-t0}')
```

```console
$ ./venv39/bin/python t.py
0.01629471778869629
$ ./venv310/bin/python t.py
8.569908380508423
```

--

___
Python tracker 
<https://bugs.python.org/issue44246>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

[Anthony Sottile]

Anthony Sottile  added the comment:

also https://github.com/miurahr/aqtinstall/issues/221

(this links to importlib-metadata tracker, not sure how you missed it)

--

___
Python tracker 
<https://bugs.python.org/issue44246>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

[Anthony Sottile]

Anthony Sottile  added the comment:

I also need `.sort(key=...)` for what it's worth, the error in this issue was 
just the first encountered

I think my only satisfactory outcome would be:

- the original api returns actual dicts
- the sub-api returns actual lists
- the new select is implemented as a separate *new* api without changing the 
existing api

--

___
Python tracker 
<https://bugs.python.org/issue44246>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

[Anthony Sottile]

Anthony Sottile  added the comment:

> Which types are sneaky and look like built-in types but do not act like them?

well for starters, there's the tuple subclass which pretends to be a dict.  but 
it violates substitutability for both `tuple` and `Mapping` so it's not useful 
in either contexts.  mypy complains about incorrect types in overrides for 
both.  the worst part of this is that the `__getitem__` moves from O(1) to O(N) 
(in some private code this makes importlib.metadata on 3.10 _10x slower than on 
3.9_).  next there's the `EntryPoints` tuple subclass which looks like a tuple 
but doesn't at all act like one (__getitem__ fails substitutability for 
example) -- this is an api break with 3.9 which returned a `list` (can't 
`.sort()` `.extend(...)`, etc. any more)

> Given that backports are available

I don't think this is appropriate.  re-introducing a backport brings in a tree 
of dependencies that have been shaky at best with backward compatibility.  in 
other words, using standard library importlib.metadata provides significantly 
improved compatibility and stability over the moving target backport (and its 
tree of dependencies, zipp being the one that breaks the most often from 
experience).  you'll notice I closed the flake8 PRs specifically because I 
didn't want to reintroduce the backport.  This backport also globally 
monkeypatches the import machinery breaking any other consumer as well -- in a 
tool as popular as flake8 I can't really make that global mutation decision for 
all of the other consumers.

> The typing issues

these were *trivially solved* by a dictionary comprehension using 
`entrypoint.name` -- it really did not need a full rework and break of the api 
to solve (could have just deprecated the `__iter__` which I actually suggested 
on the *original implementation* way back in 3.8)

> The new APIs are not only easier to describe with types

the types describing the new apis require *significant* `# type: ignore`s to 
pass mypy because they violate basic substitutability.  they also cannot be 
used in any of the contexts they were appropriate for in <3.10 (Dict[str, 
List[EntryPoint]] or List[Entrypoint] depending on the api).

> I'm not aware of a single breakage.

I'm sorry but you have to have realized from the many issues on 
importlib-metadata or the many issues linking to your deprecation issue that 
there is significant toil caused directly by your change.  cpython is meant to 
be a stable substrate to build upon, please do not force the community to 
shoulder the burden of your poor api decisions.

> even without requiring an updated importlib_metadata backport.

your proposed change introduced a different, unrelated package.  not without 
its own maintenance problems (an additional dependency that has to ~work 
indefinitely, a hack at best to support this breaking change)

> Given the amount of adoption already, reverting these changes is likely to 
> cause more disruption than moving forward with them.

I disagree, bigger things have been reverted (see: __future__.annotations)

>  So far, the only example presented above appears contrived and not an actual 
> concern

I promise you this is not a contrived case, if you look at your issue tracker 
it has been reported before and by others.  For every issue reported there's 
likely tens or hundreds of others which are not reported.

> It's common for Python minor releases to introduce undocumented changes that 
> are technically incompatible

I've seen this as a rationalization for intentional surprise breaking changes 
but I don't buy it.

Additionally your comments about (paraphrased) "the testsuite didn't 
demonstrate this usecase so I'm free to change it" are frankly a cop out -- 
your api returned a dict and a list, that's part of the public api -- changing 
that is a breaking change.

--

___
Python tracker 
<https://bugs.python.org/issue44246>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

[Anthony Sottile]

Anthony Sottile  added the comment:

personally I think they should be reverted -- they were not committed in spirit 
with the backwards compatibility policy: 
https://www.python.org/dev/peps/pep-0387/#making-incompatible-changes

- I don't think they were discussed thoroughly, and when opposition was 
presented it was not listened to thoroughly: 
https://github.com/python/importlib_metadata/pull/278
- the change significantly complicates importlib.metadata with lots of sneaky 
types (they all look like builtin types but do not act like them)
- it simultaneously introduces new apis and old apis which will both be around 
for extended periods of time but immediately moves to DeprecationWarning
- the new apis aren't remarkably better than the old apis -- the motivation was 
written as "typing issues" but without citing actual issues. in fact, the new 
items are significantly more complicated to type properly
- the change breaks many significantly important projects, from perusing 
related issues it's at the very least flake8, pandas, virtualenv, astropy, 
pytest, hypothesis -- and hundreds more from a quick github code search

--

___
Python tracker 
<https://bugs.python.org/issue44246>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

[Anthony Sottile]

Anthony Sottile  added the comment:

the "what's new" mentions nothing of this break, nor the aggressive deprecation 
warnings, nor the various shifted interfaces (such as this one): 
https://docs.python.org/3.10/whatsnew/3.10.html#importlib-metadata

--

___
Python tracker 
<https://bugs.python.org/issue44246>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44247] bpo stacktrace linkifier does not understand 3.10+

[Anthony Sottile]

New submission from Anthony Sottile :

for instance in this issue: https://bugs.python.org/issue44246

it links to 3.1/... instead of 3.10/...

--
messages: 394549
nosy: Anthony Sottile
priority: normal
severity: normal
status: open
title: bpo stacktrace linkifier does not understand 3.10+

___
Python tracker 
<https://bugs.python.org/issue44247>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

[Anthony Sottile]

New submission from Anthony Sottile :

this is breaking code that's unfortunately out of my control (vendor) -- also 
it looks really wrong

```python
import importlib.metadata
print('looks like a list:')
print(importlib.metadata.distribution('pip').entry_points)
print('first item:')
print(importlib.metadata.distribution('pip').entry_points[0])
```

output in 3.9:

```console
$ ./venv39/bin/python t.py
looks like a list:
[EntryPoint(name='pip', value='pip._internal.cli.main:main', 
group='console_scripts'), EntryPoint(name='pip3', 
value='pip._internal.cli.main:main', group='console_scripts'), 
EntryPoint(name='pip3.8', value='pip._internal.cli.main:main', 
group='console_scripts')]
first item:
EntryPoint(name='pip', value='pip._internal.cli.main:main', 
group='console_scripts')
```

```console
$ venv310/bin/python t.py
looks like a list:
(EntryPoint(name='pip', value='pip._internal.cli.main:main', 
group='console_scripts'), EntryPoint(name='pip3', 
value='pip._internal.cli.main:main', group='console_scripts'), 
EntryPoint(name='pip3.8', value='pip._internal.cli.main:main', 
group='console_scripts'))
first item:
Traceback (most recent call last):
  File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 217, in 
__getitem__
return next(iter(self.select(name=name)))
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/y/t.py", line 5, in 
print(importlib.metadata.distribution('pip').entry_points[0])
  File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 219, in 
__getitem__
raise KeyError(name)
KeyError: 0
```

--
messages: 394548
nosy: Anthony Sottile
priority: normal
severity: normal
status: open
title: 3.10 beta 1: breaking change in importlib.metadata entry points
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue44246>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44184] crash on windows invoking flake8

[Anthony Sottile]

Anthony Sottile  added the comment:

that version of flake8 uses multiprocessing (even for 1 file) -- would the ast 
objects be involved in that way? (pyflakes also makes reference cyles to handle 
"parent" relationships)

--

___
Python tracker 
<https://bugs.python.org/issue44184>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44184] crash on windows invoking flake8

[Anthony Sottile]

Anthony Sottile  added the comment:

would it maybe be helpful to bisect a history where the dataclasses / inspect 
import change is introduced earlier?  this would perhaps help pinpoint the 
other commit which is causing this?

--

___
Python tracker 
<https://bugs.python.org/issue44184>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44184] crash on windows invoking flake8

[Anthony Sottile]

Anthony Sottile  added the comment:

vstinner I showed the directions above, but here they are again:


venv\Scripts\pip install flake8==3.6.0
venv\Scripts\pip install -e . --no-deps  # ignore the conflict, but fix pyflakes

--

___
Python tracker 
<https://bugs.python.org/issue44184>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44184] crash on windows invoking flake8

[Anthony Sottile]

Anthony Sottile  added the comment:

ah sorry, the branch got squash-merged

this is the equivalent revision after the merge: 
f3b1b44bf3d2d5927004fa1c2fcf1ab2def816b9

--

___
Python tracker 
<https://bugs.python.org/issue44184>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44184] crash on windows invoking flake8

[Anthony Sottile]

Anthony Sottile  added the comment:

everything in this virtualenv is pure python so I don't think it's a faulty 
third party extension module

--

___
Python tracker 
<https://bugs.python.org/issue44184>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44184] crash on windows invoking flake8 under tox

[Anthony Sottile]

Anthony Sottile  added the comment:

I can reproduce it outside of tox using:

venv\Scripts\pip install flake8==3.6.0
venv\Scripts\pip install -e . --no-deps  # ignore the conflict, but fix pyflakes

C:\Users\asott\AppData\Local\Temp\y\pyflakes>venv\Scripts\flake8.exe setup.py

C:\Users\asott\AppData\Local\Temp\y\pyflakes>echo %ERRORLEVEL%
-1073741819

--

___
Python tracker 
<https://bugs.python.org/issue44184>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44184] crash on windows invoking flake8 under tox

[Anthony Sottile]

New submission from Anthony Sottile :

I installed python using the installers from python.org -- I originally 
reproduced this using github actions using pyflakes's testsuite

>ver

Microsoft Windows [Version 10.0.19041.985]

>venv\Scripts\python --version
Python 3.10.0b1

C:\Users\asott\AppData\Local\Temp\y\pyflakes>venv\Scripts\python --version 
--version
Python 3.10.0b1 (tags/v3.10.0b1:ba42175, May  3 2021, 20:22:30) [MSC v.1928 64 
bit (AMD64)]

to reproduce:

```
git clone https://github.com/pycqa/pyflakes
cd pyflakes
git checkout b02ba019e16f7c156ec63c2ea05c627a0fe86c48
```

# install tox somehow
```
C:\python310\python -m venv venv
venv\Scripts\pip install tox
```

here are the versions I have at that point:

```
>venv\Scripts\pip freeze --all
appdirs==1.4.4
colorama==0.4.4
distlib==0.3.1
filelock==3.0.12
packaging==20.9
pip==21.1.1
pluggy==0.13.1
py==1.10.0
pyparsing==2.4.7
setuptools==56.0.0
six==1.16.0
toml==0.10.2
tox==3.23.1
virtualenv==20.4.6
```

then run this a few times:

`venv\Scripts\tox -e py310`

even with `setenv = PYTHONFAULTHANDLER=1` I couldn't get a trace, though maybe 
that's a linux thing?

it occasionally crashes like this:

```
py310 run-test: commands[2] | flake8 pyflakes setup.py
ERROR: InvocationError for command 
'C:\Users\asott\AppData\Local\Temp\y\pyflakes\.tox\py310\Scripts\flake8.EXE' 
pyflakes setup.py (exited with code 3221225477)
```

from some googling:

> in hex is 0xc005 which is the Windows code for an access violation


I don't do much development on windows so I'm passing the torch to someone who 
knows more :)

--
messages: 393999
nosy: Anthony Sottile
priority: normal
severity: normal
status: open
title: crash on windows invoking flake8 under tox
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue44184>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44043] 3.10 b1 armhf Bus Error in hashlib test: test_gil

[Anthony Sottile]

Anthony Sottile  added the comment:

oddly enough, when I add `-X faulthandler` it passes

___

I did some research on the error message and it looks like the ubuntu 
maintainers have found the same thing and reported it here:

https://bugs.python.org/issue36445

this points at (intentional?) misaligned accesses being a problem on arm

looks like they've provided a patch as well here: 
https://bugs.python.org/issue36515

--

___
Python tracker 
<https://bugs.python.org/issue44043>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44043] 3.10 b1 armhf Bus Error in hashlib test: test_gil

[Anthony Sottile]

New submission from Anthony Sottile :

terribly sorry, I don't have much information to go off on this other than the 
build logs from deadsnakes.

I retried this build twice and it seems it consistently fails in this position:

https://launchpadlibrarian.net/537139233/buildlog_ubuntu-bionic-armhf.python3.10_3.10.0~b1-1+bionic2_BUILDING.txt.gz

The relevant logs from the build:

```
./python -m test --pgo --timeout=1200 || true
0:00:00 load avg: 2.37 Run tests sequentially (timeout: 20 min)
0:00:00 load avg: 2.37 [ 1/44] test_array
0:00:11 load avg: 2.23 [ 2/44] test_base64
0:00:15 load avg: 2.13 [ 3/44] test_binascii
0:00:16 load avg: 2.13 [ 4/44] test_binop
0:00:17 load avg: 2.13 [ 5/44] test_bisect
0:00:18 load avg: 2.13 [ 6/44] test_bytes
0:01:18 load avg: 1.41 [ 7/44] test_bz2 -- test_bytes passed in 1 min
0:01:25 load avg: 1.35 [ 8/44] test_cmath
0:01:28 load avg: 1.35 [ 9/44] test_codecs
0:01:52 load avg: 1.23 [10/44] test_collections
0:02:10 load avg: 1.16 [11/44] test_complex
0:02:16 load avg: 1.15 [12/44] test_dataclasses
0:02:23 load avg: 1.14 [13/44] test_datetime
0:03:18 load avg: 1.09 [14/44] test_decimal -- test_datetime passed in 54.9 sec
0:05:06 load avg: 1.01 [15/44] test_difflib -- test_decimal passed in 1 min 47 
sec
0:05:24 load avg: 1.01 [16/44] test_embed
0:05:27 load avg: 1.01 [17/44] test_float
0:05:32 load avg: 1.01 [18/44] test_fstring
0:05:43 load avg: 1.00 [19/44] test_functools
0:05:47 load avg: 1.00 [20/44] test_generators
0:05:51 load avg: 1.00 [21/44] test_hashlib
Fatal Python error: Bus error

Current thread 0xf7901220 (most recent call first):
  File "/<>/Lib/test/test_hashlib.py", line 842 in test_gil
  File "/<>/Lib/unittest/case.py", line 549 in _callTestMethod
  File "/<>/Lib/unittest/case.py", line 592 in run
  File "/<>/Lib/unittest/case.py", line 652 in __call__
  File "/<>/Lib/unittest/suite.py", line 122 in run
  File "/<>/Lib/unittest/suite.py", line 84 in __call__
  File "/<>/Lib/unittest/suite.py", line 122 in run
  File "/<>/Lib/unittest/suite.py", line 84 in __call__
  File "/<>/Lib/unittest/suite.py", line 122 in run
  File "/<>/Lib/unittest/suite.py", line 84 in __call__
  File "/<>/Lib/test/support/testresult.py", line 169 in run
  File "/<>/Lib/test/support/__init__.py", line 959 in _run_suite
  File "/<>/Lib/test/support/__init__.py", line 1082 in 
run_unittest
  File "/<>/Lib/test/libregrtest/runtest.py", line 210 in 
_test_module
  File "/<>/Lib/test/libregrtest/runtest.py", line 246 in 
_runtest_inner2
  File "/<>/Lib/test/libregrtest/runtest.py", line 282 in 
_runtest_inner
  File "/<>/Lib/test/libregrtest/runtest.py", line 154 in _runtest
  File "/<>/Lib/test/libregrtest/runtest.py", line 194 in runtest
  File "/<>/Lib/test/libregrtest/main.py", line 423 in 
run_tests_sequential
  File "/<>/Lib/test/libregrtest/main.py", line 521 in run_tests
  File "/<>/Lib/test/libregrtest/main.py", line 694 in _main
  File "/<>/Lib/test/libregrtest/main.py", line 641 in main
  File "/<>/Lib/test/libregrtest/main.py", line 719 in main
  File "/<>/Lib/test/__main__.py", line 2 in 
  File "/<>/Lib/runpy.py", line 86 in _run_code
  File "/<>/Lib/runpy.py", line 196 in _run_module_as_main
```

--
components: Build
messages: 392973
nosy: Anthony Sottile
priority: normal
severity: normal
status: open
title: 3.10 b1 armhf Bus Error in hashlib test: test_gil
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue44043>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

pytest 6.2.4 has been released!

[Anthony Sottile]

pytest-6.2.4
===

pytest 6.2.4 has just been released to PyPI.

This is a bug-fix release, being a drop-in replacement. To upgrade::

  pip install --upgrade pytest

The full changelog is available at
https://docs.pytest.org/en/stable/changelog.html.

Thanks to all of the contributors to this release:

* Anthony Sottile
* Bruno Oliveira
* Christian Maurer
* Florian Bruhin
* Ran Benita


Happy testing,
The pytest Development Team
___
Python-announce-list mailing list -- python-announce-list@python.org
To unsubscribe send an email to python-announce-list-le...@python.org
https://mail.python.org/mailman3/lists/python-announce-list.python.org/
Member address: arch...@mail-archive.com

[issue43933] Regression in python3.10 with traceback frame having lineno of -1

[Anthony Sottile]

Anthony Sottile  added the comment:

I've released pytest 6.2.4 for the other breakage (which is unrelated to this 
bpo -- it was the asname source position issue)

--

___
Python tracker 
<https://bugs.python.org/issue43933>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40465] Deprecate the optional *random* argument to random.shuffle()

[Anthony Sottile]

Change by Anthony Sottile :


--
nosy:  -Anthony Sottile

___
Python tracker 
<https://bugs.python.org/issue40465>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43933] Regression in python3.10 with traceback frame having lineno of -1

[Anthony Sottile]

Anthony Sottile  added the comment:

ah yeah that's the `asname` ast change -- if you use the unreleased main branch 
it has a fix for that

--

___
Python tracker 
<https://bugs.python.org/issue43933>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43994] change representation of match as / capture as `Name(..., ctx=Store())`

[Anthony Sottile]

Anthony Sottile  added the comment:

at least for static analysis of other python constructs it's very convenient to 
know from a `Name` node alone whether it's being used in a read or write 
context -- without this information an ast traversal needs to maintain more 
information about whether it's a read or write context

for pyflakes this is especially important as it needs to know what names are 
defined in scope (and referenced in scope) to produce diagnostic messages

for other tools like `dead` / `vulture` it's useful to identify introduced and 
referenced names similarly

the `as` in `with` does and the target for assignment expressions so I would 
expect the similar constructs in `match` to do so as well

`Name` nodes are also useful for better diagnostic messages as they contain 
positioning information, which isn't easily extracted from `MatchAs`, etc. -- 
if I recall correctly, the `asname` for imports was recently extended to add 
this information for the same purpose

--

___
Python tracker 
<https://bugs.python.org/issue43994>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43994] change representation of match as / capture as `Name(..., ctx=Store())`

[Anthony Sottile]

Anthony Sottile  added the comment:

and actually, now that I look close it would be useful for `MatchStar` and 
`MatchMapping` to also use a `Name(..., ctx=Store())` for their respective 
parameters as well

--

___
Python tracker 
<https://bugs.python.org/issue43994>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43994] change representation of match as / capture as `Name(..., ctx=Store())`

[Anthony Sottile]

Anthony Sottile  added the comment:

I'm suggesting instead of:

MatchAs(pattern=None, name='foo')

to have

MatchAs(pattern=None, name=Name('foo', ctx=Store()))

--

___
Python tracker 
<https://bugs.python.org/issue43994>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43994] change representation of match as / capture as `Name(..., ctx=Store())`

[Anthony Sottile]

New submission from Anthony Sottile :

I'm looking at adding support to `match` for pyflakes, and the first impression 
I have is that `MatchAs` is unnecessarily different from `Name` with 
`ctx=Store()`

if it were represented as the latter pyflakes would not require special 
handling of `match`, it would work the same as the current code

I suspect other static analysis tools would benefit from a change as well

--
messages: 392531
nosy: Anthony Sottile
priority: normal
severity: normal
status: open
title: change representation of match as / capture as `Name(..., ctx=Store())`
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue43994>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43933] Regression in python3.10 with traceback frame having lineno of -1

[Anthony Sottile]

Anthony Sottile  added the comment:

can confirm that pytest no longer crashes

--

___
Python tracker 
<https://bugs.python.org/issue43933>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38605] [typing] PEP 563: Postponed evaluation of annotations: enable it by default in Python 3.11

[Anthony Sottile]

Change by Anthony Sottile :


--
nosy: +Anthony Sottile
nosy_count: 14.0 -> 15.0
pull_requests: +24316
stage: resolved -> patch review
pull_request: https://github.com/python/cpython/pull/25596

___
Python tracker 
<https://bugs.python.org/issue38605>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43933] Regression in python3.10 with traceback frame having lineno of -1

[Anthony Sottile]

New submission from Anthony Sottile :

This is breaking pytest for failed assertions: 
https://github.com/pytest-dev/pytest/pull/8227

It also breaks the traceback in the output below

Here's a minimal example:

```python
class Boom:
def __enter__(self):
return self
def __exit__(self, *_):
raise AssertionError('boom!')


def main() -> int:
with Boom():
raise AssertionError('hi')


if __name__ == '__main__':
exit(main())
```

On python3.9 you get this:

```
Traceback (most recent call last):
  File "/home/asottile/workspace/cpython/t.py", line 10, in main
raise AssertionError('hi')
AssertionError: hi

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/asottile/workspace/cpython/t.py", line 14, in 
exit(main())
  File "/home/asottile/workspace/cpython/t.py", line 10, in main
raise AssertionError('hi')
  File "/home/asottile/workspace/cpython/t.py", line 5, in __exit__
raise AssertionError('boom!')
AssertionError: boom!
```

output in python3.10:

```
Traceback (most recent call last):
  File "/home/asottile/workspace/cpython/t.py", line 10, in main
raise AssertionError('hi')
AssertionError: hi

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/asottile/workspace/cpython/t.py", line 14, in 
exit(main())
  File "/home/asottile/workspace/cpython/t.py", line -1, in main
  File "/home/asottile/workspace/cpython/t.py", line 5, in __exit__
raise AssertionError('boom!')
AssertionError: boom!
```

Notice the second to last frame is now missing and invalid (line -1)

I bisected this and found this is the culprit:


https://github.com/python/cpython/commit/3bd6035b6baf1a7d51b7cc2c6bb2c81886236b67
https://github.com/python/cpython/pull/24202

```
3bd6035b6baf1a7d51b7cc2c6bb2c81886236b67 is the first bad commit
commit 3bd6035b6baf1a7d51b7cc2c6bb2c81886236b67
Author: Mark Shannon 
Date:   Wed Jan 13 12:05:43 2021 +

bpo-42908: Mark cleanup code at end of try-except and with artificial 
(#24202)

* Mark bytecodes at end of try-except as artificial.

* Make sure that the CFG is consistent throughout optimiization.

* Extend line-number propagation logic so that implicit returns after 
'try-except' or 'with' have the correct line numbers.

* Update importlib

 Lib/test/test_dis.py  |2 +-
 Lib/test/test_sys_settrace.py |   40 +
 Python/compile.c  |  135 +-
 Python/importlib.h| 3153 ++---
 Python/importlib_external.h   | 4489 -
 Python/importlib_zipimport.h  | 1013 +-
 6 files changed, 4473 insertions(+), 4359 deletions(-)
bisect run success
```

which appears to be due to bpo-42908

--
messages: 391795
nosy: Anthony Sottile, Mark.Shannon, pablogsal
priority: normal
severity: normal
status: open
title: Regression in python3.10 with traceback frame having lineno of -1
versions: Python 3.10, Python 3.11

___
Python tracker 
<https://bugs.python.org/issue43933>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43782] Failure to build from source on ppc64le on ubuntu xenial

[Anthony Sottile]

Anthony Sottile  added the comment:

a rebuild succeeded so I'm ok closing this :shrug:

--
stage:  -> resolved
status: pending -> closed

___
Python tracker 
<https://bugs.python.org/issue43782>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43833] Unexpected Parsing of Numeric Literals Concatenated with Boolean Operators

[Anthony Sottile]

Anthony Sottile  added the comment:

here's quite a few other cases as well -- I'd love for this to be clarified in 
PEP8 such that I can rationalize crafting a rule for it in `pycodestyle` -- 
https://github.com/PyCQA/pycodestyle/issues/371

--
nosy: +Anthony Sottile

___
Python tracker 
<https://bugs.python.org/issue43833>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43782] Failure to build from source on ppc64le on ubuntu xenial

[Anthony Sottile]

Anthony Sottile  added the comment:

hmmm strange, the only changes in 3.9.4 are a revert -- perhaps this is a 
flakey bug in gcc and not actionable

I've clicked rebuild on my build, maybe it'll magically fix it

plus xenial is almost end of lifed so I doubt anyone cares about this strange 
architecture on an old platform anyway

--

___
Python tracker 
<https://bugs.python.org/issue43782>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43782] Failure to build from source on ppc64le on ubuntu xenial

[Anthony Sottile]

New submission from Anthony Sottile :

I realize this is unlikely to be a helpful report, but something that changed 
between 3.9.3 and 3.9.4 has caused the build to break on (admittedly a strange 
platform) ppc64le


I attached the build log (zipped because otherwise it's too big ?)

The live URL is here: 
https://launchpadlibrarian.net/532585040/buildlog_ubuntu-xenial-ppc64el.python3.9_3.9.4-1+xenial1_BUILDING.txt.gz

Probably the most relevant part of the bug report is this bit, though I'm 
guessing so I don't really know what is useful and what is not.

```
Preprocessed source stored into /tmp/ccIkITd0.out file, please attach this to 
your bugreport.
=== BEGIN GCC DUMP ===
// Target: powerpc64le-linux-gnu
// Configured with: ../src/configure -v --with-pkgversion='Ubuntu/IBM 
5.4.0-6ubuntu1~16.04.12' --with-bugurl=file:///usr/share/doc/gcc-5/README.Bugs 
--enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr 
--program-suffix=-5 --enable-shared --enable-linker-build-id 
--libexecdir=/usr/lib --without-included-gettext --enable-threads=posix 
--libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu 
--enable-libstdcxx-debug --enable-libstdcxx-time=yes 
--with-default-libstdcxx-abi=new --enable-gnu-unique-object 
--disable-libquadmath --enable-plugin --with-system-zlib 
--disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo 
--with-java-home=/usr/lib/jvm/java-1.5.0-gcj-5-ppc64el/jre --enable-java-home 
--with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-5-ppc64el 
--with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-5-ppc64el 
--with-arch-directory=ppc64le --with-ecj-jar=/usr/share/java/eclipse-ecj.jar 
--enable-objc-gc --enable-secureplt --with-cpu=power8 --ena
 ble-targets=powerpcle-linux --disable-multilib --enable-multiarch 
--disable-werror --with-long-double-128 --enable-checking=release 
--build=powerpc64le-linux-gnu --host=powerpc64le-linux-gnu 
--target=powerpc64le-linux-gnu
// Thread model: posix
// gcc version 5.4.0 20160609 (Ubuntu/IBM 5.4.0-6ubuntu1~16.04.12) 
// 
// ../Python/ceval.c: In function 'is_tstate_valid':
// ../Python/ceval.c:5694:1: internal compiler error: Segmentation fault
//  }
//  ^
// Please submit a full bug report,
// with preprocessed source if appropriate.
// See  for instructions.
```

--
components: Build
files: buildlog.tgz
messages: 390580
nosy: Anthony Sottile
priority: normal
severity: normal
status: open
title: Failure to build from source on ppc64le on ubuntu xenial
versions: Python 3.9
Added file: https://bugs.python.org/file49946/buildlog.tgz

___
Python tracker 
<https://bugs.python.org/issue43782>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8978] "tarfile.ReadError: file could not be opened successfully" if compiled without zlib

[Anthony Sottile]

Anthony Sottile  added the comment:

I took a stab at improving the error message (see the linked PR)

$ ./python -c 'import tarfile; tarfile.open("Lib/test/testtar.tar.xz")'
Traceback (most recent call last):
  File "", line 1, in 
  File "/home/asottile/workspace/cpython/Lib/tarfile.py", line 1620, in open
raise ReadError(f"file could not be opened successfully:\n{error_msgs}")
tarfile.ReadError: file could not be opened successfully:
- method gz: ReadError('not a gzip file')
- method bz2: CompressionError('bz2 module is not available')
- method xz: CompressionError('lzma module is not available')
- method tar: ReadError('truncated header')

--

___
Python tracker 
<https://bugs.python.org/issue8978>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8978] "tarfile.ReadError: file could not be opened successfully" if compiled without zlib

[Anthony Sottile]

Change by Anthony Sottile :


--
nosy: +Anthony Sottile
nosy_count: 7.0 -> 8.0
pull_requests: +23611
pull_request: https://github.com/python/cpython/pull/24850

___
Python tracker 
<https://bugs.python.org/issue8978>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3353] make built-in tokenizer available via Python C API

[Anthony Sottile]

Anthony Sottile  added the comment:

I haven't looked into or thought about that yet, it might not be possible

It might also make sense to build new tokenize.py apis avoiding the 
`readline()` api -- I always found it painful to work with

--

___
Python tracker 
<https://bugs.python.org/issue3353>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3353] make built-in tokenizer available via Python C API

[Anthony Sottile]

Anthony Sottile  added the comment:

Either works for me, would you be able to point me to the starting bits as to 
how `_ast` becomes builtin?

--

___
Python tracker 
<https://bugs.python.org/issue3353>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3353] make built-in tokenizer available via Python C API

[Anthony Sottile]

Anthony Sottile  added the comment:

private api sounds fine too -- I thought it was necessary to implement the 
module (as it needs external linkage) but if it isn't then even better

--

___
Python tracker 
<https://bugs.python.org/issue3353>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3353] make built-in tokenizer available via Python C API

[Anthony Sottile]

Anthony Sottile  added the comment:

you already have that right now because the `tokenize` module is exposed. 
(except that every change to the tokenization requires it to be implemented 
once in C and once in python)

it's much more frustrating when the two differ as well

I don't think all the internals of the C tokenization need to be exposed, my 
main goals would be:

- expose enough information to reimplement Lib/tokenize.py
- replace Lib/tokenize.py with the C tokenizer

and the reasons would be:

- eliminate the (potential) drift and complexity between the two
- get a fast tokenizer


Unlike the AST, the tokenization changes much less frequently (last major 
addition I can remember is the `@` operator


We can hide almost all of the details of the tokenization behind an opaque 
struct and getter functions

--

___
Python tracker 
<https://bugs.python.org/issue3353>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3353] make built-in tokenizer available via Python C API

[Anthony Sottile]

Anthony Sottile  added the comment:

I'm interested in it because the `tokenize` module is painfully slow

--

___
Python tracker 
<https://bugs.python.org/issue3353>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3353] make built-in tokenizer available via Python C API

[Anthony Sottile]

Anthony Sottile  added the comment:

Serhiy Storchaka is this still blocked?  it's been a few years on either this 
or the linked issue and I'm reaching for this one :)

--
nosy: +Anthony Sottile

___
Python tracker 
<https://bugs.python.org/issue3353>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43013] IDLE: update code, mostly by cleanups of 2.x or 2to3 artifacts

[Anthony Sottile]

Anthony Sottile  added the comment:

> Anthony: removing 'set' from 'list(set(interable))' is wrong if 'set' were 
> added to remove duplicates.

It's not removed, it's changed to a set comprehension (which was added in 2.7 
and 3.0)

pyupgrade is very battle tested, having been run on pip, pytest, and many open 
source projects

--

___
Python tracker 
<https://bugs.python.org/issue43013>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43013] IDLE: update code, mostly by cleanups of 2.x or 2to3 artifacts

[Anthony Sottile]

Anthony Sottile  added the comment:

just noticed this PR -- you might be interested in pyupgrade which automates 
some of these things -- here's (for example) `git ls-files -- Lib/idlelib | 
grep '\.py$' | xargs pyupgrade --py36-plus

```diff
diff --git a/Lib/idlelib/calltip_w.py b/Lib/idlelib/calltip_w.py
index 1e0404aa49..278546064a 100644
--- a/Lib/idlelib/calltip_w.py
+++ b/Lib/idlelib/calltip_w.py
@@ -25,7 +25,7 @@ def __init__(self, text_widget):
 text_widget: a Text widget with code for which call-tips are desired
 """
 # Note: The Text widget will be accessible as self.anchor_widget
-super(CalltipWindow, self).__init__(text_widget)
+super().__init__(text_widget)
 
 self.label = self.text = None
 self.parenline = self.parencol = self.lastline = None
@@ -54,7 +54,7 @@ def position_window(self):
 return
 self.lastline = curline
 self.anchor_widget.see("insert")
-super(CalltipWindow, self).position_window()
+super().position_window()
 
 def showtip(self, text, parenleft, parenright):
 """Show the call-tip, bind events which will close it and reposition 
it.
@@ -73,7 +73,7 @@ def showtip(self, text, parenleft, parenright):
 self.parenline, self.parencol = map(
 int, self.anchor_widget.index(parenleft).split("."))
 
-super(CalltipWindow, self).showtip()
+super().showtip()
 
 self._bind_events()
 
@@ -143,7 +143,7 @@ def hidetip(self):
 # ValueError may be raised by MultiCall
 pass
 
-super(CalltipWindow, self).hidetip()
+super().hidetip()
 
 def _bind_events(self):
 """Bind event handlers."""
diff --git a/Lib/idlelib/debugger.py b/Lib/idlelib/debugger.py
index ccd03e46e1..53dba376af 100644
--- a/Lib/idlelib/debugger.py
+++ b/Lib/idlelib/debugger.py
@@ -49,9 +49,9 @@ def __frame2message(self, frame):
 filename = code.co_filename
 lineno = frame.f_lineno
 basename = os.path.basename(filename)
-message = "%s:%s" % (basename, lineno)
+message = f"{basename}:{lineno}"
 if code.co_name != "?":
-message = "%s: %s()" % (message, code.co_name)
+message = f"{message}: {code.co_name}()"
 return message
 
 
@@ -213,7 +213,7 @@ def interaction(self, message, frame, info=None):
 m1 = "%s" % str(type)
 if value is not None:
 try:
-m1 = "%s: %s" % (m1, str(value))
+m1 = "{}: {}".format(m1, str(value))
 except:
 pass
 bg = "yellow"
diff --git a/Lib/idlelib/debugobj.py b/Lib/idlelib/debugobj.py
index 5a4c997884..71d01c7070 100644
--- a/Lib/idlelib/debugobj.py
+++ b/Lib/idlelib/debugobj.py
@@ -87,7 +87,7 @@ def GetSubList(self):
 continue
 def setfunction(value, key=key, object=self.object):
 object[key] = value
-item = make_objecttreeitem("%r:" % (key,), value, setfunction)
+item = make_objecttreeitem(f"{key!r}:", value, setfunction)
 sublist.append(item)
 return sublist
 
diff --git a/Lib/idlelib/editor.py b/Lib/idlelib/editor.py
index a4d0c95362..5faebb5742 100644
--- a/Lib/idlelib/editor.py
+++ b/Lib/idlelib/editor.py
@@ -37,12 +37,12 @@
 def _sphinx_version():
 "Format sys.version_info to produce the Sphinx version string used to 
install the chm docs"
 major, minor, micro, level, serial = sys.version_info
-release = '%s%s' % (major, minor)
-release += '%s' % (micro,)
+release = f'{major}{minor}'
+release += f'{micro}'
 if level == 'candidate':
-release += 'rc%s' % (serial,)
+release += f'rc{serial}'
 elif level != 'final':
-release += '%s%s' % (level[0], serial)
+release += '{}{}'.format(level[0], serial)
 return release
 
 
@@ -939,7 +939,7 @@ def update_recent_files_list(self, new_file=None):
 rf_list = []
 file_path = self.recent_files_path
 if file_path and os.path.exists(file_path):
-with open(file_path, 'r',
+with open(file_path,
   encoding='utf_8', errors='replace') as rf_list_file:
 rf_list = rf_list_file.readlines()
 if new_file:
@@ -1446,7 +1446,7 @@ def newline_and_indent_event(self, event):
 else:
 self.reindent_to(y.compute_backslash_indent())
 else:
-assert 0, "bogus continuation type %r" % (c,)
+assert 0, f"bogus continuation type {c!r}"
 return "break"
 
 # This line starts a brand new 

[issue43014] tokenize spends a lot of time in `re.compile(...)`

[Anthony Sottile]

Anthony Sottile  added the comment:

attached out3.pstats / out3.svg which represent the optimization using 
lru_cache instead

--
Added file: https://bugs.python.org/file49764/out3.svg

___
Python tracker 
<https://bugs.python.org/issue43014>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43014] tokenize spends a lot of time in `re.compile(...)`

[Anthony Sottile]

Change by Anthony Sottile :


Added file: https://bugs.python.org/file49763/out3.pstats

___
Python tracker 
<https://bugs.python.org/issue43014>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43014] tokenize spends a lot of time in `re.compile(...)`

[Anthony Sottile]

Anthony Sottile  added the comment:

admittedly anecdotal but here's another data point in addition to the profiles 
attached

test.test_tokenize suite before:

$ ./python -m test.test_tokenize
..
--
Ran 78 tests in 77.148s

OK


test.test_tokenize suite after:

$ ./python -m test.test_tokenize
..
--
Ran 78 tests in 61.269s

OK

--

___
Python tracker 
<https://bugs.python.org/issue43014>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43014] tokenize spends a lot of time in `re.compile(...)`

[Anthony Sottile]

Change by Anthony Sottile :


--
keywords: +patch
pull_requests: +23130
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/24311

___
Python tracker 
<https://bugs.python.org/issue43014>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43014] tokenize spends a lot of time in `re.compile(...)`

[Anthony Sottile]

Change by Anthony Sottile :


Added file: https://bugs.python.org/file49762/out2.svg

___
Python tracker 
<https://bugs.python.org/issue43014>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43014] tokenize spends a lot of time in `re.compile(...)`

[Anthony Sottile]

Change by Anthony Sottile :


Added file: https://bugs.python.org/file49761/out2.pstats

___
Python tracker 
<https://bugs.python.org/issue43014>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43014] tokenize spends a lot of time in `re.compile(...)`

[Anthony Sottile]

Change by Anthony Sottile :


Added file: https://bugs.python.org/file49760/out.svg

___
Python tracker 
<https://bugs.python.org/issue43014>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43014] tokenize spends a lot of time in `re.compile(...)`

[Anthony Sottile]

New submission from Anthony Sottile :

I did some profiling (attached a few files here with svgs) of running this 
script:

```python
import io
import tokenize

# picked as the second longest file in cpython
with open('Lib/test/test_socket.py', 'rb') as f:
bio = io.BytesIO(f.read())


def main():
for _ in range(10):
bio.seek(0)
for _ in tokenize.tokenize(bio.readline):
pass

if __name__ == '__main__':
exit(main())
```


the first profile is before the optimization, the second is after the 
optimization

The optimization takes the execution from ~6300ms to ~4500ms on my machine 
(representing a 28% - 39% improvement depending on how you calculate it)

(I'll attach the pstats and svgs after creation, seems I can only attach one 
file at once)

--
components: Library (Lib)
files: out.pstats
messages: 385572
nosy: Anthony Sottile
priority: normal
severity: normal
status: open
title: tokenize spends a lot of time in `re.compile(...)`
type: performance
versions: Python 3.10, Python 3.9
Added file: https://bugs.python.org/file49759/out.pstats

___
Python tracker 
<https://bugs.python.org/issue43014>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36675] Doctest directives and comments missing from code samples

[Anthony Sottile]

Anthony Sottile  added the comment:

should the minimum sphinx version be bumped and/or this reverted: 
https://github.com/python/cpython/blob/f7f0ed59bcc41ed20674d4b2aa443d3b79e725f4/Doc/conf.py#L48

this change breaks, for example, sphinx 1.8.5 while building for ubuntu 20.04

The directive used here requires sphinx>=3.2.0

I notice some other attempts have been made to make the docs more compatible 
with sphinx 1.x in this release as well so there might be conflicting 
directions here: 
https://github.com/python/cpython/commit/b63a620014b67a6e63d10783149c41baaf59def8

--
nosy: +Anthony Sottile

___
Python tracker 
<https://bugs.python.org/issue36675>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42806] Incorrect offsets in new parser for f-string substitutions

[Anthony Sottile]

Anthony Sottile  added the comment:

by the way, here's a much smaller case which has a similar problem

```
x = (
f" {test(t)}"
)
```

--

___
Python tracker 
<https://bugs.python.org/issue42806>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42806] Incorrect offsets in new parser for f-string substitutions

[Anthony Sottile]

Anthony Sottile  added the comment:

I didn't initially, but now I have and it has the same bug in the current 
master: 3bf05327c2b25d42b92795d9d280288c22a0963d

--

___
Python tracker 
<https://bugs.python.org/issue42806>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42806] Incorrect offsets in new parser for f-string substitutions

[Anthony Sottile]

Anthony Sottile  added the comment:

+peg parser individuals to nosy

--
nosy: +gvanrossum, lys.nikolaou, pablogsal

___
Python tracker 
<https://bugs.python.org/issue42806>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42806] Incorrect offsets in new parser for f-string substitutions

[Anthony Sottile]

New submission from Anthony Sottile :

here's a sample file.  this is distilled down from an example which broke a 
code formatter here: https://github.com/asottile/add-trailing-comma/issues/106

def f():
x = (
'PERL_MM_OPT', (
f'INSTALL-BASE={shlex.quote(venv)} '
f'wat'
),
)


A diff from astpretty (https://github.com/asottile/astpretty) between the old 
and new parser:

```console
$ diff -u <(.tox/py39/bin/python -X oldparser -m astpretty perl.py) 
<(.tox/py39/bin/python -m astpretty perl.py)
--- /dev/fd/63  2021-01-01 15:58:09.108060968 -0800
+++ /dev/fd/62  2021-01-01 15:58:09.108060968 -0800
@@ -35,19 +35,19 @@
 end_col_offset=22,
 value=Call(
 lineno=4,
-col_offset=32,
+col_offset=16,
 end_lineno=4,
-end_col_offset=49,
+end_col_offset=33,
 func=Attribute(
 lineno=4,
-col_offset=32,
+col_offset=16,
 end_lineno=4,
-end_col_offset=43,
-value=Name(lineno=4, 
col_offset=32, end_lineno=4, end_col_offset=37, id='shlex', ctx=Load()),
+end_col_offset=27,
+value=Name(lineno=4, 
col_offset=16, end_lineno=4, end_col_offset=21, id='shlex', ctx=Load()),
 attr='quote',
 ctx=Load(),
 ),
-args=[Name(lineno=4, 
col_offset=44, end_lineno=4, end_col_offset=48, id='venv', ctx=Load())],
+args=[Name(lineno=4, 
col_offset=28, end_lineno=4, end_col_offset=32, id='venv', ctx=Load())],
 keywords=[],
 ),
 conversion=-1,
```

the old parser is correct here, and the new parser is wrong

notably it thinks that the call inside the f-string substitution is positioned 
at column 16 which is the position of the f-string start

this bug is also present in 3.10:

```console
$ venv/bin/python3.10 --version
Python 3.10.0a3
$ diff -u <(.tox/py39/bin/python -X oldparser -m astpretty perl.py) 
<(venv/bin/python3.10 -m astpretty perl.py)
--- /dev/fd/63  2021-01-01 15:59:54.59968 -0800
+++ /dev/fd/62  2021-01-01 15:59:54.59968 -0800
@@ -35,19 +35,19 @@
 end_col_offset=22,
 value=Call(
 lineno=4,
-col_offset=32,
+col_offset=16,
 end_lineno=4,
-end_col_offset=49,
+end_col_offset=33,
 func=Attribute(
 lineno=4,
-col_offset=32,
+col_offset=16,
 end_lineno=4,
-end_col_offset=43,
-value=Name(lineno=4, 
col_offset=32, end_lineno=4, end_col_offset=37, id='shlex', ctx=Load()),
+end_col_offset=27,
+value=Name(lineno=4, 
col_offset=16, end_lineno=4, end_col_offset=21, id='shlex', ctx=Load()),
 attr='quote',
 ctx=Load(),
 ),
-args=[Name(lineno=4, 
col_offset=44, end_lineno=4, end_col_offset=48, id='venv', ctx=Load())],
+args=[Name(lineno=4, 
col_offset=28, end_lineno=4, end_col_offset=32, id='venv', ctx=Load())],
 keywords=[],
 ),
 conversion=-1,
```

--
components: Interpreter Core
messages: 384208
nosy: Anthony Sottile
priority: normal
severity: normal

[issue42175] long lines from interactive stdin are truncated

[Anthony Sottile]

Anthony Sottile  added the comment:

indeed! that's it -- you learn something every day :)

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue42175>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42175] long lines from interactive stdin are truncated

[Anthony Sottile]

New submission from Anthony Sottile :

this was originally pointed out in a comment on my youtube video here: 
https://youtu.be/Xf_82stIbB8


here's a reproduction:

1. produce 5000 characters of output: python3 -c 'print("a"*5000)'
2. copy that to the clipboard
3. run this: python3 -c 'import sys; print(len(sys.stdin.read()))'
4. paste
5. press enter
6. ^D

I get the following:

$ python3 -c 'import sys; print(len(sys.stdin.read()))'
...
4096

but I expect the value to be 5001 (+1 for the \n)

--
messages: 379783
nosy: Anthony Sottile
priority: normal
severity: normal
status: open
title: long lines from interactive stdin are truncated
type: behavior
versions: Python 3.10, Python 3.8, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue42175>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40492] -m cProfile -o f.pstats with a script that does chdir() writes to the changed directory and not `.`

[Anthony Sottile]

Change by Anthony Sottile :


--
pull_requests: +21718
pull_request: https://github.com/python/cpython/pull/22755

___
Python tracker 
<https://bugs.python.org/issue40492>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40492] -m cProfile -o f.pstats with a script that does chdir() writes to the changed directory and not `.`

[Anthony Sottile]

Change by Anthony Sottile :


--
pull_requests: +21715
pull_request: https://github.com/python/cpython/pull/22752

___
Python tracker 
<https://bugs.python.org/issue40492>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41973] Docs: TypedDict is now of type function instead of class

[Anthony Sottile]

Anthony Sottile  added the comment:

Seems to be due to https://bugs.python.org/issue40187

--
nosy: +Anthony Sottile, serhiy.storchaka

___
Python tracker 
<https://bugs.python.org/issue41973>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2651] Strings passed to KeyError do not round trip

[Anthony Sottile]

Anthony Sottile  added the comment:

(I know this is old and closed but I've just run into the same thing with 
pwd.getpwnam in issue41767)


KeyError: "getpwnam(): name not found: 'test'"


would it be possible / make sense to extend `KeyError` with a 
named-only-argument which would preserve the raw string in this case?

something like (yeah the name is probably not great, I haven't had time to 
think much about this)

raise KeyError("getpwnam(): name not found 'test'", preserve_msg=True)

--
nosy: +Anthony Sottile

___
Python tracker 
<https://bugs.python.org/issue2651>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41767] KeyError exception is more difficult to read due to quoting

[Anthony Sottile]

Change by Anthony Sottile :


--
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue41767>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41767] KeyError exception is more difficult to read due to quoting

[Anthony Sottile]

New submission from Anthony Sottile :

For example:

>>> import pwd
>>> pwd.getpwnam("test")
Traceback (most recent call last):
  File "", line 1, in 
KeyError: "getpwnam(): name not found: 'test'"


An improvement would be something which doesn't have the extra set of quotes 
such as:

>>> import pwd
>>> pwd.getpwnam("test")
Traceback (most recent call last):
  File "", line 1, in 
KeyError: getpwnam(): name not found: 'test'



I understand this is due to the `KeyError` repr, I wonder if something could be 
done to improve this (maybe a subclass of KeyError? maybe a new  optional 
named-only argument to KeyError?)


Seems there are others which could be improved here as well:

>>> import unicodedata
>>> unicodedata.lookup('zzz')
Traceback (most recent call last):
  File "", line 1, in 
KeyError: "undefined character name 'zzz'"

--
messages: 376754
nosy: Anthony Sottile
priority: normal
severity: normal
status: open
title: KeyError exception is more difficult to read due to quoting
versions: Python 3.10, Python 3.8, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue41767>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20684] inspect.getfullargspec (etc) incorrectly follows __wrapped__ chains

[Anthony Sottile]

Change by Anthony Sottile :


--
nosy: +Anthony Sottile
nosy_count: 3.0 -> 4.0
pull_requests: +20834
pull_request: https://github.com/python/cpython/pull/21100

___
Python tracker 
<https://bugs.python.org/issue20684>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41260] datetime: strftime method takes different keyword argument: fmt (pure) or format (C)

[Anthony Sottile]

Anthony Sottile  added the comment:

awesome, I'm going to work through this with someone in my discord as a demo / 
mentorship opportunity -- hope that's ok!

--

___
Python tracker 
<https://bugs.python.org/issue41260>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41260] datetime: strftime method takes different keyword argument: fmt (pure) or format (C)

[Anthony Sottile]

New submission from Anthony Sottile :

C:

https://github.com/python/cpython/blob/8b33961e4bc4020d8b2d5b949ad9d5c669300e89/Modules/_datetimemodule.c#L3183

pure python:

https://github.com/python/cpython/blob/8b33961e4bc4020d8b2d5b949ad9d5c669300e89/Lib/datetime.py#L927


this makes it difficult to properly type in mypy:

https://github.com/python/typeshed/blob/209b6bb127f61fe173a60776e23883ac450cf1c8/stdlib/2and3/datetime.pyi#L55

and calling with `.strftime(fmt=...)` or `.strftime(format=...)` is inconsistent

(that said, it should _probably_ be a positional-only argument)

--
components: Extension Modules, Library (Lib)
messages: 373407
nosy: Anthony Sottile
priority: normal
severity: normal
status: open
title: datetime: strftime method takes different keyword argument: fmt (pure) 
or format (C)
type: behavior
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue41260>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com