[issue45054] json module should issue warning about duplicate keys

[Dong-hee Na]

Change by Dong-hee Na :


--
nosy: +corona10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45056] compiler: Unnecessary None in co_consts

[Dong-hee Na]

Change by Dong-hee Na :


--
nosy: +corona10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45020] Freeze all modules imported during startup.

[hai shi]

Change by hai shi :


--
nosy: +shihai1991

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44394] [security] CVE-2013-0340 "Billion Laughs" fixed in Expat >=2.4.0: Update vendored copy to expat 2.4.1

[Ned Deily]

Ned Deily  added the comment:


New changeset 79101b890ee021a901a8b6837a3a320d57adb725 by Łukasz Langa in 
branch '3.7':
[3.7] bpo-44394: Update libexpat copy to 2.4.1 (GH-26945) (GH-28042)
https://github.com/python/cpython/commit/79101b890ee021a901a8b6837a3a320d57adb725


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45021] Race condition in thread.py

[hai shi]

hai shi  added the comment:

Is it a defined behavior?
I got this sentence from pep-3148: Deadlock can occur when the callable 
associated with a Future waits on the results of another Future.

--
nosy: +shihai1991

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45052] WithProcessesTestSharedMemory.test_shared_memory_basics fails on Windows

[Eryk Sun]

Eryk Sun  added the comment:

> It may be a bug in the constructor of SharedMemory. It ignores 
> the size argument on Windows.

The `size` argument is always ignored when `create` is false, on all platforms, 
not that I understand the reason for it. The difference compared to POSIX is 
that the size of a mapping that's backed by the paging file is rounded up to a 
multiple of the page size in Windows (e.g. 4 KiB).

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45056] compiler: Unnecessary None in co_consts

[Inada Naoki]

Inada Naoki  added the comment:

I think LOAD_CONST None + RETURN_VALUE is added here. And it removed by 
optimize_cfg().

https://github.com/python/cpython/blob/793f55bde9b0299100c12ddb0e6949c6eb4d85e5/Python/compile.c#L7795-L7797

I don't know how easy to remove this unnecessary None.
But LOAD_COMMON_CONST idea can remove all None from co_consts.
https://github.com/iritkatriel/cpython/pull/27

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45056] compiler: Unnecessary None in co_consts

[Guido van Rossum]

Guido van Rossum  added the comment:

I wonder what the 3.10 compiler does different to cause this -- in 3.9 that 
extra None is not in co_consts.

(Note: Mark is on vacation for 2 weeks.)

--
nosy: +Mark.Shannon, gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44983] Wrong definition of a starred expression in the Language Reference

[Takuo Matsuoka]

Takuo Matsuoka  added the comment:

Thanks Éric Araujo, for the information. Actually, I sought for a
similar issue here in the tracker, but didn't find one filed, so this
report appears to be unique.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42414] unable to document fields of dataclass

[Laurie Opperman]

Laurie Opperman  added the comment:

No new mailing list thread, but there is one from January 2020: 
https://mail.python.org/archives/list/python-id...@python.org/thread/RHB6XFGFVM66AZTRKNTBAKFEVVEYUDD3/

--
nosy: +Epic_Wink

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36521] Consider removing docstrings from co_consts in code objects

[Inada Naoki]

Inada Naoki  added the comment:

This is WIP pull request. https://github.com/methane/cpython/pull/35
Some tests are failing because of bpo-36521.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45056] compiler: Unnecessary None in co_consts

[Inada Naoki]

New submission from Inada Naoki :

Python 3.10 compiler adds None to co_consts even when None is not used at all.

```
$ cat x1.py
def foo():
"docstring"
return 42

import dis
dis.dis(foo)
print(foo.__code__.co_consts)

$ python3.9 x1.py
  3   0 LOAD_CONST   1 (42)
  2 RETURN_VALUE
('docstring', 42)

$ python3.10 x1.py
  3   0 LOAD_CONST   1 (42)
  2 RETURN_VALUE
('docstring', 42, None)
```

--
components: Interpreter Core
keywords: 3.10regression
messages: 400683
nosy: methane
priority: normal
severity: normal
status: open
title: compiler: Unnecessary None in co_consts
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36521] Consider removing docstrings from co_consts in code objects

[Inada Naoki]

Inada Naoki  added the comment:

I grepped top 5000 downloaded packages and I can not find any real use of 
PyFunction_New(WithQualName).
So I don't know what is current workflow of PyFunction_New.

My current wip implementation adds new API (e.g. PyFunction_NewWithDoc()).
Old API keep using co_consts[0] for docstring for backward compatibility.

Adding code.co_doc is not free.

* All code objects have one additional pointer. So it eats memory.
* Unmarshal need to call `r_object()` for all code objects. So it increase 
startup time.

Note that code objects is not for only functions. Class, modules, lambdas, 
comprehensions uses code objects without docstring.
And if PEP 649 is accepted, even function annotations will use code objects. It 
will double the number of code objects in the highly annotated source files.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39218] Assertion failure when calling statistics.variance() on a float32 Numpy array

[Raymond Hettinger]

Change by Raymond Hettinger :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39218] Assertion failure when calling statistics.variance() on a float32 Numpy array

[Raymond Hettinger]

Raymond Hettinger  added the comment:


New changeset 793f55bde9b0299100c12ddb0e6949c6eb4d85e5 by Raymond Hettinger in 
branch 'main':
bpo-39218: Improve accuracy of variance calculation (GH-27960)
https://github.com/python/cpython/commit/793f55bde9b0299100c12ddb0e6949c6eb4d85e5


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45055] Fresh build on Windows fails the first time for zlib.c

[Guido van Rossum]

New submission from Guido van Rossum :

When I make a fresh checkout of the main branch on Windows and type 
"pcbuild\build" it starts downloading some distributions (e.g. sqlite) and then 
starts building. Fine. But at some point there's a whole bunch of errors that 
seem to come from building zlibmodule.c. Re-running pcbuild\build then 
downloads some extra thing and then everything builds to completion.

First set of downloads and selected logs:

Using py -3.9 (found 3.9 with py.exe)
Fetching external libraries...
Fetching bzip2-1.0.6...   
Fetching sqlite-3.35.5.0...
Fetching xz-5.2.2...
Fetching zlib-1.2.11...
Traceback (most recent call last):
  File "C:\Users\gvanrossum\deepfreeze\PCbuild\get_external.py", line 60, in 

main()
  File "C:\Users\gvanrossum\deepfreeze\PCbuild\get_external.py", line 56, in 
main
extract_zip(args.externals_dir, zip_path).replace(final_name)
  File 
"C:\Users\gvanrossum\AppData\Local\Programs\Python\Python39\lib\pathlib.py", 
line 1395, in 
replace
self._accessor.replace(self, target)
PermissionError: [WinError 5] Access is denied: 
'C:\\Users\\gvanrossum\\deepfreeze\\PCbuild\\..\\externals\\cpython-source-deps-zlib-1.2.11'
 -> 'C:\\Users\\gvanrossum\\deepfreeze\\PCbuild\\..\\externals\\zlib-1.2.11'
Fetching external binaries...
Fetching libffi-3.3.0...
Fetching openssl-bin-1.1.1l...
Fetching tcltk-8.6.11.0...
Finished.
Using "C:\Program Files (x86)\Microsoft Visual 
Studio\2019\Enterprise\MSBuild\Current\Bin\msbuild.exe" (found in the Visual 
Studio installation)
Using py -3.9 (found 3.9 with py.exe)

C:\Users\gvanrossum\deepfreeze>"C:\Program Files (x86)\Microsoft Visual 
Studio\2019\Enterprise\MSBuild\Current\Bin\msbuild.exe" 
"C:\Users\gvanrossum\deepfreeze\PCbuild\pcbuild.proj" /t:Build /m /nologo /v:m 
/clp:summary /p:Configuration=Release /p:Platform=x64 /p:IncludeExternals=true 
/p:IncludeCTypes=true /p:IncludeSSL=true /p:IncludeTkinter=true 
/p:UseTestMarker= /p:GIT="C:\Program Files\Git\cmd\git.exe"
  Killing any running python.exe instances...
  Regenerate pycore_ast.h pycore_ast_state.h Python-ast.c
  C:\Users\gvanrossum\deepfreeze\Python\Python-ast.c, 
C:\Users\gvanrossum\deepfreeze\Include\inte
  rnal\pycore_ast.h, 
C:\Users\gvanrossum\deepfreeze\Include\internal\pycore_ast_state.h regenerat
  ed.
  Regenerate opcode.h opcode_targets.h
  Include\opcode.h regenerated from Lib\opcode.py
  Jump table written into Python\opcode_targets.h
  Regenerate token-list.inc token.h token.c token.py
  Generated sources are up to date
  Getting build info from "C:\Program Files\Git\cmd\git.exe"
  Building heads/deepfreeze:044e8d866f deepfreeze
  _abc.c

...

Errors:

  Compiling...
  thread.c
  traceback.c
  zlibmodule.c
C:\Users\gvanrossum\deepfreeze\Modules\zlibmodule.c(10,10): fatal error C1083: 
Cannot open includ
e file: 'zlib.h': No such file or directory 
[C:\Users\gvanrossum\deepfreeze\PCbuild\pythoncore.vc
xproj]
  adler32.c
c1 : fatal error C1083: Cannot open source file: 
'C:\Users\gvanrossum\deepfreeze\externals\zlib-1 
.2.11\adler32.c': No such file or directory 
[C:\Users\gvanrossum\deepfreeze\PCbuild\pythoncore.vc 
xproj]
  compress.c
c1 : fatal error C1083: Cannot open source file: 
'C:\Users\gvanrossum\deepfreeze\externals\zlib-1
.2.11\compress.c': No such file or directory 
[C:\Users\gvanrossum\deepfreeze\PCbuild\pythoncore.v
cxproj]
  crc32.c
c1 : fatal error C1083: Cannot open source file: 
'C:\Users\gvanrossum\deepfreeze\externals\zlib-1
.2.11\crc32.c': No such file or directory 
[C:\Users\gvanrossum\deepfreeze\PCbuild\pythoncore.vcxp 
roj]
  deflate.c
c1 : fatal error C1083: Cannot open source file: 
'C:\Users\gvanrossum\deepfreeze\externals\zlib-1
.2.11\deflate.c': No such file or directory 
[C:\Users\gvanrossum\deepfreeze\PCbuild\pythoncore.vc 
xproj]
  infback.c
c1 : fatal error C1083: Cannot open source file: 
'C:\Users\gvanrossum\deepfreeze\externals\zlib-1
.2.11\infback.c': No such file or directory 
[C:\Users\gvanrossum\deepfreeze\PCbuild\pythoncore.vc 
xproj]
  inffast.c
c1 : fatal error C1083: Cannot open source file: 
'C:\Users\gvanrossum\deepfreeze\externals\zlib-1
.2.11\inffast.c': No such file or directory 
[C:\Users\gvanrossum\deepfreeze\PCbuild\pythoncore.vc 
xproj]
  inflate.c
c1 : fatal error C1083: Cannot open source file: 
'C:\Users\gvanrossum\deepfreeze\externals\zlib-1
.2.11\inflate.c': No such file or directory 
[C:\Users\gvanrossum\deepfreeze\PCbuild\pythoncore.vc 
xproj]
  inftrees.c
c1 : fatal error C1083: Cannot open source file: 
'C:\Users\gvanrossum\deepfreeze\externals\zlib-1
.2.11\inftrees.c': No such file or directory 
[C:\Users\gvanrossum\deepfreeze\PCbuild\pythoncore.v 
cxproj]
  trees.c
c1 : fatal error C1083: Cannot open source file: 
'C:\Users\gvanrossum\deepfreeze\externals\zlib-1
.2.11\trees.c': No such file or directory 
[C:\Users\gvanrossum\deepfreeze\PCbuild\pythoncore.vcxp 
roj]
  uncompr.c
c1 : fatal error C1083: Cannot open source file: 
'C:\Users\gvanrossum\deepfreeze\externals\zlib-1

[issue45054] json module should issue warning about duplicate keys

[Raymond Hettinger]

Change by Raymond Hettinger :


--
nosy: +bob.ippolito

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45019] Freezing modules has manual steps but could be automated.

[Eric Snow]

Eric Snow  added the comment:

I'm just waiting for the buildbots to finish.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45054] json module should issue warning about duplicate keys

[Kevin Mills]

New submission from Kevin Mills :

The json module will allow the following without complaint:

import json
d1 = {1: "fromstring", "1": "fromnumber"}
string = json.dumps(d1)
print(string)
d2 = json.loads(string)
print(d2)

And it prints:

{"1": "fromstring", "1": "fromnumber"}
{'1': 'fromnumber'}

This would be extremely confusing to anyone who doesn't already know that JSON 
keys have to be strings. Not only does `d1 != d2` (which the documentation does 
mention as a possibility after a round trip through JSON), but `len(d1) != 
len(d2)` and `d1['1'] != d2['1']`, even though '1' is in both.

I suggest that if json.dump or json.dumps notices that it is producing a JSON 
document with duplicate keys, it should issue a warning. Similarly, if 
json.load or json.loads notices that it is reading a JSON document with 
duplicate keys, it should also issue a warning.

--
components: Library (Lib)
messages: 400678
nosy: Zeturic
priority: normal
severity: normal
status: open
title: json module should issue warning about duplicate keys
type: enhancement
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45019] Freezing modules has manual steps but could be automated.

[Guido van Rossum]

Guido van Rossum  added the comment:

Is this ready to close?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45019] Freezing modules has manual steps but could be automated.

[Eric Snow]

Eric Snow  added the comment:


New changeset 044e8d866fdde3804bdb2282c7d23a8074de8f6f by Eric Snow in branch 
'main':
bpo-45019: Add a tool to generate list of modules to include for frozen modules 
(gh-27980)
https://github.com/python/cpython/commit/044e8d866fdde3804bdb2282c7d23a8074de8f6f


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21302] time.sleep (floatsleep()) should use clock_nanosleep() on Linux

[Roundup Robot]

Change by Roundup Robot :


--
keywords: +patch
nosy: +python-dev
nosy_count: 4.0 -> 5.0
pull_requests: +26521
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/28077

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45020] Freeze all modules imported during startup.

[Eric Snow]

Eric Snow  added the comment:

On Mon, Aug 30, 2021 at 2:22 PM Guido van Rossum  wrote:
> I propose to only opt in by default in **PGO builds**. After all what we're 
> doing is another extreme optimization.
>
> It should always be possible to opt in using some -X flag (e.g. to debug the 
> freeze import loader) and it should also always be possible to opt *out* (for 
> those cases where you want to edit an installed stdlib module in-place in 
> anger).
>
> I don't know how the -X mechanism works exactly but probably we could make 
> the spelling
>
>   python -X freeze=on|off
>
> with a dynamic default.

+1 to all that

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docs lie)

[Raymond Hettinger]

Raymond Hettinger  added the comment:

> So I really *do* want to see the ability of __float__ 
> to return a non-float eventually removed.

Note, the __str__ method on strings does not require an exact str.

class S:
def __str__(self):
return self

print(type(str(S('hello world'

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44756] In ./Doc, "make html" and "make build" should depend on "make venv"

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset 9ef1843892500e209afe5d916db26ecdbccd5e62 by Łukasz Langa in 
branch '3.9':
bpo-44756: Remove misleading NEWS entries of a change that was reverted before 
release (GH-28075)
https://github.com/python/cpython/commit/9ef1843892500e209afe5d916db26ecdbccd5e62


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44756] In ./Doc, "make html" and "make build" should depend on "make venv"

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset 5246dbc2a12bf8e64e18efee2fdce02a350bbf09 by Łukasz Langa in 
branch 'main':
bpo-44756: Remove misleading NEWS entries of a change that was reverted before 
release (GH-28075)
https://github.com/python/cpython/commit/5246dbc2a12bf8e64e18efee2fdce02a350bbf09


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39452] Improve the __main__ module documentation

[Guido van Rossum]

Guido van Rossum  added the comment:

Thanks, the rewrite is great!

I have one nit: did you consider which of these two idioms is better?

if __name__ == "__main__":
main()

vs.

if __name__ == "__main__":
sys.exit(main())

Your docs seem to promote the second, whereas I've usually preferred the 
former. Was this a considered choice on your part?

--
nosy: +gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37596] Reproducible pyc: frozenset is not serialized in a deterministic order

[Guido van Rossum]

Guido van Rossum  added the comment:

Thanks! This comes right in time, because we're working on freezing many more 
modules, and modules containing frozen sets didn't have a consistent frozen 
representation. Now they do!

(See issue45019, issue45020)

--
nosy: +gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45041] [sqlite3] simplify executescript()

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset e34bb409197d72711ae2c6197f9d8305533034d4 by Erlend Egeberg 
Aasland in branch 'main':
bpo-45041: Increase coverage for sqlite3.Cursor.executescript() (GH-28074)
https://github.com/python/cpython/commit/e34bb409197d72711ae2c6197f9d8305533034d4


--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24139] Use sqlite3 extended error codes

[Erlend E. Aasland]

Change by Erlend E. Aasland :


--
pull_requests: +26520
pull_request: https://github.com/python/cpython/pull/28076

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36521] Consider removing docstrings from co_consts in code objects

[Guido van Rossum]

Guido van Rossum  added the comment:

I think we shouldn't change *which* code object contains the docstring 
(changing anything about that is likely to disturb someone's workflow in a way 
that's hard to fix) -- only how PyFunction_New finds that docstring in the code 
object (if that breaks someone's workflow, the fix will be obvious).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44756] In ./Doc, "make html" and "make build" should depend on "make venv"

[Łukasz Langa]

Change by Łukasz Langa :


--
pull_requests: +26519
pull_request: https://github.com/python/cpython/pull/28075

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45041] [sqlite3] simplify executescript()

[Erlend E. Aasland]

Change by Erlend E. Aasland :


--
pull_requests: +26518
pull_request: https://github.com/python/cpython/pull/28074

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45020] Freeze all modules imported during startup.

[Guido van Rossum]

Guido van Rossum  added the comment:

FWIW, I'd be okay with doing the -X flag in a separate PR.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45020] Freeze all modules imported during startup.

[Guido van Rossum]

Guido van Rossum  added the comment:

> At this point, here are the open questions I'm seeing:

> + The editing-stdlib-.py-files problem: [...]

> + Compatibility: [...]

? + Penalty for too many frozen modules: [...]

> FWIW, I think the ideal mechanism for a dev build will be to opt in to using 
> frozen modules (instead of the source modules).  Otherwise it is too easy for 
> the unaware contributor to waste substantial time figuring out why their 
> changes are not getting used.

Agreed. I don't care much about people (even myself) editing installed modules. 
But I care a lot about people who do a git checkout and build from source 
editing modules and testing them without doing an install (Personally I never 
install what I build from source except to test the installation process.)

> Consequently, here's my order of preference for ignoring frozen modules:

> 1. use Py_DEBUG as an opt-out flag
>(if we think contributors are editing stdlib modules on a build without 
> Py_DEBUG then that isn't good enough)
> 2. automatically skip frozen modules if it's a dev build, with an explicit 
> configure flag to opt in to frozen modules.

I propose to only opt in by default in **PGO builds**. After all what we're 
doing is another extreme optimization.

It should always be possible to opt in using some -X flag (e.g. to debug the 
freeze import loader) and it should also always be possible to opt *out* (for 
those cases where you want to edit an installed stdlib module in-place in 
anger).

I don't know how the -X mechanism works exactly but probably we could make the 
spelling

  python -X freeze=on|off

with a dynamic default.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37179] asyncio loop.start_tls() provide support for TLS in TLS

[Jordan Borean]

Change by Jordan Borean :


--
pull_requests: +26517
pull_request: https://github.com/python/cpython/pull/28073

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45020] Freeze all modules imported during startup.

[Guido van Rossum]

Guido van Rossum  added the comment:

[Gregory Szorc]


> What do you set __file__ to? [...]

Exactly. I think it should not be set, just like it's not set for builtin 
modules.

> I have some observations about the implications of this. I typed up a long 
> comment but then realized someone would probably complain about me 
> distracting from the technical parts of this issue. Which forum is most 
> appropriate for this less technical commentary? (I want to lay out the case 
> for building out an official importer far more advanced than frozen importer.)

That seems to be something for python-dev. (There used to be an "import-sig" 
but the last mention I have from it is about that list shutting down for lack 
of traffic.) Or if you're still looking for more brainstorming you could try 
python-ideas first.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45020] Freeze all modules imported during startup.

[Guido van Rossum]

Guido van Rossum  added the comment:

> FWIW, I asked Ned Batchelder about this and he said this approach 
> ("fullcoverage" [1]) was added to support running coverage on the stdlib. 
> [...]

The docs you pointed out in [3] (where it talks about a *horrible hack you 
should never use" :-) should be amended with something explaining that "you 
need to comment out the line "" from frozen_module.py for this to 
work, else the frozen version of the encodings module will take priority over 
the imposter from "fullcoverage"."

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43124] [security] smtplib multiple CRLF injection

[Ned Deily]

Ned Deily  added the comment:


New changeset 29d97d17fb7adab3b0df9e178b73f70292d1cf64 by Miss Islington (bot) 
in branch '3.6':
[3.6] bpo-43124: Fix smtplib multiple CRLF injection (GH-25987) (GH-28038)
https://github.com/python/cpython/commit/29d97d17fb7adab3b0df9e178b73f70292d1cf64


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16379] SQLite error code not exposed to python

[Erlend E. Aasland]

Erlend E. Aasland  added the comment:

Thanks, Torsten for this nice suggestion, Daniel & Aviv for the initial 
patches, Gerhard & Ezio for helping improving the API, and Pablo, Asif, Hai 
Shi, & Michael for reviewing and merging!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43124] [security] smtplib multiple CRLF injection

[Ned Deily]

Ned Deily  added the comment:


New changeset d2cc04cd3024869101e894f73307944d98d187c8 by Miss Islington (bot) 
in branch '3.7':
[3.7] bpo-43124: Fix smtplib multiple CRLF injection (GH-25987) (GH-28037)
https://github.com/python/cpython/commit/d2cc04cd3024869101e894f73307944d98d187c8


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45020] Freeze all modules imported during startup.

[Gregory Szorc]

Gregory Szorc  added the comment:

> For stdlib modules it wouldn't be a big problem to set __file__ on
> frozen modules.
> Would that be enough to solve the problem?

What do you set __file__ to? Do you still materialize the .py[c] files on disk 
and set to that? (This would make the most sense.) If you support setting 
__file__ on a frozen module, how does this work at the C level? A stdlib module 
would want a different __file__ from a 3rd party module using the frozen module 
API. This would seemingly require enhancements to the frozen modules C API or 
some kind of hackery only available to stdlib frozen modules.

>> When I investigated freezing the standard library for PyOxidizer, I ran into 
>> a rash
>> of problems. The frozen importer doesn't behave like PathFinder. It doesn't
>> (didn't?) set some common module-level attributes
> This is mostly fixable for stdlib modules.  Which attributes would
> need to be added?  Are there other missing behaviors?

It was a few years ago and I can't recall specifics. I just remember that after 
encountering a couple unrelated limitations with the frozen importer I decided 
it was far too primitive to work as a general importer and decided I'd have to 
roll my own.

> Good point.  Supporting more of the FileLoader API on the frozen
> loader is something to look into, at least for stdlib modules.

>> I agree that we should shore up the frozen importer -- probably in a 
>> separate PR though.
>> (@Eric: do you think this is worth its own bpo issue?)
> Yeah.

I have some observations about the implications of this. I typed up a long 
comment but then realized someone would probably complain about me distracting 
from the technical parts of this issue. Which forum is most appropriate for 
this less technical commentary? (I want to lay out the case for building out an 
official importer far more advanced than frozen importer.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45052] WithProcessesTestSharedMemory.test_shared_memory_basics fails on Windows

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

It may be a bug in the constructor of SharedMemory. It ignores the size 
argument on Windows.

--
components: +Library (Lib), Windows
nosy: +davin, paul.moore, pitrou, serhiy.storchaka, steve.dower, tim.golden, 
zach.ware

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45001] Date parsing helpers in email module incorrectly raise IndexError for some malformed inputs

[Ned Deily]

Ned Deily  added the comment:

Thanks for the PR!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45001] Date parsing helpers in email module incorrectly raise IndexError for some malformed inputs

[Ned Deily]

Ned Deily  added the comment:


New changeset da9d6c554697414b1d275c8502e00a07c2ce06e6 by Miss Islington (bot) 
in branch '3.6':
bpo-45001: Make email date parsing more robust against malformed input 
(GH-27946) (GH-27976)
https://github.com/python/cpython/commit/da9d6c554697414b1d275c8502e00a07c2ce06e6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45053] MD5SumTests.test_checksum_fodder fails on Windows

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Test is failing because TESTFN contains now non-ASCII characters.

The path is written to stdout using the default stdout encoding on Windows 
(like cp1252), but test searches the path encoded with UTF-8. This test should 
fail also on other platforms with non-UTF-8 locale.

The simplest way to "fix" the test is using TESTFN_ASCII instead of TESTFN.

But there is also an issue in the script itself. It fails or produces a 
mojibake when the filesystem encoding and the stdout encoding do not match. 
There are similar issues in other scripts which output file names.

--
components: +Unicode
nosy: +ezio.melotti, serhiy.storchaka, vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45001] Date parsing helpers in email module incorrectly raise IndexError for some malformed inputs

[Ned Deily]

Ned Deily  added the comment:


New changeset e9b85afd7dc004460f6d914375ab67d617a8a7ff by Miss Islington (bot) 
in branch '3.7':
bpo-45001: Make email date parsing more robust against malformed input 
(GH-27946) (GH-27975)
https://github.com/python/cpython/commit/e9b85afd7dc004460f6d914375ab67d617a8a7ff


--
nosy: +ned.deily

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43398] [sqlite3] sqlite3.connect() segfaults if given a faulty Connection factory

[Erlend E. Aasland]

Erlend E. Aasland  added the comment:

> Is something missing here?

No; all good. Thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16379] SQLite error code not exposed to python

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset 86d8b465231473f850cc5e906013ba8581ddb503 by Erlend Egeberg 
Aasland in branch 'main':
bpo-16379: expose SQLite error codes and error names in `sqlite3` (GH-27786)
https://github.com/python/cpython/commit/86d8b465231473f850cc5e906013ba8581ddb503


--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34561] Replace list sorting merge_collapse()?

[Tim Peters]

Tim Peters  added the comment:

New runstack.py mostly adds comments about a surprise: the idea that 
length-adaptive ShiversSort eeks out better results than powersort appears 
nearly unique to the specific "0.80" cutoff used in the random-case generation 
code to pick between two uniform distributions. Change that cutoff, and 
powersort almost always does better.

So powersort remains the best known overall, although shivers4 remains 
competitive with it.

--
Added file: https://bugs.python.org/file50246/runstack.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14088] sys.executable generating canonical path

[Ned Deily]

Ned Deily  added the comment:

> I'm not against making Python even better: attempt to normalize the path ;-)

I would be very cautious about doing that. I'm pretty sure it would break some 
existing code.

--
nosy: +ned.deily

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45042] Many multiprocessing tests are silently skipped since 3.9

[Nikita Sobolev]

Nikita Sobolev  added the comment:

Two new issues created:
1. https://bugs.python.org/issue45053
2. https://bugs.python.org/issue45052

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45053] MD5SumTests.test_checksum_fodder fails on Windows

[Nikita Sobolev]

Nikita Sobolev  added the comment:

I would love to work on this issue :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45053] MD5SumTests.test_checksum_fodder fails on Windows

[Nikita Sobolev]

New submission from Nikita Sobolev :

While working on https://github.com/python/cpython/pull/28060 we've noticed 
that `test.test_tools.test_md5sum.MD5SumTests.test_checksum_fodder` fails on 
Windows:

```
==
FAIL: test_checksum_fodder (test.test_tools.test_md5sum.MD5SumTests)
--
Traceback (most recent call last):
  File "D:\a\cpython\cpython\lib\test\test_tools\test_md5sum.py", line 41, in 
test_checksum_fodder
self.assertIn(part.encode(), out)
^
AssertionError: b'@test_1772_tmp\xc3\xa6' not found in 
b'd38dae2eb1ab346a292ef6850f9e1a0d @test_1772_tmp\xe6\\md5sum.fodder\r\n'
```

For now it is ignored.

Related issue: https://bugs.python.org/issue45042

--
components: Tests
messages: 400648
nosy: sobolevn
priority: normal
severity: normal
status: open
title: MD5SumTests.test_checksum_fodder fails on Windows
type: behavior
versions: Python 3.10, Python 3.11, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45052] WithProcessesTestSharedMemory.test_shared_memory_basics fails on Windows

[Nikita Sobolev]

Nikita Sobolev  added the comment:

I would like to work on this issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45052] WithProcessesTestSharedMemory.test_shared_memory_basics fails on Windows

[Nikita Sobolev]

New submission from Nikita Sobolev :

While working on https://github.com/python/cpython/pull/28060 we've noticed 
that 
`test.test_multiprocessing_spawn.WithProcessesTestSharedMemory.test_shared_memory_basics`
 fails on Windows:

```
==
FAIL: test_shared_memory_basics 
(test.test_multiprocessing_spawn.WithProcessesTestSharedMemory)
--
Traceback (most recent call last):
  File "D:\a\cpython\cpython\lib\test\_test_multiprocessing.py", line 3794, in 
test_shared_memory_basics
self.assertEqual(sms.size, sms2.size)
^
AssertionError: 512 != 4096
```

For now it is ignored.

Related issue: https://bugs.python.org/issue45042

--
components: Tests
messages: 400646
nosy: sobolevn
priority: normal
severity: normal
status: open
title: WithProcessesTestSharedMemory.test_shared_memory_basics fails on Windows
type: behavior
versions: Python 3.10, Python 3.11, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44895] refleak test failure in test_exceptions

[Irit Katriel]

Irit Katriel  added the comment:

I think the leak happens when we invoke GC while the recursion limit is 
exceeded. It goes way if make this change:


def recurse_in_body_and_except():
try:
recurse_in_body_and_except()
except RecursionError as e:
gc.disable()
recurse_in_body_and_except()
gc.enable()


I also added a __del__ to B and saw it being called when recursion limit is 
small (though adding the __del__ makes the leak go away. Actually just adding 
"def f(): pass" is enough to make it go away).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43913] unittest module cleanup functions not run unless tearDownModule() is defined

[Łukasz Langa]

Łukasz Langa  added the comment:

Thanks for the fix, Serhiy, and Ryan for reporting the problem! ✨  ✨

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43398] [sqlite3] sqlite3.connect() segfaults if given a faulty Connection factory

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Is something missing here?

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43398] [sqlite3] sqlite3.connect() segfaults if given a faulty Connection factory

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset f62763d26755260c31c717fb396550e00eb6b2a0 by Erlend Egeberg 
Aasland in branch 'main':
bpo-43398: Add test for defect connection factories (GH-27966)
https://github.com/python/cpython/commit/f62763d26755260c31c717fb396550e00eb6b2a0


--
nosy: +pablogsal
status: pending -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43913] unittest module cleanup functions not run unless tearDownModule() is defined

[miss-islington]

miss-islington  added the comment:


New changeset d65fad04fad1a73b6bb17bcb08ca6f0a24376952 by Miss Islington (bot) 
in branch '3.10':
bpo-43913: Fix bugs in cleaning up classes and modules in unittest. (GH-28006)
https://github.com/python/cpython/commit/d65fad04fad1a73b6bb17bcb08ca6f0a24376952


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44019] operator.call/operator.__call__

[Antony Lee]

Antony Lee  added the comment:

> I'm not convinced that operator.caller() would be useful to me.

To be clear, as noted above, I have realized that the semantics I initially 
proposed (now known as "caller") are not particularly useful; the semantics I 
am proposing (and implementing in the linked PR) are `call(f, *args, **kwargs) 
== f(*args, **kwargs)`.

> I don't see how operator.caller() implements an existing "intrinsic operators 
> of Python".

Agreed; on the other hand function calling is much more intrinsic(?!)

> Can't you use functools.partial() for that?

How do you propose to do that?  Perhaps I am missing an easy solution...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45020] Freeze all modules imported during startup.

[Eric Snow]

Eric Snow  added the comment:

> * tricks to inject hooks ASAP (e.g. coverage.py swaps the encodings module) 
> may lose their entry point

FWIW, I asked Ned Batchelder about this and he said this approach 
("fullcoverage" [1]) was added to support running coverage on the stdlib.  It 
doesn't affect other users of coverage.py.  He didn't have any info on where 
this is used currently, though I'm pretty sure we do run coverage in CI.  
Furthermore, the devguide talks about running coverage.py on the stdlib and 
initially indicates that modules imported during startup are not covered. [2]  
However, it does have a section talking about using "fullcoverage" to cover 
startup modules.

I expect that any solution we make for contributors editing stdlib source files 
will resolve the potential issue for coverage.py.


[1] https://github.com/nedbat/coveragepy/tree/master/coverage/fullcoverage
[2] https://devguide.python.org/coverage/?highlight=coverage#measuring-coverage
[3] 
https://devguide.python.org/coverage/?highlight=coverage#coverage-results-for-modules-imported-early-on

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45020] Freeze all modules imported during startup.

[Eric Snow]

Eric Snow  added the comment:

At this point, here are the open questions I'm seeing:

+ The editing-stdlib-.py-files problem:
   * use a command-line flag to opt-out of frozen modules?
   * use a build flag to opt out (e.g. a configure flag or a new Py_NO_FROZEN 
or even Py_DEBUG)?
   * ignore frozen modules if it's a dev build?
   * (note: importlib._bootstrap and importlib._bootstrap_external must always 
be frozen, regardless of a flag)
   * accommodate users of an installed Python that sometimes edit stdlib 
modules while debugging code?
   * always emit a warning if a frozen module is ignored (in favor of the 
source module)?

+ Compatibility:
   * set __file__ (and __path__) on frozen modules?
   * store the hash of the source file for frozen modules (in the frozen .h 
files)?
   * support more of the FileLoader API on the frozen loader?

+ Penalty for too many frozen modules:
   * should we only freeze the UTF-8 encoding?
   * should we use something other than linear search for looking up frozen 
modules?

FWIW, I think the ideal mechanism for a dev build will be to opt in to using 
frozen modules (instead of the source modules).  Otherwise it is too easy for 
the unaware contributor to waste substantial time figuring out why their 
changes are not getting used.

Consequently, here's my order of preference for ignoring frozen modules:

1. use Py_DEBUG as an opt-out flag
   (if we think contributors are editing stdlib modules on a build without 
Py_DEBUG then that isn't good enough)
2. automatically skip frozen modules if it's a dev build, with an explicit 
configure flag to opt in to frozen modules

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43913] unittest module cleanup functions not run unless tearDownModule() is defined

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset 9827710a400848c9430ed364ed5d2d54f0926701 by Serhiy Storchaka in 
branch '3.9':
[3.9] bpo-43913: Fix bugs in cleaning up classes and modules in unittest. 
(GH-28006) (GH-28071)
https://github.com/python/cpython/commit/9827710a400848c9430ed364ed5d2d54f0926701


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45020] Freeze all modules imported during startup.

[Eric Snow]

Eric Snow  added the comment:

On Fri, Aug 27, 2021 at 11:14 PM Larry Hastings  wrote:
> [snip] On the other hand: if we made a viable tool that could consume some 
> arbitrary
> set of .py files and produce a C file, and said C file could then be compiled 
> into a
> shared library, end users could enjoy this speedup over the subset of the 
> standard
> library their program used, and perhaps even their own source tree(s).

Yeah, that would be interesting to investigate.

On Sat, Aug 28, 2021 at 5:17 AM Marc-Andre Lemburg
 wrote:
> Eric's approach, as I understand it, is pretty much what PyRun does.
> [further details]

It's reassuring to hear that the approach is known to be viable. :)

> In fact, you save quite a bit of disk space compared to a full Python 
> installation and
> additionally benefit from the memory mapping the OS does for sharing access 
> to the
> marshal'ed byte code between processes.

That's a good point.

> That said, some things don't work with such an approach, e.g. a few packages
> include additional data files which they expect to find on disk. Since those 
> are
> not available anymore, they fail.
>
> For PyRun I have patched some of those packages to include the data in form of
> Python modules instead, so that it gets frozen as well, e.g. the Python 
> grammar files.

For stdlib modules it wouldn't be a big problem to set __file__ on
frozen modules.
Would that be enough to solve the problem?

On Sat, Aug 28, 2021 at 5:41 PM Gregory Szorc  wrote:
> When I investigated freezing the standard library for PyOxidizer, I ran into 
> a rash
> of problems. The frozen importer doesn't behave like PathFinder. It doesn't
> (didn't?) set some common module-level attributes

This is mostly fixable for stdlib modules.  Which attributes would
need to be added?  Are there other missing behaviors?

> Also, when I last looked at the CPython source, the frozen importer performed
> a linear scan of its indexed C array performing strcmp() on each entry until 
> it
> found what it was looking for. So adding hundreds of modules could result in
> sufficient overhead and justify using a more efficient lookup algorithm.
> (PyOxidizer uses Rust's HashMap to index modules by name.)

Yeah, we noticed this too.  I wasn't sure it was something to worry
about at first because we're not freezing the entire stdlib.  We're
freezing on the order of 10, plus all the (80+) encoding modules.  I
figured we could look at an alternative to that linear search
afterward if it made sense.

> * Make sure you run unit tests against the frozen modules. If you don't do 
> this, subtle differences in how the different importers behave will lead to 
> problems.

We'll do what we already do with importlib: run the tests against both
the frozen and the source modules.  Thanks for the reminder to do this
though!

On Sat, Aug 28, 2021 at 5:53 PM Gregory Szorc  wrote:
> Oh, PyOxidizer also ran into more general issues with the frozen importer in 
> that
> it broke various importlib APIs. e.g. because the frozen importer only 
> supports
> bytecode, you can't use .__loader__.get_source() to obtain the source of a 
> module.
> This makes tracebacks more opaque and breaks legitimate API consumers relying
> on these importlib interfaces.

Good point.  Supporting more of the FileLoader API on the frozen
loader is something to look into, at least for stdlib modules.

> The fundamental limitations with the frozen importer are why I implemented my
> own meta path importer (implemented in pure Rust), which is more fully 
> featured,
> like the PathFinder importer that most people rely on today. That importer is
> available on PyPI (https://pypi.org/project/oxidized-importer/) and has its 
> own API
> to facilitate PyOxidizer-like functionality
> (https://pyoxidizer.readthedocs.io/en/stable/oxidized_importer.html) if anyone
> wants to experiment with it.

Awesome!  I'll take a look.

On Sat, Aug 28, 2021 at 6:14 PM Guido van Rossum  wrote:
> I agree that we should shore up the frozen importer -- probably in a separate 
> PR though.
> (@Eric: do you think this is worth its own bpo issue?)

Yeah.

-eric

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45051] wrote optimised async_test.py for async lib

[Zachary Ware]

New submission from Zachary Ware :

Ayush, your PRs are starting to look like spam.  Please slow down and ask for 
advice on what contributions would be welcome.  We have several resources 
available for this, such as the core-mentors...@python.org mailing list, the 
discuss.python.org discussion forum, not to mention the devguide which even 
includes a more complete list of places to get help: 
https://devguide.python.org/help/

I'm closing this PR and issue; please hold off on submitting more until you 
have followed the above advice.

--
nosy: +zach.ware
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45051] wrote optimised async_test.py for async lib

[Ayush Parikh]

Change by Ayush Parikh :


--
nosy: Ayushparikh-code
priority: normal
pull_requests: 26516
severity: normal
status: open
title: wrote optimised async_test.py for async lib
type: enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45025] Reliance on C bit fields in C API is undefined behavior

[Gregory Szorc]

Gregory Szorc  added the comment:

My use case for these low-level APIs is to write tests for low-level 
string/encoding handling in my custom use of the PyPreConfig and PyConfig 
structs. I wanted to verify that exact byte sequences were turned into specific 
representations inside of Python strings. This includes ensuring that certain 
byte sequences retain their appropriate "character" width in internal storage.

I know there are alternative ways of performing this testing. But testing 
against the actual data structure used internally by CPython seemed the most 
precise since it isolates problems to the "store in Python" side of the problem 
and not "what does Python do once the data is stored."

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43913] unittest module cleanup functions not run unless tearDownModule() is defined

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
pull_requests: +26515
pull_request: https://github.com/python/cpython/pull/28071

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43913] unittest module cleanup functions not run unless tearDownModule() is defined

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 9.0 -> 10.0
pull_requests: +26514
pull_request: https://github.com/python/cpython/pull/28070

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43913] unittest module cleanup functions not run unless tearDownModule() is defined

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset 08d9e597c8ef5a2b26375ac954fdf224f5d82c3c by Serhiy Storchaka in 
branch 'main':
bpo-43913: Fix bugs in cleaning up classes and modules in unittest. (GH-28006)
https://github.com/python/cpython/commit/08d9e597c8ef5a2b26375ac954fdf224f5d82c3c


--
nosy: +lukasz.langa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45050] created unittest file analyze_text.py

[Serhiy Storchaka]

New submission from Serhiy Storchaka :

It is unneeded.

--
nosy: +serhiy.storchaka
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45050] created unittest file analyze_text.py

[Ayush Parikh]

Change by Ayush Parikh :


--
nosy: Ayushparikh-code
priority: normal
pull_requests: 26513
severity: normal
status: open
title: created unittest file analyze_text.py
type: enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44887] test_input_tty hangs when run multiple times in the same process on macOS 10.15

[STINNER Victor]

STINNER Victor  added the comment:

Is it related to https://bugs.python.org/issue41034 ?

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45045] Optimize mapping patterns of structural pattern matching

[Dong-hee Na]

Dong-hee Na  added the comment:

> https://github.com/brandtbucher/patmaperformance

Nice benchmark suite, I will take a look :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45020] Freeze all modules imported during startup.

[Eric Snow]

Eric Snow  added the comment:

On Fri, Aug 27, 2021 at 6:29 PM Guido van Rossum  wrote:
> The plot thickens. By searching my extensive GMail archives for Jeethu Rao I 
> found
> an email from Sept. 14 to python-dev by Larry Hastings titled "Store startup 
> modules
> as C structures for 20%+ startup speed improvement?"

Thanks for finding that, Guido!

On Fri, Aug 27, 2021 at 6:37 PM Guido van Rossum  wrote:
> Either way it's a suboptimal experience for people contributing to those 
> modules. But
> we stand to gain a ~20% startup time improvement.

Agreed, and I think a solution shouldn't be too hard to reach.

On Fri, Aug 27, 2021 at 7:48 PM Larry Hastings  wrote:
> In experimenting with the prototype, I observed that simply calling stat() to 
> ensure
> the frozen .py file hadn't changed on disk lost us about half the performance 
> win
> from this approach.

Yeah, this is an approach others had suggested and I'd considered.  We
have other solutions available that don't have that penalty.

On Fri, Aug 27, 2021 at 8:08 PM Larry Hastings  wrote:
> There should be a boolean flag that enables/disables cached copies of .py 
> files from
> Lib/.  You should be able to turn it off with either an environment variable 
> or a
> command-line option, and when it's off it skips all the internal cached stuff 
> and
> uses the normal .py / .pyc machinery.
>
> With that in place, it'd be great to pre-cache all the .py files 
> automatically read in
> at startup.

Yeah, something along these lines should be good enough.

> [snip]
> But then I'm not sure this is a very good analogy--the workflow for making 
> Clinic
> changes is very different from people hacking on Lib/*.py.

Agreed.

On Fri, Aug 27, 2021 at 10:06 PM Guido van Rossum
 wrote:
> [snip]
> FWIW in my attempts to time this, it looks like the perf benefits of Eric's 
> approach are
> close to those of deep-freezing. And deep-freezing causes much more bloat of 
> the
> source code and of the resulting binary.

The question of freeze vs deep-freeze (i.e. is deep-freeze better
enough) is one we can discuss separately, and your point here is
probably the fundamental center of that discussion.  However, I don't
think it has a lot of bearing on the change proposed in this issue.

> [snip]
> I think the only solution here was hinted at in the python-dev thread from 
> 2018: have
> a command-line flag to turn it on or off (e.g. -X deepfreeze=1/0) and have a 
> policy for
> what the default for that flag should be (e.g. on by default in production 
> builds, off by
> default in developer builds -- anything that doesn't use 
> --enable-optimizations).

Agreed.

> [snip]
> it wasn't so clear that code objects should be immutable -- that realization 
> came later,
> when Greg Stein proposed making them ROM-able. That didn't work out, but the
> notion that code objects should be strictly mutable (to the python user, at 
> least)
> was born

This sounds like an interesting story.  Do you have any mailing list
links handy?  (Otherwise I can search the archives.)

> In fact, Eric's approach freezes everything in the encodings package, which 
> turns out
> to be a lot of files and a lot of code (lots of simple data tables expressed 
> in code), and
> I found that for basic startup time, it's best not to deep-freeze the 
> encodings module
> except for __init__.py, aliases.py and utf_8.py.

Yeah, this is something to consider.  FWIW, in my testing, dropping
encodings.* from the
list of frozen modules reduced the performance gains (from 20 ms to 21 ms).

-eric

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44019] operator.call/operator.__call__

[STINNER Victor]

STINNER Victor  added the comment:

> An actual use case I had for such an operator was collecting a bunch of 
> callables in a list and wanting to dispatch them to 
> concurrent.futures.Executor.map, i.e. something like 
> `executor.map(operator.call, funcs)` (to get the parallelized version of 
> `[func() for func in funcs]`).

Can't you use functools.partial() for that?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44019] operator.call/operator.__call__

[STINNER Victor]

STINNER Victor  added the comment:

Python 2.7 had apply(func, args, kwargs) which called func(*args, **kwargs).
https://docs.python.org/2.7/library/functions.html#apply

There is also functools.partial(func, *args, **kwargs)(*args2, **kwargs2) which 
calls func(*args, *args2, **kwargs, **kwargs2).
https://docs.python.org/dev/library/functools.html#functools.partial

operator.methodcaller(name, /, *args, **kwargs)(obj) calls getattr(obj, 
name)(*args, **kwargs).
https://docs.python.org/dev/library/operator.html#operator.methodcaller

I'm not convinced that operator.caller() would be useful to me. Why do you 
consider that it belongs to the stdlib? It is a common pattern? Did you see in 
this pattern in the current stdlib?

Can't you easily implement such helper function in a few lines of Python?

operator documentation says: "The operator module exports a set of efficient 
functions corresponding to the intrinsic operators of Python". I don't see how 
operator.caller() implements an existing "intrinsic operators of Python".

methodcaller() can be implemented in 4 lines of Python, as shown in its 
documentation:
---
def methodcaller(name, /, *args, **kwargs):
def caller(obj):
return getattr(obj, name)(*args, **kwargs)
return caller
---

--
nosy: +brett.cannon, rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45045] Optimize mapping patterns of structural pattern matching

[Brandt Bucher]

Brandt Bucher  added the comment:

I'm also in the process of creating some pattern matching benchmarks. You might 
find them useful for benching optimizations like this in the future:

https://github.com/brandtbucher/patmaperformance

In particular, I'm curious to see the impact of this change on "patma_holdem". 
I bet it's quite good.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45045] Optimize mapping patterns of structural pattern matching

[Brandt Bucher]

Brandt Bucher  added the comment:

Thanks, this is awesome.

FYI, there's probably more low-hanging fruit like this scattered about the 
pattern matching implementation. It's all very new code, mostly written by one 
person ;).

(I wouldn't worry too much about the pattern compiler, though... I'm currently 
finishing up a total rewrite of that!)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45025] Reliance on C bit fields in C API is undefined behavior

[STINNER Victor]

STINNER Victor  added the comment:

> PyUnicode_KIND does *not* expose the implementation details to the programmer.

PyUnicode_KIND() is very specific to the exact PEP 393 implementation. 
Documentation of this field:
---
/* Character size:

   - PyUnicode_WCHAR_KIND (0):

 * character type = wchar_t (16 or 32 bits, depending on the
   platform)

   - PyUnicode_1BYTE_KIND (1):

 * character type = Py_UCS1 (8 bits, unsigned)
 * all characters are in the range U+-U+00FF (latin1)
 * if ascii is set, all characters are in the range U+-U+007F
   (ASCII), otherwise at least one character is in the range
   U+0080-U+00FF

   - PyUnicode_2BYTE_KIND (2):

 * character type = Py_UCS2 (16 bits, unsigned)
 * all characters are in the range U+-U+ (BMP)
 * at least one character is in the range U+0100-U+

   - PyUnicode_4BYTE_KIND (4):

 * character type = Py_UCS4 (32 bits, unsigned)
 * all characters are in the range U+-U+10
 * at least one character is in the range U+1-U+10
 */
unsigned int kind:3;
---

I don't think that PyUnicode_KIND() makes sense if CPython uses UTF-8 tomorrow.


> If the internal representation os strings is switched to use masks and shifts 
> instead of bitfields, PyUnicode_KIND (and others) can be adapted to the new 
> details without breaking API compatibility.

PyUnicode_KIND() was exposed in the *public* C API because unicodeobject.h 
provides functions as macros for best performances, and these macros use 
PyUnicode_KIND() internally.

Macros like PyUnicode_READ(kind, data, index) are also designed for best 
performances with the exact PEP 393 implementation.

The public C API should only contain PyUnicode_READ_CHAR(unicode, index): this 
macro doesn't use "kind" or "data" which are (again) specific to the PEP 393.

In the CPython implementation, we should use the most efficient code, it's fine 
to use macros accessing directly structures.

But for the public C API, I would recommend to only provide abstractions, even 
if there are a little bit slower.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25867] os.stat raises exception when using unicode and no locale is set

[STINNER Victor]

STINNER Victor  added the comment:

> It's doing this now, so seems like it has been fixed: % env -i ./python.exe 
> (...)

Right. It's correct to close the issue.

The PEP 540 added a UTF-8 Mode. This mode is enabled if Python is started with 
the "C" or "POSIX" locale (LC_CTYPE category). If the UTF-8 Mode is enabled, 
Python uses UTF-8 for its "filesystem" encoding:

* https://docs.python.org/dev/library/os.html#python-utf-8-mode
* 
https://docs.python.org/dev/glossary.html#term-filesystem-encoding-and-error-handler

Moreover, the PEP 538 also tries to use a UTF-8 variable of "C" and "POSIX" 
locales, which also fix this issue.

I documented how Python configures its "filesystem encoding" at:
https://docs.python.org/dev/c-api/init_config.html#c.PyConfig.filesystem_encoding

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45025] Reliance on C bit fields in C API is undefined behavior

[Petr Viktorin]

Petr Viktorin  added the comment:

PyUnicode_KIND does *not* expose the implementation details to the programmer.

If the internal representation os strings is switched to use masks and shifts 
instead of bitfields, PyUnicode_KIND (and others) can be adapted to the new 
details without breaking API compatibility.
And that switch would fix this issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45025] Reliance on C bit fields in C API is undefined behavior

[STINNER Victor]

STINNER Victor  added the comment:

> In order to avoid undefined behavior, Python's C API should avoid all use of 
> bit fields.

See also the PEP 620. IMO more generally, the C API should not expose 
structures, but provide ways to access it through getter and setter functions.

See bpo-40120 "Undefined C behavior going beyond end of struct via a [1] 
arrays" which is a similar issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37596] Reproducible pyc: frozenset is not serialized in a deterministic order

[Brandt Bucher]

Change by Brandt Bucher :


--
pull_requests: +26512
pull_request: https://github.com/python/cpython/pull/28068

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45025] Reliance on C bit fields in C API is undefined behavior

[STINNER Victor]

STINNER Victor  added the comment:

> The macro PyUnicode_KIND is part of the documented public C API.

IMO it was a mistake to expose it as part of the public C API. This is an 
implementation detail which should not be exposed. The C API should not expose 
*directly* how characters are stored in memory, but provide an abstract way to 
read and write Unicode characters.

The PEP 393 implementation broke the old C API in many ways because it exposed 
too many implementation details. Sadly, the new C API is... not better :-(

If tomorrow, CPython is modified to use UTF-8 internally (as PyPy does), the C 
API will likely be broken *again* in many (new funny) ways.

11 years after the PEP 393 (Python 3.3), we only start fixing the old C API :-( 
The work will be completed in 2 or 3 Python releases (Python 3.12 or 3.13):

* https://www.python.org/dev/peps/pep-0623/
* https://www.python.org/dev/peps/pep-0624/

The C API for Unicode strings is causing a lot of issues in PyPy which uses 
UTF-8 internally. C extensions can fail to build on PyPy if they use functions 
(macros) like PyUnicode_KIND().

--
nosy: +methane, serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14088] sys.executable generating canonical path

[STINNER Victor]

STINNER Victor  added the comment:

> The path is not required to be normalized.

I'm talking about the current implementation.

I'm not against making Python even better: attempt to normalize the path ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14088] sys.executable generating canonical path

[STINNER Victor]

STINNER Victor  added the comment:

test_sys:

def test_executable(self):
# sys.executable should be absolute
self.assertEqual(os.path.abspath(sys.executable), sys.executable)

IMO the test is wrong. It must test: 
test.assertTrue(os.path.isabs(sys.executable)).

The path is not required to be normalized.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45025] Reliance on C bit fields in C API is undefined behavior

[Petr Viktorin]

Petr Viktorin  added the comment:

The macro PyUnicode_KIND is part of the documented public C API. It accesses 
the bit field "state.kind" directly.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[STINNER Victor]

STINNER Victor  added the comment:

For the very specific case of os.environ.clear(), the C function clearenv() 
could be used if available.

While clearenv() is available in the glibc, it's not a POSIX function.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45025] Reliance on C bit fields in C API is undefined behavior

[STINNER Victor]

STINNER Victor  added the comment:

> At least the PyASCIIObject struct in Include/cpython/unicodeobject.h uses bit 
> fields. Various preprocessor macros like PyUnicode_IS_ASCII() and 
> PyUnicode_KIND() access this struct's bit field.

What is your use case? Which functions do you need?

You should not access directly the PyASCIIObject structure. Python provides 
many functions to access the content of a Unicode string object.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17120] Mishandled _POSIX_C_SOURCE and _XOPEN_SOURCE in pyconfig.h

[STINNER Victor]

STINNER Victor  added the comment:

On my Fedora 34, running ./configure creates pyconfig.h with these values:

$ grep -E '_POSIX_C_SOURCE|_XOPEN_SOURCE' -B1 pyconfig.h
/* Define to activate features from IEEE Stds 1003.1-2008 */
#define _POSIX_C_SOURCE 200809L
--
/* Define to the level of X/Open that your system supports */
#define _XOPEN_SOURCE 700
--
/* Define to activate Unix95-and-earlier features */
#define _XOPEN_SOURCE_EXTENDED 1


Relevant lines in configure.ac:
---
if test $define_xopen_source = yes
then
  # X/Open 7, incorporating POSIX.1-2008
  AC_DEFINE(_XOPEN_SOURCE, 700,
Define to the level of X/Open that your system supports)

  # On Tru64 Unix 4.0F, defining _XOPEN_SOURCE also requires
  # definition of _XOPEN_SOURCE_EXTENDED and _POSIX_C_SOURCE, or else
  # several APIs are not declared. Since this is also needed in some
  # cases for HP-UX, we define it globally.
  AC_DEFINE(_XOPEN_SOURCE_EXTENDED, 1,
Define to activate Unix95-and-earlier features)

  AC_DEFINE(_POSIX_C_SOURCE, 200809L, Define to activate features from IEEE 
Stds 1003.1-2008)
fi
---

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[STINNER Victor]

STINNER Victor  added the comment:

Attached set_unset_env.c program calls putenv("=hello world") and then 
unsetenv("").

On my Fedora 34 with glibc-2.33-20.fc34.x86_64, putenv() succeed, but 
unsetenv() fails.
---
$ gcc set_unset_env.c -g -o set_unset_env && ./set_unset_env
putenv("=hello world") -> hello world
ERROR: unsetenv("") failed: [error 22] Invalid argument
---

By the way, getenv() fails to find an environment variable if its name is 
empty: I reimplemented getenv() using the 'environ' variable for my test.

--
Added file: https://bugs.python.org/file50245/set_unset_env.c

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45042] Many multiprocessing tests are silently skipped since 3.9

[Łukasz Langa]

Łukasz Langa  added the comment:

FYI, there seem to be two Windows-specific regressions since the tests were 
unintentionally disabled, namely test_shared_memory_basics and 
test_checksum_fodder.

Following Serhiy's advice, I elect to have those tests skipped for now on 
Windows and fix them through separate issues. We will be releasing 3.9.7 
according to schedule today.

--
nosy: +lukasz.langa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[Irit Katriel]

Irit Katriel  added the comment:

I see, so intercepting the assignment is not enough. Reopening.

--
resolution: out of date -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[STINNER Victor]

STINNER Victor  added the comment:

By the way:
---
The env command from GNU coreutils supports setting the environment variable 
with an empty name but not unsetting it. That's a bug.

$ env '=wibble' env |grep wibble 
=wibble
$ env '=wibble' env -u '' env
env: cannot unset `': Invalid argument
---

https://unix.stackexchange.com/questions/178522/unsetting-environment-variable-with-an-empty-name

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[STINNER Victor]

STINNER Victor  added the comment:

The following command still fails on the Python main branch on Linux:
---
$ env -i =value ./python -c 'import pprint, os; pprint.pprint(os.environ); del 
os.environ[""]'
environ({'': 'value', 'LC_CTYPE': 'C.UTF-8'})
Traceback (most recent call last):
  File "", line 1, in 
  File "/home/vstinner/python/main/Lib/os.py", line 689, in __delitem__
unsetenv(encodedkey)

OSError: [Errno 22] Invalid argument
---

'del os.environ[""]' calls unsetenv("") which fails with EINVAL.

Python is a thin wrapper to C functions setenv() and unsetenv(), and raises an 
exception when a C function fails. It works as expected.

Python exposes the variable with an empty name which is found in the 
environment variables: again, it works as expected.

I don't see how Python could do better, since the glibc unsetenv() fails with 
EINVAL if the string is empty. It is even a documented behaviour, see the 
unsetenv() manual page:
---
ERRORS

   EINVAL name is NULL, points to a string of length 0, or contains an '=' 
character.
---

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20658] os.environ.clear() fails with empty keys (posix.unsetenv)

[Irit Katriel]

Change by Irit Katriel :


--
stage: test needed -> resolved
status: pending -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45021] Race condition in thread.py

[STINNER Victor]

Change by STINNER Victor :


--
nosy:  -vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com