[issue17110] sys.argv docs should explaining how to handle encoding issues

[miss-islington]

miss-islington  added the comment:


New changeset 5b80cb5584a72044424f2d82d0ae79c720f24c47 by Miss Islington (bot) 
in branch '3.7':
bpo-17110: doc: add note how to get bytes from sys.argv (GH-12602)
https://github.com/python/cpython/commit/5b80cb5584a72044424f2d82d0ae79c720f24c47


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17110] sys.argv docs should explaining how to handle encoding issues

[miss-islington]

Change by miss-islington :


--
pull_requests: +12559

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17110] sys.argv docs should explaining how to handle encoding issues

[Inada Naoki]

Inada Naoki  added the comment:


New changeset 38f4e468d4b1e135c67337c18ae142193ba8 by Inada Naoki in branch 
'master':
bpo-17110: doc: add note how to get bytes from sys.argv (GH-12602)
https://github.com/python/cpython/commit/38f4e468d4b1e135c67337c18ae142193ba8


--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36438] Python 3.5.7 import error on Cross compile

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Python 3.5 and 3.6 only get security fixes.  Please test on 3.7 or even better 
3.8.  Your questions should be directed to python-list.

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36426] exec() issue when used inside function

[Terry J. Reedy]

Terry J. Reedy  added the comment:

This is not a bug; the behavior is covered in the docs.  I am surprised that 
this works in 2.7, as it has the same warning as 3.7:
"
locals()
Update and return a dictionary representing the current local symbol table. 
Free variables are returned by locals() when it is called in function blocks, 
but not in class blocks.
Note The contents of this dictionary should not be modified; changes may not 
affect the values of local and free variables used by the interpreter."

--
nosy: +terry.reedy
type: behavior -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35224] PEP 572: Assignment Expressions

[Vedran Čačić]

Vedran Čačić  added the comment:

Carol, if you're willing to go into the lion's den that is Python-ideas with 
this, you have my eternal gratitude. :-)

Steven, sorry, there really is no rush. I really don't think I ever said there 
is. However, I think it would be much easier to change the behavior while the 
thing is in alpha. Isn't that the purpose of alpha? 

(If I'm wrong here, please disregard. I would be fine to see this happening few 
years from now. I believe in "Python in the limit", not actual versions, but 
too many times I have been said "what you ask makes sense in the ideal world, 
but that ship has sailed long ago".)

Yes, of course (name := expression) is an improvement over what we have now, 
and I'm grateful for that. It's just that when I explain it to my students (" = 
is just assignment, := is for assignment and displaying"), I have no good 
reason to tell them why they must put the parentheses---except "Python is too 
worried you will make a mistake", and that just doesn't seem like something 
Python usually does. (That's something Java would do to people.:)

Yes, Jupyter sometimes does allow things that would otherwise be SyntaxErrors 
(though much less than they used to, since Python gave them a lot of headache 
by introducing decorators---if you remember that story;), and going to Jupyter 
was the next thing on my mind after I'm rejected here. I just thought it would 
be much easier to just allow this "at the source", so Jupyter people don't have 
to think "what if they finally allow toplevel walruses, but with different 
semantics (e.g., printing result even if it is None)?".

Carol and Victor, I'm sorry I have usurped a bugtracker issue for this 
discussion. First, I really thought this is about implementation of assignment 
expressions, and this is the best place to put it. Second, I didn't expect a 
discussion---I thought it would be either "that makes no sense, go away" or 
"yeah, good idea, we'll do it". For the next such issue (there will probably be 
one:), do you suggest that the more appropriate thing would be to open a new 
issue?

(Let me just reiterate that I'm not going to python-ideas. You probably can't 
understand how stressful that place is, but believe me, it is. I'm not the only 
one that thinks so. If that's the only sanctioned method to improve Python, 
even when it is about details, then I'll just withdraw from the game.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36411] Python 3 f.tell() gets out of sync with file pointer in binary append+read mode

[Terry J. Reedy]

Terry J. Reedy  added the comment:

'Python 3.9' does not exist yet.  This choice is for patches that should only 
be applied to 3.9 when available (like future deprecations or removals).

--
nosy: +terry.reedy
versions:  -Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36467] IDLE to help with invisible characters

[Terry J. Reedy]

Terry J. Reedy  added the comment:

With respect to Shell, this is a request that IDLE move further away from 
strictly imitating Python running in a dumb terminal and better help users with 
intelligent action and information.  I want both to improve editing and make 
entry of a statement  in Shell more or less the same as as entering one in the 
Editor. 

Goal for this issue: make specific error visible and allow specific correction 
without retyping the complete line.  When?  see below.

First, what invisible characters are an issue? Do you mean truly no-space 
invisible or displayed as spaces?  For the moment, I assume the issue is ascii 
control chars.

On Windows, some can be entered directly with KeyPress-Alt, up to 3 keypad 
digits, KeyRelease-Alt.  But there are various interactions with 
interpretations as DOS graphics characters (alt-1 = light smiley face, alt-01 = 
\x01 displayed as box), old keyboard codes (alt-3 = HOME key), and IDLE 
bindings (alt-2 = dark smiley face + IDLE zoomheight, alt-02 = \x02 displayed 
as space + zoomheight, maybe twice).  (I checked that zoomheight return 
'break'.)  A mess.

Once a control char is entered into tk text, which is most easily done with an 
escapes within a string, some appear as half-width boxes, which interact a bit 
badly with fixed-width ascii; some appear as spaces, the subject of this issue; 
and a few otherwise (I think at least one on one OS is no-space invisible, 
which would also be the subject of this issue).  By experiment, details depend 
on the OS.

Realistic test case: 'for iin [1]: pass'.  The char before 'in' is \x02, 
displayed by tk Text on Windows as ' '.  Perhaps the user only meant to zoom or 
unzoom the window and was not aware of the difference between the regular 2-@ 
key and numpad 2.

When this code is submitted and the symtax error detected, the control char 
gets a red background.  In the editor, the cursor is placed after the error 
char after the SyntexError box is dismissed.  An experienced user should know 
to delete the 'space' and enter a real space, but still would not understand 
the error or how it got there.

In Shell, one currently has to retrieve the statement as the new prompt and 
move the cursor to the false space.  Before implementing the fix below, I would 
like to change Shell so one can edit syntax errors in-place, just as in the 
editor.  IDLE compiles code for syntax checking before submitting the code 
object to be exec-ed.  There is no need to freeze the code, display a 
traceback, and display a new prompt.

Possible improvement 1: check a marked chars or scan the entire reported error 
line for control chars other than newline and tab in a string and replace any 
found with red-background '\xnn' escapes.

At present, I would prefer to reserve bulk scanning for bulk text entry via 
pasting or file reading.  For interactive entry ...

Possible improvement 2: IDLE already checks each character typed to see whether 
it is a '.' in code, '/' or (on Windows) r'\\' in string, or '\n' anywhere.  
(And it also checks syntax coloring.)  Add a check for controls chars and 
immediately replace with red escape (or whatever else we decide on).  Leave the 
cursor immediately after the error.  Put an error message either on the status 
bar or in a non-modal popup similar to a calltip.

I already want to get rid of the SyntaxError box and replace it with a display 
of just the error message, as above, without the boilerplate around it.

Neither check will catch problemmatic non-ascii chars displayed as no-space or 
a space. This set likely depends on the font and OS and you did not specify 
this an an issue.  IDLE cannot see what is on the screen.
  
In any case, worrying about non-ascii should be a followup to an initial patch. 
 If we do, we can selectively include problematic visible chars, such as smart 
quotes.

--
nosy: +cheryl.sabella
stage:  -> test needed
type:  -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36482] let struct's internal cache use FIFO policy

[Ma Lin]

Ma Lin  added the comment:

FYI, re module's cache is using FIFO policy:

https://github.com/python/cpython/blob/v3.8.0a3/Lib/re.py#L288-L293

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36482] let struct's internal cache use FIFO policy

[Ma Lin]

Change by Ma Lin :


--
keywords: +patch
pull_requests: +12558
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36482] let struct's internal cache use FIFO policy

[Ma Lin]

New submission from Ma Lin :

Currently, when the cache is full, the entire cache is cleared.
This patch let it use FIFO policy.

Inada Naoki, Raymond Hettinger, could you review this patch? Thanks.
No hurry, just do it when you have time.

--
components: Library (Lib)
messages: 339168
nosy: Ma Lin, inada.naoki, rhettinger
priority: normal
severity: normal
status: open
title: let struct's internal cache use FIFO policy
type: behavior
versions: Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36481] telnetlib process_rawq() callback

[Roundup Robot]

Change by Roundup Robot :


--
keywords: +patch
pull_requests: +12557
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36478] backport of pickle fixes to Python 3.5.7 uses C99 for loops

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +larry, vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34160] ElementTree not preserving attribute order

[STINNER Victor]

Change by STINNER Victor :


--
nosy:  -vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34160] ElementTree not preserving attribute order

[STINNER Victor]

STINNER Victor  added the comment:

I set the priority to release blocker to make sure that the issue got enough 
attention. Raymond started a thread on python-dev, so I reset the priority:
https://mail.python.org/pipermail/python-dev/2019-March/156709.html

Moreover, the change is now documented in "What's New in Python 3.8?" 
documentation ("Changes in the Python API" section):
https://github.com/python/cpython/commit/06e1e688228013f411aca6309562ef80df6ce5d3

--
priority: release blocker -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36481] telnetlib process_rawq() callback

[Gökhan Öztürk]

New submission from Gökhan Öztürk :

telnetlib.Telnet class requires a callback so that the raw data that comes from 
socket can be processed first.

This will be useful for the compressed incoming data. Most of the MUD servers 
have MCCP V2 (byte: 86) protocol that send compressed data. But as for now. the 
only option is to create new class that inherits telnetlib.Telnet and override 
process_rawq()

--
components: IO
messages: 339166
nosy: Gökhan Öztürk
priority: normal
severity: normal
status: open
title: telnetlib process_rawq() callback
type: enhancement
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36480] .strip() unexpected output on Windows

[Eric V. Smith]

Eric V. Smith  added the comment:

Please provide the exact code to duplicate the problem.

I suspect this is a problem with your code, not with str.strip and not with 
Windows.

--
nosy: +eric.smith
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36480] .strip() unexpected output on Windows

[78]

New submission from 78 <78alphadevi...@gmail.com>:

Using .strip() on windows leads to the first and last character of a word being 
deleted.

Magenta.zip becomes agenta.zi

--
components: Windows
messages: 339164
nosy: 78Alpha, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: .strip() unexpected output on Windows
type: behavior
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36085] Enable better DLL resolution

[Steve Dower]

Steve Dower  added the comment:

Leaving this in commit review for a couple of days, then I'll close.

--
stage: patch review -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36085] Enable better DLL resolution

[Steve Dower]

Steve Dower  added the comment:


New changeset 2438cdf0e932a341c7613bf4323d06b91ae9f1f1 by Steve Dower in branch 
'master':
bpo-36085: Enable better DLL resolution on Windows (GH-12302)
https://github.com/python/cpython/commit/2438cdf0e932a341c7613bf4323d06b91ae9f1f1


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35224] PEP 572: Assignment Expressions

[STINNER Victor]

Change by STINNER Victor :


--
nosy:  -vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35947] Update libffi_msvc to current version of libffi

[Steve Dower]

Steve Dower  added the comment:


New changeset 32119e10b792ad7ee4e5f951a2d89ddbaf111cc5 by Steve Dower (Paul 
Monson) in branch 'master':
bpo-35947: Update Windows to the current version of libffi (GH-11797)
https://github.com/python/cpython/commit/32119e10b792ad7ee4e5f951a2d89ddbaf111cc5


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35224] PEP 572: Assignment Expressions

[STINNER Victor]

STINNER Victor  added the comment:

The bug tracker is not the appropriate place to discuss a PEP. This issue is 
about the implementation of the PEP.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35224] PEP 572: Assignment Expressions

[Steven D'Aprano]

Steven D'Aprano  added the comment:

You are one person, who has used this feature for what, a month elapsed 
time? 300 person-hours actual experience with it? Allowing top-level 
unparenthisized walrus expressions will affect hundreds of thousands of 
people, for collectively millions of hours over a decade or more of 
elapsed time. What's the rush about lifting this restriction?

If the restriction turns out to be "pointless", then we can remove it 
later, and no harm done. You say this is ugly in the notebooks:

(variable := expression)

but it is surely still an improvement over the status quo:

variable = expression; variable

But if we remove it now, and it turns out that it wasn't as pointless as 
you thought, then we're stuck with a design mistake that will be very 
hard to fix without breaking people's code.

I'm glad you've found an excellent use-case for unbracketed assignment 
expressions, and I don't oppose your suggested change, I'm just 
advocating caution.

Besides, Jypiter already allows interactive code that would be a syntax 
error outside of their environment. They can probably relax that 
restriction within Jypiter, while still leaving the language alone.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36085] Enable better DLL resolution

[Steve Dower]

Steve Dower  added the comment:

So symlinking didn't work (Python is too clever for that these days ;) ), but 
straight copying the exe and required DLLs is fine.

It puts python.exe, python38.dll and vcruntime140.dll (properly discovered of 
course) into a temp directory, puts _sqlite3.pyd into a subdirectory and only 
allows imports from that directory and the pure stdlib (for encodings). Then we 
test with add_dll_directory(), then copy sqlite3.dll in and test again without.

Assuming tests all pass, I consider this complete now.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33356] Windows 10 buildbot: test__xxsubinterpreters.test_already_running() fails randomly

[Eric Snow]

Eric Snow  added the comment:

I'll take a look when I get a chance.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36427] Document that PyEval_RestoreThread and PyGILState_Ensure can terminate the calling thread

[Eric Snow]

Eric Snow  added the comment:

> Currently PyEval_RestoreThread and its callers (mainly PyGILState_Ensure)
> can terminate the thread if the interpreter is finalizing:

s/interpreter/runtime/

Most likely (guaranteed?) it will be in the main interpreter, but it is 
actually not triggered by interpreter finalization (though it probably should 
be; I've opened #36479).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36479] Exit threads when interpreter is finalizing rather than runtime.

[Eric Snow]

New submission from Eric Snow :

Currently when a thread acquires the GIL, it subsequently exits if the runtime 
is finalizing.  This helps with some cases like with stopping daemon threads.

This behavior should instead be triggered by the thread's interpreter 
finalizing rather than the runtime.  This implies the following changes:

* in Py_EndInterpreter() move "interp-finalizing = 1;" to right after the call 
to "wait_for_thread_shutdown()"
* in Py_FinalizeEx() add "interp-finalizing = 1;" right after the call to 
"wait_for_thread_shutdown()"
* update _PyEval_EvalFrameDefault() (the eval loop) to check 
"interp->finalizing" instead of the runtime
* likewise update PyEval_RestoreThread() (as well as PyEval_AcquireThread() and 
PyEval_AcquireLock(); see #36475)

The check will actually need to change from this:

  if (_Py_IsFinalizing() && !_Py_CURRENTLY_FINALIZING(tstate)) {

to look like this:

  if (interp->finalizing && !_Py_CURRENTLY_FINALIZING(tstate)) {

If we could guarantee that only the main thread will ever call Py_FinalizeEx() 
then it would look more like this:

  if (interp->finalizing && tstate.id != _PyRuntime.main_thread {

--
components: Interpreter Core
messages: 339155
nosy: eric.snow
priority: normal
severity: normal
stage: needs patch
status: open
title: Exit threads when interpreter is finalizing rather than runtime.
type: behavior
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36478] backport of pickle fixes to Python 3.5.7 uses C99 for loops

[Anthony Sottile]

Change by Anthony Sottile :


--
keywords: +patch
pull_requests: +12556
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36427] Document that PyEval_RestoreThread and PyGILState_Ensure can terminate the calling thread

[Eric Snow]

Eric Snow  added the comment:

> > if (_Py_IsFinalizing() && !_Py_CURRENTLY_FINALIZING(tstate))
>
> _Py_IsFinalizing() check is redundant :-)

Not really:

* _Py_IsFinalizing() checks if the runtime is finalizing
* _Py_CURRENTLY_FINALIZING checks if the current thread is the one
running finalization

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36427] Document that PyEval_RestoreThread and PyGILState_Ensure can terminate the calling thread

[Eric Snow]

Eric Snow  added the comment:

Related:

* #36475: "PyEval_AcquireLock() and PyEval_AcquireThread() do not handle 
runtime finalization properly."
* #36476: "Runtime finalization assumes all other threads have exited."

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36478] backport of pickle fixes to Python 3.5.7 uses C99 for loops

[Anthony Sottile]

New submission from Anthony Sottile :

While building python 3.5.7 for https://github.com/deadsnakes

../Modules/_pickle.c: In function 'PyMemoTable_Copy':
../Modules/_pickle.c:677:5: error: 'for' loop initial declarations are only 
allowed in C99 mode
 for (size_t i = 0; i < self->mt_allocated; i++) {
 ^
../Modules/_pickle.c:677:5: note: use option -std=c99 or -std=gnu99 to compile 
your code
../Modules/_pickle.c: In function '_pickle_PicklerMemoProxy_copy_impl':
../Modules/_pickle.c:4207:5: error: 'for' loop initial declarations are only 
allowed in C99 mode
 for (size_t i = 0; i < memo->mt_allocated; ++i) {
 ^
../Modules/_pickle.c: In function 'Unpickler_set_memo':
../Modules/_pickle.c:6794:9: error: 'for' loop initial declarations are only 
allowed in C99 mode
 for (size_t i = 0; i < new_memo_size; i++) {
 ^
../Modules/_pickle.c:6842:9: error: 'for' loop initial declarations are only 
allowed in C99 mode
 for (size_t i = new_memo_size - 1; i != SIZE_MAX; i--) {
 ^
make[1]: *** [Modules/_pickle.o] Error 1


This cherry-pick caused the regression: 
https://github.com/python/cpython/commit/ef33dd6036aafbd3f06c1d56e2b1a81dae3da63c

--
components: Build
messages: 339152
nosy: Anthony Sottile
priority: normal
severity: normal
status: open
title: backport of pickle fixes to Python 3.5.7 uses C99 for loops
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36157] Document PyInterpreterState_Main().

[Eric Snow]

Eric Snow  added the comment:

As I noted on the PR, this might be a good chance to make sure the C-API docs 
are clear about what the "main" interpreter is.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36097] Use only public C-API in _xxsubinterpreters module.

[Eric Snow]

Change by Eric Snow :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36225] Lingering subinterpreters should be implicitly cleared on shutdown

[Eric Snow]

Eric Snow  added the comment:

test__xxsubinterpreters is a great place for tests that exercise use of 
subinterpreters, including most lifecycle operations.  There are also one or 
two subinterpreter-related tests in test_embed.  However, for this issue the 
interplay with runtime finalization means tests should probably stay with other 
tests that exercise Py_FinalizeEx().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36225] Lingering subinterpreters should be implicitly cleared on shutdown

[Eric Snow]

Eric Snow  added the comment:

Interestingly, I noticed this independently today. :)

Here's what I wrote in #36477 (which I've closed as a duplicate):

When using subinterpreters, any that exist when Py_FinalizeEx() is called do 
not appear to get cleaned up during runtime finalization.  Maybe I've been 
looking at the code too much and I'm missing something. :)

This really isn't a problem except for embedders that use subinterpreters 
(where we're leaking memory).  However, even with the "python" executable it 
can have an impact because the subinterpreters' non-daemon threads will exit 
later than expected. (see #36469 & #36476)

The solution would be to finalize all subinterpreters at the beginning of 
Py_FinalizeEx(), right before the call to wait_for_thread_shutdown().  This 
means calling Py_EndInterpreter() for all the runtime's interpreters (except 
the main one).  It would also mean setting a flag 
(_PyRuntime.interpreters.finalizing?) right before that to disallow creation of 
any more subinterptreters.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36477] Subinterpreters are not finalized during runtime finalization.

[Eric Snow]

Change by Eric Snow :


--
resolution:  -> duplicate
stage: needs patch -> resolved
status: open -> closed
superseder:  -> Lingering subinterpreters should be implicitly cleared on 
shutdown

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31182] Suggested Enhancements to zipfile & tarfile command line interfaces

[Brett Cannon]

Brett Cannon  added the comment:

I'm going to agree w/ Serhiy and say thanks to Steve for the idea but for 
maintainability purposes we should keep the CLI of both modules simple and only 
for critical operations.

--
nosy: +brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31182] Suggested Enhancements to zipfile & tarfile command line interfaces

[Brett Cannon]

Change by Brett Cannon :


--
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36469] Stuck during interpreter exit, attempting to take the GIL

[Eric Snow]

Eric Snow  added the comment:

@Remy, aside from the recommendations I've made, I'm not sure what else we can 
do to help.  Before we close the issue, I'd really like to ensure that one of 
those threads is holding the GIL still.  It would definitely be a problem if a 
thread exited while still holding the GIL.  The only way I can think of is to 
trace through the code. [1]


[1] https://github.com/python/cpython/tree/v3.5.3

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36476] Runtime finalization assumes all other threads have exited.

[Eric Snow]

Eric Snow  added the comment:

FYI, I've opened issue36477 to deal with the subinterpreters case.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36477] Subinterpreters are not finalized during runtime finalization.

[Eric Snow]

New submission from Eric Snow :

When using subinterpreters, any that exist when Py_FinalizeEx() is called do 
not appear to get cleaned up during runtime finalization.  Maybe I've been 
looking at the code too much and I'm missing something. :)

This really isn't a problem except for embedders that use subinterpreters 
(where we're leaking memory).  However, even with the "python" executable it 
can have an impact because the subinterpreters' non-daemon threads will exit 
later than expected. (see #36469 & #36476)

The solution would be to finalize all subinterpreters at the beginning of 
Py_FinalizeEx(), right before the call to wait_for_thread_shutdown().  This 
means calling Py_EndInterpreter() for all the runtime's interpreters (except 
the main one).  It would also mean setting a flag 
(_PyRuntime.interpreters.finalizing?) right before that to disallow creation of 
any more subinterptreters.

--
components: Interpreter Core
messages: 339144
nosy: eric.snow
priority: normal
severity: normal
stage: needs patch
status: open
title: Subinterpreters are not finalized during runtime finalization.
type: behavior
versions: Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36469] Stuck during interpreter exit, attempting to take the GIL

[Eric Snow]

Eric Snow  added the comment:

I've also opened issues #36476 and #36477.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36476] Runtime finalization assumes all other threads have exited.

[Eric Snow]

New submission from Eric Snow :

Among the first 3 things that happen in Py_FinalizeEx() are, in order:

1. wait for all non-daemon threads (of the main interpreter) to finish
2. call registered atexit funcs
3. mark the runtime as finalizing

At that point the only remaining Python threads are:

* the main thread (where finalization is happening)
* daemon threads
* non-daemon threads created in atexit functions
* any threads belonging to subinterpreters

The next time any of those threads (aside from main) acquire the GIL, we expect 
that they will exit via a call to PyThread_exit_thread() (caveat: issue 
#36475).  However, we have no guarantee on when that will happen, if ever.  
Such lingering threads can cause problems, including crashes and deadlock (see 
issue #36469).

I don't know what else we can do, beyond what we're already doing.  Any ideas?

--
components: Interpreter Core
messages: 339143
nosy: eric.snow
priority: normal
severity: normal
status: open
title: Runtime finalization assumes all other threads have exited.
type: behavior
versions: Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36468] Treeview: wrong color change

[Ned Deily]

Ned Deily  added the comment:

Thanks for the report.  The difference in behavior appears to be due to a 
change in Tk. Using the python.org 3.7.2 or .3 installers which use Tk 8.6.8, 
the colors are displayed.  But if I use a Python linked with Tk 8.6.9, the bars 
are not colored.  You can run the following commands in Python to verify which 
version of Tk is in use:

import tkinter
root = tkinter.Tk()
root.tk.call('info', 'patchlevel')

In any case, there isn't likely anything Python can do about it as tkinter is 
pretty much a thin wrapper around calls to Tk and it's difficult to know what 
the expected Tk behavior is.  If you want to pursue this issue, I suggest 
bringing it up on a Tk forum or checking whether there is a Tk issue about it.

But I had luck! Doing a quick search, I found this Tk ticket which seems to 
describe your problem and does confirm that the behavior change was introduced 
in 8.6.9:

https://core.tcl.tk/tk/info/509cafafae

--
nosy: +ned.deily
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36475] PyEval_AcquireLock() and PyEval_AcquireThread() do not handle runtime finalization properly.

[Eric Snow]

Change by Eric Snow :


--
components: +Interpreter Core

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36466] Adding a way to strip annotations from compiled bytecode

[Ivan Levkivskyi]

Change by Ivan Levkivskyi :


--
nosy: +gvanrossum, levkivskyi

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36469] Stuck during interpreter exit, attempting to take the GIL

[Eric Snow]

Eric Snow  added the comment:

Looking at the stack traces for all your threads (super helpful, BTW), I saw 4 
groups:

* (1) main thread (blocked on GIL during runtime finalization)
   + Thread 1
* (12) blocked on GIL in call to PyEval_RestoreThread() (socket, select, 
time.sleep(), file.read())
* (3) waiting for a connection on a socket (with GIL not held, presumably)
   + Thread 5
   + Thread 12
   + Thread 14
* (1) blocked on a threading.Lock
   + Thread 7
* (1) blocked on "head" lock when cleaning up after execution
   + Thread 18

So there's a third lock involved in this deadlock.  It isn't actually clear to 
me (without further digging) which thread actually holds the GIL.  I'd guess 
it's one of the last two (though it could be one of the 3 waiting on a socket). 
 However, in each of those cases I would not expect the GIL to be held at that 
point.

Regardless, the race likely involves the threading.Lock being held late in 
finalization.  It could be that you are not releasing it somewhere where you 
should be.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36470] dataclasses.replace raises an exception if InitVar with default argument is not provided.

[Ivan Levkivskyi]

Change by Ivan Levkivskyi :


--
nosy: +levkivskyi

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36469] Stuck during interpreter exit, attempting to take the GIL

[Eric Snow]

Eric Snow  added the comment:

I've opened issue36475 for the two C-API functions that do not cause daemon 
threads to exit.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36469] Stuck during interpreter exit, attempting to take the GIL

[Remy Noel]

Remy Noel  added the comment:

Oh, also, i do not use any C extension (apart from the one i mentionned), so i 
do not acquire/release the GIL directly (a component of the standard library 
would do so). 

The demon threads, mainly spend their time listening to sockets and running 
short subprocesses (all in pure python).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36475] PyEval_AcquireLock() and PyEval_AcquireThread() do not handle runtime finalization properly.

[Eric Snow]

New submission from Eric Snow :

Daemon threads keep running until they finish or until finalization starts.  
For the latter, there is a check right after the thread acquires the GIL which 
causes the thread to exit if runtime finalization has started. [1]  However, 
there are functions in the C-API that facilitate acquiring the GIL, but do not 
cause the thread to exit during finalization:

  PyEval_AcquireLock()
  PyEval_AcquireThread()

Daemon threads that acquire the GIL through these can cause a deadlock during 
finalization.  (See issue #36469.)  They should probably be updated to match 
what PyEval_RestoreThread() does.


[1] see PyEval_RestoreThread() and the eval loop, in PyEval_EvalFrameEx()

--
messages: 339138
nosy: eric.snow
priority: normal
severity: normal
stage: test needed
status: open
title: PyEval_AcquireLock() and PyEval_AcquireThread() do not handle runtime 
finalization properly.
type: behavior
versions: Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36469] Stuck during interpreter exit, attempting to take the GIL

[Remy Noel]

Remy Noel  added the comment:

Thank you a lot for this detailed answer.

Does the "causes of exit" may terminate the thread without releasing the GIL ?
Because as far as i can tell, none of the threads seems to own the GIL (i only 
checked _PyThreadState_Current though there might be a better way to find the 
GIL owner).
Therefore, the question is whether thread B is still alive after tB2. and, if 
so, whether we can find it. (Or whether we can understand why it left without 
releasing the GIL).

Is there any variable i may check to dig this further ?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36474] RecursionError resets trace function set via sys.settrace

[daniel hahler]

daniel hahler  added the comment:

Discovered via / relevant for coverage's PyTracer: 
https://github.com/nedbat/coveragepy/issues/787.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36474] RecursionError resets trace function set via sys.settrace

[daniel hahler]

New submission from daniel hahler :

A RecursionError causes the trace function set via `sys.settrace` to get 
removed/unset.

Given the following script:

```
import sys


def trace(*args):
print("trace", args)
return trace

sys.settrace(trace)


def f():
f()


print(sys.gettrace())
try:
f()
except Exception as exc:
print(exc)
print(sys.gettrace())
```

Running it will output:
```

trace (, 'call', None)
trace (, 'line', None)
trace (, 'call', None)
trace (, 'line', None)
…
trace (, 'call', None)
trace (, 'line', None)
trace maximum recursion depth exceeded while getting the repr of an object
None

```

--
components: Interpreter Core
messages: 339135
nosy: blueyed
priority: normal
severity: normal
status: open
title: RecursionError resets trace function set via sys.settrace
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36469] Stuck during interpreter exit, attempting to take the GIL

[Eric Snow]

Eric Snow  added the comment:

Here are some things that would likely help:

* in the main thread stop your daemon threads if you hit SystemExit
* make sure daemon threads release/acquire the GIL frequently (so they notice 
finalization)
* make sure daemon threads otherwise exit promptly (no long running C code)
* stop using daemon threads
* use a newer version of Python (may be fixed there)

I'm going take a look at master to see if it has a similar possible problem 
with daemon threads and runtime finalization.  If there is then I'll likely 
open a separate issue (and reference it here).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36469] Stuck during interpreter exit, attempting to take the GIL

[Eric Snow]

Eric Snow  added the comment:

At this point I think it's likely that the problem relates to how daemon 
threads are handled during runtime finalization.

What normally happens in the main thread of the "python3" executable is this:

1. Python runtime initializes
2. main interpreter initializes
3. the requested code is executed (in this case via PyRun_SimpleFileExFlags)
4. the runtime is finalized
   a. wait for all non-daemon threads to finish
   b. call registered atexit funcs
   c. mark the runtime as finalizing
   d. ...

>From the stack trace you gave, the main thread is definitely past step 4c in 
>the runtime finalization process.

Note the following:

* marking the runtime as finalizing is meant to cause all remaining daemon 
threads to exit the next time they take the GIL
* further, runtime finalization assumes that all daemon threads will have 
finished soon after step 4c
* not all C-API functions for acquiring the GIL actually cause the current 
thread to exit if the runtime is finalizing (see below)
* step 4a applies only to the main interpreter; using subinterpreters 
complicates the situation a little (threads get finalized with each 
subinterpreter later on)
* any thread created in an atexit func (4b) can cause problems

Cause thread to exit if runtime is finalizing:

* PyEval_RestoreThread()
* PyEval_EvalFrameEx()  (the eval loop)

Do not cause thread to exit if runtime is finalizing:

* PyEval_InitThreads()
* PyEval_ReInitThreads()
* PyEval_AcquireLock()
* PyEval_AcquireThread()

Regardless, from what you've reported it looks like the following is happening:

m1. main thread starts
m2. thread A (daemon) created
m3. thread B (daemon) created (by main thread or thread A)
m4. code in main thread raises SystemExit
m5. the Python runtime begins finalization (incl. steps 4a-4c above)
m6. the main thread starts cleaning up the main interpreter
m7. it starts cleaning up the main interpreter's threads
m8. it acquires the "head" lock
m9. it starts cleaning up the frame tied to one of those threads
m10. a socket object in that frame gets destroyed
m11. a warning is issued (presumably about an unclosed socket)

tB1. thread B (still running) acquires GIL
tB2. it does not release the GIL

m12. creating the warning causes a function to get called (socket.__repr__)
m13. this causes the main thread to try to acquire the GIL
m14. it blocks (waiting for thread B)

tA1. thread A (still running) finishes and starts cleaning itself up
tA2. it tries to acquire the "head" lock
tA3. it blocks (waiting for the main thread)


Notable:

* at step m7 the code assumes that all the interpreter's threads have exited at 
that point
* with the verbose flag to "python3", the runtime will print a warning if any 
thread is still running when its (finalizing) interpreter tries to clean it up

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36469] Stuck during interpreter exit, attempting to take the GIL

[Remy Noel]

Remy Noel  added the comment:

std modules: atexit, json, os, signal, socket, ssl, subprocess, sys, time, 
threading, xmlrpc.client
The non-standard extension: psutil, PIL (only for image encoding though).

The amount of threads may vary, but i may have around 20 threads at all time. 
Most of them are indeed demon threads.

I have one atexit handler: i executes a few subprocess and pereform a few 
xmlrpc calls then exit. In this case, the handler go fully executed.

There are signal handlers, but none of them got called.

No monkeypatching is involved :)

I only browsed the patch up until the 3.5 head. (i guess il lacked to courage 
to go up to 3.7).

I tried to write a reproduction case, but i failed to run into the error. Of 
course, i will try to improve it if i get a clue about a way to increase the 
likelyness of the problem.
Sadly, all i have right now is a process i can attach to.

--
type: behavior -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35224] PEP 572: Assignment Expressions

[Carol Willing]

Carol Willing  added the comment:

@veky As a Jupyter notebook maintainer, I can see your point and I suspect some 
would like it. I'm not sure how big a benefit it would be for folks based on 
current notebook usage and practices. I just don't know. It's worth a 
discussion, but it should take place on python-ideas first to see how much 
traction your proposal would have.

Let's keep this issue focused on the implementation of 572 as accepted.

--
nosy: +willingc

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36463] python37.dll crashing 0xc000041d

[Brett Cannon]

Brett Cannon  added the comment:

Please provide code which consistently causes the crash, otherwise your use of 
ctypes makes the very likely not Python's fault and nearly impossible to debug 
without a working example.

--
nosy: +brett.cannon
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36463] python37.dll crashing 0xc000041d

[Brett Cannon]

Change by Brett Cannon :


--
nosy:  -brett.cannon
status: pending -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36448] Message "You will need to rebuild pythoncore to see the changes" should mention "make regen-all"

[Steve Dower]

Steve Dower  added the comment:

Merged. It doesn't need a backport - the target audience is CI users who will 
see this on master first.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
type:  -> compile error
versions:  -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36472] Some old PR with CLA not signed

[Brett Cannon]

Brett Cannon  added the comment:

I'm already going through the "CLA not signed" issues and cleaning them up, so 
there's no need for anyone to duplicate my work.

And this is not a steering council concern but a core-workflow concern.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36448] Message "You will need to rebuild pythoncore to see the changes" should mention "make regen-all"

[Steve Dower]

Steve Dower  added the comment:


New changeset 3396d1e0ca858065c5bb7e5a9737be6ffc4028f7 by Steve Dower (Jeroen 
Demeyer) in branch 'master':
bpo-36448: mention 'make regen-all' in error message (GH-12585)
https://github.com/python/cpython/commit/3396d1e0ca858065c5bb7e5a9737be6ffc4028f7


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35224] PEP 572: Assignment Expressions

[Miro Hrončok]

Change by Miro Hrončok :


--
nosy:  -hroncok

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34915] LWPCookieJar.save() creates *.lwp file in 644 mode

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

I guess this is a good choice and distutils stores .pypirc [0] in this manner 
that has username and password. 

[0] 
https://github.com/python/cpython/blob/2f54908afc5665937d763510b4430f10cf764641/Lib/distutils/config.py#L45

--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36469] Stuck during interpreter exit, attempting to take the GIL

[Eric Snow]

Eric Snow  added the comment:

* Are any signals (or signal handlers) involved?
* Are you monkeypatching any builtins, builtin/stdlib modules (or any parts of 
those)?

Keep in mind that a number of bugs have been fixed in later releases related to 
the various things I've asked about.  For instance, see issue #30703.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36470] dataclasses.replace raises an exception if InitVar with default argument is not provided.

[Greg Kuhn]

Greg Kuhn  added the comment:

Fixed title

--
title: dataclasses replace raises an exception with an empty -> 
dataclasses.replace raises an exception if InitVar with default argument is not 
provided.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30703] Non-reentrant signal handler (test_multiprocessing_forkserver hangs)

[Eric Snow]

Change by Eric Snow :


--
nosy: +eric.snow

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35707] time.sleep() should support objects with __float__

[Jeroen Demeyer]

Jeroen Demeyer  added the comment:

Is anybody willing to review PR 11636?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36448] Message "You will need to rebuild pythoncore to see the changes" should mention "make regen-all"

[Jeroen Demeyer]

Jeroen Demeyer  added the comment:

> I'd propose adding "%0D%0A%0D%0AIf you are developing on another platform, 
> try make regen-all and commit the updated files"

I updated the PR with wording similar to that. I don't want to bikeshed too 
much about the precise wording. If you disagree with my variant, I'll use your 
proposal verbatim.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35224] PEP 572: Assignment Expressions

[Vedran Čačić]

Vedran Čačić  added the comment:

Sorry, I don't have the energy for endless discussions without any result that 
almost always happen there. If you - of all people - don't see an obvious 
benefit of this (not even a feature - just a removal of a quite pointless 
limitation), then I'm probably wrong and there's no point in that.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36469] Stuck during interpreter exit, attempting to take the GIL

[Eric Snow]

Eric Snow  added the comment:

Also:

* are any of your threads daemon threads?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35983] tp_dealloc trashcan shouldn't be called for subclasses

[Jeroen Demeyer]

Jeroen Demeyer  added the comment:

I created an additional PR 12607 with some more changes to the, in particular 
to make the old backwards-compatibility trashcan macros safer. This should be 
seen as part of the same bugfix but I decided to make a new PR because PR 11841 
had several positive reviews already.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36469] Stuck during interpreter exit, attempting to take the GIL

[Eric Snow]

Eric Snow  added the comment:

As Inada-san indicated, the problem might be resolved already.  So without the 
option of reproducing the problem, it will be hard to resolve this.

Here's some information you could provide that might help narrow down the scope 
a bit:

* what (stdlib/third-party/internal) modules are you using, particularly 
extension modules?
* how many threads are you using at once?
* how many atexit handlers do you have and what are they each doing?

--
nosy: +eric.snow
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36473] Detect all dictionary changes during iteration

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

This is a duplicate of issue6017, issue19332 and issue29420.

--
nosy: +pitrou, rhettinger, serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36473] Detect all dictionary changes during iteration

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35550] Some define guards for Solaris are wrong

[Jesús Cea Avión]

Change by Jesús Cea Avión :


--
nosy: +jcea

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36469] Stuck during interpreter exit, attempting to take the GIL

[Remy Noel]

Remy Noel  added the comment:

The bug happens about once every two weeks on a script that is fired more than 
10K times a day.

Sadly, i can't update the whole production environment to try it on the latest. 
(and i was unable to trigger the bug by myself).

I was hoping we could find inconsistencies in the hanging process that could 
lead to clues about the origin of the error.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36452] Detect dict iteration "overflow" when changing keys

[Thomas Perl]

Change by Thomas Perl :


--
pull_requests: +12555

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36473] Detect all dictionary changes during iteration

[Thomas Perl]

Change by Thomas Perl :


--
keywords: +patch
pull_requests: +12554
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36473] Detect all dictionary changes during iteration

[Thomas Perl]

New submission from Thomas Perl :

On top of issue 36452, I noticed some other corner cases that are still not 
handled. For one, the patch (Github PR 12596) only handles iterating over keys, 
but not iterating over values or items:

==
a = {0: 0}
it = iter(a.values())
print('Length hint:', it.__length_hint__())
print(next(it))
print('Length hint:', it.__length_hint__())
del a[0]
a[1] = 99
print(next(it))
print('Length hint:', it.__length_hint__())
==

Replace a.values() there with a.items() -- same issue. Note that PR 12596 fixes 
the a.keys() case (same as iterating over "a" directly).

Applying the "di->len == 0" check in dictiter_iternextvalue() and 
dictiter_iternextitem() would fix those two cases above, but would still not 
fix the following case:

==
a = {0: 'a', 1: 'b', 2: 'c'}

it = iter(a)
i = next(it)
print('got first:', i)
del a[1]
a[1] = 'd'
i = next(it)
print('got second:', i)
i = next(it)
print('got third:', i)

try:
i = next(it)
raise RuntimeError(f'got fourth: {i}')
except StopIteration:
print('stop iteration')
==

The reason for this is that the iteration count (3 in this case) isn't 
modified, but the dict's keys are still changed, and the iteration order is as 
follows:

==
got first: 0
got second: 2
got third: 1
stop iteration
==

Note that the key 1 there is first deleted and then set.

I'll add a Github PR that tries to solve these corner cases too by tracking 
dict keys modification.

--
components: Interpreter Core
messages: 339115
nosy: thomas.perl
priority: normal
severity: normal
status: open
title: Detect all dictionary changes during iteration
type: behavior
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36471] PEP 432, PEP 587: Add _Py_RunMain()

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 2f54908afc5665937d763510b4430f10cf764641 by Victor Stinner in 
branch 'master':
bpo-36471: Add _Py_RunMain() (GH-12618)
https://github.com/python/cpython/commit/2f54908afc5665937d763510b4430f10cf764641


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36472] Some old PR with CLA not signed

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

Brett would like to reduce the number of PRs, maybe we need to ask to the 
council about that point.

--
nosy: +matrixise

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36471] PEP 432, PEP 587: Add _Py_RunMain()

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +12553
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36469] Stuck during interpreter exit, attempting to take the GIL

[Inada Naoki]

Inada Naoki  added the comment:

Can you reproduce it on 3.7 or master branch?

Python 3.5 is security fix only mode now.
https://devguide.python.org/#status-of-python-branches

--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35224] PEP 572: Assignment Expressions

[Guido van Rossum]

Guido van Rossum  added the comment:

@veky -- please take this up on python-ideas.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36472] Some old PR with CLA not signed

[Emmanuel Arias]

Emmanuel Arias  added the comment:

> I would request this to be discussed in other channels like 
> https://discuss.python.org/c/core-workflow or zulip core-workflow chat or 
> mailing list.

Ok. It good for you if I send this issue to 
https://discuss.python.org/c/core-workflow?

Then, I close this issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36472] Some old PR with CLA not signed

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

There were cases in the past where CLA was signed but there was mismatch in the 
bpo details and GitHub usernames linked due to which bot didn't change it to 
CLA signed. I would request this to be discussed in other channels like 
https://discuss.python.org/c/core-workflow or zulip core-workflow chat or 
mailing list. This issue could be closed since there is nothing that could be 
done in the bug tracker here. These problems might be solved by using CLA 
assistant. Closing a PR needs relevant access and I guess Brett is also 
reviewing PRs where CLA is not signed.

--
nosy: +brett.cannon, xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36472] Some old PR with CLA not signed

[Emmanuel Arias]

New submission from Emmanuel Arias :

Hello everybody!

Making a searching on GitHub: is:open label:"CLA not signed" 

We can see that there are some PR opened with CLA not signed label.
The newest is 22 days ago and the oldest is from May 8, 2017.

I can closed it if you consider apropiate

--
messages: 339108
nosy: eamanu
priority: normal
severity: normal
status: open
title: Some old PR with CLA not signed
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36471] PEP 432, PEP 587: Add _Py_RunMain()

[STINNER Victor]

STINNER Victor  added the comment:

I'm working on a PR to implement it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36471] PEP 432, PEP 587: Add _Py_RunMain()

[STINNER Victor]

New submission from STINNER Victor :

The PEP 587 adds new functions taking command line arguments:

Py_PreInitializeFromArgs(config, argc, argv)
Py_PreInitializeFromWideArgs(config, argc, argv)
Py_InitializeFromArgs(config, argc, argv)
Py_InitializeFromWideArgs(config, argc, argv)

I propose to add _Py_RunMain() (currently private, until PEP 587 is accepted) 
to be able to implement something similar to Py_Main() but with way more 
configuration options:

Example to create an "isolated Python" in a few line of C:
---
#include 

int main(int argc, char *argv[])
{
_PyCoreConfig config = _PyCoreConfig_INIT;
config.isolated = 1;

_PyInitError err = _Py_InitializeFromArgs(, argc, argv);
if (_Py_INIT_FAILED(err)) {
_Py_ExitInitError(err);
}

return _Py_RunMain();
}
---

It's like the regular "python3" program, except that it's always isolated:
---
$ my-isolated-python
Python 3.8.0a3+ (heads/run_main-dirty:d167362ecc, Mar 29 2019, 13:03:30) 
>>> 1+1
2
>>> import sys
>>> sys.flags.isolated
1
---

Using _Py_RunMain(), it becomes trivial to implement something like PEP 432's 
"system python" idea ;-)

--
components: Interpreter Core
messages: 339106
nosy: ncoghlan, vstinner
priority: normal
severity: normal
status: open
title: PEP 432, PEP 587: Add _Py_RunMain()
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36470] dataclasses replace raises an exception with an empty

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36470] dataclasses replace raises an exception with an empty

[Greg Kuhn]

New submission from Greg Kuhn :

I have a snippet below which runs fine on python 3.7.0 but raises a ValueError 
exception on 3.7.1. I believe it's related to 
https://bugs.python.org/issue33805.

The error: c:\python\lib\dataclasses.py:1219: ValueError

The script:

from dataclasses import replace, dataclass, InitVar

@dataclass
class Test:
a:int = 1
b:InitVar[int] = None

def __post_init__(self, b):
if b is not None:
self.a = b


if __name__ == '__main__':
t = Test()
t1 = Test(b=5)
assert t1.a == 5

t2 = replace(t1, **{})
print(t2)

--
components: Interpreter Core
messages: 339105
nosy: Greg Kuhn
priority: normal
severity: normal
status: open
title: dataclasses replace raises an exception with an empty
type: behavior
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28375] cgi.py spam in Apache server logs

[yoch]

yoch  added the comment:

Same issue here (python 3.6). This is very annoying, especially in case of 
large entries in FieldStorage, because the whole data is written to the log.

Example:
FieldStorage('image', 'upload.jpg', b'...can be very long...')

--
components: +Library (Lib)
nosy: +yoch.melka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27797] ASCII file with UNIX line conventions and enough lines throws SyntaxError when ASCII-compatible codec is declared

[Inada Naoki]

Change by Inada Naoki :


--
resolution:  -> duplicate
stage: needs patch -> resolved
status: open -> closed
superseder:  -> SyntaxError: encoding problem: iso-8859-1 on Windows

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36468] Treeview: wrong color change

[SilentGhost]

Change by SilentGhost :


--
nosy: +gpolo, serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36469] Stuck during interpreter exit, attempting to take the GIL

[Remy Noel]

New submission from Remy Noel :

I have a script (sadly, I can't publish it) spawning multiple threads that, in 
rare occurences, does not manage to exit properly and get stuck forever.

More precisely, this seems to happen during Interpreter exit: The atexit 
callbacks are called sucessfully, and we then have multiple threads that are 
all atempting to get the GIL why None seems to owns it (_PyThreadState_Current 
is always '{_value = 0}' while gil_locked is '{_value = 1}').

The main thread stack looks like this:

#0  pthread_cond_timedwait@@GLIBC_2.3.2 () at 
../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#1  0x55d00997ce6a in PyCOND_TIMEDWAIT (cond=0x55d009e8ddc0 , 
mut=0x55d009e8dd80 , us=) at ../Python/condvar.h:103
#2  take_gil () at ../Python/ceval_gil.h:224
#3  0x55d00998580b in PyEval_EvalFrameEx () at ../Python/ceval.c:1273
#4  0x55d00996f16f in _PyEval_EvalCodeWithName.lto_priv.1929 (qualname=0x0, 
name=, closure=0x0, kwdefs=0x0, defcount=0, defs=0x0, kwcount=0, 
kws=, argcount=, 
args=, locals=, globals=, 
_co=) at ../Python/ceval.c:4033
#5  PyEval_EvalCodeEx () at ../Python/ceval.c:4054
#6  0x55d0099b90e3 in function_call.lto_priv () at 
../Objects/funcobject.c:627
#7  0x55d009a02e17 in PyObject_Call () at ../Objects/abstract.c:2166
#8  0x55d00992034e in method_call.lto_priv () at 
../Objects/classobject.c:330
#9  0x55d009a02e17 in PyObject_Call () at ../Objects/abstract.c:2166
#10 0x55d00996df7d in PyEval_CallObjectWithKeywords () at 
../Python/ceval.c:4595
#11 0x55d009a5d05d in slot_tp_repr () at ../Objects/typeobject.c:5992
#12 0x55d0099c9685 in PyObject_Repr () at ../Objects/object.c:482
#13 0x55d0099aa6be in unicode_fromformat_arg (vargs=0x7ffc2ca81110, 
f=0x55d009a8a837 "R", writer=0x7ffc2ca810b0) at ../Objects/unicodeobject.c:2645
#14 PyUnicode_FromFormatV () at ../Objects/unicodeobject.c:2710
#15 0x55d009a572bc in PyErr_WarnFormat () at ../Python/_warnings.c:895
#16 0x55d0098840bb in sock_dealloc (s=0x7f43000fc528) at 
../Modules/socketmodule.c:4177
#17 0x55d0099d031d in subtype_dealloc.lto_priv () at 
../Objects/typeobject.c:1209
#18 0x55d0099b68f7 in frame_dealloc.lto_priv () at 
../Objects/frameobject.c:431
#19 0x55d0098ab7b1 in PyThreadState_Clear (tstate=0x55d00bee8a70) at 
../Python/pystate.c:386
#20 0x55d009a4d08a in PyInterpreterState_Clear () at ../Python/pystate.c:118
#21 0x55d009a4e1d2 in Py_Finalize () at ../Python/pylifecycle.c:633
#22 0x55d009a4e2a8 in Py_Exit (sts=sts@entry=0) at 
../Python/pylifecycle.c:1465
#23 0x55d009a4e38e in handle_system_exit () at ../Python/pythonrun.c:602
#24 0x55d009a4e3f6 in PyErr_PrintEx () at ../Python/pythonrun.c:612
#25 0x55d009a4f667 in PyErr_Print () at ../Python/pythonrun.c:508
#26 PyRun_SimpleFileExFlags () at ../Python/pythonrun.c:401
#27 0x55d009a7c2e7 in run_file (p_cf=0x7ffc2ca814fc, 
filename=0x55d00bb01140 L"...", fp=0x55d00bb62e60) at ../Modules/main.c:318
#28 Py_Main () at ../Modules/main.c:768
#29 0x55d00990bd71 in main () at ../Programs/python.c:65
#30 0x7f430b7cd2e1 in __libc_start_main (main=0x55d00990bc90 , 
argc=11, argv=0x7ffc2ca81708, init=, fini=, 
rtld_fini=, stack_end=0x7ffc2ca816f8)
at ../csu/libc-start.c:291
#31 0x55d009a12a7a in _start ()

We can see it is trying to get the GIL while finalizing (as it is emitting a 
warning when destroying a socket). However, this prevents any other thread to 
get deleted since the first thread holds the head_lock. For instance we have 
thread 18 trying to get the head lock:

Thread 18 (Thread 0x7f4302ffd700 (LWP 21117)):
#0  0x7f430c6aa536 in futex_abstimed_wait_cancelable (private=0, 
abstime=0x0, expected=0, futex_word=0x55d00bb014c0) at 
../sysdeps/unix/sysv/linux/futex-internal.h:205
#1  do_futex_wait (sem=sem@entry=0x55d00bb014c0, abstime=0x0) at 
sem_waitcommon.c:111
#2  0x7f430c6aa5e4 in __new_sem_wait_slow (sem=0x55d00bb014c0, abstime=0x0) 
at sem_waitcommon.c:181
#3  0x55d00994d2d5 in PyThread_acquire_lock_timed () at 
../Python/thread_pthread.h:352
#4  0x55d009a4dcec in tstate_delete_common () at ../Python/pystate.c:418
#5  0x55d009a4dd88 in PyThreadState_DeleteCurrent () at 
../Python/pystate.c:457
#6  0x55d009a482a4 in t_bootstrap () at ../Modules/_threadmodule.c:1027
#7  0x7f430c6a2494 in start_thread (arg=0x7f4302ffd700) at 
pthread_create.c:333
#8  0x7f430b895acf in clone () at 
../sysdeps/unix/sysv/linux/x86_64/clone.S:97


I attached the full stacktrace of the 18 threads.

I a not sure wether we either shouldn't try to lock the GIL while finalizing or 
if i somehow just happen to have run into a thread aqcuiring the GIL without 
releasing it.

python version is 3.5.3.

I kept the problematic process running and can extract any information you may 
want from it.

--
components: Interpreter Core, Library (Lib)
files: stacktraces
messages: 339103
nosy: mocramis
priority: normal
severity: normal

[issue36463] python37.dll crashing 0xc000041d

[Savagery]

Savagery  added the comment:

crash is caused by the WndProc definitions, seems having python code that does 
anything inside of those WINFUNCTYPE ctypes callbacks increasing the frequency 
of the python crash with positive correlation to amount of python code in the 
function.

--
components: +ctypes -Windows

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36463] python37.dll crashing 0xc000041d

[Savagery]

Change by Savagery :


--
components: +Windows

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com