date:20200403



Change by Karthikeyan Singaravelan :


--
nosy: +cjw296, xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40176] unterminated string literal tokenization error messages could be better

2020-04-03 Thread Ammar Askar



Ammar Askar  added the comment:

Just re-posting this here from the open PR. Rust's handling of this seems nice 
and beginner friendly:

  error: unterminated double quote string
   --> src/main.rs:2:19
|
  2 |   let message = "Hello world
|  ___^
  3 | | println!(message);
  4 | | }
| |_^

Like Serhiy suggested, it points to the /start/ of the string, rather than the 
EOL and the message seems nice too.

--
nosy: +ammar2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40177] Python Language Reference Documentation

2020-04-03 Thread Steven D'Aprano



Steven D'Aprano  added the comment:

Oh never mind, I'm just going to slink away now and stop posting corrections 
when distracted...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40177] Python Language Reference Documentation

2020-04-03 Thread Steven D'Aprano



Steven D'Aprano  added the comment:

Oops, that should be

... ``x == x``, ``3 < x``, and ``x > 3`` are all false ...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40177] Python Language Reference Documentation

2020-04-03 Thread Steven D'Aprano



Steven D'Aprano  added the comment:

How about this?


"The not-a-number values ``float('NaN')`` and ``decimal.Decimal('NaN')`` are
special. Not-a-number values always compare unordered and unequal to any other 
value, including themselves. For example, if ``x = float('NaN')``, then ``x == 
x``, ``3 < x``, and ``x < 3`` are all false, but ``x != x`` is true. This 
behavior is compliant with the IEEE 754 standard."


Due to technology problems at my end, I cannot submit a PR for this, but this 
should be an easy issue. If somebody wants to use the text above as a start, 
please feel free to do so.

--
nosy: +steven.daprano

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40120] Undefined C behavior going beyond end of struct via a [1] arrays.



Gregory P. Smith  added the comment:

PyBytesObject could become a struct that contains a single opaque internal 
struct.

there is some code out there that references PyBytesObjects internals by field 
name but my searches across a broad swath of code so far seem to suggest that 
is so rare this change may be plausible (more research needed).  People are 
using the PyBytes_ macros and APIs.  yay!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40120] Undefined C behavior going beyond end of struct via a [1] arrays.



Gregory P. Smith  added the comment:

agreed, being opaque seems ideal.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40176] unterminated string literal tokenization error messages could be better

2020-04-03 Thread Alex Gaynor



Alex Gaynor  added the comment:

Here's my suggestion:

End of line reached without finding the end of string literal. Are you missing 
a closing quote?

--
nosy: +alex

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40177] Python Language Reference Documentation



Change by Karthikeyan Singaravelan :


--
nosy: +mark.dickinson, rhettinger, stutzbach

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40177] Python Language Reference Documentation

2020-04-03 Thread Arnuld


New submission from Arnuld :

In section "6.10.1 Value comparisons", it is written:

https://docs.python.org/3/reference/expressions.html

"The not-a-number values float('NaN') and decimal.Decimal('NaN') are
special. Any ordered comparison of a number to a not-a-number value is
false. A counter-intuitive implication is that not-a-number values are
not equal to themselves. For example, if x = float('NaN'), 3 < x, x <
3, x == x, x != x are all false. This behavior is compliant with IEEE
754."

Last comparison "x != x" does not return False, it returns True. Here is the 
output from my iPython interpeter I am using on Arch Linux (latest as of today):

In [86]: x == y
Out[86]: False

In [87]: x != y
Out[87]: True


I verified the bug it on Wikipedia too:

https://en.wikipedia.org/wiki/NaN#Comparison_with_NaN

--
assignee: docs@python
components: Documentation
files: Screenshot_2020-04-04 6 Expressions — Python 3 8 2 documentation.png
messages: 365742
nosy: ArnuldOnData, docs@python
priority: normal
severity: normal
status: open
title: Python Language Reference Documentation
type: enhancement
versions: Python 3.7, Python 3.8, Python 3.9
Added file: https://bugs.python.org/file49033/Screenshot_2020-04-04 6 
Expressions — Python 3 8 2 documentation.png

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40139] mimetypes module racy



Terry J. Reedy  added the comment:

3.5.3 is not the most recent 3.5.  Anyway, 3.5 and 3.6 and soon 3.7 only get 
security patches.  So it needs to be determined if there are failure with 3.8 
and 3.9.

--
nosy: +terry.reedy
versions:  -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38689] IDLE crashes when KeyError is raised during calltip generation



miss-islington  added the comment:


New changeset 15337726e5b92976c2815d05c514804e9aa49a8c by Miss Islington (bot) 
in branch '3.8':
bpo-38689: avoid IDLE hanging when calltip fails getting a signature (GH-17152)
https://github.com/python/cpython/commit/15337726e5b92976c2815d05c514804e9aa49a8c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38689] IDLE crashes when KeyError is raised during calltip generation



miss-islington  added the comment:


New changeset 681044a0ab6c93554ff8d003c7f9fe5fdb0c83ba by Miss Islington (bot) 
in branch '3.7':
bpo-38689: avoid IDLE hanging when calltip fails getting a signature (GH-17152)
https://github.com/python/cpython/commit/681044a0ab6c93554ff8d003c7f9fe5fdb0c83ba


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40132] Mechanism to control who owns package names on PyPI?



Terry J. Reedy  added the comment:

PyPI is a separate project from CPython and has its own repository, tracker, 
and developers.

--
nosy: +terry.reedy
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40127] Documentation of SSL library



Change by Terry J. Reedy :


--
versions:  -Python 2.7, Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38689] IDLE crashes when KeyError is raised during calltip generation



Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 3.0 -> 4.0
pull_requests: +18715
pull_request: https://github.com/python/cpython/pull/19353

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38689] IDLE crashes when KeyError is raised during calltip generation



Change by miss-islington :


--
pull_requests: +18716
pull_request: https://github.com/python/cpython/pull/19354

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38689] IDLE crashes when KeyError is raised during calltip generation



Terry J. Reedy  added the comment:


New changeset 52013e5b6d5ca32eef5a3d65ecdf7db89cefc2fd by Tal Einat in branch 
'master':
bpo-38689: avoid IDLE hanging when calltip fails getting a signature (GH-17152)
https://github.com/python/cpython/commit/52013e5b6d5ca32eef5a3d65ecdf7db89cefc2fd


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40160] documentation example of os.walk should be less destructive

2020-04-03 Thread John Taylor



John Taylor  added the comment:

I made the suggested change to just print the os.remove() statements (instead 
of executing them) and also removed the 'skip news'.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40160] documentation example of os.walk should be less destructive

2020-04-03 Thread Kyle Stanley

Kyle Stanley  added the comment:

Serhiy Storchaka wrote:
> I do not think there is clearer example of topdown=False than recursive 
> remove.
>
> If you think that this example is destructive, consider how destructive is 
> any possible example for shutil.rmtree()!

I concur with Serhiy. If the example is changed to just print out the file and 
directory path, as the PR proposes to do, it seems to defeat the purpose of 
using `topdown=False` (and the code example) in the first place.

If anything, I would suggest adding succinct comments or a note that very 
briefly explains how one could see a visual demonstration of what it does 
without removing any files or directories. For example: 
``print(f"os.remove({os.path.join(root, name)})")``.

--
nosy: +aeros

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40175] Add support for removing zip entry in ZipInfo

2020-04-03 Thread Kyle Stanley



Kyle Stanley  added the comment:

> ZipInfo currently only allow adding entries to zip archives but doesn't have 
> the ability to remove entries from them - pretty useful feature.

This public API enhancement seems substantial enough to warrant a python-ideas 
ML thread or topic in the "Ideas" category of discuss.python.org. It would help 
to demonstrate some real-world examples where one may want to programmatically 
remove entries.

Also, added zipfile maintainers to the nosy list.

--
nosy: +aeros, alanmcintyre, serhiy.storchaka, twouters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40158] MSBuild Extensions in CPython NuGet Package has Bad Expression



Steve Dower  added the comment:


New changeset e6685ad05385f8cb492e8e1c7c07889a94517f55 by Miss Islington (bot) 
in branch '3.8':
bpo-40158: Fix CPython MSBuild Properties in NuGet Package (GH-19343)
https://github.com/python/cpython/commit/e6685ad05385f8cb492e8e1c7c07889a94517f55


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40158] MSBuild Extensions in CPython NuGet Package has Bad Expression



Steve Dower  added the comment:


New changeset 7f70456b92c9ff0bcc4df2a2cec213ab2a897591 by Miss Islington (bot) 
in branch '3.7':
bpo-40158: Fix CPython MSBuild Properties in NuGet Package (GH-19343)
https://github.com/python/cpython/commit/7f70456b92c9ff0bcc4df2a2cec213ab2a897591


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40145] Pyshellext room for binary size improvement



Steve Dower  added the comment:

In general, no settings rely on both Platform AND Configuration, so you likely 
only need to check one or the other.

Look at PCbuild/pyproject.props for the best examples. IIRC, properties are set 
assuming Win32/Release, and then we override those that need to be different 
for either Debug or x64/ARM builds. Any settings that you think are worth 
applying to all projects can go in this file, while any that are strictly for 
pyshellext can go in its vcxproj (but using a similar style).

Let's not start a CMake debate here - there are plenty of other issues on here 
about that. We currently maintain MSBuild files by hand, and it works well.

More commits on the same PR are fine. We squash merge at the end, so the 
history only remains in the commit message if we need to put it there.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40176] unterminated string literal tokenization error messages could be better



Serhiy Storchaka  added the comment:

It could be even better. Inside the tokenizer we know where the string literal 
starts and what quotes it uses. The line and the offset of the *start* of the 
literal can be set in a SyntaxError.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40160] documentation example of os.walk should be less destructive



Serhiy Storchaka  added the comment:

I do not think there is clearer example of topdown=False than recursive remove.

If you think that this example is destructive, consider how destructive is any 
possible example for shutil.rmtree()!

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40126] Incorrect error handling in unittest.mock



Change by Serhiy Storchaka :


--
pull_requests: +18714
pull_request: https://github.com/python/cpython/pull/19351

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40164] Upgrade Windows and macOS installer builds to OpenSSL 1.1.1f



Steve Dower  added the comment:

I've pushed new binaries for OpenSSL 1.1.1f on Windows. I'll try and to the 
rest over the weekend, but if someone else wants to do the PCbuild PR feel free.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40160] documentation example of os.walk should be less destructive

2020-04-03 Thread Raymond Hettinger



Raymond Hettinger  added the comment:

One possibility is a gathering cumulative directory statistics that include 
totals from all descendants (i.e. how many bytes of files would you save by 
removing the directory with rm -rf).

Outside of aggregating statistics, the normal reason to use topdown=False is 
when paths are being mutated.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40126] Incorrect error handling in unittest.mock



Serhiy Storchaka  added the comment:

I think that the current code is not correct. __exit__ should not be called if 
__enter__ is failed. If some __enter__ implementation calls other __enter__s it 
should manually call corresponding __exit__s.

--
nosy: +michael.foord, ncoghlan, serhiy.storchaka
versions: +Python 3.8, Python 3.9 -Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40145] Pyshellext room for binary size improvement

2020-04-03 Thread Alexander Riccio



Alexander Riccio  added the comment:

Oh, uh, also, do you prefer I add a commit or a new branch & PR?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40145] Pyshellext room for binary size improvement

2020-04-03 Thread Alexander Riccio



Alexander Riccio  added the comment:

Ahh, ok. Even though I question the usefulness of manually maintaining MSBuild 
files instead of something like CMake, I can work with that. Is there a 
preferred way to do it? It looks like I can do a 
Condition="'$(Configuration)|$(Platform)'=='Release|x64'" and merge a bunch of 
things under each one.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40110] multiprocessing.Pool.imap() should be lazy

2020-04-03 Thread Tim Peters



Tim Peters  added the comment:

Whenever there's parallel processing with communication, there's always the 
potential for producers to pump out data faster than consumers can process 
them.  But builtin primitives generally don't try to address that directly.  
They don't - and can't - know enough about the application's intent.

Instead, as I briefly alluded to earlier, mediation (when needed) is frequently 
accomplished by users by explicit use of bounded queues.  When process A 
produces data for process B, it sends the data over a bounded queue.  Nothing 
is done to slow A down, except that when a bounded queue is full, an attempt to 
enqueue a new piece of data blocks until B removes some old data from the 
queue.  That's a dead easy, painless, and foolproof way to limit A's speed to 
the rate at which B can consume data.  Nothing in A's logic changes - it's the 
communication channel that applies the brakes.

I'll attach `pipe.py` to illustrate.  It constructs a 10-stage pipeline.  The 
first process in the chain *could* produce data at a ferocious rate - but the 
bounded queue connecting it to the next process slows it to the rate at which 
the second process runs.  All the other processes in the pipeline grab data 
from the preceding process via a bounded queue, work on it for a second (just a 
sleep(1) here), then enqueue the result for the next process in the pipeline.  
The main program loops, pulling data off the final process as fast as results 
show up.

So, if you run it, you'll see that new data is produced once per second, then 
when the pipeline is full final results are delivered once per second.  When 
the first process is done, results continue to be pulled off one per second 
until the pipeline is drained.

The queues here have maximum size 1, just to show that it works.  In practice, 
a larger bound is usually used, to allow for that processes in real life often 
take varying amounts of time depending on the data they're currently 
processing.  Letting a handful of work items queue up keeps processes busy - if 
processing some piece of data goes especially fast, fine, there may be more in 
the input queue waiting to go immediately.  How many is "a handful"?  Again, 
that's something that can't in general be guessed - the programmer has to apply 
their best judgment based on their deep knowledge of intended application 
behavior.

Just to be cute ;-) , the code here uses imap() to start the 10 worker 
processes.  The result of calling imap() is never used, it's just an easy way 
to apply a Pool to the task.  And it absolutely relies on that imap() consumes 
its iterable (range(NSTAGES)) as fast as it can, to get the NSTAGES=10 worker 
processes running.

--
Added file: https://bugs.python.org/file49032/pipe.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40118] os.stat in linux shows the wrong inode

2020-04-03 Thread Ammar Askar



Ammar Askar  added the comment:

As noted in the documentation for os.stat 
(https://docs.python.org/3.9/library/os.html#os.stat):

> This function normally follows symlinks; to stat a symlink add the argument 
> follow_symlinks=False, or use lstat().

--
nosy: +ammar2
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40126] Incorrect error handling in unittest.mock

2020-04-03 Thread Barry McLarnon



Change by Barry McLarnon :


--
components: +Library (Lib) -Tests

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40158] MSBuild Extensions in CPython NuGet Package has Bad Expression



Change by Steve Dower :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40158] MSBuild Extensions in CPython NuGet Package has Bad Expression



Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 7.0 -> 8.0
pull_requests: +18712
pull_request: https://github.com/python/cpython/pull/19349

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40158] MSBuild Extensions in CPython NuGet Package has Bad Expression



Change by miss-islington :


--
pull_requests: +18713
pull_request: https://github.com/python/cpython/pull/19350

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40158] MSBuild Extensions in CPython NuGet Package has Bad Expression



Steve Dower  added the comment:


New changeset 6e623ff9d251e0ce86e9b18a01bfd6f067079d7a by Chris Martinez in 
branch 'master':
bpo-40158: Fix CPython MSBuild Properties in NuGet Package (GH-19343)
https://github.com/python/cpython/commit/6e623ff9d251e0ce86e9b18a01bfd6f067079d7a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40141] Add line and column information for keywords in the AST



Pablo Galindo Salgado  added the comment:


New changeset 40cf35c5b070b3f33aae58a996fea0e8291a8616 by Pablo Galindo in 
branch 'master':
bpo-40141: Include the value in the column position for keyword AST nodes 
(GH-19348)
https://github.com/python/cpython/commit/40cf35c5b070b3f33aae58a996fea0e8291a8616


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40147] Move checking for duplicated keywords to the compiler



Change by Pablo Galindo Salgado :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40141] Add line and column information for keywords in the AST



Change by Pablo Galindo Salgado :


--
pull_requests: +18711
pull_request: https://github.com/python/cpython/pull/19348

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40147] Move checking for duplicated keywords to the compiler



Pablo Galindo Salgado  added the comment:


New changeset 254ec783411d9d16e51f1116f98918be2ef0e884 by Pablo Galindo in 
branch 'master':
bpo-40147: Move the check for duplicate keywords to the compiler (GH-19289)
https://github.com/python/cpython/commit/254ec783411d9d16e51f1116f98918be2ef0e884


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36541] Make lib2to3 grammar better match Python, support the := walrus



Gregory P. Smith  added the comment:

Support for `:=` is in, are we still lacking `f(**not x)` support?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36541] Make lib2to3 grammar better match Python, support the := walrus



Gregory P. Smith  added the comment:

master/3.9 changeset:
https://github.com/python/cpython/commit/3c3aa4516c70753de06bb142b6793d01330fcf0f

3.8 changeset: 
https://github.com/python/cpython/commit/1098671e4e5ec1513247f05598158eaa3428c5be

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36541] Make lib2to3 grammar better match Python, support the := walrus



Gregory P. Smith  added the comment:


New changeset 96c5f5a3a3fabf43e8114d0dbc30bed409da1ba6 by Tim Hatch in branch 
'3.7':
[3.7] bpo-36541: lib2to3: Support named assignment expressions (GH-12702) 
(GH-19317)
https://github.com/python/cpython/commit/96c5f5a3a3fabf43e8114d0dbc30bed409da1ba6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40170] [C API] Make PyTypeObject structure an opaque structure in the public C API

2020-04-03 Thread Andy Lester



Change by Andy Lester :


--
nosy: +petdance

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39576] Surprising MemoryError in `decimal` with MAX_PREC



Pablo Galindo Salgado  added the comment:

> Traceback, does it work --without-pymalloc?

No, it does also not work with `--without-pymalloc?`.

> Has anything emerged xlc-wise?

Not for now, I keep investigating and I may try to contact
IBM about this, but at this stage, I am getting more confident
that this is a bug in xlc or something going on with the platform
or the libC implementation.

I am currently chasing something weird that I saw while investigating
regarding too big malloc() calls and the data segment size on AIX that
produces a similar hang.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40176] unterminated string literal tokenization error messages could be better

2020-04-03 Thread Batuhan Taskaya



Change by Batuhan Taskaya :


--
keywords: +patch
nosy: +BTaskaya
nosy_count: 1.0 -> 2.0
pull_requests: +18710
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19346

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39943] Meta: Clean up various issues in C internals



Change by Serhiy Storchaka :


--
pull_requests: +18709
pull_request: https://github.com/python/cpython/pull/19345

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39576] Surprising MemoryError in `decimal` with MAX_PREC

2020-04-03 Thread Stefan Krah



Stefan Krah  added the comment:

Has anything emerged xlc-wise? Traceback, does it work --without-pymalloc?

I have the feeling that (in many OSS projects) the more complex xlc
issues never get resolved after the initial report.

So I'm contemplating to do the same as 

   
https://devhub.vr.rwth-aachen.de/soehrl/arbor/commit/775fe80796c717da4ed61e1cb7ace27268afc965

and disable the xlc build for _decimal.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40176] unterminated string literal tokenization error messages could be better

2020-04-03 Thread Benjamin Peterson



New submission from Benjamin Peterson :

It has been pointed out to me that the errors the tokenizer produces for 
unterminated strings, "EOL while scanning string literal" and "EOF while 
scanning triple-quoted string literal", contain parsing jargon that make it 
difficult for new users to understand the problem, likely a missing quote.

--
components: Interpreter Core
messages: 365713
nosy: benjamin.peterson
priority: normal
severity: normal
status: open
title: unterminated string literal tokenization error messages could be better
type: enhancement
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40174] HAVE_CLOCK_GETTIME not repected in pytime.c



Change by STINNER Victor :


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40174] HAVE_CLOCK_GETTIME not repected in pytime.c

2020-04-03 Thread Benjamin Peterson



Benjamin Peterson  added the comment:

Does this cause a problem? At this point, it might be preferable to just assume 
clock_gettime exists on POSIX systems.

--
nosy: +benjamin.peterson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40077] Convert static types to PyType_FromSpec()

2020-04-03 Thread hai shi



Change by hai shi :


--
pull_requests: +18708
pull_request: https://github.com/python/cpython/pull/19344

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40158] MSBuild Extensions in CPython NuGet Package has Bad Expression

2020-04-03 Thread Chris Martinez



Change by Chris Martinez :


--
keywords: +patch
pull_requests: +18707
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19343

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40038] pathlib: remove partial support for preserving accessor when modifying a path

2020-04-03 Thread Barney Gale



Change by Barney Gale :


--
keywords: +patch
pull_requests: +18706
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19342

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40077] Convert static types to PyType_FromSpec()

2020-04-03 Thread hai shi



Change by hai shi :


--
pull_requests: +18705
pull_request: https://github.com/python/cpython/pull/19341

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40131] Zipapp example has parameters in the wrong order



Karthikeyan Singaravelan  added the comment:

Thanks Leron for the report. Thanks Zackery for the patch.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions:  -Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40131] Zipapp example has parameters in the wrong order



Karthikeyan Singaravelan  added the comment:


New changeset d19162fe5b2aba48a94278baa0f569fc42932072 by Miss Islington (bot) 
in branch '3.7':
bpo-40131: Fix source and target order in zipapp example (GH-19290) (GH-19340)
https://github.com/python/cpython/commit/d19162fe5b2aba48a94278baa0f569fc42932072


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40131] Zipapp example has parameters in the wrong order



Karthikeyan Singaravelan  added the comment:


New changeset e6783981df6ae5c63f73be67cc41b1350bc0fcc6 by Miss Islington (bot) 
in branch '3.8':
bpo-40131: Fix source and target order in zipapp example (GH-19290) (GH-19339)
https://github.com/python/cpython/commit/e6783981df6ae5c63f73be67cc41b1350bc0fcc6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39537] Change line number table format



Change by Dong-hee Na :


--
nosy: +corona10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40131] Zipapp example has parameters in the wrong order



Change by miss-islington :


--
pull_requests: +18704
pull_request: https://github.com/python/cpython/pull/19340

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40131] Zipapp example has parameters in the wrong order



Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 4.0 -> 5.0
pull_requests: +18703
pull_request: https://github.com/python/cpython/pull/19339

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40131] Zipapp example has parameters in the wrong order



Karthikeyan Singaravelan  added the comment:


New changeset bd6a4c3d72828d3d0e13922e165998539d24f8bc by Zackery Spytz in 
branch 'master':
bpo-40131: Fix source and target order in zipapp example (GH-19290)
https://github.com/python/cpython/commit/bd6a4c3d72828d3d0e13922e165998539d24f8bc


--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40126] Incorrect error handling in unittest.mock

2020-04-03 Thread Barry McLarnon



Barry McLarnon  added the comment:

After further investigation, it seems this was fixed in 
https://github.com/python/cpython/commit/436c2b0d67da68465e709a96daac7340af3a5238

However, this fix was as part of an unrelated changeset and in a different 
function in 3.8+, and was never rolled back to 3.7 and below. PR opened to add 
the missing attribute instantiation to 3.7.

--
nosy:  -python-dev
versions:  -Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40126] Incorrect error handling in unittest.mock

2020-04-03 Thread Roundup Robot



Change by Roundup Robot :


--
keywords: +patch
nosy: +python-dev
nosy_count: 1.0 -> 2.0
pull_requests: +18701
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19338

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40105] Updating zip comment doesn't truncate the zip file

2020-04-03 Thread mzr



Change by mzr :


--
pull_requests: +18700
pull_request: https://github.com/python/cpython/pull/19337

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40149] test_threading leaked [38, 38, 38] references, sum=114



Dong-hee Na  added the comment:

Running 
from abc import ABCMeta
on the subinterpreter makes same size of leak.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40175] Add support for removing zip entry in ZipInfo

2020-04-03 Thread nergall2



Change by nergall2 :


--
keywords: +patch
pull_requests: +18699
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19336

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40175] Add support for removing zip entry in ZipInfo

2020-04-03 Thread nergall2



New submission from nergall2 :

ZipInfo currently only allow adding entries to zip archives but doesn't have 
the ability to remove entries from them - pretty useful feature.

--
components: Library (Lib)
messages: 365704
nosy: nergall2
priority: normal
severity: normal
status: open
title: Add support for removing zip entry in ZipInfo
type: enhancement
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40174] HAVE_CLOCK_GETTIME not repected in pytime.c

2020-04-03 Thread Jérôme


New submission from Jérôme :

Hi,

In the file Python/pytime.c (line 886 and others), functions and constants that 
I infer should be declared by HAVE_CLOCK_GETTIME are called without #ifdef.

Best regards,
Jérôme.

--
components: Interpreter Core
messages: 365703
nosy: jerome.hamm
priority: normal
severity: normal
status: open
title: HAVE_CLOCK_GETTIME not repected in pytime.c
type: compile error
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40105] Updating zip comment doesn't truncate the zip file

2020-04-03 Thread Yudi



Change by Yudi :


--
keywords: +patch
pull_requests: +18698
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19333

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40172] ZipInfo corrupts file names in some old zip archives

2020-04-03 Thread Yudi



Change by Yudi :


--
keywords: +patch
pull_requests: +18697
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19335

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40173] test.support.import_fresh_module fails to correctly block submodules when fresh is specified

2020-04-03 Thread Paul Ganssle



New submission from Paul Ganssle :

It seems that test.support.import_fresh_module gets tripped up with its module 
blocking when you attempt to get a fresh copy of a submodule of a module where 
you are also importing the module that you are trying to block (bit of a doozy 
of a sentence there...). So, for example, with the following configuration in 
mymodule/__init__.py:

from .other import other

try:
from ._a import attr
except ImportError:
from ._b import attr

(Assuming _a.attr = "A" and _b.attr = "B"), if you attempt to do:

m = test.support.import_fresh_module("mymodule", 
fresh=("mymodule._other",), blocked=("mymodule._a"))

Then you'll find that m.attr is pulled from _a.attr. Here's a small script to 
demonstrate:

from test.support import import_fresh_module
import sys

def import_ab(fresh_other):
fresh = ("mymodule._other", ) if fresh_other else ()

mods_out = []
for to_block in "_b", "_a":
blocked = (f"mymodule.{to_block}",)

mods_out.append(import_fresh_module("mymodule",
fresh=fresh, blocked=blocked))
return mods_out


for fresh_other in [True, False]:
mymodule_a, mymodule_b = import_ab(fresh_other)

qualifier = "With" if fresh_other else "Without"
print(f"{qualifier} a fresh import of mymodule._other")

print(f"a: {mymodule_a.attr}")
print(f"b: {mymodule_b.attr}")
print()

When you run it with a suitably configured module on Python 3.8:

$ python importer.py 
With a fresh import of mymodule._other
a: A
b: A

Without a fresh import of mymodule._other
a: A
b: B

It also happens if you add `mymodule._a` or `mymodule._b` to the fresh list 
when you are trying to block the other one.

I *think* the problem is that in the step where _save_and_remove_module is 
called on fresh_name (see here: 
https://github.com/python/cpython/blob/76db37b1d37a9daadd9e5b320f2d5a53cd1352ec/Lib/test/support/__init__.py#L328-L329),
 it's necessarily populating `sys.modules` with a fresh import of the top-level 
module we're trying to import (mymodule) *before* the blocking goes into 
effect, then the final call to importlib.import_module just hits that cache.

I think either of the following options will fix this issue:

1. Switching the order of how "fresh" and "blocked" are resolved or
2. Deleting `sys.modules[name]` if it exists immediately before calling 
`importlib.import_module(name)

That said, I'm still having some weird statefulness problems if I block a C 
module's import and *then* block a Python module's import, so there may be some 
other underlying pathology to the current approach.

--
components: Tests
files: test_support_repro.zip
messages: 365702
nosy: brett.cannon, eric.snow, ncoghlan, p-ganssle
priority: normal
severity: normal
status: open
title: test.support.import_fresh_module fails to correctly block submodules 
when fresh is specified
type: behavior
versions: Python 3.7, Python 3.8, Python 3.9
Added file: https://bugs.python.org/file49031/test_support_repro.zip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40105] Updating zip comment doesn't truncate the zip file

2020-04-03 Thread mzr



Change by mzr :


--
nosy: +mzr

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40172] ZipInfo corrupts file names in some old zip archives

2020-04-03 Thread Yudi



New submission from Yudi :

Some old zip files that don't yet use unicode file names might have entries 
with characters beyond the ascii range.
ZipInfo seems to encode these file names with 'cp437' codepage (correct for old 
zips) but decode them back with 'ascii' code page which might corrupt them.

--
components: Library (Lib)
files: example.zip
messages: 365701
nosy: yudilevi
priority: normal
severity: normal
status: open
title: ZipInfo corrupts file names in some old zip archives
type: behavior
versions: Python 3.8
Added file: https://bugs.python.org/file49030/example.zip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40112] AIX: xlc - default path changed and no longer recognized



Change by STINNER Victor :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40171] Attempting to import inaccessible package imports an empty package

2020-04-03 Thread Konrad Borowski



Konrad Borowski  added the comment:

The attached example works on Python 2, but it breaks with Python 3. This may 
be a regression.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40171] Attempting to import inaccessible package imports an empty package

2020-04-03 Thread Konrad Borowski



New submission from Konrad Borowski :

The attached shell program returns `AttributeError: module 'mod' has no 
attribute 'x'`. I would rather expect `ImportError` instead of loading an empty 
package due to permission error causing Python module to be inaccessible.

--
files: demonstration
messages: 365699
nosy: xfix
priority: normal
severity: normal
status: open
title: Attempting to import inaccessible package imports an empty package
type: behavior
versions: Python 3.8
Added file: https://bugs.python.org/file49029/demonstration

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40112] AIX: xlc - default path changed and no longer recognized



STINNER Victor  added the comment:


New changeset 76db37b1d37a9daadd9e5b320f2d5a53cd1352ec by Michael Felt in 
branch 'master':
bpo-40112: distutils test_search_cpp: Fix logic to determine if C compiler is 
xlc on AIX (GH-19225)
https://github.com/python/cpython/commit/76db37b1d37a9daadd9e5b320f2d5a53cd1352ec


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40143] shutil.rmtree will frequently fail on Windows under heavy load due to racy deletion

2020-04-03 Thread Eryk Sun

Eryk Sun added the comment:

> It's inherently racy, since deleting a file on Windows *doesn't
> actually delete it*, instead it marks the file for deletion. The
> system will eventually get around to deleting it, but under heavy
> load, this might be sometime after an attempt is made to delete
> the parent directory.

Commonly, WINAPI DeleteFileW and RemoveDirectoryW unlink the target file or
directory synchronously. There are cases such as a watched directory and
malware scanners that can make deleting asynchronous, but it's unlike the above
characterization. It's not like the delete operation gets queued by the
filesystem for the system to get around to sometime later.

Deleting or renaming a filesystem file/directory begins with creating a File
object that's granted delete access. This is the first hurdle, since all
existing File objects for a file have to share data access, including
read/execute, write/append, and delete/rename access. Sharing delete/rename
access is uncommon, and trying to delete an open file fails with
ERROR_SHARING_VIOLATION (32).

If the caller has a handle with delete access, the next hurdle is being allowed
to set the delete disposition on the underlying file/link control block
(FCB/LCB) in the filesystem. This request will be denied with
ERROR_ACCESS_DENIED (5) if the file is flagged as readonly or is currently
memory-mapped as data or image. In these cases, a file can still be renamed
within the filesystem, which is useful if there is a known destination path.

Assuming that setting the FCB/LCB delete disposition succeeds, then, with
classic Windows delete semantics (as opposed to POSIX semantics in available in
Windows 10), the file will be unlinked when the last File object that
references the FCB/LCB gets cleaned up. This in turn occurs when the last
handle for the last File object gets closed.

WINAPI CloseHandle calls the NtClose system function, which synchronously calls
a kernel object's close method, if implemented. A File object has a close
method that, if it's the last handle for the object in the system,
synchronously calls the filesystem device stack with an IRP_MJ_CLEANUP request.
You can see how a filesystem cleanup function works in the source of the
fastfat sample driver [1]. Pay attention to blocks in FatCommonCleanup that
check the flag FCB_STATE_DELETE_ON_CLOSE in the UserDirectoryOpen and
UserFileOpen cases. Note that even if the cleanup function were to complete
asynchronously with STATUS_PENDING, the close method of the File object itself
waits for completion. So the bases are covered to ensure deleting works
synchronously in the common case when a file is referenced only by the handle
that's used to delete it. This excludes the cases of pre-existing references
that share delete access and implicit interference from malware scanners that
inject themselves
into the filesystem device stack.

In Windows 10, NTFS supports POSIX delete semantics [2], i.e.
FILE_DISPOSITION_DELETE | FILE_DISPOSITION_POSIX_SEMANTICS. In this case, a
delete request still has to pass the hurdles of the file sharing mode, readonly
flag, and data/image file mappings. What changes is that the file will be
unlinked as soon as the deleting handle is closed. Existing opens can continue
to access data in the file.

DeleteFileW in Windows 10 first tries to use a POSIX delete, though this is
still undocumented. If a POSIX delete fails (e.g. it's a FAT32 filesystem),
DeleteFileW falls back on a classic delete.

RemoveDirectoryW is limited to a classic delete in Windows 10, so it's subject
to race conditions with watched directories. In particular, Explorer keeps
directories open to watch for changes. It shares delete access, which allows
RemoveDirectoryW to successfully set the delete disposition of a watched
directory. In turn, when a watch fails when the delete disposition is set,
Explorer immediately closes its handle. This unlinks the directory, but it's a
race condition.

When I have time, I'll check whether NTFS supports POSIX delete semantics on
directories -- by directly calling NtSetInformationFile:
FileDispositionInformationEx. If it does, then probably a future version of
Windows will try to use a POSIX delete in RemoveDirectoryW, just as DeleteFileW
current does. That will eliminate the problem with watched directories -- at
least on the system volume (C:), which is required to use NTFS.

[1]:
https://github.com/microsoft/Windows-driver-samples/blob/6c1981b8504329521343ad00f32daa847fa6083a/filesys/fastfat/cleanup.c#L47
[2]:
https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/ntddk/ns-ntddk-_file_disposition_information_ex

--
nosy: +eryksun

___
Python tracker

___
___
Python-bugs-list mailing list
Unsubscribe:

[issue40160] documentation example of os.walk should be less destructive

2020-04-03 Thread John Taylor



John Taylor  added the comment:

I would prefer an example that does not actually modify the file system.  Is 
there any way this could be achieved, yet still demonstrate why topdown=False 
is necessary?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40149] test_threading leaked [38, 38, 38] references, sum=114



Dong-hee Na  added the comment:

gc: collectable 
gc: collectable 
gc: collectable 
gc: collectable 

is not collected for the first time.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37095] [Feature Request]: Add zstd support in tarfile

2020-04-03 Thread Ugra Dániel


Change by Ugra Dániel :


--
nosy: +daniel.ugra

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40163] multissl doesn't support tarballs in /source/old/



STINNER Victor  added the comment:

> talk to OpenSSL upstream.

Do you mean continue to provide old versions in /source/ directory as well? 
Maybe they move tarballs to /source/old/ on purpose, to force users to use the 
latest versions which get fixes for new vulnerabilities?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40140] test_builtin crashes when runned in parallel mode on solaris



STINNER Victor  added the comment:

> Well, if needed I can create one but looks like it is going be an obsoleted 
> OS soon :/

I dislike the idea of making Python codes to support Solaris, since Solaris 
vendor doesn't support it anymore... Moreover, it's closed source, and most 
core devs don't have access to Solaris (ex: me).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40140] test_builtin crashes when runned in parallel mode on solaris



STINNER Victor  added the comment:


New changeset 0961dbdea2a449fc5b7d77610d6d10e6036fbdf3 by Victor Stinner in 
branch '3.7':
bpo-40140: test_builtin.PtyTests registers SIGHUP handler (GH-19314) (GH-19316) 
(GH-19318)
https://github.com/python/cpython/commit/0961dbdea2a449fc5b7d77610d6d10e6036fbdf3


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40120] Undefined C behavior going beyond end of struct via a [1] arrays.



STINNER Victor  added the comment:

For me, the most sane option is to make structures opaque in the C API, and 
then flexible array members.

I did something similar for atomic types. First, we got tons of build isssues 
with various C compilers and then with C++ compilers. I moved the header to our 
"internal C API", so basically I removed it from the public C API. Since that 
time, we stopped to get bug reports about pyatomic.h :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40170] [C API] Make PyTypeObject structure an opaque structure in the public C API



Change by Dong-hee Na :


--
nosy: +corona10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40170] [C API] Make PyTypeObject structure an opaque structure in the public C API



New submission from STINNER Victor :

Leaking the PyTypeObject structure in the C API indirectly causes ABI issue 
(especially for statically allocated types), cause practical issues when old 
fields are removed and new fields are added (ex: tp_vectorcall addition and 
tp_print removal caused a lot of troubles with C code generated by Cython: see 
bpo-37250), prevents us to add feature and experiment optimization.

I don't expect that we will be able to make PyTypeObject opaque soon. The 
purpose of this issue is to track the work done towards this goal.

I propose to slowly prepare the Python code base, the C API and third party 
code (especially Cython) to make PyTypeObject structure opaque.

We have to identify most common code patterns which access directly 
PyTypeObject fields, provide helper functions, and ease the migration to 
solutions which don't access directly PyTypeObject.

See also bpo-39573: "Make PyObject an opaque structure in the limited C API" 
and bpo-39947 "Make the PyThreadState structure opaque (move it to the internal 
C API)".

Longer rationale about making structures of the C API opaque:

* https://pythoncapi.readthedocs.io/
* https://pythoncapi.readthedocs.io/bad_api.html
* https://pythoncapi.readthedocs.io/optimization_ideas.html

--

Multiple practical issues are preventing us to make PyTypeObject opaque right 
now.


(*) Many C extension modules are still using statically allocated types: there 
is an on-going effort in bpo-40077 to convert C extension modules one by one to 
PyType_FromSpec().


(*) Py_TYPE(obj)->tp_name is commonly accessed to format an error message. 
Example:

PyErr_Format(PyExc_TypeError, "exec() globals must be a dict, not 
%.100s",
 Py_TYPE(globals)->tp_name);

I worked on bpo-34595 and started a discussion on python-dev to propose to add 
a new %T formatter to PyUnicode_FromFormatV() and so indirectly to 
PyUnicode_FromFormat() and PyErr_Format():
https://mail.python.org/archives/list/python-...@python.org/thread/HKYUMTVHNBVB5LJNRMZ7TPUQKGKAERCJ/#3UAMHYG6UF4MPLXBZORHO4JVKUBRUZ53

Sadly, we failed to reach a consensus and I gave up on this idea. We should 
reconsider this idea.

We need to agree on how types should be formatted:

* just the name without any dot "type_name",
* qualified name "something.type_name",
* fully qualified name "module.something.type_name"

There is also the question of breaking applications which rely on the current 
exact error message. And the question of removing legacy "%.100s" which was 
used before Python was able to allocate a buffer large enough to arbitrary 
string length. When an error is formatted in pure Python, names are never 
truncated.


(*) Call the function of the parent type when a method is overriden in a 
subclass. Example with PyTypeObject.tp_free called in a deallocator:

static void
abc_data_dealloc(_abc_data *self)
{
PyTypeObject *tp = Py_TYPE(self);
...
tp->tp_free(self);
Py_DECREF(tp);
}


(*) The PEP 384 provides the most generic PyType_GetSlot() but it's not 
convenient to use: need to handle error (NULL), need to cast the void* into the 
expected type (error prone cast), etc.

We should slowly add more and more helper functions for most common use cases. 
We can try to convert a few C extension modules of the Python stdlib to see 
which use cases are the most common.

Hopefully, many use cases are already abstracted by widely used functions like 
PyNumber_Add(), PySequence_Size(), etc.


(*) Likely other issues that I forgot.

--
components: C API
messages: 365689
nosy: vstinner
priority: normal
severity: normal
status: open
title: [C API] Make PyTypeObject structure an opaque structure in the public C 
API
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39573] Make PyObject an opaque structure in the limited C API



STINNER Victor  added the comment:

I created bpo-40170 "[C API] Make PyTypeObject structure an opaque structure in 
the public C API".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24132] Direct sub-classing of pathlib.Path

2020-04-03 Thread Piotr Dobrogost



Change by Piotr Dobrogost :


--
nosy: +piotr.dobrogost

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9253] argparse: optional subparsers

2020-04-03 Thread brent s.



Change by brent s. :


--
nosy: +bsaner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36906] Compile time textwrap.dedent() equivalent for str or bytes literals

2020-04-03 Thread Miguel Amaral



Miguel Amaral  added the comment:

A related issue(which I believe has no topic in this forum yet) is substituting 
an expression that results in a multiline string into a multiline f-string 
while matching its indentation.
If a new type of string prefix is made to auto-dedent, maybe the substitutions 
should match the local indentation.
Some related stackoverflow posts:

https://stackoverflow.com/questions/36739667/python-templates-for-generating-python-code-with-proper-multiline-indentation

https://stackoverflow.com/a/57189263/2976410

I.e. ideally we would have:
```python
def make_g_code():
  nl='\n'
  return d"""\
def g():
  {nl.join(something(i) for i in range(n))}
  return something_else
"""
```
This still has issues. Newline needs to be put into a variable, for instance. 
In the case of using this template for languages, great many use braces for 
delimiting blocks and those need to be escaped inside f-strings.

An implementation that works with spaces only (does not suit my case where 
mixed indentation is possible) is here:

http://code.activestate.com/recipes/578835-string-templates-with-adaptive-indenting/

Please let me know if this is the wrong place to comment on this issue.

--
nosy: +Miguel Amaral

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40155] AIX: test_builtin.test_input_no_stdout_fileno() hangs

2020-04-03 Thread Michael Felt



Michael Felt  added the comment:

Thanks for the fix!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40143] shutil.rmtree will frequently fail on Windows under heavy load due to racy deletion



Steve Dower  added the comment:

What about renaming the base directory in place? Moving things across drives 
doesn't help, and we can't reasonably determine a suitable location for temp 
files other than by leaving them where they are.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40158] MSBuild Extensions in CPython NuGet Package has Bad Expression



Steve Dower  added the comment:

Either of those fixes look good. I normally use IO.Path personally, which might 
be because it's been around longer, but anything that works on VS 2017 and 
later should be fine.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40122] The implementation and documentation of "dis.findlables" don't match