date:20200706



Change by Joannah Nanjekye :


--
keywords: +patch
pull_requests: +20515
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/21370

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41225] Add a test for get_id() in the symtable module



New submission from Joannah Nanjekye :

The method get_id() in the symtable module is implemented and documented but 
not tested.

--
messages: 373201
nosy: nanjekyejoannah
priority: normal
severity: normal
status: open
title: Add a test for get_id() in the symtable module

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41217] Obsolete note for default asyncio event loop on Windows

2020-07-06 Thread Emmanuel Arias



Change by Emmanuel Arias :


--
nosy: +eamanu

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41224] Document is_annotate() in symtable and update doc strings



Change by Joannah Nanjekye :


--
keywords: +patch
pull_requests: +20514
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/21369

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41224] Document is_annotate() in symtable and update doc strings



Change by Joannah Nanjekye :


--
title: Document is_annotated() in the symtable module -> Document is_annotate() 
in symtable and update doc strings

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41224] Document is_annotated() in the symtable module



New submission from Joannah Nanjekye :

The function is_annotated() in symtable is not documented. Am using this
opportunity to also add docstrings to the methods that didnot have them already.

--
assignee: docs@python
components: Documentation
messages: 373200
nosy: docs@python, nanjekyejoannah
priority: normal
severity: normal
status: open
title: Document is_annotated() in the symtable module
type: enhancement
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41210] LZMADecompressor.decompress(FORMAT_RAW) truncate output when input is paticular LZMA+BCJ data

2020-07-06 Thread Hiroshi Miura



Hiroshi Miura  added the comment:

>Compression filters:
>FILTER_LZMA1 (for use with FORMAT_ALONE)
>FILTER_LZMA2 (for use with FORMAT_XZ and FORMAT_RAW)

I look into past discussion  BPO-6715 when lzma module proposed.
https://bugs.python.org/issue6715

There is an only comment about FORMAT_ALONE and LZMA1 here 
https://bugs.python.org/issue6715#msg92174

> .lzma is actually not a format. It is just the raw output of the LZMA1
> coder. XZ instead is a container format for the LZMA2 coder, which
probably means LZMA+some metadata.

It said FORMAT_ALONE decode .lzma archive which use LZMA1 as coder and 
FORMAT_XZ decode .xz archive which use LZMA2 as coder.
There are no discussion about FORMAT_RAW.

This indicate an opposite relation between two things.
FORMAT_ALONE should use with LZMA1.
FORMAT_XZ should use with LZMA2. 

FORMAT_RAW actually no limitation against LZMA1/2.

Here is another discussion about lzma_raw_encoder and LZMA1.
A xz/liblzma maintainer Lasse suggest lzma_raw_encoder is usable for LZMA1.
https://sourceforge.net/p/lzmautils/discussion/708858/thread/cd04b6ace0/#6050


I think we need fix the document.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

2020-07-06 Thread Guido van Rossum



Guido van Rossum  added the comment:

There's no python-dev discussion; if you want more feedback I recommend 
starting on python-ideas first (on either forum you may expect pushback because 
this is not about a proposed change to Python or its workflow).

The Lib/ast.py module will continue to be the official API for the standard 
AST. It is a simple wrapper around the builtin parser (at least in CPython -- I 
don't actually know to what extent other Python implementations support it, but 
they certainly *could*). And in 3.9 and later the AST is already being produced 
using the *new* parser.

We want to deprecate lib2to3 because nobody is interested in maintaining it., 
Having it in the stdlib, with its strict backwards compatibility requirements, 
makes it difficult to do a good job at updating it. This is why it's been 
forked repeatedly -- once forked, the owner of the fork can make changes 
easily, preserving the API perfectly (if so desired) and maintaining 
compatibility with older Python versions.

My own thoughts are that libraries like LibCST and parso have two sides: an API 
for the AST, and a way to parse source code into an AST. Usually the parsing 
API is incredibly simple -- e.g. a function to parse a file and another 
function to parse a string. And there's no reason for the AST API to change 
just because the parsing algorithm has changed.

Finally, we already have a (rough) Python implementation of the PEG parser too 
-- in fact it's included in Tools/peg_generator (and used to regenerate the 
metaparser). This reads the same grammar format (i.e. Grammar/python.gram) and 
generates Python code instead of C code to do the parsing. It's easy to 
retarget the tokenizer of the generated Python code.

So a decent way forward might be to pick one of the 3rd party libraries 
(perhaps parso, which is itself a fork of lib2to3 and what LibCST builds on) 
and update its parser to use a PEG parser generated using the PEG generator 
from Tools/peg_generator (which people are welcome to fork).

This might be a summer-of-code-sized project.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41220] add optional make_key argument to lru_cache



Change by Raymond Hettinger :


--
Removed message: https://bugs.python.org/msg373196

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41220] add optional make_key argument to lru_cache



Raymond Hettinger  added the comment:

>>> from functools import lru_cache
>>> def my_make_key(my_list):
...   return my_list[0]
...
>>> @lru_cache(128, make_key=my_make_key)
... def cached_func(my_list):
...   return sum(my_list)
...
>>> cached_func([10, 20, 30])
60
>>> cached_func([10, 11, 12]) # <-- Why would we want this to return 60?
60

This seems unsafe.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41219] Mimetypes doesn't support audio/webm

2020-07-06 Thread Roundup Robot



Change by Roundup Robot :


--
keywords: +patch
nosy: +python-dev
nosy_count: 1.0 -> 2.0
pull_requests: +20513
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/21368

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41220] add optional make_key argument to lru_cache



Raymond Hettinger  added the comment:

What is the given example trying to accomplish?  It looks like every possible 
input list would be cached using just the first element of the list.  Would 
cached_func([10, 20, 30]) and cached_func([10, 11, 12]) return the same sum?  
If so, why would that be desirable?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41220] add optional make_key argument to lru_cache



Change by Raymond Hettinger :


--
Removed message: https://bugs.python.org/msg373194

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41215] AIX: build fails for xlc/xlC since new PEG parser



miss-islington  added the comment:


New changeset edeaf61b6827ab3a8673aff1fb7717917f08f003 by Miss Islington (bot) 
in branch '3.9':
bpo-41215: Make assertion in the new parser more strict (GH-21364)
https://github.com/python/cpython/commit/edeaf61b6827ab3a8673aff1fb7717917f08f003


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41220] add optional make_key argument to lru_cache



Raymond Hettinger  added the comment:

What is the given example trying to accomplish?  It looks like every possible 
input list would be cached using just the first element of the list.  Would 
cached_func([10, 20, 30]) and cached_func([10, 11, 12]) return the same sum?  
If not, why would that be desireable?

--
assignee:  -> rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41218] PyCF_ALLOW_TOP_LEVEL_AWAIT + list comprehension set .CO_COROUTINE falg.



miss-islington  added the comment:


New changeset b71ff9a5b6e60ee1209a04d2e0e58d9a2e341db3 by Miss Islington (bot) 
in branch '3.8':
bpo-41218: Improve the test cases for test_compile_top_level_await_no_coro 
(GH-21363)
https://github.com/python/cpython/commit/b71ff9a5b6e60ee1209a04d2e0e58d9a2e341db3


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22239] asyncio: nested event loop

2020-07-06 Thread mike bayer



mike bayer  added the comment:

yes so if you have async/await all internal, are you saying you can make that 
work for synchronous code *without* running the event loop?  that is, some kind 
of container that just does the right thing?  my concern with that would still 
be performance.When asyncio was based on yield and exception throws, that 
was a lot of overhead to add to functions and that was what my performance 
testing some years back showed.   w/ async/await I'm sure things have been 
optimized, but in general when i have function a() -> b() -> c(), I am trying 
to iron as much Python overhead as I possibly can out of that and I'd be 
concerned that the machinery to work through async/await would add latency.   
additionally if it was async/await internally but then i need to access the 
majority of Python DBAPIs that are sync, I need a thread pool anyway, right?  
which is also another big barrier to jump over.

It seems you were involved with urllib3's approach to use a code rewriter 
rather than a runtime approach based on the discussion at 
https://github.com/urllib3/urllib3/issues/1323 , but it's not clear if Python 2 
compatibility was the only factor or if the concern of "writing a giant shim" 
was also.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41215] AIX: build fails for xlc/xlC since new PEG parser



Lysandros Nikolaou  added the comment:


New changeset 782f44b8fb07ec33cee148b2b6b4cf53024fe0cd by Lysandros Nikolaou in 
branch 'master':
bpo-41215: Make assertion in the new parser more strict (GH-21364)
https://github.com/python/cpython/commit/782f44b8fb07ec33cee148b2b6b4cf53024fe0cd


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41215] AIX: build fails for xlc/xlC since new PEG parser



Change by miss-islington :


--
pull_requests: +20512
pull_request: https://github.com/python/cpython/pull/21367

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41218] PyCF_ALLOW_TOP_LEVEL_AWAIT + list comprehension set .CO_COROUTINE falg.



Change by Pablo Galindo Salgado :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41218] PyCF_ALLOW_TOP_LEVEL_AWAIT + list comprehension set .CO_COROUTINE falg.



Change by miss-islington :


--
pull_requests: +20510
pull_request: https://github.com/python/cpython/pull/21365

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41218] PyCF_ALLOW_TOP_LEVEL_AWAIT + list comprehension set .CO_COROUTINE falg.



Change by miss-islington :


--
pull_requests: +20511
pull_request: https://github.com/python/cpython/pull/21366

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41218] PyCF_ALLOW_TOP_LEVEL_AWAIT + list comprehension set .CO_COROUTINE falg.



Pablo Galindo Salgado  added the comment:


New changeset c2c1f1f906cdeb40576880d4b6a4f8fcbc016eb8 by Pablo Galindo in 
branch 'master':
bpo-41218: Improve the test cases for test_compile_top_level_await_no_coro 
(GH-21363)
https://github.com/python/cpython/commit/c2c1f1f906cdeb40576880d4b6a4f8fcbc016eb8


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41218] PyCF_ALLOW_TOP_LEVEL_AWAIT + list comprehension set .CO_COROUTINE falg.



Pablo Galindo Salgado  added the comment:


New changeset 6488a4a3c9790040059fc5d293e518f193daac8d by Pablo Galindo in 
branch '3.9':
[3.9] bpo-41218: Only mark async code with CO_COROUTINE. (GH-21357) (GH-21362)
https://github.com/python/cpython/commit/6488a4a3c9790040059fc5d293e518f193daac8d


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41215] AIX: build fails for xlc/xlC since new PEG parser



Lysandros Nikolaou  added the comment:

> Can you prepare a PR?

Done.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41215] AIX: build fails for xlc/xlC since new PEG parser



Change by Lysandros Nikolaou :


--
pull_requests: +20509
pull_request: https://github.com/python/cpython/pull/21364

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39107] Upgrade tcl/tk to 8.6.10 (Windows and maxOS)

2020-07-06 Thread Aivar Annamaa



Aivar Annamaa  added the comment:

According to the comments under https://github.com/python/cpython/pull/18982 
there is no point in creating a PR, so I'll just share my experiences.

I was able to build current master with Tcl/Tk 8.6.10 both on Windows 10 
(64-bit) and on macOS Catalina. I ran the tests on Windows and no 
Tkinter-related tests failed. I tested it manually on both platforms by poking 
around in IDLE and Thonny IDE and saw no (big) issues. (The only regression I 
saw in Thonny was Toplevel tooltips gaining a title bar, but I could solve it 
by adding a "wm_overrideredirect(1)"). 

One of the new features on macOS -- support for dark mode -- looked nice. 
Half-baked emoji support had not regressed at least.

I really hope you'll find time to include Tk 8.6.10 in the last beta of Python 
3.9!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22239] asyncio: nested event loop

2020-07-06 Thread Yury Selivanov



Yury Selivanov  added the comment:

> I think this is a really critical technique to have so that libraries that 
> mediate between a user-facing facade and TCP based backends no longer have to 
> make a hard choice about if they are to support sync vs. async (or async with 
> an optional sync facade around it).

If this works for such a big and elaborate framework as SQLA, we can definitely 
highlight this as a valid approach and even add a link to a blog post from the 
docs. We'll need to add an asyncio specific FAQ page for that or something 
similar.

Another approach, which would probably be a nonstarter for SQLA, is to use 
async/await for literally everything internally, and provide a tiny synchronous 
facade on top.  Funny thing you don't even need an event loop for that, just 
the basic understanding of how coroutines work internally.  I used this to 
create the edgedb-python package which has both sync and async first-class 
support with one code base.  Sync is even faster there in simple throughput 
benchmarks (as expected).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40360] Deprecate lib2to3 (and 2to3) for future removal

2020-07-06 Thread Peter Ludemann

Peter Ludemann added the comment:

Looking at the suggested successor tools (redbaron, libCST, parso, awpa) ...
all of them appear to use some variant of pgen2. But at some point Python will
be using a PEG approach (PEP 617), and therefor the pgen2 approach apparently
won't work.

For a number of projects, it's important to have a parse tree that contains all
the "whitespace" information (indent, dedent, comment, newline, etc.) As far as
I can tell, the new PEG parser won't provide that, and it seems that none of
the successor tools will be able to handle future versions of Python syntax.

So, three questions:
1. Am I right that all proposed replacements (redbaron, libCST, parso, awpa)
use some variation of the LL(1) and therefore will have trouble in the future?
2. Are there any plans (either part of the core development or as a project)
for one of these replacements that is PEG-based? (Or a new project?)
3. Is Lib/ast.py going to continue being supported? (I infer that it will, with
the change from LL(1) to PEG being mostly transparent -
https://mail.python.org/archives/list/python-...@python.org/thread/HOZ2RI3FXUEMAT4XAX4UHFN4PKG5J5GR/#4D3B2NM2JMV2UKIT6EV5Q2A6XK2HXDEH
)

If Lib/ast.py continues to be supported, I think I can see a way of providing
functionality similar to lib2to3 (in terms of an AST-ish thing with
"whitespace" from the source, sufficient for tools such as yapf, black,
pykythe, pytype, mypy, etc.) as a kind of wrapper to ast.py.
I suppose I should discuss this idea on python-dev? Is there an ongoing
discussion? (I couldn't find any but might have been using the wrong search
terms)

___
Python tracker

___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41177] ConvertingList and ConvertingTuple lack iterators and ConvertingDict lacks items()

2020-07-06 Thread Brett Hannigan



Brett Hannigan  added the comment:

O.K. CLA is now signed and if I check on the "check-yourself" with my github 
user it is showing that I have signed it now.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22239] asyncio: nested event loop

2020-07-06 Thread mike bayer



mike bayer  added the comment:

>  This recipe was one of the reasons why I added `loop.set_task_factory` 
> method to the spec, so that it's possible to implement this in an *existing* 
> event loop like uvloop. But ultimately asyncio is flexible enough to let 
> users use their own event loop which can be compatible with greenlets among 
> other improvements.

Right, when I sought to look at this, I know that my users want to use the 
regular event loop in asyncio or whatever system they are using.


> Ultimately, asyncio will probably never ship with greenlets integration 
> enabled by default, but we should definitely make it possible (if there are 
> some limitations now).  It doesn't seem to me that nested event loops are 
> needed for this, right?

So right, the approach I came up with does not need nested event loops and it's 
my vague understanding that nesting event loops is more difficult to debug, 
because you have these two separate loops handing off to each other.

What is most striking about my recipe is that it never even leaves the default 
event loop.  Originally my first iteration when I was trying to get it working, 
I had a separate thread going on, as it seemed intuitive that "of course you 
need a thread to bridge async and sync code" but then I erased the "Thread()" 
part around it and then it just worked anyway.   Like it's simple enough that 
shipping this as a third party library is almost not even worth it, you can 
just drop this in wherever.   If different libraries had their own drop-in of 
this, they would even work together.   greenlet is really like totally 
cheating.  

the philosophical thing here is, usually in my many twitter debates on the 
subject, folks get into how they like the explicit async and await keywords and 
they like that IO is explicit.   So I'm seeking to keep these people happy and 
give then "async def execute(sql)", and use an async DB driver, but then the 
library that these folks are using is internally not actually explicit IO.  But 
they don't have to see that, I've done all the work of the "implicit IO" stuff 
and in a library it's not the same kind of problem anyway. I think this is 
a really critical technique to have so that libraries that mediate between a 
user-facing facade and TCP based backends no longer have to make a hard choice 
about if they are to support sync vs. async (or async with an optional sync 
facade around it).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41215] AIX: build fails for xlc/xlC since new PEG parser



Pablo Galindo Salgado  added the comment:

> Pablo, on second thought, should we maybe change the assertion to 
> `assert(name_len > 0)`?

Yup, although not sure how that can happen (but I agree that's the point of 
assertions *wink*) Can you prepare a PR?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41202] Allow to provide custom exception handler to asyncio.run()

2020-07-06 Thread Yury Selivanov



Yury Selivanov  added the comment:

The idiomatic way:


async def main():
  loop = asyncio.get_running_loop()
  loop.set_exception_handler(...)
  # other code

asyncio.run(main())

We don't want to add new arguments to asyncio.run as there would be too many.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41218] PyCF_ALLOW_TOP_LEVEL_AWAIT + list comprehension set .CO_COROUTINE falg.



miss-islington  added the comment:


New changeset 41db8ffc59566b8552f9cce4452ee8afad00aa63 by Miss Islington (bot) 
in branch '3.8':
bpo-41218: Only mark async code with CO_COROUTINE. (GH-21357)
https://github.com/python/cpython/commit/41db8ffc59566b8552f9cce4452ee8afad00aa63


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41215] AIX: build fails for xlc/xlC since new PEG parser



Lysandros Nikolaou  added the comment:

Pablo, on second thought, should we maybe change the assertion to 
`assert(name_len > 0)`?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41215] AIX: build fails for xlc/xlC since new PEG parser



Pablo Galindo Salgado  added the comment:

Michael, could you check the latest master and 3.9 HEAD? If you don;t see the 
problem anymore, we can close this issue :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22239] asyncio: nested event loop

2020-07-06 Thread Yury Selivanov



Yury Selivanov  added the comment:

Thanks for posting this, Mike.

> Vague claims of "framework X is faster because it's async" appear, impossible 
> to confirm as it is unknown how much of their performance gains come from the 
> "async" aspect and how much of it is that they happened to rewrite a new 
> framework from scratch in a completely different way (hint: it's the latter).

These kind of claims are not specific to async vs. sync. They are all over the 
place for every two pieces of comparable technologies. While novice users might 
base their technology choice purely on such benchmarks, it's less of an issue 
for startups/tech companies.

That said, I agree with most of your points so far.

> The asyncpg project, one of the few asyncio database drivers that exists, 
> notes in its FAQ "asyncpg uses asynchronous execution model and API, which is 
> fundamentally incompatible with SQLAlchemy" [2], yet we know this is not true 
>  because SQLAlchemy works just fine with gevent and eventlet, with no 
> architectural changes at all.

But it is true. Making asynchronous network requests in asyncio requires 
async/await or using callbacks and it's not possible to do them, say, from 
__getattr__ (you mention this yourself).  This is what that particular comment 
is about, nothing more. Using gevent and eventlet as examples in this 
particular context isn't helping you. Apologies for nitpicking, I know it's not 
the point of this discussion.

> A day later, someone took the same idea and got Flask to work in an asyncio 
> event loop at [5].  The general idea of using greenlet in this way is also 
> present at [6], so I won't be patenting this idea today as oremanj can claim 
> prior art.

Yes, this approach definitely works and I even did that in production myself a 
few years ago with https://github.com/1st1/greenio (it's terribly outdated now).

> The recipe is simple and so far appears to be very effective.

This recipe was one of the reasons why I added `loop.set_task_factory` method 
to the spec, so that it's possible to implement this in an *existing* event 
loop like uvloop. But ultimately asyncio is flexible enough to let users use 
their own event loop which can be compatible with greenlets among other 
improvements.

Ultimately, asyncio will probably never ship with greenlets integration enabled 
by default, but we should definitely make it possible (if there are some 
limitations now).  It doesn't seem to me that nested event loops are needed for 
this, right?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41218] PyCF_ALLOW_TOP_LEVEL_AWAIT + list comprehension set .CO_COROUTINE falg.



Change by Pablo Galindo Salgado :


--
pull_requests: +20508
pull_request: https://github.com/python/cpython/pull/21363

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41218] PyCF_ALLOW_TOP_LEVEL_AWAIT + list comprehension set .CO_COROUTINE falg.



Change by Pablo Galindo Salgado :


--
nosy: +pablogsal
nosy_count: 3.0 -> 4.0
pull_requests: +20507
pull_request: https://github.com/python/cpython/pull/21362

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41218] PyCF_ALLOW_TOP_LEVEL_AWAIT + list comprehension set .CO_COROUTINE falg.

2020-07-06 Thread Łukasz Langa


Łukasz Langa  added the comment:


New changeset bd46174a5a09a54e5ae1077909f923f56a7cf710 by Matthias Bussonnier 
in branch 'master':
bpo-41218: Only mark async code with CO_COROUTINE. (#21357)
https://github.com/python/cpython/commit/bd46174a5a09a54e5ae1077909f923f56a7cf710


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41218] PyCF_ALLOW_TOP_LEVEL_AWAIT + list comprehension set .CO_COROUTINE falg.



Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 2.0 -> 3.0
pull_requests: +20506
pull_request: https://github.com/python/cpython/pull/21361

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41223] `object`-backed `memoryview`'s `tolist` errors

2020-07-06 Thread jakirkham



New submission from jakirkham :

When working with an `object`-backed `memoryview`, it seems we are unable to 
coerce it to a `list`. This would be useful as it would provide a way to get 
the underlying `object`'s into something a bit easier to work with.

```
In [1]: import numpy

In [2]: a = numpy.array(["abc", "def", "ghi"], dtype=object)

In [3]: m = memoryview(a)   

In [4]: m.tolist()  
---
NotImplementedError   Traceback (most recent call last)
 in 
> 1 m.tolist()

NotImplementedError: memoryview: format O not supported
```

--
messages: 373175
nosy: jakirkham
priority: normal
severity: normal
status: open
title: `object`-backed `memoryview`'s `tolist` errors
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35786] get_lock() method is not present for Values created using multiprocessing.Manager()



Change by Steve Dower :


--
components:  -Windows
nosy:  -steve.dower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35786] get_lock() method is not present for Values created using multiprocessing.Manager()

2020-07-06 Thread Ian Jacob Bertolacci



Ian Jacob Bertolacci  added the comment:

What's being done about this?
I would say this is less "misleading documentation" and more "incorrect 
implementation"

There is also not an obvious temporary work-around.

--
nosy: +IanBertolacci

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41215] AIX: build fails for xlc/xlC since new PEG parser

2020-07-06 Thread Michael Felt


Michael Felt  added the comment:

I also tested xlC v16, and then it just hung, for hours - while all of you were 
being more productive. 

I’ll kickoff my bot again, and see how it goes. 

Sent from my iPhone

> On 6 Jul 2020, at 21:31, miss-islington  wrote:
> 
> 
> miss-islington  added the comment:
> 
> 
> New changeset 1ac0cbca369f16f9191833dd54536482fb141a98 by Pablo Galindo in 
> branch 'master':
> bpo-41215: Don't use NULL by default in the PEG parser keyword list (GH-21355)
> https://github.com/python/cpython/commit/1ac0cbca369f16f9191833dd54536482fb141a98
> 
> 
> --
> 
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41217] Obsolete note for default asyncio event loop on Windows

2020-07-06 Thread Zackery Spytz



Change by Zackery Spytz :


--
keywords: +patch
nosy: +ZackerySpytz
nosy_count: 4.0 -> 5.0
pull_requests: +20505
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/21360

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41207] distutils.command.build_ext raises FileNotFoundError

2020-07-06 Thread Jason R. Coombs



Change by Jason R. Coombs :


--
keywords: +patch
pull_requests: +20504
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/21359

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41207] distutils.command.build_ext raises FileNotFoundError

2020-07-06 Thread Jason R. Coombs



Jason R. Coombs  added the comment:

I learned the magical incantation to port commits from pypa/distutils to 
CPython:

```
cpython bugfix/41207-rewrite-filenotfound $ git -C ~/p/pypa/distutils 
format-patch HEAD~2 --stdout | git am --directory Lib   
 
Applying: Add test for spawn when exe is missing. Ref pypa/distutils#3.
Applying: Replace OSError with DistutilsExecError. Fixes pypa/distutils#3 and 
pypa/setuptools#2228 and bpo-41207.
```

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40742] Doc: Parallel build break audit table

2020-07-06 Thread Julien Palard



Change by Julien Palard :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40742] Doc: Parallel build break audit table

2020-07-06 Thread Julien Palard



Julien Palard  added the comment:


New changeset a103e73ce8d34e3af5f556ee9090ce89249d565e by Julien Palard in 
branch 'master':
bpo-40742: Doc: fix parallel build. (GH-21237)
https://github.com/python/cpython/commit/a103e73ce8d34e3af5f556ee9090ce89249d565e


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41207] distutils.command.build_ext raises FileNotFoundError

2020-07-06 Thread Jason R. Coombs



Jason R. Coombs  added the comment:

Sure. I'll submit patches.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41014] warning: 'sqlite3_trace' is deprecated

2020-07-06 Thread Ronald Oussoren



Ronald Oussoren  added the comment:

I agree, this is a "feature" of the SQLite headers as shipped by Apple.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed
type: enhancement -> compile error

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41218] PyCF_ALLOW_TOP_LEVEL_AWAIT + list comprehension set .CO_COROUTINE falg.



Change by Pablo Galindo Salgado :


--
nosy: +lukasz.langa
priority: normal -> release blocker

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39562] Asynchronous comprehensions don't work in asyncio REPL



Pablo Galindo Salgado  added the comment:

> As the changes here has been released maybe you want to reclose and marked 
> https://bugs.python.org/issue41218 as blocker it should also be fixed by
https://github.com/python/cpython/pull/21357

Thanks for the heads up, Matthias!

Yes, I think is the best thing to do. For some reason, I thought this was not 
still released :(

--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41221] Output of print() might get truncated in unbuffered mode

2020-07-06 Thread Manuel Jacob


Manuel Jacob  added the comment:

2147479552 is the 0x7000 bytes limit documented for write() on Linux 
(source: https://man7.org/linux/man-pages/man2/write.2.html). The limit could 
be even smaller in other circumstances or other systems.

I’m adding Victor Stinner to the nosy list, as he added the code limiting 
writes to the console on Windows in e0daff1c61e323d2a39dd8241de67082d1f10fd7, 
and he might have an opinion on the topic.

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39375] Document os.environ[x] = y and os.putenv() as thread unsafe



Change by Joannah Nanjekye :


--
nosy: +nanjekyejoannah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32528] Change base class for futures.CancelledError

2020-07-06 Thread Bruce Merry



Bruce Merry  added the comment:

FYI this has just bitten me after updating my OS to one that ships Python 3.8. 
It is code that was written with asyncio cancellation in mind and which 
expected CancelledError to be caught with "except Exception" (the exception 
block unwound incomplete operations before re-raising the exception).

It's obviously too late to do anything about Python 3.8, but I'm mentioning 
this as a data point in support of having a deprecation period if similar 
changes are made in future.

On the plus side, while fixing up my code and checking all instances of "except 
Exception" I found some places where this change did fix latent cancellation 
bugs. So I'm happy with the change, just a little unhappy that it came as a 
surprise.

--
nosy: +bmerry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41222] Undocumented behaviour change of POpen.stdout.readine with bufsize=0 or =1

2020-07-06 Thread Yann Dirson



New submission from Yann Dirson :

On a POpen object created with bufsize=0, stdout.readline() does a buffered 
reading with python3, whereas in 2.7 it did char-by-char reading.  See attached 
example.

As a result, a poll including the stdout object suffers a behaviour change when 
stdout is ready for writing and there is more than one line of data available.  
In both cases we get notified by poll() that data is available on the fd and we 
can stdout.readline() and get back to our polling loop.  Then:

* with python2 poll() then returns immediately and stdout.readline() will then 
return the next line

* with python3 poll() now blocks

Running the attached example under strace reveals the underlying difference:

 write(4, "go\n", 3) = 3
 poll([{fd=5, events=POLLIN|POLLERR|POLLHUP}], 1, -1) = 1 ([{fd=5, 
revents=POLLIN}])
-read(5, "x", 1) = 1
-read(5, "x", 1) = 1
-read(5, "x", 1) = 1
-read(5, "x", 1) = 1
-read(5, "x", 1) = 1
-read(5, "x", 1) = 1
-read(5, "x", 1) = 1
-read(5, "x", 1) = 1
-read(5, "x", 1) = 1
-read(5, "x", 1) = 1
-read(5, "x", 1) = 1
-read(5, "x", 1) = 1
-read(5, "\n", 1)= 1
-fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(0x88, 0x2), ...}) = 0
+read(5, "\nyyy\naaa"..., 8192) = 74
 write(1, ">\n", 14) = 14


We can see a buffered read, which explains the behaviour difference.

Changing to bufsize=1, strace does not show a difference here.

This is especially troubling, as the first note in 
https://docs.python.org/3/library/io.html#class-hierarchy mentions that even in 
buffered mode there is an unoptimized readline() implementation.

--
components: IO
files: testproc-unbuffered.py
messages: 373165
nosy: yann
priority: normal
severity: normal
status: open
title: Undocumented behaviour change of POpen.stdout.readine with bufsize=0 or 
=1
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8
Added file: https://bugs.python.org/file49299/testproc-unbuffered.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] [CVE-2020-15523] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath



Steve Dower  added the comment:

Announcement post: 
https://mail.python.org/archives/list/security-annou...@python.org/thread/C5RIXC2ZIML3NOEIOGFPA6ISGU5L2QXL/

CVE-2020-15523 is an invalid search path in Python 3.6 and later on 
Windows. It occurs during Py_Initialize() when the runtime attempts to 
pre-load python3.dll. If Py_SetPath() has been called, the expected 
location is not set, and locations elsewhere on the user's system will 
be searched.

This issue is not triggered when running python.exe. It only applies 
when CPython has been embedded in another application.

Issue: https://bugs.python.org/issue29778
Patch: https://github.com/python/cpython/pull/21297

The next patched releases will be: 3.9.0b5, 3.8.4, 3.7.9 (source only), 
3.6.12 (source only)

Other than applying the patch, applications may mitigate the 
vulnerability by explicitly calling LoadLibrary() on their copy of 
python3.dll before calling Py_Initialize(). Even with the patch applied, 
applications should include a copy of python3.dll alongside their main 
Python DLL.

Thanks to Eric Gantumur for detecting and reporting the issue to the 
Python Security Response Team.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41014] warning: 'sqlite3_trace' is deprecated



Joannah Nanjekye  added the comment:

I think the same reasoning in the discussion from this thread: 
https://bugs.python.org/issue41018 applies to this too.

--
nosy: +nanjekyejoannah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37724] [[Errno 17] File exists: ] # Try create directories that are not part of the archive with



Joannah Nanjekye  added the comment:

This is marked 2.7 is it still relevant?

--
nosy: +nanjekyejoannah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41215] AIX: build fails for xlc/xlC since new PEG parser



miss-islington  added the comment:


New changeset 1ac0cbca369f16f9191833dd54536482fb141a98 by Pablo Galindo in 
branch 'master':
bpo-41215: Don't use NULL by default in the PEG parser keyword list (GH-21355)
https://github.com/python/cpython/commit/1ac0cbca369f16f9191833dd54536482fb141a98


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41215] AIX: build fails for xlc/xlC since new PEG parser



miss-islington  added the comment:


New changeset 54f115dd533653c43b3c5541bf5936b22e484474 by Pablo Galindo in 
branch '3.9':
[3.9] bpo-41215: Don't use NULL by default in the PEG parser keyword list 
(GH-21355) (GH-21356)
https://github.com/python/cpython/commit/54f115dd533653c43b3c5541bf5936b22e484474


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39562] Asynchronous comprehensions don't work in asyncio REPL

2020-07-06 Thread Matthias Bussonnier



Matthias Bussonnier  added the comment:

Pablo, I see you reopened and marked as blocker, 

As the changes here has been released maybe you want to reclose and marked 
https://bugs.python.org/issue41218 as blocker it should also be fixed by
https://github.com/python/cpython/pull/21357

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41218] PyCF_ALLOW_TOP_LEVEL_AWAIT + list comprehension set .CO_COROUTINE falg.

2020-07-06 Thread Matthias Bussonnier



Change by Matthias Bussonnier :


--
keywords: +patch
pull_requests: +20503
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/21357

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41162] Clear audit hooks after destructors



Steve Dower  added the comment:

Merged the initial fix, but we now need to find any exploitable paths that 
remain.

Considering how late in finalization they now run, they're very unlikely to 
succeed at doing anything interesting. However, they'd also qualify as bugs 
(potentially crashes), so we'll want to find ways to make them run earlier 
anyway.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41218] PyCF_ALLOW_TOP_LEVEL_AWAIT + list comprehension set .CO_COROUTINE falg.

2020-07-06 Thread Matthias Bussonnier



Matthias Bussonnier  added the comment:

https://bugs.python.org/issue39562 seem to have triggered that.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] [CVE-2020-15523] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath



Steve Dower  added the comment:

Fixes are in. Also adding the CVE number to the bug title.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
title: _Py_CheckPython3 uses uninitialized dllpath when embedder sets module 
path with Py_SetPath -> [CVE-2020-15523] _Py_CheckPython3 uses uninitialized 
dllpath when embedder sets module path with Py_SetPath

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

2020-07-06 Thread Ned Deily



Ned Deily  added the comment:


New changeset 46cbf6148a46883110883488d3e9febbe46ba861 by Steve Dower in branch 
'3.6':
[3.6] bpo-29778: Ensure python3.dll is loaded from correct locations when 
Python is embedded (GH-21298) (#21354)
https://github.com/python/cpython/commit/46cbf6148a46883110883488d3e9febbe46ba861


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41215] AIX: build fails for xlc/xlC since new PEG parser



Change by Pablo Galindo Salgado :


--
pull_requests: +20502
pull_request: https://github.com/python/cpython/pull/21356

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath



Change by Steve Dower :


--
pull_requests: +20500
pull_request: https://github.com/python/cpython/pull/21354

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41215] AIX: build fails for xlc/xlC since new PEG parser



Change by Pablo Galindo Salgado :


--
keywords: +patch
pull_requests: +20501
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/21355

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41215] AIX: build fails for xlc/xlC since new PEG parser



Pablo Galindo Salgado  added the comment:

> Pablo, I can do that as well, if you don't have the time.

Thanks! I had the PR ready, just needed some time to push it :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41220] add optional make_key argument to lru_cache

2020-07-06 Thread Karthikeyan Singaravelan



Change by Karthikeyan Singaravelan :


--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41220] add optional make_key argument to lru_cache

2020-07-06 Thread Bar Harel



Bar Harel  added the comment:

May I suggest calling it key? Will comply with other Python functions.

--
nosy: +bar.harel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41176] revise Tkinter mainloop dispatching flag behavior

2020-07-06 Thread Richard Sheridan



Richard Sheridan  added the comment:

I'm planning to write the long-awaited Tkinter Internals section of the docs. 
(https://github.com/python/cpython/blame/master/Doc/library/tk.rst#L48) I've 
spent too much time at this point to let it all go down the memory hole. 
Unfortunately, I don't know how ALL of the internals work. Is there someone 
else we could add to nosy that might be interested in writing some subsections?

Also, should this extended docs contribution be a new issue or rolled in with 
this one? Much abbreviated documentation of the new methods in this PR could be 
added to tkinter.rst. The new docs issue would be dependent on this issue since 
I won't be able to complete the docs until we have finished discussing what the 
future behavior of threads waiting for `dispatching` will be (error & poll vs 
Tcl_ConditionWait).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41221] Output of print() might get truncated in unbuffered mode

2020-07-06 Thread Manuel Jacob


New submission from Manuel Jacob :

Without unbuffered mode, it works as expected:

% python -c "import sys; sys.stdout.write('x'*4294967296)" | wc -c
4294967296

% python -c "import sys; print('x'*4294967296)" | wc -c 
4294967297

With unbuffered mode, writes get truncated to 2147479552 bytes on my Linux 
machine:

% python -u -c "import sys; sys.stdout.write('x'*4294967296)" | wc -c   
2147479552

% python -u -c "import sys; print('x'*4294967296)" | wc -c 
2147479553

I didn’t try, but it’s probably an even bigger problem on Windows, where writes 
might be limited to 32767 bytes: 
https://github.com/python/cpython/blob/v3.9.0b4/Python/fileutils.c#L1585

Without unbuffered mode, `sys.stdout.buffer` is a `io.BufferedWriter` object.

% python -c 'import sys; print(sys.stdout.buffer)'
<_io.BufferedWriter name=''>

With unbuffered mode, `sys.stdout.buffer` is a `io.FileIO` object.

% python -u -c 'import sys; print(sys.stdout.buffer)' 
<_io.FileIO name='' mode='wb' closefd=False>

`io.BufferedWriter` implements the `io.BufferedIOBase` interface. 
`io.BufferedIOBase.write()` is documented to write all passed bytes. 
`io.FileIO` implements the `io.RawIOBase` interface. `io.RawIOBase.write()` is 
documented to be able to write less bytes than passed.

`io.TextIOWrapper.write()` is not documented to write all characters it has 
been passed, but e.g. `print()` relies on that.

To fix the problem, it has to be ensured that either
* `sys.stdout.buffer` is an object that guarantees that all bytes passed to its 
`write()` method are written (e.g. deriving from `io.BufferedIOBase`), or
* `io.TextIOWrapper` calls the `write()` method of its underlying binary stream 
until all bytes have been written, or
* users of `io.TextIOWrapper` call `write()` until all characters have been 
written.

In the first two possibilities it probably makes sense to tighten the contract 
of `io.TextIOBase.write` to guarantee that all passed characters are written.

--
components: IO
messages: 373151
nosy: mjacob
priority: normal
severity: normal
status: open
title: Output of print() might get truncated in unbuffered mode
type: behavior
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

2020-07-06 Thread Ned Deily



Ned Deily  added the comment:


New changeset 110dd153662a13b8ae1bb06348e5b1f118ab26d7 by Steve Dower in branch 
'3.7':
[3.7] bpo-29778: Ensure python3.dll is loaded from correct locations when 
Python is embedded (GH-21297) (#21298)
https://github.com/python/cpython/commit/110dd153662a13b8ae1bb06348e5b1f118ab26d7


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41215] AIX: build fails for xlc/xlC since new PEG parser



Lysandros Nikolaou  added the comment:

Pablo, I can do that as well, if you don't have the time.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41215] AIX: build fails for xlc/xlC since new PEG parser



Pablo Galindo Salgado  added the comment:

Well, in any case, I will submit a patch today to the parser generator to 
substitute the NULL for single-element arrays which seems to work.

Michael, could you confirm that this patch works for you:

diff --git a/Parser/pegen/parse.c b/Parser/pegen/parse.c
index 9d3ac575df..e5511bf815 100644
--- a/Parser/pegen/parse.c
+++ b/Parser/pegen/parse.c
@@ -1,6 +1,6 @@
-// @generated by pegen.py from ./Grammar/python.gram
-#include "pegen.h"

+#include "pegen.h"
+// @generated by pegen.py from ./Grammar/python.gram
 #if defined(Py_DEBUG) && defined(Py_BUILD_CORE)
 extern int Py_DebugFlag;
 #define D(x) if (Py_DebugFlag) x;
@@ -9,8 +9,8 @@ extern int Py_DebugFlag;
 #endif
 static const int n_keyword_lists = 15;
 static KeywordToken *reserved_keywords[] = {
-NULL,
-NULL,
+(KeywordToken[]) {{NULL, -1}},
+(KeywordToken[]) {{NULL, -1}},
 (KeywordToken[]) {
 {"if", 510},
 {"in", 518},
@@ -65,11 +65,11 @@ static KeywordToken *reserved_keywords[] = {
 {"nonlocal", 509},
 {NULL, -1},
 },
-NULL,
-NULL,
-NULL,
-NULL,
-NULL,
+(KeywordToken[]) {{NULL, -1}},
+(KeywordToken[]) {{NULL, -1}},
+(KeywordToken[]) {{NULL, -1}},
+(KeywordToken[]) {{NULL, -1}},
+(KeywordToken[]) {{NULL, -1}},
 (KeywordToken[]) {
 {"__peg_parser__", 531},
 {NULL, -1},
@@ -24735,7 +24735,6 @@ _PyPegen_parse(Parser *p)
 // Initialize keywords
 p->keywords = reserved_keywords;
 p->n_keyword_lists = n_keyword_lists;
-
 // Run parser
 void *result = NULL;
 if (p->start_rule == Py_file_input) {

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

2020-07-06 Thread Łukasz Langa


Łukasz Langa  added the comment:


New changeset aa7f7756149a10c64d01f583b71e91814db886ab by Miss Islington (bot) 
in branch '3.8':
bpo-29778: Ensure python3.dll is loaded from correct locations when Python is 
embedded (GH-21297) (GH-21352)
https://github.com/python/cpython/commit/aa7f7756149a10c64d01f583b71e91814db886ab


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41215] AIX: build fails for xlc/xlC since new PEG parser

2020-07-06 Thread David Edelsohn



David Edelsohn  added the comment:

Maybe XLC was being overly aggressive with speculation and it now is fixed. I 
can't tell if Michael's earlier comment meant that it no longer crashes with 
XLC v16.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22239] asyncio: nested event loop

2020-07-06 Thread mike bayer



mike bayer  added the comment:

hey there,

I seem to have two cents to offer so here it is.An obscure issue in the 
Python bug tracker is probably not the right place for this so consider this as 
an early draft of something that maybe I'll talk about more elsewhere.

> This basically divides code into two islands - async and non-async

yes, this is the problem, and at the bottom of this apparently somewhat ranty 
comment is a solution, and the good news is that it does not require Python or 
asyncio be modified.  My concern is kind of around how it is that everyone has 
been OK with the current state of affairs for so long, why it is that "asyncio 
is fundamentally incompatible with library X" is considered to be acceptable, 
and also how easy it was to find a workaround, this is not something I would 
have expected to come up with.  Kind of like you don't expect to invent Velcro 
or windshield wipers.

asyncio's approach is what those of us in the library/framework community call 
"explicit async", you have to mark functions that will be doing IO and the 
points at which IO occurs must also be marked.Long ago it was via callback 
functions, then asyncio turned it into decorators and yields, and finally 
pep492 turned it into async/await, and it is very nicely done.  It is of course 
a feature of asyncio that writing out async/await means your code can in theory 
be clearer as to where IO occurs and all that, and while I don't totally buy 
that myself, I'm of course in favor of that style of coding being available, it 
definitely has its own kind of self-satisfaction built in when you do it.  
That's all great.

But as those of us in the library/framework community also know, asyncio's 
approach essentially means, libraries like Flask, Django, my own SQLAlchemy, 
etc. are all automatically "non-workable" with the asyncio approach; while 
these libraries can certainly have asyncio endpoints added to them, the task as 
designed is not that simple, since to go from an asyncio endpoint all the way 
through library code that doesn't care about async and then down into a 
networking library that again has asyncio endpoints, the publishing of "async" 
and the "await" or yield approach must be wired all the way through every 
function and method.  This is all despite that when you're not at the 
endpoints, the points at which IO occurs is fully predictable such that 
libraries like gevent don't need you to write it.   So we are told that 
libraries have to have full end-to-end rewrites of all their code to work this 
way, or otherwise maintain two codebases, or something like that.

The side effect of this is that a whole bunch of library and framework authors 
now get to create all new libraries and frameworks, which do exactly the same 
thing as all the existing libraries and frameworks, except they sprinkle the 
"async/await" keywords throughout middle tiers as required.  Vague claims of 
"framework X is faster because it's async" appear, impossible to confirm as it 
is unknown how much of their performance gains come from the "async" aspect and 
how much of it is that they happened to rewrite a new framework from scratch in 
a completely different way (hint: it's the latter).

Or in other cases, as if to make it obvious how much the "async/await" keywords 
come down to being more or less boilerplate for the "middle" parts of 
libraries, the urllib3 project wrote the "unasync" project [1] so that they can 
simply maintain two separate codebases, one that has "async/await" and  the 
other which just search-and-replaced them out.

SQLAlchemy has not been "replaced" by this trend as asyncio database libraries 
have not really taken off in Python, and there are very few actual async 
drivers.   Some folks have written SQLAlchemy-async libraries that use 
SQLAlchemy's expression system while they have done the tedious, redundant and 
impossible-to-maintain work of replicating enough of SQLAlchemy's execution 
internals such that a modest "sqlalchemy-like" experience with asyncio can be 
reproduced. But these libraries are closed out from all of the fixes and 
improvements that occur to SQLAlchemy itself, as well as that these systems 
likely target a smaller subset of SQLAlchemy's behaviors and features in any 
case.They certainly can't get the ORM working as the ORM runs lots of SQL 
executions internally, all of which would have to propagate their "asyncness" 
outwards throughout hundreds of functions.

The asyncpg project, one of the few asyncio database drivers that exists, notes 
in its FAQ "asyncpg uses asynchronous execution model and API, which is 
fundamentally incompatible with SQLAlchemy" [2], yet we know this is not true  
because SQLAlchemy works just fine with gevent and eventlet, with no 
architectural changes at all.  Using libraries like SQLAlchemy or Django with a 
non-blocking IO, event-based model is commonplace.   It's the "explicit" part 
of it that is hard, which is because of

[issue41215] AIX: build fails for xlc/xlC since new PEG parser

2020-07-06 Thread Michael Felt



Michael Felt  added the comment:

Note: - two different systems, different HW, different OS levels.

xlc-v11, master : commit deb016224cc506503fb05e821a60158c83918ed4 (HEAD
-> master, upstream/master, upstream/HEAD)

Segmentation fault in _PyEval_EvalFrameDefault at line 941 in file
"../git/py39-3.10/Include/object.h" ($t1)
"object.h" has only 659 lines
(dbx) where
_PyEval_EvalFrameDefault(tstate = 0x300af594, f = 0x0017, throwflag
= 15), line 941 in "object.h"
_PyEval_EvalCode(tstate = 0x1001c454, _co = 0x30092520, globals =
0x2ff20190, locals = 0x10324213, args = 0x100b2798, argcount = 44,
kwnames = 0x2ff201b0, kwargs = 0x2228228f, kwcount = 0, kwstep = 2, defs
= (nil), defcount = 0, kwdefs = (nil), closure = (nil), name = (nil),
qualname = (nil)), line 40 in "pycore_ceval.h"
_PyEval_EvalCodeWithName(_co = 0x0003, globals = 0x2ff20270, locals
= 0x2ff20220, args = 0x2004426c, argcount = 806222336, kwnames = (nil),
kwargs = 0x2ff20220, kwcount = 0, kwstep = 2, defs = (nil), defcount =
0, kwdefs = (nil), closure = (nil), name = (nil), qualname = (nil)),
line 4416 in "ceval.c"
PyEval_EvalCodeEx(_co = 0x10110c74, globals = 0x20095072, locals =
(nil), args = 0x300af598, argcount = 806024596, kws = 0x200950aa,
kwcount = 537481208, defs = 0x300bcf10, defcount = 0, kwdefs = (nil),
closure = (nil)), line 4415 in "ceval.c"
builtin___build_class__(self = 0x100b8bdc, args = (nil), nargs =
804389664, kwnames = 0x2004426c), line 222 in "bltinmodule.c"
cfunction_vectorcall_FASTCALL_KEYWORDS(func = 0x20060d88, args =
0x30046448, nargsf = 805553488, kwnames = 0x3004abb8), line 459 in
"methodobject.c"
_PyEval_EvalFrameDefault(tstate = 0x01ff, f = 0x200012d0, throwflag
= 80439), line 628 in "abstract.h"
_PyEval_EvalCode(tstate = 0x100d4eb4, _co = 0x3003f440, globals =
0x300b2ca8, locals = 0x300b516c, args = 0x300b5168, argcount =
805586012, kwnames = 0x30044450, kwargs = 0x30001028, kwcount = 0,
kwstep = 2, defs = (nil), defcount = 0, kwdefs = (nil), closure = (nil),
name = (nil), qualname = (nil)), line 40 in "pycore_ceval.h"
_PyEval_EvalCodeWithName(_co = 0x3004abd8, globals = 0x300ac910, locals
= 0x2ff205b0, args = 0x2004426c, argcount = 269191840, kwnames =
0x2ff2060c, kwargs = 0x, kwcount = 0, kwstep = 2, defs = (nil),
defcount = 0, kwdefs = (nil), closure = (nil), name = (nil), qualname =
(nil)), line 4416 in "ceval.c"
PyEval_EvalCodeEx(_co = 0x10090ab0, globals = 0x0004, locals =
0x30062be8, args = (nil), argcount = 805624984, kws = 0x20060d88,
kwcount = 804390432, defs = 0x0004, defcount = 0, kwdefs = (nil),
closure = (nil)), line 4415 in "ceval.c"
PyEval_EvalCode(co = 0x103a1254, globals = 0x103a153e, locals =
0x300d1a48), line 857 in "ceval.c"
builtin_exec_impl(module = 0x300b8ec8, source = 0x300671d8, globals =
0x2002af08, locals = 0x300af170), line 1035 in "bltinmodule.c"
builtin_exec(module = 0x30059f6c, args = 0x30050d20, nargs = 804390720),
line 371 in "bltinmodule.c.h"
cfunction_vectorcall_FASTCALL(func = 0x101c1120, args = 0x3006906c,
nargsf = 805736544, kwnames = 0x2004426c), line 443 in "methodobject.c"
PyVectorcall_Call(callable = 0xd0129a34, tuple = 0x3008c028, kwargs =
0x2ff207f0), line 249 in "call.c"
_PyObject_Call(tstate = 0x100b259c, callable = 0x30027ce2, args =
0x2ff20850, kwargs = 0x000c), line 265 in "call.c"
do_call_core(tstate = (nil), func = (nil), callargs = 0x3003c550, kwdict
= 0x3004abb8), line 5142 in "ceval.c"
_PyEval_EvalFrameDefault(tstate = 0x, f = 0x300b2268, throwflag
= 805759592), line 3603 in "ceval.c"
_PyEval_EvalCode(tstate = 0x3005e9d8, _co = 0x30062290, globals = (nil),
locals = 0x20026000, args = 0x300ac690, argcount = 805331176, kwnames =
0x2ff20a60, kwargs = 0x422822cf, kwcount = 0, kwstep = 1, defs = (nil),
defcount = 0, kwdefs = (nil), closure = (nil), name = 0x300197c8,
qualname = 0x300197c8), line 40 in "pycore_ceval.h"
_PyFunction_Vectorcall(func = 0x3005dbac, stack = (nil), nargsf =
805553488, kwnames = 0x30040960), line 417 in "call.c"
_PyEval_EvalFrameDefault(tstate = 0x10075ac4, f = 0x30090e10, throwflag
= 804391856), line 628 in "abstract.h"
function_code_fastcall(tstate = 0x300a4568, co = 0x3004a1b0, args =
0x2000af50, nargs = 1, globals = 0x3004a688), line 40 in "pycore_ceval.h"
_PyEval_EvalFrameDefault(tstate = (nil), f = 0x0001, throwflag =
804392192), line 628 in "abstract.h"
function_code_fastcall(tstate = 0x3009116c, co = 0x30031176, args =
0x30031160, nargs = 805622640, globals = (nil)), line 40 in "pycore_ceval.h"
_PyEval_EvalFrameDefault(tstate = 0x10075ac4, f = 0x20060d88, throwflag
= 804392528), line 628 in "abstract.h"
function_code_fastcall(tstate = 0x0094, co = 0x00ab, args =
0x3003c550, nargs = 805702704, globals = 0x3004abd8), line 40 in
"pycore_ceval.h"
_PyEval_EvalFrameDefault(tstate = (nil), f = (nil), throwflag =
804393056), line 628 in "abstract.h"
function_code_fastcall(tstate = 0x3004ab90, co = 0x3004a7c8, args =
0x2ff21020, nargs = 537150060, globals = 0x10073f98), line 40 in

[issue41215] AIX: build fails for xlc/xlC since new PEG parser



Pablo Galindo Salgado  added the comment:

> I don't believe that this is an XLC bug, but I suspect that it is undefined 
> behavior / implementation-defined behavior.

I was looking at the C99 standard but could not find anything that makes this 
undefined. Do you know what this construct is contradicting the standard on?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath



miss-islington  added the comment:


New changeset 4981fe36c7477303de830e8dca929a02caaaffe4 by Miss Islington (bot) 
in branch '3.9':
bpo-29778: Ensure python3.dll is loaded from correct locations when Python is 
embedded (GH-21297)
https://github.com/python/cpython/commit/4981fe36c7477303de830e8dca929a02caaaffe4


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41220] add optional make_key argument to lru_cache

2020-07-06 Thread Itay azolay



Change by Itay azolay :


--
keywords: +patch
pull_requests: +20499
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/21353

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41220] add optional make_key argument to lru_cache

2020-07-06 Thread Itay azolay



New submission from Itay azolay :

I'd like to add optional argument to lru_cache.
This argument is a user given function that will replace the default behaviour 
of creating a key from the args/kwds of the function.

for example:

def my_make_key(my_list):
  return my_list[0]

@lru_cache(128, make_key=my_make_key)
def cached_func(my_list):
  return sum(my_list)

This will creating a cached function that accepts immutable.
Also, It will allow user to add custom functions from knowledge about the 
expected function input, without the need to create custom classes and/or 
overriding __hash__

--
components: Library (Lib)
messages: 373141
nosy: Itayazolay
priority: normal
severity: normal
status: open
title: add optional make_key argument to lru_cache
type: enhancement
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41219] Mimetypes doesn't support audio/webm

2020-07-06 Thread cere blanco



New submission from cere blanco :

Mimetypes doesn't support audio/webm

--
components: Library (Lib)
messages: 373140
nosy: cere blanco
priority: normal
severity: normal
status: open
title: Mimetypes doesn't support audio/webm
type: enhancement
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41208] An exploitable segmentation fault in marshal module

2020-07-06 Thread Iman Sharafodin



Iman Sharafodin  added the comment:

Sure. Thank you.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath



Change by miss-islington :


--
pull_requests: +20498
pull_request: https://github.com/python/cpython/pull/21352

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath



Steve Dower  added the comment:


New changeset dcbaa1b49cd9062fb9ba2b9d49555ac6cd8c60b5 by Steve Dower in branch 
'master':
bpo-29778: Ensure python3.dll is loaded from correct locations when Python is 
embedded (GH-21297)
https://github.com/python/cpython/commit/dcbaa1b49cd9062fb9ba2b9d49555ac6cd8c60b5


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath