[Python-Dev] [RELEASE] Python 2.7.14 release candidate 1

[Benjamin Peterson]

I'm happy to announce the immediate availability of Python 2.7.14
release candidate 1, a testing release for the latest bugfix release in
the Python 2.7 series.

Downloads of source code and binaries are at
https://www.python.org/downloads/release/python-2714rc1/

Please consider testing the release with your libraries and applications
and reporting any bugs to 
https://bugs.python.org

A final release is expected in 3 weeks.

Regards,
Benjamin
2.7 release manager
(on behalf of 2.7's contributors)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 550 v4

[Nathaniel Smith]

On Sat, Aug 26, 2017 at 7:58 AM, Elvis Pranskevichus  wrote:
> On Saturday, August 26, 2017 2:34:29 AM EDT Nathaniel Smith wrote:
>> On Fri, Aug 25, 2017 at 3:32 PM, Yury Selivanov
>  wrote:
>> > Coroutines and Asynchronous Tasks
>> > -
>> >
>> > In coroutines, like in generators, context variable changes are
>> > local>
>> > and are not visible to the caller::
>> > import asyncio
>> >
>> > var = new_context_var()
>> >
>> > async def sub():
>> > assert var.lookup() == 'main'
>> > var.set('sub')
>> > assert var.lookup() == 'sub'
>> >
>> > async def main():
>> > var.set('main')
>> > await sub()
>> > assert var.lookup() == 'main'
>> >
>> > loop = asyncio.get_event_loop()
>> > loop.run_until_complete(main())
>>
>> I think this change is a bad idea. I think that generally, an async
>> call like 'await async_sub()' should have the equivalent semantics
>> to a synchronous call like 'sync_sub()', except for the part where
>> the former is able to contain yields. Giving every coroutine an LC
>> breaks that equivalence. It also makes it so in async code, you
>> can't necessarily refactor by moving code in and out of
>> subroutines. Like, if we inline 'sub' into 'main', that shouldn't
>> change the semantics, but...
>
> If we could easily, we'd given each _normal function_ its own logical
> context as well.

I mean... you could do that. It'd be easy to do technically, right?
But it would make the PEP useless, because then projects like decimal
and numpy couldn't adopt it without breaking backcompat, meaning they
couldn't adopt it at all.

The backcompat argument isn't there in the same way for async code,
because it's new and these functions have generally been broken there
anyway. But it's still kinda there in spirit: there's a huge amount of
collective knowledge about how (synchronous) Python code works, and
IMO async code should match that whenever possible.

> What we are talking about here is variable scope leaking up the call
> stack.  I think this is a bad pattern.  For decimal context-like uses
> of the EC you should always use a context manager.  For uses like Web
> request locals, you always have a top function that sets the context
> vars.

It's perfectly reasonable to have a script where you call
decimal.setcontext or np.seterr somewhere at the top to set the
defaults for the rest of the script. Yeah, maybe it'd be a bit cleaner
to use a 'with' block wrapped around main(), and certainly in a
complex app you want to stick to that, but Python isn't just used for
complex apps :-). I foresee confused users trying to figure out why
np.seterr suddenly stopped working when they ported their app to use
async.

This also seems like it makes some cases much trickier. Like, say you
have an async context manager that wants to manipulate a context
local. If you write 'async def __aenter__', you just lost -- it'll be
isolated. I think you have to write some awkward thing like:

def __aenter__(self):
coro = self._real_aenter()
coro.__logical_context__ = None
return coro

It would be really nice if libraries like urllib3/requests supported
async as an option, but it's difficult because they can't drop support
for synchronous operation and python 2, and we want to keep a single
codebase. One option I've been exploring is to write them in
"synchronous style" but with async/await keywords added, and then
generating a py2-compatible version with a script that strips out
async/await etc. (Like a really simple 3to2 that just works at the
token level.) One transformation you'd want to apply is replacing
__aenter__ -> __enter__, but this gets much more difficult if we have
to worry about elaborate transformations like the above...

If I have an async generator, and I set its __logical_context__ to
None, then do I also have to set this attribute on every coroutine
returned from calling __anext__/asend/athrow/aclose?

>>
>> I think I see the motivation: you want to make
>>
>>await sub()
>>
>> and
>>
>>await ensure_future(sub())
>>
>> have the same semantics, right? And the latter has to create a Task
>
> What we want is for `await sub()` to be equivalent to
> `await asyncio.wait_for(sub())` and to `await asyncio.gather(sub())`.

I don't feel like there's any need to make gather() have exactly the
same semantics as a regular call -- it's pretty clearly a
task-spawning primitive that runs all of the given coroutines
concurrently, so it makes sense that it would have task-spawning
semantics rather than call semantics.

wait_for is a more unfortunate case; there's really no reason for it
to create a Task at all, except that asyncio made the decision to
couple cancellation and Tasks, so if you want one then you're stuck
with the other.

Yury's made some comments about stealing Trio's cancellation system
and adding it to asyncio -- I don't know how serious he was. If he did
then it would let you use

Re: [Python-Dev] PEP 550 v4

[Nathaniel Smith]

On Sat, Aug 26, 2017 at 10:25 AM, Eric Snow  wrote:
> With threads we have a directed graph of execution, rooted at the root
> thread, branching with each new thread and merging with each .join().
> Each thread gets its own copy of each threading.local, regardless of
> the relationship between branches (threads) in the execution graph.
>
> With async (and generators) we also have a directed graph of
> execution, rooted in the calling thread, branching with each new async
> call.  Currently there is no equivalent to threading.local for the
> async execution graph.  This proposal involves adding such an
> equivalent.
>
> However, the proposed solution isn''t quite equivalent, right?  It
> adds a concept of lookup on the chain of namespaces, traversing up the
> execution graph back to the root.  threading.local does not do this.
> Furthermore, you can have more than one threading.local per thread.
> From what I read in the PEP, each node in the execution graph has (at
> most) one Execution Context.
>
> The PEP doesn't really say much about these differences from
> threadlocals, including a rationale.  FWIW, I think such a COW
> mechanism could be useful.  However, it does add complexity to the
> feature.  So a clear explanation in the PEP of why it's worth it would
> be valuable.

You might be interested in these notes I wrote to motivate why we need
a chain of namespaces, and why simple "async task locals" aren't
sufficient:

https://github.com/njsmith/pep-550-notes/blob/master/dynamic-scope.ipynb

They might be a bit verbose to include directly in the PEP, but
Yury/Elvis, feel free to steal whatever if you think it'd be useful.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 550 v4

[Ethan Furman]

On 08/26/2017 12:12 PM, David Mertz wrote:

I'm convinced by the new section explaining why a single value is better than a 
namespace.  Nonetheless, it would feel
more "Pythonic" to me to create a property `ContextVariable.val` whose getter 
and setter was `.lookup()` and `.set()`
(or maybe `._lookup()` and `._set()`).

Lookup might require a more complex call signature in rare cases, but the large 
majority of the time it would simply be
`var.val`, and that should be the preferred API IMO.  That provides a nice parallel 
between `var.name `
and `var.val` also.


+1 to the property solution.

--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 550 v4

[francismb]

Hi,

> Multithreaded Code
> --
> 
> In multithreaded code, context variables behave like thread locals::
> 
> var = new_context_var()
> 
> def sub():
> assert var.lookup() is None  # The execution context is empty
>  # for each new thread.
> var.set('sub')
> 
> def main():
> var.set('main')
> 
> thread = threading.Thread(target=sub)
> thread.start()
> thread.join()
> 
> assert var.lookup() == 'main'
> 
it's by design that the execution context for new threads to be empty or
should it be possible to set it to some initial value? Like e.g:

 var = new_context_var('init')

 def sub():
 assert var.lookup() == 'init'
 var.set('sub')

 def main():
 var.set('main')

 thread = threading.Thread(target=sub)
 thread.start()
 thread.join()

 assert var.lookup() == 'main'


Thanks,
--francis
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 550 v4

[Antoine Pitrou]

On Sat, 26 Aug 2017 13:23:00 -0400
Yury Selivanov  wrote:
> >
> > A good point; however, ChainMap, which behaves similarly as far as lookup
> > goes, uses "get" and does not have a "lookup" method.  I think we lose more
> > than we gain by changing that method name.  
> 
> ChainMap is constrained to be a Mapping-like object, but I get your
> point.  Let's see what others say about the "lookup()".  It is kind of
> an experiment to try a name and see if it fits.

I like "get()" better than "lookup()".

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 550 v4

[francismb]

Hi,

> 
> For the purpose of this section, we define *execution context* as an
> opaque container of non-local state that allows consistent access to
> its contents in the concurrent execution environment.
> 

May be nonsense/non-practical :-), but :

- How extrapolates this concept to the whole interpreter (container of
...non-local state --> interpreter state; concurrent execution
environment --> python interpreter, ...) ?

Thank in advance !
-- francis
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 550 v4

[David Mertz]

I'm convinced by the new section explaining why a single value is better
than a namespace.  Nonetheless, it would feel more "Pythonic" to me to
create a property `ContextVariable.val` whose getter and setter was
`.lookup()` and `.set()` (or maybe `._lookup()` and `._set()`).

Lookup might require a more complex call signature in rare cases, but the
large majority of the time it would simply be `var.val`, and that should be
the preferred API IMO.  That provides a nice parallel between `var.name`
and `var.val` also.

On Sat, Aug 26, 2017 at 11:22 AM, Barry Warsaw  wrote:

> On Aug 26, 2017, at 14:15, Yury Selivanov  wrote:
> >
> > Elvis just added it:
> > https://www.python.org/dev/peps/pep-0550/#replication-of-
> threading-local-interface
>
> Thanks, that’s exactly what I was looking for.  Great summary of the issue.
> >
> >> That said, if we have to use method lookup, then I agree that `.get()`
> is a better choice than `.lookup()`.  But in that case, would it be
> possible to add an optional `default=None` argument so that you can specify
> a marker object for a missing value?  I worry that None might be a valid
> value in some cases, but that currently can’t be distinguished from
> “missing”.
> >
> > Nathaniel has a use case where he needs to know if the value is in the
> > topmost LC or not.
> >
> > One way to address that need is to have the following signature for
> lookup():
> >
> >   lookup(*, default=None, traverse=True)
> >
> > IMO "lookup" is a slightly better name in this particular context.
>
> Given that signature (which +1), I agree.  You could add keywords for
> debugging lookup fairly easily too.
>
> Cheers,
> -Barry
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> mertz%40gnosis.cx
>
>


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 550 v4

[Sven R. Kunze]

On 26.08.2017 19:23, Yury Selivanov wrote:

ChainMap is constrained to be a Mapping-like object, but I get your
point.  Let's see what others say about the "lookup()".  It is kind of
an experiment to try a name and see if it fits.


I like "get" more. ;-)


Best,
Sven

PS:
This might be a result of still leaning towards attribute access despite 
the discussion you referenced.
I still don't think complicating and reinventing terminology (which 
basically results in API names) buys us much.


And I am still with Ethan, a context stack is just a ChainMap. Renaming 
basic methods won't hide that fact. That's my only criticism of the PEP. 
The rest is fine and useful.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 550 v4

[Barry Warsaw]

On Aug 26, 2017, at 14:15, Yury Selivanov  wrote:
> 
> Elvis just added it:
> https://www.python.org/dev/peps/pep-0550/#replication-of-threading-local-interface

Thanks, that’s exactly what I was looking for.  Great summary of the issue.
> 
>> That said, if we have to use method lookup, then I agree that `.get()` is a 
>> better choice than `.lookup()`.  But in that case, would it be possible to 
>> add an optional `default=None` argument so that you can specify a marker 
>> object for a missing value?  I worry that None might be a valid value in 
>> some cases, but that currently can’t be distinguished from “missing”.
> 
> Nathaniel has a use case where he needs to know if the value is in the
> topmost LC or not.
> 
> One way to address that need is to have the following signature for lookup():
> 
>   lookup(*, default=None, traverse=True)
> 
> IMO "lookup" is a slightly better name in this particular context.

Given that signature (which +1), I agree.  You could add keywords for debugging 
lookup fairly easily too.

Cheers,
-Barry



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 550 v4

[Yury Selivanov]

On Sat, Aug 26, 2017 at 12:30 PM, Barry Warsaw  wrote:
> I agree with David; this PEP has really gotten to a great place and the new 
> organization makes it much easier to understand.
>
>> On Aug 25, 2017, at 22:19, Ethan Furman  wrote:
>>
>> Why "lookup" and not "get" ?  Many APIs use "get" and it's functionality is 
>> well understood.
>
> I have the same question as Sven as to why we can’t have attribute access 
> semantics.  I probably asked that before, and you probably answered, so maybe 
> if there’s a specific reason why this can’t be supported, the PEP should 
> include a “rejected ideas” section explaining the choice.

Elvis just added it:
https://www.python.org/dev/peps/pep-0550/#replication-of-threading-local-interface

>
> That said, if we have to use method lookup, then I agree that `.get()` is a 
> better choice than `.lookup()`.  But in that case, would it be possible to 
> add an optional `default=None` argument so that you can specify a marker 
> object for a missing value?  I worry that None might be a valid value in some 
> cases, but that currently can’t be distinguished from “missing”.

Nathaniel has a use case where he needs to know if the value is in the
topmost LC or not.

One way to address that need is to have the following signature for lookup():

   lookup(*, default=None, traverse=True)

IMO "lookup" is a slightly better name in this particular context.

Yury
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 550 v4

[Yury Selivanov]

Hi Eric,

On Sat, Aug 26, 2017 at 1:25 PM, Eric Snow  wrote:
> Thanks for the update.  Comments in-line below.
>
> -eric
>
> On Fri, Aug 25, 2017 at 4:32 PM, Yury Selivanov  
> wrote:
>> [snip]
>>
>> This PEP adds a new generic mechanism of ensuring consistent access
>> to non-local state in the context of out-of-order execution, such
>> as in Python generators and coroutines.
>>
>> Thread-local storage, such as ``threading.local()``, is inadequate for
>> programs that execute concurrently in the same OS thread.  This PEP
>> proposes a solution to this problem.
>
>> [snip]
>>
>> In regular, single-threaded code that doesn't involve generators or
>> coroutines, context variables behave like globals::
>>
>> [snip]
>>
>> In multithreaded code, context variables behave like thread locals::
>>
>> [snip]
>>
>> In generators, changes to context variables are local and are not
>> visible to the caller, but are visible to the code called by the
>> generator.  Once set in the generator, the context variable is
>> guaranteed not to change between iterations::
>
> With threads we have a directed graph of execution, rooted at the root
> thread, branching with each new thread and merging with each .join().
> Each thread gets its own copy of each threading.local, regardless of
> the relationship between branches (threads) in the execution graph.
>
> With async (and generators) we also have a directed graph of
> execution, rooted in the calling thread, branching with each new async
> call.  Currently there is no equivalent to threading.local for the
> async execution graph.  This proposal involves adding such an
> equivalent.

Correct.

>
> However, the proposed solution isn''t quite equivalent, right?  It
> adds a concept of lookup on the chain of namespaces, traversing up the
> execution graph back to the root.  threading.local does not do this.
> Furthermore, you can have more than one threading.local per thread.
> From what I read in the PEP, each node in the execution graph has (at
> most) one Execution Context.
>
> The PEP doesn't really say much about these differences from
> threadlocals, including a rationale.  FWIW, I think such a COW
> mechanism could be useful.  However, it does add complexity to the
> feature.  So a clear explanation in the PEP of why it's worth it would
> be valuable.

Currently, the  PEP covers the proposed mechanism in-depth, explaining
why every detail of the spec is the way it is.  But I think it'd be
valuable to highlight differences from theading.local() in a separate
section.  We'll think about adding one.

Yury
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 550 v4

[Eric Snow]

On Sat, Aug 26, 2017 at 11:19 AM, Yury Selivanov
 wrote:
> This is similar to namedtuples, and nobody really complains about them.

FWIW, there are plenty of complaints on python-ideas about this (and
never a satisfactory solution).  :)  That said, I don't think it is as
big a deal here since the target audience is much smaller.

-eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 550 v4

[Eric Snow]

Thanks for the update.  Comments in-line below.

-eric

On Fri, Aug 25, 2017 at 4:32 PM, Yury Selivanov  wrote:
> [snip]
>
> This PEP adds a new generic mechanism of ensuring consistent access
> to non-local state in the context of out-of-order execution, such
> as in Python generators and coroutines.
>
> Thread-local storage, such as ``threading.local()``, is inadequate for
> programs that execute concurrently in the same OS thread.  This PEP
> proposes a solution to this problem.

> [snip]
>
> In regular, single-threaded code that doesn't involve generators or
> coroutines, context variables behave like globals::
>
> [snip]
>
> In multithreaded code, context variables behave like thread locals::
>
> [snip]
>
> In generators, changes to context variables are local and are not
> visible to the caller, but are visible to the code called by the
> generator.  Once set in the generator, the context variable is
> guaranteed not to change between iterations::

With threads we have a directed graph of execution, rooted at the root
thread, branching with each new thread and merging with each .join().
Each thread gets its own copy of each threading.local, regardless of
the relationship between branches (threads) in the execution graph.

With async (and generators) we also have a directed graph of
execution, rooted in the calling thread, branching with each new async
call.  Currently there is no equivalent to threading.local for the
async execution graph.  This proposal involves adding such an
equivalent.

However, the proposed solution isn''t quite equivalent, right?  It
adds a concept of lookup on the chain of namespaces, traversing up the
execution graph back to the root.  threading.local does not do this.
Furthermore, you can have more than one threading.local per thread.
>From what I read in the PEP, each node in the execution graph has (at
most) one Execution Context.

The PEP doesn't really say much about these differences from
threadlocals, including a rationale.  FWIW, I think such a COW
mechanism could be useful.  However, it does add complexity to the
feature.  So a clear explanation in the PEP of why it's worth it would
be valuable.

> [snip]
>
> The following new Python APIs are introduced by this PEP:
>
> 1. The ``sys.new_context_var(name: str='...')`` function to create
>``ContextVar`` objects.
>
> 2. The ``ContextVar`` object, which has:
>
>* the read-only ``.name`` attribute,
>* the ``.lookup()`` method which returns the value of the variable
>  in the current execution context;
>* the ``.set()`` method which sets the value of the variable in
>  the current execution context.
>
> 3. The ``sys.get_execution_context()`` function, which returns a
>copy of the current execution context.
>
> 4. The ``sys.new_execution_context()`` function, which returns a new
>empty execution context.
>
> 5. The ``sys.new_logical_context()`` function, which returns a new
>empty logical context.
>
> 6. The ``sys.run_with_execution_context(ec: ExecutionContext,
>func, *args, **kwargs)`` function, which runs *func* with the
>provided execution context.
>
> 7. The ``sys.run_with_logical_context(lc:LogicalContext,
>func, *args, **kwargs)`` function, which runs *func* with the
>provided logical context on top of the current execution context.

#1-4 are consistent with a single EC per Python thread.  However, #5-7
imply that more than one EC per thread is supported, but only one is
active in the current execution stack (notably the EC is rooted at the
calling frame).  threading.local provides a much simpler mechanism but
does not support the chained context (COW) semantics...
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 550 v4

[Yury Selivanov]

On Sat, Aug 26, 2017 at 1:23 PM, Ethan Furman  wrote:
> On 08/26/2017 09:25 AM, Yury Selivanov wrote:
>>
>> On Fri, Aug 25, 2017 at 10:19 PM, Ethan Furman wrote:
>
>
 A *context variable* is an object representing a value in the
 execution context.  A new context variable is created by calling
 the ``new_context_var()`` function.  A context variable object has
 two methods:

 * ``lookup()``: returns the value of the variable in the current
 execution context;

 * ``set()``: sets the value of the variable in the current
 execution context.
>>>
>>>
>>>
>>> Why "lookup" and not "get" ?  Many APIs use "get" and it's functionality
>>> is
>>> well understood.
>>
>>
>> ContextVar.set(value) method writes the `value` to the *topmost LC*.
>>
>> ContextVar.lookup() method *traverses the stack* until it finds the LC
>> that has a value.  "get()" does not reflect this subtle semantics
>> difference.
>
>
> A good point; however, ChainMap, which behaves similarly as far as lookup
> goes, uses "get" and does not have a "lookup" method.  I think we lose more
> than we gain by changing that method name.

ChainMap is constrained to be a Mapping-like object, but I get your
point.  Let's see what others say about the "lookup()".  It is kind of
an experiment to try a name and see if it fits.

Yury
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 550 v4

[Ethan Furman]

On 08/26/2017 09:25 AM, Yury Selivanov wrote:

On Fri, Aug 25, 2017 at 10:19 PM, Ethan Furman wrote:



A *context variable* is an object representing a value in the
execution context.  A new context variable is created by calling
the ``new_context_var()`` function.  A context variable object has
two methods:

* ``lookup()``: returns the value of the variable in the current
execution context;

* ``set()``: sets the value of the variable in the current
execution context.



Why "lookup" and not "get" ?  Many APIs use "get" and it's functionality is
well understood.


ContextVar.set(value) method writes the `value` to the *topmost LC*.

ContextVar.lookup() method *traverses the stack* until it finds the LC
that has a value.  "get()" does not reflect this subtle semantics
difference.


A good point; however, ChainMap, which behaves similarly as far as lookup goes, uses "get" and does not have a "lookup" 
method.  I think we lose more than we gain by changing that method name.


--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 550 v4

[Yury Selivanov]

On Sat, Aug 26, 2017 at 1:10 PM, David Mertz  wrote:
> Would it be possible/desirable to make the default a unique string value
> like a UUID or a stringified counter?

Sure, or we could just use the id of ContextVar.

In the end, when we want to introspect the EC while debugging, we
would see something like this:

{
   ContextVar(name='518CDD4F-D676-408F-B968-E144F792D055'): 42,
   ContextVar(name='decimal_context'): DecimalContext(precision=2),
   ContextVar(name='7A44D3BE-F7A1-40B7-BE51-7DFFA7E0E02F'): 'spam'
}

That's why I think it's easier to force users always specify the name:

my_var = sys.new_context_var('my_var')

This is similar to namedtuples, and nobody really complains about them.

Yury
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 550 v4

[Yury Selivanov]

On Sat, Aug 26, 2017 at 2:34 AM, Nathaniel Smith  wrote:
> On Fri, Aug 25, 2017 at 3:32 PM, Yury Selivanov  
> wrote:
>> Coroutines and Asynchronous Tasks
>> -
>>
>> In coroutines, like in generators, context variable changes are local
>> and are not visible to the caller::
>>
>> import asyncio
>>
>> var = new_context_var()
>>
>> async def sub():
>> assert var.lookup() == 'main'
>> var.set('sub')
>> assert var.lookup() == 'sub'
>>
>> async def main():
>> var.set('main')
>> await sub()
>> assert var.lookup() == 'main'
>>
>> loop = asyncio.get_event_loop()
>> loop.run_until_complete(main())
>
> I think this change is a bad idea. I think that generally, an async
> call like 'await async_sub()' should have the equivalent semantics to
> a synchronous call like 'sync_sub()', except for the part where the
> former is able to contain yields.

That exception is why the semantics cannot be equivalent.

> Giving every coroutine an LC breaks
> that equivalence. It also makes it so in async code, you can't
> necessarily refactor by moving code in and out of subroutines.

I'll cover the refactoring argument later in this email.

[..]
> It also adds non-trivial overhead, because now lookup() is O(depth of
> async callstack), instead of O(depth of (async) generator nesting),
> which is generally much smaller.

I don't think it's non-trivial though:

First, we have a cache in ContextVar which makes lookup O(1) for any
tight code that uses libraries like decimal and numpy.

Second, most of the LCs in the chain will be empty, so even the
uncached lookup will still be fast.

Third, you will usually have your "with my_context()" block right
around your code (or within a few awaits distance), otherwise it will
be hard to reason what's the context.  And if, occasionally, you have
a one single "var.lookup()" call that won't be cached, the cost of it
will still be measured in microseconds.

Finally, the easy to follow semantics is the main argument for the
change (even at the cost of making "get()" a bit slower in corner
cases).

>
> I think I see the motivation: you want to make
>
>await sub()
>
> and
>
>await ensure_future(sub())
>
> have the same semantics, right?

Yes.

> And the latter has to create a Task
> and split it off into a new execution context, so you want the former
> to do so as well? But to me this is like saying that we want
>
>sync_sub()
>
> and
>
>thread_pool_executor.submit(sync_sub).result()

This example is very similar to:

await sub()

and

await create_task(sub())

So it's really about making the semantics for coroutines be predictable.

> (And fwiw I'm still not convinced we should give up on 'yield from' as
> a mechanism for refactoring generators.)

I don't get this "refactoring generators" and "refactoring coroutines" argument.

Suppose you have this code:

  def gen():
 i = 0
 for _ in range(3):
 i += 1
 yield i
 for _ in range(5):
 i += 1
 yield i

You can't refactor gen() by simply copying/pasting parts of its body
into a separate generator:

  def count3():
 for _ in range(3):
 i += 1
 yield

  def gen():
 i = 0

 yield from count3()

 for _ in range(5):
 i += 1
 yield i

The above won't work for some obvious reasons: 'i' is a nonlocal
variable for 'count3' block of code. Almost exactly the same thing
will happen with the current PEP 550 specification, which is a *good*
thing.

'yield from' and 'await' are not about refactoring. They can be used
for splitting large generators/coroutines into a set of smaller ones,
sure. But there's *no* magical, always working, refactoring mechanism
that allows to do that blindly.

>
>> To establish the full semantics of execution context in couroutines,
>> we must also consider *tasks*.  A task is the abstraction used by
>> *asyncio*, and other similar libraries, to manage the concurrent
>> execution of coroutines.  In the example above, a task is created
>> implicitly by the ``run_until_complete()`` function.
>> ``asyncio.wait_for()`` is another example of implicit task creation::
>>
>> async def sub():
>> await asyncio.sleep(1)
>> assert var.lookup() == 'main'
>>
>> async def main():
>> var.set('main')
>>
>> # waiting for sub() directly
>> await sub()
>>
>> # waiting for sub() with a timeout
>> await asyncio.wait_for(sub(), timeout=2)
>>
>> var.set('main changed')
>>
>> Intuitively, we expect the assertion in ``sub()`` to hold true in both
>> invocations, even though the ``wait_for()`` implementation actually
>> spawns a task, which runs ``sub()`` concurrently with ``main()``.
>
> I found this example confusing -- you talk about sub() and main()
> running concurrently, but ``wait_for`` blocks main() until sub() has
> finished running, right?

Right.  Before we continue, let

Re: [Python-Dev] PEP 550 v4

[David Mertz]

Would it be possible/desirable to make the default a unique string value
like a UUID or a stringified counter?

On Aug 26, 2017 9:35 AM, "Yury Selivanov"  wrote:

On Sat, Aug 26, 2017 at 12:56 AM, David Mertz  wrote:
> This is now looking really good and I can understands it.

Great!

>
> One question though. Sometimes creation of a context variable is done
with a
> name argument, other times not. E.g.
>
>   var1 = new_context_var('var1')
>   var = new_context_var()

We were very focused on making the High-level Specification as
succinct as possible, omitting some API details that are not important
for understanding the semantics.

"name" argument is not optional and will be required.  If it's
optional, people will not provide it, making it very hard to
introspect the context when we want it.

I guess we'll just update the High-level Specification section to use
the correct signature of "new_context_var".

Yury
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Scope, not context? (was Re: PEP 550 v3 naming)

[Barry Warsaw]

On Aug 26, 2017, at 12:43, francismb  wrote:
> 
>>> I propose RaymondLuxuryYach_t, but we’ll have to pronounce it 
>>> ThroatwobblerMangrove.
>> 
>> Sorry, but I think we should prononce it ThroatwobblerMangrove.
>> 
> too hard to pronounce but at least is unique, I would prefer thredarena
> but I see naming is hard ... :-)

Oh mollusks, I thought you said bacon.

-Barry



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Scope, not context? (was Re: PEP 550 v3 naming)

[francismb]

>> I propose RaymondLuxuryYach_t, but we’ll have to pronounce it 
>> ThroatwobblerMangrove.
> 
> Sorry, but I think we should prononce it ThroatwobblerMangrove.
> 
too hard to pronounce but at least is unique, I would prefer thredarena
but I see naming is hard ... :-)

Thanks!
--francis
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] how to dump cst

[Emily E Morehouse]

Hi,

I believe the parser module
 allows
for generation of a CST, though the documentation is not specific about the
type of syntax tree used. The old compiler module
 utilized it for this
purpose and contains explicit documentation about getting a CST from the
parser module.

-- EMV

On Sat, Aug 26, 2017 at 1:53 AM, yang chen  wrote:

> Hello, everyone.
>
>
> I'm learning python source code, I used to print info when I debug code,
> but it's not convenient. I want to print CST, AST, ByteCode with iPython,
> wirte some code, then print it's CST、AST、 BYTECODE, and then to read the
> source code.
>
> But, I can't dump the CST tree which will transform to AST, I know I can
> print CST tree by modifying python source code, but how to dump the CST
> tree in iPython just like dumping AST with ast package.
>
> 1. get AST with ast package2. get ByteCode with dis package
>
> 3. but how get cst??
>
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> emilyemorehouse%40gmail.com
>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 550 v4

[Yury Selivanov]

On Sat, Aug 26, 2017 at 12:56 AM, David Mertz  wrote:
> This is now looking really good and I can understands it.

Great!

>
> One question though. Sometimes creation of a context variable is done with a
> name argument, other times not. E.g.
>
>   var1 = new_context_var('var1')
>   var = new_context_var()

We were very focused on making the High-level Specification as
succinct as possible, omitting some API details that are not important
for understanding the semantics.

"name" argument is not optional and will be required.  If it's
optional, people will not provide it, making it very hard to
introspect the context when we want it.

I guess we'll just update the High-level Specification section to use
the correct signature of "new_context_var".

Yury
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 550 v4

[Yury Selivanov]

On Sat, Aug 26, 2017 at 9:33 AM, Sven R. Kunze  wrote:
[..]
> Why not the same interface as thread-local storage? This has been the
> question which bothered me from the beginning of PEP550. I don't understand
> what inventing a new way of access buys us here.

This was covered at length in these threads:

https://mail.python.org/pipermail/python-ideas/2017-August/046888.html
https://mail.python.org/pipermail/python-ideas/2017-August/046889.html

I forgot to add a subsection to "Design Consideration" with a summary
of that thread. Will be fixed in the next revision.

Yury
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 550 v4

[Barry Warsaw]

I agree with David; this PEP has really gotten to a great place and the new 
organization makes it much easier to understand.

> On Aug 25, 2017, at 22:19, Ethan Furman  wrote:
> 
> Why "lookup" and not "get" ?  Many APIs use "get" and it's functionality is 
> well understood.

I have the same question as Sven as to why we can’t have attribute access 
semantics.  I probably asked that before, and you probably answered, so maybe 
if there’s a specific reason why this can’t be supported, the PEP should 
include a “rejected ideas” section explaining the choice.

That said, if we have to use method lookup, then I agree that `.get()` is a 
better choice than `.lookup()`.  But in that case, would it be possible to add 
an optional `default=None` argument so that you can specify a marker object for 
a missing value?  I worry that None might be a valid value in some cases, but 
that currently can’t be distinguished from “missing”.

I’d also like a debugging interface, such that I can ask “context_var.get()” 
and get some easy diagnostics about the resolution order.

Cheers,
-Barry



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 550 v4

[Yury Selivanov]

On Sat, Aug 26, 2017 at 6:22 AM, Stefan Behnel  wrote:
> Hi,
>
> I'm aware that the current implementation is not final, but I already
> adapted the coroutine changes for Cython to allow for some initial
> integration testing with real external (i.e. non-Python coroutine) targets.
> I haven't adapted the tests yet, so the changes are currently unused and
> mostly untested.
>
> https://github.com/scoder/cython/tree/pep550_exec_context
>
> I also left some comments in the github commits along the way.

Huge thanks for thinking about how this proposal will work for Cython
and trying it out.  Although I must warn you that the last reference
implementation is very outdated, and the implementation we will end up
with will be very different (think a total rewrite from scratch).

Yury
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 550 v4

[Yury Selivanov]

On Fri, Aug 25, 2017 at 10:19 PM, Ethan Furman  wrote:
> All in all, I like it.  Nice job.

Thanks!

>
> On 08/25/2017 03:32 PM, Yury Selivanov wrote:
>
>> A *context variable* is an object representing a value in the
>> execution context.  A new context variable is created by calling
>> the ``new_context_var()`` function.  A context variable object has
>> two methods:
>>
>> * ``lookup()``: returns the value of the variable in the current
>>execution context;
>>
>> * ``set()``: sets the value of the variable in the current
>>execution context.
>
>
> Why "lookup" and not "get" ?  Many APIs use "get" and it's functionality is
> well understood.

ContextVar.set(value) method writes the `value` to the *topmost LC*.

ContextVar.lookup() method *traverses the stack* until it finds the LC
that has a value.  "get()" does not reflect this subtle semantics
difference.

Yury
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 550 v4: Decimal examples and performance (was: Re: PEP 550 v4)

[Yury Selivanov]

On Sat, Aug 26, 2017 at 7:45 AM, Stefan Krah  wrote:
>
> Hi,
>
> thanks, on the whole this is *much* easier to understand.

Thanks!

> I'll add some comments on the decimal examples. The thing is, decimal
> is already quite tricky and people do read PEPs long after they have
> been accepted, so they should probably reflect best practices.

Agree.

[..]
>> Some languages, that support coroutines or generators, recommend
>> passing the context manually as an argument to every function, see [1]_
>> for an example.  This approach, however, has limited use for Python,
>> where there is a large ecosystem that was built to work with a TLS-like
>> context.  Furthermore, libraries like ``decimal`` or ``numpy`` rely
>> on context implicitly in overloaded operator implementations.
>
> I'm not sure why this approach has limited use for decimal:
>
>
> from decimal import *
>
> def fractions(precision, x, y):
> ctx = Context(prec=precision)
> yield ctx.divide(Decimal(x), Decimal(y))
> yield ctx.divide(Decimal(x), Decimal(y**2))
>
> g1 = fractions(precision=2, x=1, y=3)
> g2 = fractions(precision=6, x=2, y=3)
> print(list(zip(g1, g2)))

Because you have to know the limitations of implicit decimal context
to make this choice. Most people don't (at least from my experience).

> This is the first thing I'd do when writing async-safe code.

Because you know the decimal module very well :)

>
> Again, people do read PEPs.  So if an asyncio programmer without any
> special knowledge of decimal reads the PEP, he probably assumes that
> localcontext() is currently the only option, while the safer and
> easy-to-reason-about context methods exist.

I agree.

>
>
>> Now, let's revisit the decimal precision example from the `Rationale`_
>> section, and see how the execution context can improve the situation::
>>
>> import decimal
>>
>> decimal_prec = new_context_var()  # create a new context variable
>>
>> # Pre-PEP 550 Decimal relies on TLS for its context.
>> # This subclass switches the decimal context storage
>> # to the execution context for illustration purposes.
>> #
>> class MyDecimal(decimal.Decimal):
>> def __init__(self, value="0"):
>> prec = decimal_prec.lookup()
>> if prec is None:
>> raise ValueError('could not find decimal precision')
>> context = decimal.Context(prec=prec)
>> super().__init__(value, context=context)
>
> As I understand it, the example creates a context with a custom precision
> and attempts to use that context to create a Decimal.
>
> This doesn't switch the actual decimal context. Secondly, the precision in
> the context argument to the Decimal() constructor has no effect --- the
> context there is only used for error handling.
>
> Lastly, if the constructor *did* use the precision, one would have to be
> careful about double rounding when using MyDecimal().
>
>
> I get that this is supposed to be for illustration only, but please let's
> be careful about what people might take away from that code.

In the next iteration of the PEP we'll remove decimal examples and
replace them with something with simpler semantics.  This is clearly
the best choice now.

>> This generic caching approach is similar to what the current C
>> implementation of ``decimal`` does to cache the the current decimal
>> context, and has similar performance characteristics.
>
> I think it'll work, but can we agree on hard numbers like max 2% slowdown
> for the non-threaded case and 4% for applications that only use threads?

I'd be *very* surprised if wee see any noticeable slowdown at all. The
way ContextVars will implement caching is very similar to the trick
you use now.

Yury
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 550 v4: Decimal examples and performance (was: Re: PEP 550 v4)

[Stefan Krah]

Hi,

thanks, on the whole this is *much* easier to understand.

I'll add some comments on the decimal examples. The thing is, decimal
is already quite tricky and people do read PEPs long after they have
been accepted, so they should probably reflect best practices.


On Fri, Aug 25, 2017 at 06:32:22PM -0400, Yury Selivanov wrote:
> Unfortunately, TLS does not work well for programs which execute
> concurrently in a single thread.  A Python generator is the simplest
> example of a concurrent program.  Consider the following::
> 
> def fractions(precision, x, y):
> with decimal.localcontext() as ctx:
> ctx.prec = precision
> yield Decimal(x) / Decimal(y)
> yield Decimal(x) / Decimal(y**2)
> 
> g1 = fractions(precision=2, x=1, y=3)
> g2 = fractions(precision=6, x=2, y=3)
> 
> items = list(zip(g1, g2))
> 
> The expected value of ``items`` is::

"Many people (wrongly) expect the values of ``items`` to be::" ;)

> 
> [(Decimal('0.33'), Decimal('0.67')),
>  (Decimal('0.11'), Decimal('0.22'))]


> Some languages, that support coroutines or generators, recommend
> passing the context manually as an argument to every function, see [1]_
> for an example.  This approach, however, has limited use for Python,
> where there is a large ecosystem that was built to work with a TLS-like
> context.  Furthermore, libraries like ``decimal`` or ``numpy`` rely
> on context implicitly in overloaded operator implementations.

I'm not sure why this approach has limited use for decimal:


from decimal import *

def fractions(precision, x, y):
ctx = Context(prec=precision)
yield ctx.divide(Decimal(x), Decimal(y))
yield ctx.divide(Decimal(x), Decimal(y**2))

g1 = fractions(precision=2, x=1, y=3)
g2 = fractions(precision=6, x=2, y=3)
print(list(zip(g1, g2)))


This is the first thing I'd do when writing async-safe code.

Again, people do read PEPs.  So if an asyncio programmer without any
special knowledge of decimal reads the PEP, he probably assumes that
localcontext() is currently the only option, while the safer and
easy-to-reason-about context methods exist.


> Now, let's revisit the decimal precision example from the `Rationale`_
> section, and see how the execution context can improve the situation::
> 
> import decimal
> 
> decimal_prec = new_context_var()  # create a new context variable
> 
> # Pre-PEP 550 Decimal relies on TLS for its context.
> # This subclass switches the decimal context storage
> # to the execution context for illustration purposes.
> #
> class MyDecimal(decimal.Decimal):
> def __init__(self, value="0"):
> prec = decimal_prec.lookup()
> if prec is None:
> raise ValueError('could not find decimal precision')
> context = decimal.Context(prec=prec)
> super().__init__(value, context=context)

As I understand it, the example creates a context with a custom precision
and attempts to use that context to create a Decimal.

This doesn't switch the actual decimal context. Secondly, the precision in
the context argument to the Decimal() constructor has no effect --- the
context there is only used for error handling.

Lastly, if the constructor *did* use the precision, one would have to be
careful about double rounding when using MyDecimal().


I get that this is supposed to be for illustration only, but please let's
be careful about what people might take away from that code.


> This generic caching approach is similar to what the current C
> implementation of ``decimal`` does to cache the the current decimal
> context, and has similar performance characteristics.

I think it'll work, but can we agree on hard numbers like max 2% slowdown
for the non-threaded case and 4% for applications that only use threads?

I'm a bit cautious because other C-extension state-managing PEPs didn't
come close to these figures.


Stefan Krah



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] how to dump cst

[yang chen]

Hello, everyone.


I'm learning python source code, I used to print info when I debug code,
but it's not convenient. I want to print CST, AST, ByteCode with iPython,
wirte some code, then print it's CST、AST、 BYTECODE, and then to read the
source code.

But, I can't dump the CST tree which will transform to AST, I know I can
print CST tree by modifying python source code, but how to dump the CST
tree in iPython just like dumping AST with ast package.

1. get AST with ast package2. get ByteCode with dis package

3. but how get cst??
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 550 v4

[Elvis Pranskevichus]

On Saturday, August 26, 2017 2:34:29 AM EDT Nathaniel Smith wrote:
> On Fri, Aug 25, 2017 at 3:32 PM, Yury Selivanov 
 wrote:
> > Coroutines and Asynchronous Tasks
> > -
> > 
> > In coroutines, like in generators, context variable changes are
> > local> 
> > and are not visible to the caller::
> > import asyncio
> > 
> > var = new_context_var()
> > 
> > async def sub():
> > assert var.lookup() == 'main'
> > var.set('sub')
> > assert var.lookup() == 'sub'
> > 
> > async def main():
> > var.set('main')
> > await sub()
> > assert var.lookup() == 'main'
> > 
> > loop = asyncio.get_event_loop()
> > loop.run_until_complete(main())
> 
> I think this change is a bad idea. I think that generally, an async
> call like 'await async_sub()' should have the equivalent semantics
> to a synchronous call like 'sync_sub()', except for the part where
> the former is able to contain yields. Giving every coroutine an LC
> breaks that equivalence. It also makes it so in async code, you
> can't necessarily refactor by moving code in and out of
> subroutines. Like, if we inline 'sub' into 'main', that shouldn't
> change the semantics, but...

If we could easily, we'd given each _normal function_ its own logical 
context as well.

What we are talking about here is variable scope leaking up the call 
stack.  I think this is a bad pattern.  For decimal context-like uses 
of the EC you should always use a context manager.  For uses like Web 
request locals, you always have a top function that sets the context 
vars.

> 
> I think I see the motivation: you want to make
> 
>await sub()
> 
> and
> 
>await ensure_future(sub())
> 
> have the same semantics, right? And the latter has to create a Task

What we want is for `await sub()` to be equivalent to 
`await asyncio.wait_for(sub())` and to `await asyncio.gather(sub())`.

Imagine we allow context var changes to leak out of `async def`.  It's 
easy to write code that relies on this:

async def init():
var.set('foo')

async def main():
await init()
assert var.lookup() == 'foo'

If we change `await init()` to `await asyncio.wait_for(init())`, the 
code will break (and in real world, possibly very subtly).

> It also adds non-trivial overhead, because now lookup() is O(depth
> of async callstack), instead of O(depth of (async) generator
> nesting), which is generally much smaller.

You would hit cache in lookup() most of the time.


Elvis
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 442 undocumented C API functions

[Pauli Virtanen]

Hi,

PEP 442 says "Two new C API functions are provided to ease calling of
tp_finalize, especially from custom deallocators."

This includes PyObject_CallFinalizerFromDealloc. However, no
documentation of these functions appears to exist (https://bugs.python.
org/issue31276).

As far as I understand, if you have a custom tp_dealloc, you *have* to
call PyObject_CallFinalizerFromDealloc in order to get your tp_finalize
called. Is this correct?

However, since the necessary functions are undocumented, it's unclear
if they were intended to be public Python API functions. So are they
actually public functions that 3rd party extensions can call?
If not, how is tp_finalize supposed to be used?

I'd be happy if someone can clarify the issue.

-- 
Pauli Virtanen
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 550 v4

[Sven R. Kunze]

On 26.08.2017 04:19, Ethan Furman wrote:

On 08/25/2017 03:32 PM, Yury Selivanov wrote:


A *context variable* is an object representing a value in the
execution context.  A new context variable is created by calling
the ``new_context_var()`` function.  A context variable object has
two methods:

* ``lookup()``: returns the value of the variable in the current
   execution context;

* ``set()``: sets the value of the variable in the current
   execution context.


Why "lookup" and not "get" ?  Many APIs use "get" and it's 
functionality is well understood.




Why not the same interface as thread-local storage? This has been the 
question which bothered me from the beginning of PEP550. I don't 
understand what inventing a new way of access buys us here.



Python features regular attribute access for years. It's even simpler 
than method-based access.



Best,
Sven
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 550 v4

[Stefan Behnel]

Hi,

I'm aware that the current implementation is not final, but I already
adapted the coroutine changes for Cython to allow for some initial
integration testing with real external (i.e. non-Python coroutine) targets.
I haven't adapted the tests yet, so the changes are currently unused and
mostly untested.

https://github.com/scoder/cython/tree/pep550_exec_context

I also left some comments in the github commits along the way.

Stefan


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com