Re: [Python-Dev] PEP 567 -- Context Variables

[Dima Tisnek]

My 2c:
TL;DR PEP specifies implementation in some detail, but doesn't show
how proposed change can or should be used.



get()/set(value)/delete() methods: Python provides syntax sugar for
these, let's use it.
(dict: d["k"]/d["k] = value/del d["k"]; attrs: obj.k/obj.k = value/del
obj.k; inheriting threading.Local)



This PEP and 550 describe why TLS is inadequate, but don't seem to
specify how proposed context behaves in async world. I'd be most
interested in how it appears to work to the user of the new library.

Consider a case of asynchronous cache:

async def actual_lookup(name):
...

def cached_lookup(name, cache={}):
if name not in cache:
cache["name"] = shield(ensure_future(actual_lookup(name))
return cache["name"]

Unrelated (or related) asynchronous processes end up waiting on the same future:

async def called_with_user_context():
...
await cached_lookup(...)
...

Which context is propagated to actual_lookup()?
The PEP doesn't seem to state that clearly.
It appears to be first caller's context.
Is it a copy or a reference?
If first caller is cancelled, the context remains alive.



token is fragile, I believe PEP should propose a working context
manager instead.
Btw., isn't a token really a reference to
state-of-context-before-it's-cloned-and-modified?

On 13 December 2017 at 01:33, Yury Selivanov  wrote:
> Hi,
>
> This is a new proposal to implement context storage in Python.
>
> It's a successor of PEP 550 and builds on some of its API ideas and
> datastructures.  Contrary to PEP 550 though, this proposal only focuses
> on adding new APIs and implementing support for it in asyncio.  There
> are no changes to the interpreter or to the behaviour of generator or
> coroutine objects.
>
>
> PEP: 567
> Title: Context Variables
> Version: $Revision$
> Last-Modified: $Date$
> Author: Yury Selivanov 
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 12-Dec-2017
> Python-Version: 3.7
> Post-History: 12-Dec-2017
>
>
> Abstract
> 
>
> This PEP proposes the new ``contextvars`` module and a set of new
> CPython C APIs to support context variables.  This concept is
> similar to thread-local variables but, unlike TLS, it allows
> correctly keeping track of values per asynchronous task, e.g.
> ``asyncio.Task``.
>
> This proposal builds directly upon concepts originally introduced
> in :pep:`550`.  The key difference is that this PEP is only concerned
> with solving the case for asynchronous tasks, and not generators.
> There are no proposed modifications to any built-in types or to the
> interpreter.
>
>
> Rationale
> =
>
> Thread-local variables are insufficient for asynchronous tasks which
> execute concurrently in the same OS thread.  Any context manager that
> needs to save and restore a context value and uses
> ``threading.local()``, will have its context values bleed to other
> code unexpectedly when used in async/await code.
>
> A few examples where having a working context local storage for
> asynchronous code is desired:
>
> * Context managers like decimal contexts and ``numpy.errstate``.
>
> * Request-related data, such as security tokens and request
>   data in web applications, language context for ``gettext`` etc.
>
> * Profiling, tracing, and logging in large code bases.
>
>
> Introduction
> 
>
> The PEP proposes a new mechanism for managing context variables.
> The key classes involved in this mechanism are ``contextvars.Context``
> and ``contextvars.ContextVar``.  The PEP also proposes some policies
> for using the mechanism around asynchronous tasks.
>
> The proposed mechanism for accessing context variables uses the
> ``ContextVar`` class.  A module (such as decimal) that wishes to
> store a context variable should:
>
> * declare a module-global variable holding a ``ContextVar`` to
>   serve as a "key";
>
> * access the current value via the ``get()`` method on the
>   key variable;
>
> * modify the current value via the ``set()`` method on the
>   key variable.
>
> The notion of "current value" deserves special consideration:
> different asynchronous tasks that exist and execute concurrently
> may have different values.  This idea is well-known from thread-local
> storage but in this case the locality of the value is not always
> necessarily to a thread.  Instead, there is the notion of the
> "current ``Context``" which is stored in thread-local storage, and
> is accessed via ``contextvars.get_context()`` function.
> Manipulation of the current ``Context`` is the responsibility of the
> task framework, e.g. asyncio.
>
> A ``Context`` is conceptually a mapping, implemented using an
> immutable dictionary.  The ``ContextVar.get()`` method does a
> lookup in the current ``Context`` with ``self`` as a key, raising a
> ``LookupError``  or returning a default value specified in
> the constructor.
>
> The ``ContextVar.set(value)`` method clones the current ``Context``,
> 

Re: [Python-Dev] PEP 567 -- Context Variables

[Yury Selivanov]

On Tue, Dec 12, 2017 at 10:36 PM, Guido van Rossum  wrote:
> Some more feedback:
>
>> This proposal builds directly upon concepts originally introduced
>> in :pep:`550`.
>
> The phrase "builds upon" typically implies that the other resource must be
> read and understood first. I don't think that we should require PEP 550 for
> understanding of PEP 567. Maybe "This proposal is a simplified version of
> :pep:`550`." ?

I agree, "simplified version" is better.

>
>> The notion of "current value" deserves special consideration:
>> different asynchronous tasks that exist and execute concurrently
>> may have different values.  This idea is well-known from thread-local
>> storage but in this case the locality of the value is not always
>> necessarily to a thread.  Instead, there is the notion of the
>> "current ``Context``" which is stored in thread-local storage, and
>> is accessed via ``contextvars.get_context()`` function.
>> Manipulation of the current ``Context`` is the responsibility of the
>> task framework, e.g. asyncio.
>
> This begs two (related) questions:
> - If it's stored in TLS, why isn't it equivalent to TLS?
> - If it's read-only (as mentioned in the next paragraph) how can the
> framework modify it?
>
> I realize the answers are clear, but at this point in the exposition you
> haven't given the reader enough information to answer them, so this
> paragraph may confuse readers.

I'll think how to rephrase it.

>
>> Specification
>> =
>> [points 1, 2, 3]
>
> Shouldn't this also list Token? (It must be a class defined here so users
> can declare the type of variables/arguments in their code representing these
> tokens.)
>
>> The ``ContextVar`` class has the following constructor signature:
>> ``ContextVar(name, *, default=no_default)``.
>
> I think a word or two about the provenance of `no_default` would be good. (I
> think it's an internal singleton right?) Ditto for NO_DEFAULT in the C
> implementation sketch.

Fixed.

>
>> class Task:
>> def __init__(self, coro):
>
> Do we need a keyword arg 'context=None' here too? (I'm not sure what would
> be the use case, but somehow it stands out in comparison to call_later()
> etc.)

call_later() is low-level and it needs the 'context' argument as Task
and Future use it in their implementation.

It would be easy to add 'context' parameter to Task and
loop.create_task(), but I don't know about any concrete use-case for
that just yet.

>
>> CPython C API
>> -
>> TBD
>
> Yeah, what about it? :-)

I've added it: https://github.com/python/peps/pull/508/files

I didn't want to get into too much detail about the C API until I have
a working PR. Although I feel that the one I describe in the PEP now
is very close to what we'll have.

>
>> The internal immutable dictionary for ``Context`` is implemented
>> using Hash Array Mapped Tries (HAMT).  They allow for O(log N) ``set``
>> operation, and for O(1) ``get_context()`` function.  [...]
>
> I wonder if we can keep the HAMT out of the discussion at this point. I have
> nothing against it, but given that you already say you're leaving out
> optimizations and nothing in the pseudo code given here depends on them I
> wonder if they shouldn't be mentioned later. (Also the appendix with the
> perf analysis is the one thing that I think we can safely leave out, just
> reference PEP 550 for this.)

I've added a new section "Implementation Notes" that mentions HAMT and
ContextVar.get() cache.  Both refer to PEP 550's lengthy explanations.

>
>> class _ContextData
>
> Since this isn't a real class anyway I think the __mapping attribute might
> as well be named _mapping. Ditto for other __variables later.

Done.

Yury
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 567 -- Context Variables

[Guido van Rossum]

Some more feedback:

> This proposal builds directly upon concepts originally introduced
> in :pep:`550`.

The phrase "builds upon" typically implies that the other resource must be
read and understood first. I don't think that we should require PEP 550 for
understanding of PEP 567. Maybe "This proposal is a simplified version of
:pep:`550`." ?

> The notion of "current value" deserves special consideration:
> different asynchronous tasks that exist and execute concurrently
> may have different values.  This idea is well-known from thread-local
> storage but in this case the locality of the value is not always
> necessarily to a thread.  Instead, there is the notion of the
> "current ``Context``" which is stored in thread-local storage, and
> is accessed via ``contextvars.get_context()`` function.
> Manipulation of the current ``Context`` is the responsibility of the
> task framework, e.g. asyncio.

This begs two (related) questions:
- If it's stored in TLS, why isn't it equivalent to TLS?
- If it's read-only (as mentioned in the next paragraph) how can the
framework modify it?

I realize the answers are clear, but at this point in the exposition you
haven't given the reader enough information to answer them, so this
paragraph may confuse readers.

> Specification
> =
> [points 1, 2, 3]

Shouldn't this also list Token? (It must be a class defined here so users
can declare the type of variables/arguments in their code representing
these tokens.)

> The ``ContextVar`` class has the following constructor signature:
> ``ContextVar(name, *, default=no_default)``.

I think a word or two about the provenance of `no_default` would be good.
(I think it's an internal singleton right?) Ditto for NO_DEFAULT in the C
implementation sketch.

> class Task:
> def __init__(self, coro):

Do we need a keyword arg 'context=None' here too? (I'm not sure what would
be the use case, but somehow it stands out in comparison to call_later()
etc.)

> CPython C API
> -
> TBD

Yeah, what about it? :-)

> The internal immutable dictionary for ``Context`` is implemented
> using Hash Array Mapped Tries (HAMT).  They allow for O(log N) ``set``
> operation, and for O(1) ``get_context()`` function.  [...]

I wonder if we can keep the HAMT out of the discussion at this point. I
have nothing against it, but given that you already say you're leaving out
optimizations and nothing in the pseudo code given here depends on them I
wonder if they shouldn't be mentioned later. (Also the appendix with the
perf analysis is the one thing that I think we can safely leave out, just
reference PEP 550 for this.)

> class _ContextData

Since this isn't a real class anyway I think the __mapping attribute might
as well be named _mapping. Ditto for other __variables later.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 567 -- Context Variables

[Yury Selivanov]

On Tue, Dec 12, 2017 at 9:55 PM, Guido van Rossum  wrote:
> On Tue, Dec 12, 2017 at 5:35 PM, Yury Selivanov 
> wrote:
>>
>> On Tue, Dec 12, 2017 at 6:49 PM, Victor Stinner
>>  wrote:
>> > I like the overall idea and I prefer this PEP over PEP 550 since it's
>> > shorter and easier to read :-)
>> >
>> > Question: Is there an API to list all context variables?
>>
>> Context implements abc.Mapping, so 'get_context().keys()' will give
>> you a list of all ContextVars in the current context.
>
>
> This was hinted at in the PEP, but maybe an explicit example would be nice.

Sure.

>
>>
>> > Each get_context() call returns a new Context object. It may be worth
>> > to mention it. I understand why, but it's surprising that "assert
>> > get_context() is not get_context()" fails. Maybe it's a naming issue?
>> > Maybe rename it to contextvars.context()?
>>
>> I think the name is fine.  While get_context() will return a new instance
>> every time you call it, those instances will have the same context
>> variables/values in them, so I don't think it's a problem.
>
>
> I'm fine with this, but perhaps == should be supported so that those two are
> guaranteed to be considered equal? (Otherwise an awkward idiom to compare
> contexts using expensive dict() copies would be needed to properly compare
> two contexts for equality.)

I've no problem with implementing 'Context.__eq__'.  I think
abc.Mapping also implements it.

>
>>
>> > At the first read, I understood that that ctx.run() creates a new
>> > temporary context which is removed once ctx.run() returns.
>> >
>> > Now I understand that context variable values are restored to their
>> > previous values once run() completes. Am I right?
>>
>> ctx.run(func) runs 'func' in the 'ctx' context.  Any changes to
>> ContextVars that func makes will stay isolated to the 'ctx' context.
>>
>> >
>> > Maybe add a short comment to explain that?
>>
>> Added.
>
>
> The PEP still contains the following paragraph:
>
>> Any changes to the context will be contained and persisted in the
>> ``Context`` object on which ``run()`` is called on.
>
> This phrase is confusing; it could be read as implying that context changes
> made by the function *will* get propagated back to the caller of run(),
> contradicting what was said earlier. Maybe it's best to just delete it?
> Otherwise if you intend it to add something it needs to be rephrased. Maybe
> "persisted" is the key word causing confusion?

I'll remove "persisted" now, I agree it adds more confusion than
clarity.  Victor is also confused with how 'Context.run()' is
currently explained, I'll try to make it clearer.

Thank you,
Yury
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 567 -- Context Variables

[Guido van Rossum]

On Tue, Dec 12, 2017 at 5:35 PM, Yury Selivanov 
wrote:

> On Tue, Dec 12, 2017 at 6:49 PM, Victor Stinner
>  wrote:
> > I like the overall idea and I prefer this PEP over PEP 550 since it's
> > shorter and easier to read :-)
> >
> > Question: Is there an API to list all context variables?
>
> Context implements abc.Mapping, so 'get_context().keys()' will give
> you a list of all ContextVars in the current context.
>

This was hinted at in the PEP, but maybe an explicit example would be nice.


> > Each get_context() call returns a new Context object. It may be worth
> > to mention it. I understand why, but it's surprising that "assert
> > get_context() is not get_context()" fails. Maybe it's a naming issue?
> > Maybe rename it to contextvars.context()?
>
> I think the name is fine.  While get_context() will return a new instance
> every time you call it, those instances will have the same context
> variables/values in them, so I don't think it's a problem.
>

I'm fine with this, but perhaps == should be supported so that those two
are guaranteed to be considered equal? (Otherwise an awkward idiom to
compare contexts using expensive dict() copies would be needed to properly
compare two contexts for equality.)


> > At the first read, I understood that that ctx.run() creates a new
> > temporary context which is removed once ctx.run() returns.
> >
> > Now I understand that context variable values are restored to their
> > previous values once run() completes. Am I right?
>
> ctx.run(func) runs 'func' in the 'ctx' context.  Any changes to
> ContextVars that func makes will stay isolated to the 'ctx' context.
>
> >
> > Maybe add a short comment to explain that?
>
> Added.
>

The PEP still contains the following paragraph:

> Any changes to the context will be contained and persisted in the
> ``Context`` object on which ``run()`` is called on.

This phrase is confusing; it could be read as implying that context changes
made by the function *will* get propagated back to the caller of run(),
contradicting what was said earlier. Maybe it's best to just delete it?
Otherwise if you intend it to add something it needs to be rephrased. Maybe
"persisted" is the key word causing confusion?

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [python/peps] PEP 567 review and copyedits (#503)

[Yury Selivanov]

On Tue, Dec 12, 2017 at 9:07 PM, Chris Angelico  wrote:
> Redirecting comments from the PR to the ML. Everything that was
> tightly bound to the PR has been dropped.
>
> On Wed, Dec 13, 2017 at 12:15 PM, Yury Selivanov
>  wrote:
>> Most of your questions should be asked on python-dev. I'll answer them here, 
>> but if you have any follow-ups, please raise the on the ml.
>>
>>> What happens if you set, set again, then reset from the first one's token?
>>
>> The context will be reset to the state it was in before the first set, 
>> w.r.t. that variable's value.
>>
>>> A peek at the implementation shows that it simply resets the value, so 
>>> aside from having magic that allows it to represent "no value", the token 
>>> buys nothing that you couldn't get by simply returning the old value - 
>>> which is a valuable API to have. Am I reading this correctly?
>>
>> "no value" is the main feature of Token, it what makes the PEP future 
>> compatible with PEP 550.
>
> A lot of APIs are built to return the old value, not wrapped in any
> sort of opaque token. If the "no value" magic were to be exposed
> (contextvars.NO_VALUE as a specific sentinel value), this would allow
> deliberate use of "previous value" as an actual part of the API. Does
> futureproofing require that the token be opaque?

To get the previous value just use 'ContextVar.get()' before you call
'ContextVar.set()'.

I don't see a lot of value in further enhancing "Token".
Future-proofing requires us to have no ContextVar.delete() method, and
that's why we have the set/reset API.


>
>>> Implementation, _ContextData class - I don't often see "self.__mapping" in 
>>> PEPs unless the name mangling is actually needed. Is it used here? Would 
>>> "self._mapping" be as effective?
>>
>> __ is used to highlight the fact that all those attributes are private and 
>> inaccessible for Python code.
>
> Just to clarify, then: this is an artifact of the Python reference
> implementation being unable to perfectly represent the behaviour of C
> code, but had you been writing this for actual implementation in the
> stdlib, you'd have used a single underscore?

I'd still use the dunder prefix, there's nothing wrong with it IMO.

>
>>> The HAMT for Context boasts O(log N) 'set' operations. What is N? Number of 
>>> variables used? Number of times they get set? Number of times set w/o being 
>>> reset?
>>
>> Number of ContextVars set in the context.
>
> Thanks. Might be worth mentioning that;

Yeah, I'll update the PEP.

> with the
> semantically-equivalent Python code, it's obvious that the cost of the
> copy scales with the number of unique variables, but without knowing
> what the actual HAMT does, it's not so obvious that that's true there
> too. The net result is that repeatedly setting the same variable
> doesn't increase the cost - right?

Yes!

I had to balance the implementation around the following constraints:

1. get_context() must be fast, as it will be used in
asyncio.call_soon() all the time.

2. ContextVar.get() must be fast, as modules like numpy and decimal
won't use it otherwise.

3. ContextVar.set() must not be slow, or become slower and slower as
we have more and more vars on the context.

Having an immutable dict (as opposed to using 'dict.copy()') allows us
to have a super fast 'get_context()'.  We could move 'dict.copy()' to
'ContextVar.set()', but then it would make it an O(N) operation, which
isn't acceptable either.  HAMT is a way to implement an immutable
mapping with a fast O(log N) 'set()' operation.

> The PEP looks pretty good to me.

Thank you, Chris.

Yury
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [python/peps] PEP 567 review and copyedits (#503)

[Chris Angelico]

Redirecting comments from the PR to the ML. Everything that was
tightly bound to the PR has been dropped.

On Wed, Dec 13, 2017 at 12:15 PM, Yury Selivanov
 wrote:
> Most of your questions should be asked on python-dev. I'll answer them here, 
> but if you have any follow-ups, please raise the on the ml.
>
>> What happens if you set, set again, then reset from the first one's token?
>
> The context will be reset to the state it was in before the first set, w.r.t. 
> that variable's value.
>
>> A peek at the implementation shows that it simply resets the value, so aside 
>> from having magic that allows it to represent "no value", the token buys 
>> nothing that you couldn't get by simply returning the old value - which is a 
>> valuable API to have. Am I reading this correctly?
>
> "no value" is the main feature of Token, it what makes the PEP future 
> compatible with PEP 550.

A lot of APIs are built to return the old value, not wrapped in any
sort of opaque token. If the "no value" magic were to be exposed
(contextvars.NO_VALUE as a specific sentinel value), this would allow
deliberate use of "previous value" as an actual part of the API. Does
futureproofing require that the token be opaque?

>> Implementation, _ContextData class - I don't often see "self.__mapping" in 
>> PEPs unless the name mangling is actually needed. Is it used here? Would 
>> "self._mapping" be as effective?
>
> __ is used to highlight the fact that all those attributes are private and 
> inaccessible for Python code.

Just to clarify, then: this is an artifact of the Python reference
implementation being unable to perfectly represent the behaviour of C
code, but had you been writing this for actual implementation in the
stdlib, you'd have used a single underscore?

>> The HAMT for Context boasts O(log N) 'set' operations. What is N? Number of 
>> variables used? Number of times they get set? Number of times set w/o being 
>> reset?
>
> Number of ContextVars set in the context.

Thanks. Might be worth mentioning that; with the
semantically-equivalent Python code, it's obvious that the cost of the
copy scales with the number of unique variables, but without knowing
what the actual HAMT does, it's not so obvious that that's true there
too. The net result is that repeatedly setting the same variable
doesn't increase the cost - right?

Most of my questions were fairly straightforwardly answered in Yury's
response on the PR. One question about formatting has been
subsequently fixed, so all's well on that front too.

The PEP looks pretty good to me.

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 567 -- Context Variables

[Yury Selivanov]

Hi Victor,

On Tue, Dec 12, 2017 at 6:49 PM, Victor Stinner
 wrote:
> Hi Yury,
>
> I like the overall idea and I prefer this PEP over PEP 550 since it's
> shorter and easier to read :-)
>
> Question: Is there an API to list all context variables?

Context implements abc.Mapping, so 'get_context().keys()' will give
you a list of all ContextVars in the current context.

>
> Would it be possible to have a very summary of the changes in the PEP?
> I propose:
>
> """
> * Added contextvars module with ContextVar, Context and Token classes,
> and a get_context() function
> * asyncio: Added keyword-only context parameter to call_at(),
> call_later(), call_soon() methods of event loops and
> Future.add_done_callback(); Task are modified internally to maintain
> their own isolated context.
> """

Added.

>
> Each get_context() call returns a new Context object. It may be worth
> to mention it. I understand why, but it's surprising that "assert
> get_context() is not get_context()" fails. Maybe it's a naming issue?
> Maybe rename it to contextvars.context()?

I think the name is fine.  While get_context() will return a new instance
every time you call it, those instances will have the same context
variables/values in them, so I don't think it's a problem.

>
>
>> Abstract: ... This concept is similar to thread-local variables but, unlike 
>> TLS, ...
>
> nitpick: please write "Thread Local Storage (TLS)". When I read TLS, I
> understand HTTPS (Transport Layer Security) :-)

Fixed.

[..]
>> ``contextvars.Token`` is an opaque object that should be used to
>> restore the ``ContextVar`` to its previous value, or remove it from
>> the context if it was not set before.  The ``ContextVar.reset(Token)``
>> is used for that::
>>
>> old = var.set(1)
>> try:
>> ...
>> finally:
>> var.reset(old)
>
> I don't see where is the token in this example. Does set() return a
> token object? Yes according to ContextVar pseudo-code below.
>
> When I read "old", I understand that set() returns the old value, not
> an opaque token. Maybe rename "old" to "token"?

Fixed.

>
>
>> The ``Token`` API exists to make the current proposal forward
>> compatible with :pep:`550`, in case there is demand to support
>> context variables in generators and asynchronous generators in the
>> future.
>
> Cool. I like the idea of starting with something simple in Python 3.7.
> Then extend it in Python 3.8 or later (support generators), if it
> becomes popular, once the first simple (but "incomplete", without
> generators) implementation is battle-tested.
>
>
>> Any changes to any context variables that ``function`` causes, will
>> be contained in the ``ctx`` context::
>>
>> var = ContextVar('var')
>> var.set('spam')
>>
>> def function():
>> assert var.get() == 'spam'
>>
>> var.set('ham')
>> assert var.get() == 'ham'
>>
>> ctx = get_context()
>> ctx.run(function)
>>
>> assert var.get('spam')
>
> Should I read assert var.get() == 'spam' here?

Yes, fixed.

>
> At the first read, I understood that that ctx.run() creates a new
> temporary context which is removed once ctx.run() returns.
>
> Now I understand that context variable values are restored to their
> previous values once run() completes. Am I right?

ctx.run(func) runs 'func' in the 'ctx' context.  Any changes to
ContextVars that func makes will stay isolated to the 'ctx' context.

>
> Maybe add a short comment to explain that?

Added.

>
> # Call function() in the context ctx
> # and then restores context variables of ctx to their previous values
> ctx.run(function)
>
>
>> Backwards Compatibility
>> ===
>>
>> This proposal preserves 100% backwards compatibility.
>
> Ok.
>
>> Libraries that use ``threading.local()`` to store context-related
>> values, currently work correctly only for synchronous code.  Switching
>> them to use the proposed API will keep their behavior for synchronous
>> code unmodified, but will automatically enable support for
>> asynchronous code.
>
> I'm confused by this sentence. I suggest to remove it :-)
>
> Converting code to contextvars makes it immediately backward
> incompatible, no I'm not sure that it's a good it to suggest it in
> this section.

If we update decimal to use ContextVars internally, decimal will stay
100% backwards compatible.

Yury
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 565: Show DeprecationWarning in __main__

[Victor Stinner]

2017-12-13 0:24 GMT+01:00 Guido van Rossum :
> Considered disagreement is acceptable.

Sure, I'm fine with that ;-)

> Nick, congrats with PEP 565! Please update the PEP to mark it as approved
> with a link to this message as the resolution, and let's get the
> implementation into 3.7a4!

Nick wrote that he will be away, since I update his PEP:
https://github.com/python/peps/commit/355eced94cf4117492c9e1eee8f950f08e53ec90

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 567 -- Context Variables

[Victor Stinner]

Hi Yury,

I like the overall idea and I prefer this PEP over PEP 550 since it's
shorter and easier to read :-)

Question: Is there an API to list all context variables?

Would it be possible to have a very summary of the changes in the PEP?
I propose:

"""
* Added contextvars module with ContextVar, Context and Token classes,
and a get_context() function
* asyncio: Added keyword-only context parameter to call_at(),
call_later(), call_soon() methods of event loops and
Future.add_done_callback(); Task are modified internally to maintain
their own isolated context.
"""

Each get_context() call returns a new Context object. It may be worth
to mention it. I understand why, but it's surprising that "assert
get_context() is not get_context()" fails. Maybe it's a naming issue?
Maybe rename it to contextvars.context()?


> Abstract: ... This concept is similar to thread-local variables but, unlike 
> TLS, ...

nitpick: please write "Thread Local Storage (TLS)". When I read TLS, I
understand HTTPS (Transport Layer Security) :-)

Your PEP seems to be written for asyncio. Maybe it would help to
understand it to make it more explicit in the abstract... even if I
understand perfectly that it's not strictly specific to asyncio ;-)


> # Declare a context variable 'var' with the default value 42.
> var = ContextVar('var', default=42)

nitpick: I suggest to use 'name' rather than 'var', to make it obvious
that the first parameter is the variable name.

> ``contextvars.Token`` is an opaque object that should be used to
> restore the ``ContextVar`` to its previous value, or remove it from
> the context if it was not set before.  The ``ContextVar.reset(Token)``
> is used for that::
>
> old = var.set(1)
> try:
> ...
> finally:
> var.reset(old)

I don't see where is the token in this example. Does set() return a
token object? Yes according to ContextVar pseudo-code below.

When I read "old", I understand that set() returns the old value, not
an opaque token. Maybe rename "old" to "token"?


> The ``Token`` API exists to make the current proposal forward
> compatible with :pep:`550`, in case there is demand to support
> context variables in generators and asynchronous generators in the
> future.

Cool. I like the idea of starting with something simple in Python 3.7.
Then extend it in Python 3.8 or later (support generators), if it
becomes popular, once the first simple (but "incomplete", without
generators) implementation is battle-tested.


> Any changes to any context variables that ``function`` causes, will
> be contained in the ``ctx`` context::
>
> var = ContextVar('var')
> var.set('spam')
>
> def function():
> assert var.get() == 'spam'
>
> var.set('ham')
> assert var.get() == 'ham'
>
> ctx = get_context()
> ctx.run(function)
>
> assert var.get('spam')

Should I read assert var.get() == 'spam' here?

At the first read, I understood that that ctx.run() creates a new
temporary context which is removed once ctx.run() returns.

Now I understand that context variable values are restored to their
previous values once run() completes. Am I right?

Maybe add a short comment to explain that?

# Call function() in the context ctx
# and then restores context variables of ctx to their previous values
ctx.run(function)


> Backwards Compatibility
> ===
>
> This proposal preserves 100% backwards compatibility.

Ok.

> Libraries that use ``threading.local()`` to store context-related
> values, currently work correctly only for synchronous code.  Switching
> them to use the proposed API will keep their behavior for synchronous
> code unmodified, but will automatically enable support for
> asynchronous code.

I'm confused by this sentence. I suggest to remove it :-)

Converting code to contextvars makes it immediately backward
incompatible, no I'm not sure that it's a good it to suggest it in
this section.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 565: Show DeprecationWarning in __main__

[Guido van Rossum]

OK, in that case I'll just pronounce approval right here. Considered
disagreement is acceptable.

Nick, congrats with PEP 565! Please update the PEP to mark it as approved
with a link to this message as the resolution, and let's get the
implementation into 3.7a4!

On Tue, Dec 12, 2017 at 2:58 PM, Victor Stinner 
wrote:

> Hi,
>
> 2017-12-12 21:21 GMT+01:00 Guido van Rossum :
> > I'm still hoping to accept this PEP, but I don't have time to wrap my
> head
> > around -Xdev ("devmode"?) which appears to be Victor's latest pet
> project.
> > Should PEP 565 be changed to copy with devmode's behavior, or the other
> way
> > around, or should they just ignore each other? It is not clear of me what
> > the status of the mention in PEP 565 of -Xdev is -- normative or
> > informational? I really don't want to have to learn how devmode works in
> > order to be able to accept PEP 565 (or send it back for revision), so I
> am
> > asking you two to let me know.
>
> The warnings filters had a few corner cases. We discussed with Nick to
> fix them to make them simpler. We agreed on these priorities for
> command line options and environment variables:
>
> -b and -bb > -W > PYTHONWARNINGS > -X dev > default filters
>
> In release mode, the default filters became:
>
> ignore::DeprecationWarning
> ignore::PendingDeprecationWarning
> ignore::ImportWarning
> ignore::ResourceWarning
>
> ignore::BytesWarning is gone. We now rely on the fact the BytesWarning
> should not be emited without -b nor -bb in practice.
>
> It has been implemented in https://bugs.python.org/issue32230 ! (I
> just merged Nick's PR.)
>
>
> Now -X dev behaves again as my initial propopal: for warnings, "-X
> dev" simply behaves  as "-W default". (Previously, I had to hack the
> code to respect -b and -bb options, but I don't think that it's worth
> it to explain that here, it's doesn't matter anymore ;-))
>
> The PEP 565 is still different: it doesn't behaves as "-W default",
> but "-W default::DeprecationWarning:__main__". Only DeprecationWarning
> warnings are shown, whereas -X dev shows DeprecationWarning, but also
> PendingDeprecationWarning, ResourceWarning and ImportWarning.
> Moreover, -X dev shows warnings in all modules, not only __main__.
>
> You may see -X dev as a builtin linter, whereas PEP 565 seems to be
> very specific to one specific issue: display deprecation warnings, but
> only in the __main__ module.
>
> Does it help you to understand the difference?
>
>
> Note: I still dislike the PEP 565, but well, that's just my opinion ;-)
>
> Victor
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 565: Show DeprecationWarning in __main__

[Victor Stinner]

Hi,

2017-12-12 21:21 GMT+01:00 Guido van Rossum :
> I'm still hoping to accept this PEP, but I don't have time to wrap my head
> around -Xdev ("devmode"?) which appears to be Victor's latest pet project.
> Should PEP 565 be changed to copy with devmode's behavior, or the other way
> around, or should they just ignore each other? It is not clear of me what
> the status of the mention in PEP 565 of -Xdev is -- normative or
> informational? I really don't want to have to learn how devmode works in
> order to be able to accept PEP 565 (or send it back for revision), so I am
> asking you two to let me know.

The warnings filters had a few corner cases. We discussed with Nick to
fix them to make them simpler. We agreed on these priorities for
command line options and environment variables:

-b and -bb > -W > PYTHONWARNINGS > -X dev > default filters

In release mode, the default filters became:

ignore::DeprecationWarning
ignore::PendingDeprecationWarning
ignore::ImportWarning
ignore::ResourceWarning

ignore::BytesWarning is gone. We now rely on the fact the BytesWarning
should not be emited without -b nor -bb in practice.

It has been implemented in https://bugs.python.org/issue32230 ! (I
just merged Nick's PR.)


Now -X dev behaves again as my initial propopal: for warnings, "-X
dev" simply behaves  as "-W default". (Previously, I had to hack the
code to respect -b and -bb options, but I don't think that it's worth
it to explain that here, it's doesn't matter anymore ;-))

The PEP 565 is still different: it doesn't behaves as "-W default",
but "-W default::DeprecationWarning:__main__". Only DeprecationWarning
warnings are shown, whereas -X dev shows DeprecationWarning, but also
PendingDeprecationWarning, ResourceWarning and ImportWarning.
Moreover, -X dev shows warnings in all modules, not only __main__.

You may see -X dev as a builtin linter, whereas PEP 565 seems to be
very specific to one specific issue: display deprecation warnings, but
only in the __main__ module.

Does it help you to understand the difference?


Note: I still dislike the PEP 565, but well, that's just my opinion ;-)

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 565: Show DeprecationWarning in __main__

[Guido van Rossum]

Nick and Victor,

I'm still hoping to accept this PEP, but I don't have time to wrap my head
around -Xdev ("devmode"?) which appears to be Victor's latest pet project.
Should PEP 565 be changed to copy with devmode's behavior, or the other way
around, or should they just ignore each other? It is not clear of me what
the status of the mention in PEP 565 of -Xdev is -- normative or
informational? I really don't want to have to learn how devmode works in
order to be able to accept PEP 565 (or send it back for revision), so I am
asking you two to let me know.

On Wed, Dec 6, 2017 at 1:42 AM, Victor Stinner 
wrote:

> Let's discuss -Xdev implementation issue at https://bugs.python.org/
> issue32230
>
> In short, -Xdev must add its warning at the end to respect BytesWarning,
> whereas it's not possible with -W option :-(
>
> Victor
>
> Le 6 déc. 2017 09:15, "Nick Coghlan"  a écrit :
>
> On 6 December 2017 at 14:50, Nick Coghlan  wrote:
> > On 6 December 2017 at 14:34, Nick Coghlan  wrote:
> >> That said, I go agree we could offer easier to use APIs to app
> >> developers that just want to hide warnings from their users, so I've
> >> filed https://bugs.python.org/issue32229 to propose a straightforward
> >> "warnings.hide_warnings()" API that encapsulates things like checking
> >> for a non-empty sys.warnoptions list.
> >
> > I've updated the "Limitations" section of the PEP to mention that
> > separate proposal:
> > https://github.com/python/peps/commit/6e93c8d2e6ad698834578d
> 4077b92a8fc84a70f5
>
> Having rebased the PEP 565 patch atop the "-X dev" changes, I think
> that if we don't change some of the details of how `-X dev` is
> implemented, `warnings.hide_warnings` (or a comparable convenience
> API) is going to be a requirement to help app developers effectively
> manage their default warnings settings in 3.7+.
>
> The problem is that devmode doesn't currently behave the same way
> `-Wd` does when it comes to sys.warnoptions:
>
> $ ./python -Wd -c "import sys; print(sys.warnoptions);
> print(sys.flags.dev_mode)"
> ['d']
> False
> $ ./python -X dev -c "import sys; print(sys.warnoptions);
> print(sys.flags.dev_mode)"
> []
> True
>
> As currently implemented, the warnings module actually checks
> `sys.flags.dev_mode` directly during startup (or `sys._xoptions` in
> the case of the pure Python fallback), and populates the warnings
> filter differently depending on what it finds:
>
> $ ./python -c "import warnings; print('\n'.join(map(str,
> warnings.filters)))"
> ('default', None, , '__main__', 0)
> ('ignore', None, , None, 0)
> ('ignore', None, , None, 0)
> ('ignore', None, , None, 0)
> ('ignore', None, , None, 0)
> ('ignore', None, , None, 0)
>
> $ ./python -X dev -c "import warnings; print('\n'.join(map(str,
> warnings.filters)))"
> ('ignore', None, , None, 0)
> ('default', None, , None, 0)
> ('default', None, , None, 0)
>
> $ ./python -Wd -c "import warnings; print('\n'.join(map(str,
> warnings.filters)))"
> ('default', None, , None, 0)
> ('default', None, , '__main__', 0)
> ('ignore', None, , None, 0)
> ('ignore', None, , None, 0)
> ('ignore', None, , None, 0)
> ('ignore', None, , None, 0)
> ('ignore', None, , None, 0)
>
> This means the app development snippet proposed in the PEP will no
> longer do the right thing, since it will ignore the dev mode flag:
>
> if not sys.warnoptions:
> # This still runs for `-X dev`
> warnings.simplefilter("ignore")
>
> My main suggested fix would be to adjust the way `-X dev` is
> implemented to include `sys.warnoptions.append('default')` (and remove
> the direct dev_mode query from the warnings module code).
>
> However, another possible way to go would be to make the correct
> Python 3.7+-only snippet look like this:
>
> import warnings
> warnings.hide_warnings()
>
> And have the forward-compatible snippet look like this:
>
> import warnings:
> if hasattr(warnings, "hide_warnings"):
> # Accounts for `-W`, `-X dev`, and any other implementation
> specific settings
> warnings.hide_warnings()
> else:
> # Only accounts for `-W`
> import sys
> if not sys.warnoptions:
> warnings.simplefilter("ignore")
>
> (We can also do both, of course)
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>
>
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 

Re: [Python-Dev] What's the status of PEP 505: None-aware operators?

[Chris Angelico]

On Wed, Dec 13, 2017 at 6:03 AM, Guido van Rossum  wrote:
> And I'll never approve syntax to make it easier to just ignore all
> exceptions without looking at them.

Well, I certainly wouldn't advocate "except Exception: -1", but the
syntax is the same as "except KeyError: -1" which is less
unreasonable.

But PEP 463 was rejected, which means that any proposals along these
lines need to first deal with the objections to that PEP, else there's
not a lot of point discussing them.

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] What's the status of PEP 505: None-aware operators?

[Guido van Rossum]

And I'll never approve syntax to make it easier to just ignore all
exceptions without looking at them.

On Tue, Dec 12, 2017 at 12:48 AM, Chris Angelico  wrote:

> On Tue, Dec 12, 2017 at 7:39 PM, Michel Desmoulin
>  wrote:
> >
> >
> > Le 29/11/2017 à 19:02, Barry Warsaw a écrit :
> >> On Nov 29, 2017, at 12:40, David Mertz  wrote:
> >>
> >>> I think some syntax could be possible to only "catch" some exceptions
> and let others propagate.  Maybe:
> >>>
> >>>val = name.strip()[4:].upper() except (AttributeError, KeyError): -1
> >>>
> >>> I don't really like throwing a colon in an expression though.  Perhaps
> some other word or symbol could work instead.  How does this read:
> >>>
> >>>val = name.strip()[4:].upper() except -1 in (AttributeError,
> KeyError)
> >>
> >> I don’t know whether I like any of this  but I think a more
> natural spelling would be:
> >>
> >>val = name.strip()[4:].upper() except (AttributeError, KeyError) as
> -1
> >>
> >> which could devolve into:
> >>
> >>val = name.strip()[4:].upper() except KeyError as -1
> >>
> >> or:
> >>
> >>val = name.strip()[4:].upper() except KeyError # Implicit `as None`
> >>
> >> I would *not* add any spelling for an explicit bare-except equivalent.
> You would have to write:
> >>
> >>val = name.strip()[4:].upper() except Exception as -1
> >>
> >> Cheers,
> >> -Barry
> >>
> >
> > I really like this one. It's way more general. I can see a use for
> > IndexError as well (lists don't have the dict.get() method).
> >
> > Also I would prefer not to use "as" this way. In the context of an
> > exception, "as" already binds the exception to a variable so it's
> confusing.
> >
> > What about:
> >
> >
> > val = name.strip()[4:].upper() except Exception: -1
>
> That happens to be the exact syntax recommended by PEP 463 (modulo
> some distinguishing parentheses).
>
> https://www.python.org/dev/peps/pep-0463/
>
> ChrisA
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 567 -- Context Variables

[Yury Selivanov]

Hi,

This is a new proposal to implement context storage in Python.

It's a successor of PEP 550 and builds on some of its API ideas and
datastructures.  Contrary to PEP 550 though, this proposal only focuses
on adding new APIs and implementing support for it in asyncio.  There
are no changes to the interpreter or to the behaviour of generator or
coroutine objects.


PEP: 567
Title: Context Variables
Version: $Revision$
Last-Modified: $Date$
Author: Yury Selivanov 
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 12-Dec-2017
Python-Version: 3.7
Post-History: 12-Dec-2017


Abstract


This PEP proposes the new ``contextvars`` module and a set of new
CPython C APIs to support context variables.  This concept is
similar to thread-local variables but, unlike TLS, it allows
correctly keeping track of values per asynchronous task, e.g.
``asyncio.Task``.

This proposal builds directly upon concepts originally introduced
in :pep:`550`.  The key difference is that this PEP is only concerned
with solving the case for asynchronous tasks, and not generators.
There are no proposed modifications to any built-in types or to the
interpreter.


Rationale
=

Thread-local variables are insufficient for asynchronous tasks which
execute concurrently in the same OS thread.  Any context manager that
needs to save and restore a context value and uses
``threading.local()``, will have its context values bleed to other
code unexpectedly when used in async/await code.

A few examples where having a working context local storage for
asynchronous code is desired:

* Context managers like decimal contexts and ``numpy.errstate``.

* Request-related data, such as security tokens and request
  data in web applications, language context for ``gettext`` etc.

* Profiling, tracing, and logging in large code bases.


Introduction


The PEP proposes a new mechanism for managing context variables.
The key classes involved in this mechanism are ``contextvars.Context``
and ``contextvars.ContextVar``.  The PEP also proposes some policies
for using the mechanism around asynchronous tasks.

The proposed mechanism for accessing context variables uses the
``ContextVar`` class.  A module (such as decimal) that wishes to
store a context variable should:

* declare a module-global variable holding a ``ContextVar`` to
  serve as a "key";

* access the current value via the ``get()`` method on the
  key variable;

* modify the current value via the ``set()`` method on the
  key variable.

The notion of "current value" deserves special consideration:
different asynchronous tasks that exist and execute concurrently
may have different values.  This idea is well-known from thread-local
storage but in this case the locality of the value is not always
necessarily to a thread.  Instead, there is the notion of the
"current ``Context``" which is stored in thread-local storage, and
is accessed via ``contextvars.get_context()`` function.
Manipulation of the current ``Context`` is the responsibility of the
task framework, e.g. asyncio.

A ``Context`` is conceptually a mapping, implemented using an
immutable dictionary.  The ``ContextVar.get()`` method does a
lookup in the current ``Context`` with ``self`` as a key, raising a
``LookupError``  or returning a default value specified in
the constructor.

The ``ContextVar.set(value)`` method clones the current ``Context``,
assigns the ``value`` to it with ``self`` as a key, and sets the
new ``Context`` as a new current.  Because ``Context`` uses an
immutable dictionary, cloning it is O(1).


Specification
=

A new standard library module ``contextvars`` is added with the
following APIs:

1. ``get_context() -> Context`` function is used to get the current
   ``Context`` object for the current OS thread.

2. ``ContextVar`` class to declare and access context variables.

3. ``Context`` class encapsulates context state.  Every OS thread
   stores a reference to its current ``Context`` instance.
   It is not possible to control that reference manually.
   Instead, the ``Context.run(callable, *args)`` method is used to run
   Python code in another context.


contextvars.ContextVar
--

The ``ContextVar`` class has the following constructor signature:
``ContextVar(name, *, default=no_default)``.  The ``name`` parameter
is used only for introspection and debug purposes.  The ``default``
parameter is optional.  Example::

# Declare a context variable 'var' with the default value 42.
var = ContextVar('var', default=42)

``ContextVar.get()`` returns a value for context variable from the
current ``Context``::

# Get the value of `var`.
var.get()

``ContextVar.set(value) -> Token`` is used to set a new value for
the context variable in the current ``Context``::

# Set the variable 'var' to 1 in the current context.
var.set(1)

``contextvars.Token`` is an opaque object that should be used to
restore the ``ContextVar`` 

Re: [Python-Dev] Support of the Android platform

[Victor Stinner]

2017-12-12 15:02 GMT+01:00 Antoine Pitrou :
> It sounds reasonable to me, as long as someone is monitoring their
> results (that would probably be you).

There is now the buildbot-status where failures are reported. Trust
me, I like making a lot of noise when something is broken :-)

FYI I'm trying https://github.com/python/cpython/pull/1629 : I
succeeded to "cross-compile" a binary on Linux for Android. I'm now
downloading an ISO to boot an x86-64 Android VM to test this binary
;-)

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Support of the Android platform

[Antoine Pitrou]

On Tue, 12 Dec 2017 10:50:53 +0100
Xavier de Gaye  wrote:
> On 12/11/2017 04:14 PM, Victor Stinner wrote:
>  > I'm asking for precise hardware specifications since Red Hat may be
>  > able to provide one through the https://osci.io/ program.  
> 
> Is it acceptable to run the arm buildbots only on a weekly basis ?

It sounds reasonable to me, as long as someone is monitoring their
results (that would probably be you).

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Support of the Android platform

[Xavier de Gaye]

On 12/11/2017 04:14 PM, Victor Stinner wrote:
> I'm asking for precise hardware specifications since Red Hat may be
> able to provide one through the https://osci.io/ program.

Is it acceptable to run the arm buildbots only on a weekly basis ?

For all the architectures (x86_64, armv7 and arm64), the tests are run with the same Python code, built with the same tools and running on the same operating system that runs on emulators using the 
same configuration.


If we do not take into account the ctypes issue, there is only one issue (issue 26939 [1]) among all the Android issues that is specific to arm, and it is not even really an arm issue but an issue 
related to the slowness of the emulator that has been fixed by increasing in a test the switch interval with sys.setswitchinterval().


Xavier

[1] https://bugs.python.org/issue26939

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is static typing still optional?

[Eric V. Smith]

On 12/11/2017 9:25 PM, Nick Coghlan wrote:
On 11 Dec. 2017 12:26 pm, "Eric V. Smith" > wrote:




I see a couple of options:
1a: Use a default type annotation, if one is not is supplied.
typing.Any would presumably make the most sense.
1b: Use None if not type is supplied.
2: Rework the code to not require annotations at all.


1c: annotate with the string "typing.Any" (this may require a tweak to 
the rules for evaluating lazy annotations, though)


Good idea, since it needs to be supported, anyway, especially in light 
of PEP 563.


Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] What's the status of PEP 505: None-aware operators?

[Chris Angelico]

On Tue, Dec 12, 2017 at 7:39 PM, Michel Desmoulin
 wrote:
>
>
> Le 29/11/2017 à 19:02, Barry Warsaw a écrit :
>> On Nov 29, 2017, at 12:40, David Mertz  wrote:
>>
>>> I think some syntax could be possible to only "catch" some exceptions and 
>>> let others propagate.  Maybe:
>>>
>>>val = name.strip()[4:].upper() except (AttributeError, KeyError): -1
>>>
>>> I don't really like throwing a colon in an expression though.  Perhaps some 
>>> other word or symbol could work instead.  How does this read:
>>>
>>>val = name.strip()[4:].upper() except -1 in (AttributeError, KeyError)
>>
>> I don’t know whether I like any of this  but I think a more natural 
>> spelling would be:
>>
>>val = name.strip()[4:].upper() except (AttributeError, KeyError) as -1
>>
>> which could devolve into:
>>
>>val = name.strip()[4:].upper() except KeyError as -1
>>
>> or:
>>
>>val = name.strip()[4:].upper() except KeyError # Implicit `as None`
>>
>> I would *not* add any spelling for an explicit bare-except equivalent.  You 
>> would have to write:
>>
>>val = name.strip()[4:].upper() except Exception as -1
>>
>> Cheers,
>> -Barry
>>
>
> I really like this one. It's way more general. I can see a use for
> IndexError as well (lists don't have the dict.get() method).
>
> Also I would prefer not to use "as" this way. In the context of an
> exception, "as" already binds the exception to a variable so it's confusing.
>
> What about:
>
>
> val = name.strip()[4:].upper() except Exception: -1

That happens to be the exact syntax recommended by PEP 463 (modulo
some distinguishing parentheses).

https://www.python.org/dev/peps/pep-0463/

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] What's the status of PEP 505: None-aware operators?

[Michel Desmoulin]

Le 29/11/2017 à 19:02, Barry Warsaw a écrit :
> On Nov 29, 2017, at 12:40, David Mertz  wrote:
> 
>> I think some syntax could be possible to only "catch" some exceptions and 
>> let others propagate.  Maybe:
>>
>>val = name.strip()[4:].upper() except (AttributeError, KeyError): -1
>>
>> I don't really like throwing a colon in an expression though.  Perhaps some 
>> other word or symbol could work instead.  How does this read:
>>
>>val = name.strip()[4:].upper() except -1 in (AttributeError, KeyError)
> 
> I don’t know whether I like any of this  but I think a more natural 
> spelling would be:
> 
>val = name.strip()[4:].upper() except (AttributeError, KeyError) as -1
> 
> which could devolve into:
> 
>val = name.strip()[4:].upper() except KeyError as -1
> 
> or:
> 
>val = name.strip()[4:].upper() except KeyError # Implicit `as None`
> 
> I would *not* add any spelling for an explicit bare-except equivalent.  You 
> would have to write:
> 
>val = name.strip()[4:].upper() except Exception as -1
> 
> Cheers,
> -Barry
> 

I really like this one. It's way more general. I can see a use for
IndexError as well (lists don't have the dict.get() method).

Also I would prefer not to use "as" this way. In the context of an
exception, "as" already binds the exception to a variable so it's confusing.

What about:


val = name.strip()[4:].upper() except Exception: -1
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com