date:20200108

Finished creating 2 issues on the meanwhile:
bpo-39264 to fix UserDict.get().
bpo-39267 to fix dict.__missing__ & dict.get() documentation.
Both include a PR.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/IQZ4S6BBIIKIOV4NTZ633SDFZC3D5M6J/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Should we pass tstate in the VECTORCALL calling convention before making it public?

Le mer. 8 janv. 2020 à 22:15, Nick Coghlan  a écrit :
> The vectorcall convention places a significant emphasis on speed, so
> being able to do a single PyThreadState_Get() call on the initial
> transition from Python to C, and then pass the thread state explicitly
> after that would make a lot of sense.

Honestly, I'm not 100% sure that performance is a good argument in
favor of adding tstate to the calling convention, since we cannot
measure yet the overhead in term of performance of calling frequently
_PyThreadState_GET(), especially with hypothetical changes on the GIL
state API.

I can only "feel" that it might be a performance issue tomorrow.

For me a stronger argument is that it should help to make Python more
"correct" in the case of subinterpreters. Help to get the right state,
since today too many things are "implicit" (rely on "globals").


> So I think this is a good idea, but we might need to place a caveat on
> Python 3.9 that passing in a thread state that is NOT the currently
> active thread state may not work correctly yet (it's going to take a
> while to make sure that we never inadvertently call back into
> "implicit thread state" APIs from the implementation of APIs that
> accept an explicit thread state, and even after CPython itself is
> fixed, there's no telling what other extension modules might be
> doing).

In term of API, we may keep the current *API* unchanged: pass
implicitly the current Python thread state. For example,
_PyObject_Vectorcall() doesn't take a tstate argument, but gets the
current Python thread state.

static inline PyObject *
_PyObject_Vectorcall(PyObject *callable, PyObject *const *args,
 size_t nargsf, PyObject *kwnames)
{
PyThreadState *tstate = PyThreadState_GET();
return _PyObject_VectorcallTstate(tstate, callable,
  args, nargsf, kwnames);
}

(I added _PyObject_VectorcallTstate() recently :-))

We can keep this prototype for the public API, but have a *private*
API which allows to pass tstate.

My idea is that internally, tstate would be passed *explicitly*. For example:

res = PyObject_Vectorcall(func, ...); // public API, 3rd party code
vs
res = _PyObject_Vectorcall(tstate, func, ...);  // private API, Python internals

My first question is about the calling convention, not the API ;-)

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/A35VK4D7DMY5UYSGTWOINHINDY5KM3JV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Should we pass tstate in the VECTORCALL calling convention before making it public?

2020-01-08 Thread Nick Coghlan

On Thu, 9 Jan 2020 at 01:49, Victor Stinner  wrote:
> The question is now if we should "propagate" tstate to function calls
> in the latest VECTORCALL calling convention (which is currently
> private). Petr Viktorin plans to make VECTORCALL APIs public in Python
> 3.9, as planned in the PEP 590:
> https://bugs.python.org/issue39245

The vectorcall convention places a significant emphasis on speed, so
being able to do a single PyThreadState_Get() call on the initial
transition from Python to C, and then pass the thread state explicitly
after that would make a lot of sense.

So I think this is a good idea, but we might need to place a caveat on
Python 3.9 that passing in a thread state that is NOT the currently
active thread state may not work correctly yet (it's going to take a
while to make sure that we never inadvertently call back into
"implicit thread state" APIs from the implementation of APIs that
accept an explicit thread state, and even after CPython itself is
fixed, there's no telling what other extension modules might be
doing).

The transition to the full public vectorcall API already keeps the
underscore prefixed versions around as aliases, so if the public API
changes to accept an explicit thread state, then the underscore
prefixed versions can be kept as wrapper functions that implicitly use
the current thread state.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TPKHV5G2LNZYUH33E3QEIO5OIVUEWF2P/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Dict missing consistency issues.

>
> [Bar Harel]
> > As for get(). Current implementation of UserDict returns the default
> value
> > for get() if __missing__() raises KeyError.  Which sounds reasonable to
> me.
> > After all, you would logically expect __missing__ to be called for get()
> > as you tried to get a non-existing value.
>
>
> [Ethan Furman]
> I disagree.  My understanding of the purpose behind get() is to get a value
> or return the default specified in the get() call, not to create a new key
> (which the default __missing__ does).
>

There is no default __missing__. That's what the __missing__ at defaultdict
does.

Like I said before: The fact __missing__ is not called for missing values
is a bit funny in my opinion, but it doesn't actually matter cause we can't
do anything about it without breaking lots of stuff (e.g. making
defaultdict.get worthless).
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3HPG4NPXSVOB3Q5HE7TQE37JG6TUFLMC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Dict missing consistency issues.

On Wed, Jan 8, 2020 at 12:02 PM Ethan Furman  wrote:

> I disagree.  My understanding of the purpose behind get() is to get a value
> or return the default specified in the get() call, not to create a new key
> (which the default __missing__ does).
>

Right. For comparison, defaultdict.get() doesn't add the key either.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/I6RDOQDF7JN5ATIQNPVI4MRHBRHV2M4L/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Dict missing consistency issues.

2020-01-08 Thread Ethan Furman


[Bar Harel]:

I was just about to fix dict's get() to call __missing__ if a key doesn't
exist (before returning the default value) but realized that although small,
that patch can cause future issues.  Right now there's an inconsistency:

The reason for this inconsistency is because the Mapping abc and dict
behave differently.  Dict's get doesn't call __getitem__ which causes
the call not to route to __missing__.  MutableMapping's get calls __getitem__,
which UserDict implements as a check to __missing__ as well.

Should get() call __missing__?



[Guido van Rossum]

I don't think __missing__ should be called by get() -- get() already has a
way to deal with missing keys, and making it use two different mechanisms
would be weird (e.g. if get() calls __missing__, is the default value ever
used?).



[Ethan Furman]
It could be if __missing__ refused to return a value for some particular keys.

However, I don't think get() should call __missing__.


[Bar Harel]

As for get(). Current implementation of UserDict returns the default value
for get() if __missing__() raises KeyError.  Which sounds reasonable to me.
After all, you would logically expect __missing__ to be called for get()
as you tried to get a non-existing value.



[Ethan Furman]
I disagree.  My understanding of the purpose behind get() is to get a value
or return the default specified in the get() call, not to create a new key
(which the default __missing__ does).

--
~Ethan~
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/M2K2BTDGFYUMQFRBHPY5IFCDYFXDMUET/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Dict missing consistency issues.

Sounds good to me.

If you want a second opinion als Raymond Hettinger.

On Wed, Jan 8, 2020 at 11:09 Bar Harel  wrote:

> Well it depends on the consensus for __missing__, and the consensus for
> get().
>
> I believe modifying get() is out of the question as it will lead to
> breakage. If so, whether it's a quirk or not doesn't matter as we can't
> change it. Adding a comment to the docs should suffice, along with updating
> the data model to explicitly state dict.__missing__ instead of
> object.__missing__.
>
> As for __missing__, if it's only a dict thing, I don't believe Mapping
> should be aware of it's existence, which means the patch should center on
> UserDict behaving differently (thus mimicking dict's quirky behavior).
>
> Since we cannot override get() to access self.data, (as it will bypass and
> break custom __getitem__) I think we might as well modify it to use the
> __contains__ it already created.
>
> A simple check of "if k in D __getitem__" will slow only UserDict, not all
> mappings, and be an acceptable solution.
>
> That's all as long as we accept the 2 facts that __missing__ is dict-only
> and .get() will stay somewhat quirky and unexpected but documented.
>
> On Wed, Jan 8, 2020 at 8:11 PM Guido van Rossum  wrote:
>
>> Hum, it looks like the UserDict.get() behavior is an accident due to the
>> way Mapping.get() is implemented. The comment there says
>>
>> 'D.get(k[,d]) -> D[k] if k in D, else d.  d defaults to None.'
>>
>> but the implementation just tries D[k] and catches KeyError. The
>> UserDict.__contains__() implementation is explicitly code to avoid
>> __missing__, since the default Mapping.__contains__() implementation *also*
>> just calls D[k] and catches KeyError.
>>
>> More indication that the UserDict.get() behavior is an accident: in
>> test_userdict.py there's a test for __missing__, test_missing() but it
>> doesn't test get() at all, only __getitem__().
>>
>> How were you proposing to fix the situation? I could imagine changing
>> Mapping.get() to call __contains__() first. That would slow down
>> Mapping.get() slightly (the default implementation would end up calling
>> __getitem__() twice if the key is present), but I don't think that's a
>> grave concern (if people want it faster there are a zillion ways to do
>> that).
>>
>> On Wed, Jan 8, 2020 at 9:49 AM Bar Harel  wrote:
>>
>>> Sorry for the double post then :-)
>>>
>>> As for get(). Current implementation of UserDict returns the default
>>> value for get() if __missing__() raises KeyError. Which sounds
>>> reasonable to me. After all, you would logically expect __missing__ to be
>>> called for get() as you tried to get a non-existing value.
>>>
>>> If __missing__ is only a dict thing, then I'll add a note to dict's
>>> .get() method saying it doesn't call __missing__ for missing values.
>>> I can also modify UserDict to behave in a consistent manner.
>>>
>>> Does that make sense?
>>>
>>> On Wed, Jan 8, 2020 at 7:42 PM Guido van Rossum 
>>> wrote:
>>>
 (Note: We received your email twice.)

 I don't think __missing__ should be called by get() -- get() already
 has a way to deal with missing keys, and making it use two different
 mechanisms would be weird (e.g. if get() calls __missing__, is the default
 value ever used?).

 To answer your second question, __missing__ is only a dict thing, it is
 not part of Mapping.

 On Wed, Jan 8, 2020 at 8:07 AM Bar Harel  wrote:

> Hey guys,
>
> I was just about to fix dict's get() to call __missing__ if a key
> doesn't exist (before returning the default value) but realized that
> although small, that patch can cause future issues.
> Right now there's an inconsistency:
>
> >>> from collections import UserDict
> >>> class A(dict):
> ...  def __missing__(self, key):
> ...   print(key)
> ...
> >>> class B(UserDict):
> ...  def __missing__(self, key):
> ...   print("UserDict", key)
> ...
> >>> a = A()
> >>> b = B()
> >>> a.get(123)
> >>> b.get(123)
> UserDict 123
> >>> a.get(123, "abc")
> 'abc'
> >>> b.get(123, "abc")
> UserDict 123
>
> The reason for this inconsistency is because the Mapping abc and dict
> behave differently.
> Dict's get doesn't call __getitem__ which causes the call not to route
> to __missing__.
> MutableMapping's get calls __getitem__, which UserDict implements as a
> check to __missing__ as well.
>
> According to the doc, the specification requires dict's __getitem__ to
> call __missing__. It doesn't say anything about get().
>
> Should get() call __missing__?
>
> If it does, things like defaultdict.get() might break. It will however
> be more consistent with dict's specification.
> If it doesn't, we expect Mapping to not care about __missing__ as it's
> only a dict thing, which will require UserDict to override get(). Dict's
> get() wil

[Python-Dev] Re: Why doesn't venv also install python3*-config?

2020-01-08 Thread Matt Billenstein via Python-Dev

On Wed, Jan 08, 2020 at 12:26:39PM +0100, Musbur wrote:
> I'm experimenting with package development on different versions of Python
> in different virtualenvs. After running "make" I don't do "make install",
> but rather I set up virtualenvs by running /path/to/source/python -m venv
> env_dir. 

I'd suggest https://github.com/pyenv/pyenv -- this handles this more
seamlessly.

m

--
Matt Billenstein
m...@vazor.com
http://www.vazor.com/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DL45OWIVYPZLKMLVSYISPG2XNY2S3PE7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Dict missing consistency issues.

Well it depends on the consensus for __missing__, and the consensus for
get().

I believe modifying get() is out of the question as it will lead to
breakage. If so, whether it's a quirk or not doesn't matter as we can't
change it. Adding a comment to the docs should suffice, along with updating
the data model to explicitly state dict.__missing__ instead of
object.__missing__.

As for __missing__, if it's only a dict thing, I don't believe Mapping
should be aware of it's existence, which means the patch should center on
UserDict behaving differently (thus mimicking dict's quirky behavior).

Since we cannot override get() to access self.data, (as it will bypass and
break custom __getitem__) I think we might as well modify it to use the
__contains__ it already created.

A simple check of "if k in D __getitem__" will slow only UserDict, not all
mappings, and be an acceptable solution.

That's all as long as we accept the 2 facts that __missing__ is dict-only
and .get() will stay somewhat quirky and unexpected but documented.

On Wed, Jan 8, 2020 at 8:11 PM Guido van Rossum  wrote:

> Hum, it looks like the UserDict.get() behavior is an accident due to the
> way Mapping.get() is implemented. The comment there says
>
> 'D.get(k[,d]) -> D[k] if k in D, else d.  d defaults to None.'
>
> but the implementation just tries D[k] and catches KeyError. The
> UserDict.__contains__() implementation is explicitly code to avoid
> __missing__, since the default Mapping.__contains__() implementation *also*
> just calls D[k] and catches KeyError.
>
> More indication that the UserDict.get() behavior is an accident: in
> test_userdict.py there's a test for __missing__, test_missing() but it
> doesn't test get() at all, only __getitem__().
>
> How were you proposing to fix the situation? I could imagine changing
> Mapping.get() to call __contains__() first. That would slow down
> Mapping.get() slightly (the default implementation would end up calling
> __getitem__() twice if the key is present), but I don't think that's a
> grave concern (if people want it faster there are a zillion ways to do
> that).
>
> On Wed, Jan 8, 2020 at 9:49 AM Bar Harel  wrote:
>
>> Sorry for the double post then :-)
>>
>> As for get(). Current implementation of UserDict returns the default
>> value for get() if __missing__() raises KeyError. Which sounds
>> reasonable to me. After all, you would logically expect __missing__ to be
>> called for get() as you tried to get a non-existing value.
>>
>> If __missing__ is only a dict thing, then I'll add a note to dict's
>> .get() method saying it doesn't call __missing__ for missing values.
>> I can also modify UserDict to behave in a consistent manner.
>>
>> Does that make sense?
>>
>> On Wed, Jan 8, 2020 at 7:42 PM Guido van Rossum  wrote:
>>
>>> (Note: We received your email twice.)
>>>
>>> I don't think __missing__ should be called by get() -- get() already has
>>> a way to deal with missing keys, and making it use two different mechanisms
>>> would be weird (e.g. if get() calls __missing__, is the default value ever
>>> used?).
>>>
>>> To answer your second question, __missing__ is only a dict thing, it is
>>> not part of Mapping.
>>>
>>> On Wed, Jan 8, 2020 at 8:07 AM Bar Harel  wrote:
>>>
 Hey guys,

 I was just about to fix dict's get() to call __missing__ if a key
 doesn't exist (before returning the default value) but realized that
 although small, that patch can cause future issues.
 Right now there's an inconsistency:

 >>> from collections import UserDict
 >>> class A(dict):
 ...  def __missing__(self, key):
 ...   print(key)
 ...
 >>> class B(UserDict):
 ...  def __missing__(self, key):
 ...   print("UserDict", key)
 ...
 >>> a = A()
 >>> b = B()
 >>> a.get(123)
 >>> b.get(123)
 UserDict 123
 >>> a.get(123, "abc")
 'abc'
 >>> b.get(123, "abc")
 UserDict 123

 The reason for this inconsistency is because the Mapping abc and dict
 behave differently.
 Dict's get doesn't call __getitem__ which causes the call not to route
 to __missing__.
 MutableMapping's get calls __getitem__, which UserDict implements as a
 check to __missing__ as well.

 According to the doc, the specification requires dict's __getitem__ to
 call __missing__. It doesn't say anything about get().

 Should get() call __missing__?

 If it does, things like defaultdict.get() might break. It will however
 be more consistent with dict's specification.
 If it doesn't, we expect Mapping to not care about __missing__ as it's
 only a dict thing, which will require UserDict to override get(). Dict's
 get() will need to receive a doc update as well stating __missing__ is not
 called.

 Second question is: Is __missing__ only a dict thing, or is it part of
 the Mapping ABC?

 I would expect it to be a part of the Mapping ABC, with subclasses no

[Python-Dev] Re: Dict missing consistency issues.

Hum, it looks like the UserDict.get() behavior is an accident due to the
way Mapping.get() is implemented. The comment there says

'D.get(k[,d]) -> D[k] if k in D, else d.  d defaults to None.'

but the implementation just tries D[k] and catches KeyError. The
UserDict.__contains__() implementation is explicitly code to avoid
__missing__, since the default Mapping.__contains__() implementation *also*
just calls D[k] and catches KeyError.

More indication that the UserDict.get() behavior is an accident: in
test_userdict.py there's a test for __missing__, test_missing() but it
doesn't test get() at all, only __getitem__().

How were you proposing to fix the situation? I could imagine changing
Mapping.get() to call __contains__() first. That would slow down
Mapping.get() slightly (the default implementation would end up calling
__getitem__() twice if the key is present), but I don't think that's a
grave concern (if people want it faster there are a zillion ways to do
that).

On Wed, Jan 8, 2020 at 9:49 AM Bar Harel  wrote:

> Sorry for the double post then :-)
>
> As for get(). Current implementation of UserDict returns the default value
> for get() if __missing__() raises KeyError. Which sounds reasonable to me.
> After all, you would logically expect __missing__ to be called for get() as
> you tried to get a non-existing value.
>
> If __missing__ is only a dict thing, then I'll add a note to dict's .get()
> method saying it doesn't call __missing__ for missing values.
> I can also modify UserDict to behave in a consistent manner.
>
> Does that make sense?
>
> On Wed, Jan 8, 2020 at 7:42 PM Guido van Rossum  wrote:
>
>> (Note: We received your email twice.)
>>
>> I don't think __missing__ should be called by get() -- get() already has
>> a way to deal with missing keys, and making it use two different mechanisms
>> would be weird (e.g. if get() calls __missing__, is the default value ever
>> used?).
>>
>> To answer your second question, __missing__ is only a dict thing, it is
>> not part of Mapping.
>>
>> On Wed, Jan 8, 2020 at 8:07 AM Bar Harel  wrote:
>>
>>> Hey guys,
>>>
>>> I was just about to fix dict's get() to call __missing__ if a key
>>> doesn't exist (before returning the default value) but realized that
>>> although small, that patch can cause future issues.
>>> Right now there's an inconsistency:
>>>
>>> >>> from collections import UserDict
>>> >>> class A(dict):
>>> ...  def __missing__(self, key):
>>> ...   print(key)
>>> ...
>>> >>> class B(UserDict):
>>> ...  def __missing__(self, key):
>>> ...   print("UserDict", key)
>>> ...
>>> >>> a = A()
>>> >>> b = B()
>>> >>> a.get(123)
>>> >>> b.get(123)
>>> UserDict 123
>>> >>> a.get(123, "abc")
>>> 'abc'
>>> >>> b.get(123, "abc")
>>> UserDict 123
>>>
>>> The reason for this inconsistency is because the Mapping abc and dict
>>> behave differently.
>>> Dict's get doesn't call __getitem__ which causes the call not to route
>>> to __missing__.
>>> MutableMapping's get calls __getitem__, which UserDict implements as a
>>> check to __missing__ as well.
>>>
>>> According to the doc, the specification requires dict's __getitem__ to
>>> call __missing__. It doesn't say anything about get().
>>>
>>> Should get() call __missing__?
>>>
>>> If it does, things like defaultdict.get() might break. It will however
>>> be more consistent with dict's specification.
>>> If it doesn't, we expect Mapping to not care about __missing__ as it's
>>> only a dict thing, which will require UserDict to override get(). Dict's
>>> get() will need to receive a doc update as well stating __missing__ is not
>>> called.
>>>
>>> Second question is: Is __missing__ only a dict thing, or is it part of
>>> the Mapping ABC?
>>>
>>> I would expect it to be a part of the Mapping ABC, with subclasses not
>>> having to implement it. Right now it's not.
>>>
>>> Looking forward for your inputs,
>>> Bar Harel
>>> ___
>>> Python-Dev mailing list -- python-dev@python.org
>>> To unsubscribe send an email to python-dev-le...@python.org
>>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>>> Message archived at
>>> https://mail.python.org/archives/list/python-dev@python.org/message/SDXOEMAEM6KQ3CQCJVBVRT5QNSPAVU6X/
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>>
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>> *Pronouns: he/him **(why is my pronoun here?)*
>> 
>>
>

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-d

[Python-Dev] Re: Dict missing consistency issues.

Sorry for the double post then :-)

As for get(). Current implementation of UserDict returns the default value
for get() if __missing__() raises KeyError. Which sounds reasonable to me.
After all, you would logically expect __missing__ to be called for get() as
you tried to get a non-existing value.

If __missing__ is only a dict thing, then I'll add a note to dict's .get()
method saying it doesn't call __missing__ for missing values.
I can also modify UserDict to behave in a consistent manner.

Does that make sense?

On Wed, Jan 8, 2020 at 7:42 PM Guido van Rossum  wrote:

> (Note: We received your email twice.)
>
> I don't think __missing__ should be called by get() -- get() already has a
> way to deal with missing keys, and making it use two different mechanisms
> would be weird (e.g. if get() calls __missing__, is the default value ever
> used?).
>
> To answer your second question, __missing__ is only a dict thing, it is
> not part of Mapping.
>
> On Wed, Jan 8, 2020 at 8:07 AM Bar Harel  wrote:
>
>> Hey guys,
>>
>> I was just about to fix dict's get() to call __missing__ if a key doesn't
>> exist (before returning the default value) but realized that although
>> small, that patch can cause future issues.
>> Right now there's an inconsistency:
>>
>> >>> from collections import UserDict
>> >>> class A(dict):
>> ...  def __missing__(self, key):
>> ...   print(key)
>> ...
>> >>> class B(UserDict):
>> ...  def __missing__(self, key):
>> ...   print("UserDict", key)
>> ...
>> >>> a = A()
>> >>> b = B()
>> >>> a.get(123)
>> >>> b.get(123)
>> UserDict 123
>> >>> a.get(123, "abc")
>> 'abc'
>> >>> b.get(123, "abc")
>> UserDict 123
>>
>> The reason for this inconsistency is because the Mapping abc and dict
>> behave differently.
>> Dict's get doesn't call __getitem__ which causes the call not to route to
>> __missing__.
>> MutableMapping's get calls __getitem__, which UserDict implements as a
>> check to __missing__ as well.
>>
>> According to the doc, the specification requires dict's __getitem__ to
>> call __missing__. It doesn't say anything about get().
>>
>> Should get() call __missing__?
>>
>> If it does, things like defaultdict.get() might break. It will however be
>> more consistent with dict's specification.
>> If it doesn't, we expect Mapping to not care about __missing__ as it's
>> only a dict thing, which will require UserDict to override get(). Dict's
>> get() will need to receive a doc update as well stating __missing__ is not
>> called.
>>
>> Second question is: Is __missing__ only a dict thing, or is it part of
>> the Mapping ABC?
>>
>> I would expect it to be a part of the Mapping ABC, with subclasses not
>> having to implement it. Right now it's not.
>>
>> Looking forward for your inputs,
>> Bar Harel
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/SDXOEMAEM6KQ3CQCJVBVRT5QNSPAVU6X/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/MVPF5AM2TDB65DRYA3UNXMN23IFICMOW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Dict missing consistency issues.

(Note: We received your email twice.)

I don't think __missing__ should be called by get() -- get() already has a
way to deal with missing keys, and making it use two different mechanisms
would be weird (e.g. if get() calls __missing__, is the default value ever
used?).

To answer your second question, __missing__ is only a dict thing, it is not
part of Mapping.

On Wed, Jan 8, 2020 at 8:07 AM Bar Harel  wrote:

> Hey guys,
>
> I was just about to fix dict's get() to call __missing__ if a key doesn't
> exist (before returning the default value) but realized that although
> small, that patch can cause future issues.
> Right now there's an inconsistency:
>
> >>> from collections import UserDict
> >>> class A(dict):
> ...  def __missing__(self, key):
> ...   print(key)
> ...
> >>> class B(UserDict):
> ...  def __missing__(self, key):
> ...   print("UserDict", key)
> ...
> >>> a = A()
> >>> b = B()
> >>> a.get(123)
> >>> b.get(123)
> UserDict 123
> >>> a.get(123, "abc")
> 'abc'
> >>> b.get(123, "abc")
> UserDict 123
>
> The reason for this inconsistency is because the Mapping abc and dict
> behave differently.
> Dict's get doesn't call __getitem__ which causes the call not to route to
> __missing__.
> MutableMapping's get calls __getitem__, which UserDict implements as a
> check to __missing__ as well.
>
> According to the doc, the specification requires dict's __getitem__ to
> call __missing__. It doesn't say anything about get().
>
> Should get() call __missing__?
>
> If it does, things like defaultdict.get() might break. It will however be
> more consistent with dict's specification.
> If it doesn't, we expect Mapping to not care about __missing__ as it's
> only a dict thing, which will require UserDict to override get(). Dict's
> get() will need to receive a doc update as well stating __missing__ is not
> called.
>
> Second question is: Is __missing__ only a dict thing, or is it part of the
> Mapping ABC?
>
> I would expect it to be a part of the Mapping ABC, with subclasses not
> having to implement it. Right now it's not.
>
> Looking forward for your inputs,
> Bar Harel
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/SDXOEMAEM6KQ3CQCJVBVRT5QNSPAVU6X/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TCAJTQUEWLERBFXICN5FYA3KBRUPQQJN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Why doesn't venv also install python3*-config?

2020-01-08 Thread Ivan Pozdeev via Python-Dev

On 08.01.2020 14:26, Musbur wrote:

Hello,

I'm experimenting with package development on different versions of Python in different virtualenvs. After running "make" I don't do "make
install", but rather I set up virtualenvs by running /path/to/source/python -m venv env_dir. This works for as long as I don't need to
compile extensions. Once I do that I'm running into trouble because there is no python3-config binary in the venv, so it uses the "system"
python3-config which of course returns results for the /usr(/local)/ tree.

This seems to go against the idea of an encapsulated and self-contained environment. And the venv created straight from the
"not-installed" source tree works so well that having a venv/bin/python3-config whose output points into the source tree seems a logical
step. Is this an omission or is there a rationale for not doing it?

AFAIK for _inside_ a virtual environment, the convention is `python',
`python-`, `pip' etc. -- unversioned executables.

Versioned executables are the convention for UNIX _outside_ a virtual
environment.

This is codified in https://www.python.org/dev/peps/pep-0394/.

Of course I can "properly" install different Python versions by using different "configure --prefix" directories. But I like the venv so
much in general that this rubs me the wrong way.

Thanks!
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/KMG3USMQLQNI6AEX6UUS2WTNFIIIS2XY/
Code of Conduct: http://python.org/psf/codeofconduct/

--
Regards,
Ivan
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/N3E26CJGWBI4IJ4VWNXIRAG3CCVYCFPF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Why doesn't venv also install python3*-config?

Hi,

I dislike python-config for multiple reasons

You may get the wrong information if you pick the wrong python-config script :-(

IMHO we should add a new module (problem: how should it be called?
pyconfig?) which could be used using "python3 -m xxx ...". There is a
similar discussion between "pip ..." and "python3 -m pip ..."
commands: I understand that "python3 -m pip ..." is more reliable to
know which Python will be picked, especially in a virtual environment.

There are two implementations: one in Python used on macOS, one in
shell used on other platforms.

Moreover, if it's implemented in the stdlib, it can be implemented in
Python and it should be easier to maintain than a shell script. For
example, python-config has no unit test...

Victor

Le mer. 8 janv. 2020 à 17:00, Musbur  a écrit :
>
> Hello,
>
> I'm experimenting with package development on different versions of
> Python in different virtualenvs. After running "make" I don't do "make
> install", but rather I set up virtualenvs by running
> /path/to/source/python -m venv env_dir. This works for as long as I
> don't need to compile extensions. Once I do that I'm running into
> trouble because there is no python3-config binary in the venv, so it
> uses the "system" python3-config which of course returns results for the
> /usr(/local)/ tree.
>
> This seems to go against the idea of an encapsulated and self-contained
> environment. And the venv created straight from the "not-installed"
> source tree works so well that having a venv/bin/python3-config whose
> output points into the source tree seems a logical step. Is this an
> omission or is there a rationale for not doing it?
>
> Of course I can "properly" install different Python versions by using
> different "configure --prefix" directories. But I like the venv so much
> in general that this rubs me the wrong way.
>
> Thanks!
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/KMG3USMQLQNI6AEX6UUS2WTNFIIIS2XY/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NTNC4ZXNR6XFJPX5HK6VEFCZYEPP47IV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Should we pass tstate in the VECTORCALL calling convention before making it public?

2020-01-08 Thread Steve Dower


On 08Jan2020 0746, Victor Stinner wrote:

The question is now if we should "propagate" tstate to function calls
in the latest VECTORCALL calling convention (which is currently
private). 
+1, for all the same reasons I'm +1 on passing it to C functions 
explicitly. (In short, embedding can be very painful when you have to 
manipulate ambient thread state to make Python code work, and this is a 
good step towards getting rid of that requirement.)


Cheers,
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NK2AMXQXYRTG2JFBNXOP4XEUEGSVORFK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Dict missing consistency issues.

Hey guys,

I was just about to fix dict's get() to call __missing__ if a key doesn't exist 
(before returning the default value) but realized that although small, that 
patch can cause future issues.
Right now there's an inconsistency:

>>> from collections import UserDict
>>> class A(dict):
...  def __missing__(self, key):
...   print(key)
... 
>>> class B(UserDict):
...  def __missing__(self, key):
...   print("UserDict", key)
... 
>>> a = A()
>>> b = B()
>>> a.get(123)
>>> b.get(123)
UserDict 123
>>> a.get(123, "abc")
'abc'
>>> b.get(123, "abc")
UserDict 123

The reason for this inconsistency is because the Mapping abc and dict behave 
differently.
Dict's get doesn't call __getitem__ which causes the call not to route to 
__missing__.
MutableMapping's get calls __getitem__, which UserDict implements as a check to 
__missing__ as well.

According to the doc, the specification requires dict's __getitem__ to call 
__missing__. It doesn't say anything about get().

Should get() call __missing__?

If it does, things like defaultdict.get() might break. It will however be more 
consistent with dict's specification.
If it doesn't, we expect Mapping to not care about __missing__ as it's only a 
dict thing, which will require UserDict to override get(). Dict's get() will 
need to receive a doc update as well stating __missing__ is not called.

Second question is: Is __missing__ only a dict thing, or is it part of the 
Mapping ABC?

I would expect it to be a part of the Mapping ABC, with subclasses not having 
to implement it. Right now it's not.

Looking forward for your inputs,
Bar Harel
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SDXOEMAEM6KQ3CQCJVBVRT5QNSPAVU6X/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Why doesn't venv also install python3*-config?

2020-01-08 Thread Musbur


Hello,

I'm experimenting with package development on different versions of 
Python in different virtualenvs. After running "make" I don't do "make 
install", but rather I set up virtualenvs by running 
/path/to/source/python -m venv env_dir. This works for as long as I 
don't need to compile extensions. Once I do that I'm running into 
trouble because there is no python3-config binary in the venv, so it 
uses the "system" python3-config which of course returns results for the 
/usr(/local)/ tree.


This seems to go against the idea of an encapsulated and self-contained 
environment. And the venv created straight from the "not-installed" 
source tree works so well that having a venv/bin/python3-config whose 
output points into the source tree seems a logical step. Is this an 
omission or is there a rationale for not doing it?


Of course I can "properly" install different Python versions by using 
different "configure --prefix" directories. But I like the venv so much 
in general that this rubs me the wrong way.


Thanks!
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KMG3USMQLQNI6AEX6UUS2WTNFIIIS2XY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Dict missing consistency issues

Hey guys,

I was just about to fix dict's get() to call __missing__ if a key doesn't
exist (before returning the default value) but realized that although
small, that patch can cause future issues.
Right now there's an inconsistency:

>>> from collections import UserDict
>>> class A(dict):
...  def __missing__(self, key):
...   print(key)
...
>>> class B(UserDict):
...  def __missing__(self, key):
...   print("UserDict", key)
...
>>> a = A()
>>> b = B()
>>> a.get(123)
>>> b.get(123)
UserDict 123
>>> a.get(123, "abc")
'abc'
>>> b.get(123, "abc")
UserDict 123

The reason for this inconsistency is because the Mapping abc and dict
behave differently.
Dict's get doesn't call __getitem__ which causes the call not to route to
__missing__.
MutableMapping's get calls __getitem__, which UserDict implements as a
check to __missing__ as well.

According to the doc, the specification requires dict's __getitem__ to call
__missing__. It doesn't say anything about get().

Should get() call __missing__?

If it does, things like defaultdict.get() might break. It will however be
more consistent with dict's specification.
If it doesn't, we expect Mapping to not care about __missing__ as it's only
a dict thing, which will require UserDict to override get(). Dict's get()
will need to receive a doc update as well stating __missing__ is not called.

Second question is: Is __missing__ only a dict thing, or is it part of the
Mapping ABC?

I would expect it to be a part of the Mapping ABC, with subclasses not
having to implement it. Right now it's not.

Looking forward for your inputs,
Bar Harel
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JMNLIZJ6XBDCDR6PZWUN33PYOWO4ZFLP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Should we pass tstate in the VECTORCALL calling convention before making it public?

Hi,

I started to modify Python internals to pass explicitly the Python
thread state ("tstate") to internal C a functions:
https://vstinner.github.io/cpython-pass-tstate.html

Until subinterpreters will be fully implemented, it's unclear if
getting the current Python thread state using _PyThreadState_GET()
macro (which uses an atomic read) will remain efficient. For example,
right now, the "GIL state" API doesn't support subinterpreters: fixing
this may require to add a lock somewhere which may make
_PyThreadState_GET() less efficient. Sorry, I don't have numbers,
since I didn't experiment to implement these changes yet: I was
blocked by other issues. We can only guess at this point.

To me, it sounds like a good practice to pass the current Python
thread state to internal C functions. It seems like most core
developers agreed with that in my previous python-dev thread "Pass the
Python thread state to internal C functions":

https://mail.python.org/archives/list/python-dev@python.org/thread/PQBGECVGVYFTVDLBYURLCXA3T7IPEHHO/#Q4IPXMQIM5YRLZLHADUGSUT4ZLXQ6MYY

The question is now if we should "propagate" tstate to function calls
in the latest VECTORCALL calling convention (which is currently
private). Petr Viktorin plans to make VECTORCALL APIs public in Python
3.9, as planned in the PEP 590:
https://bugs.python.org/issue39245

I added explicitly Stefan Behnel in copy, since Cython should be
directly impacted by such change. Cython is the kind of project which
may benefit of having tstate accessible directly.

I started to move more and more things from "globals" to "per
interpreter". For example, the garbage collector is now "per
interpreter" (lives in PyThreadState). Small integer singletons are
now also "per singleton": int("1") are now different objects in each
interpreter, whereas they were shared previously. Later, even "None"
singleton (and all other singletons) should be made "per interpreter".
Getting a "per interpreter" object requires to state from the Python
thread state: call _PyThreadState_GET(). Avoiding _PyThreadState_GET()
calls reduces any risk of making Python slower with incoming
subinterpreter changes.

For the long term, the goal is that each subinterpreter has its own
isolated world: no Python object should be shared, no state should be
shared. The intent is to avoid any need of locking, to maximize
performance when running interpreters in parallel. Obviously, each
interpreter will have its own private GIL ;-) Py_INCREF() and
Py_DECREF() would require slow atomic operations if Python objects are
shared. If objects are not shared between interpreters, Py_INCREF()
and Py_DECREF() can remain as fast as they are today. Any additional
locking may kill performance.

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PIXJAJPWKDGHSQD65VOO2B7FDLU2QLHH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Should set objects maintain insertion order too?