Re: [Python-Dev] Comment on PEP 562 (Module __getattr__ and __dir__)

2017-11-20 Thread Serhiy Storchaka 

20.11.17 03:02, Guido van Rossum пише:
Serhiy's definition sounds recursive (defining __getattr__ to define the 
behavior of __getattr__) but Mark's suggestion makes his intention 
unclear since the error message is still the same.


It is recursive only when the '__dict__' attribute is not defined. I 
assumed that it is defined for simplicity. And if isn't defined 
hasattr(self, '__dict__') will cause a recursion too.


In any case the real C code handles this more carefully and effectively.

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Comments on PEP 560 (Core support for typing module and generic types)

2017-11-20 Thread Mark Shannon 



On 19/11/17 22:36, Ivan Levkivskyi wrote:
On 19 November 2017 at 21:06, Mark Shannon > wrote:


By far and away the largest change in PEP 560 is the change to the
behaviour of object.__getitem__. This is not mentioned in the PEP at
all, but is explicit in the draft implementation.
The implementation could implement `type.__getitem__` instead of
changing `object.__getitem__`, but that is still a major change to
the language.


Except that there is no such thing as object._getitem__. Probably you 
mean PyObject_GetItem (which is just what is done by BINARY_SUBSCR opcode).


Yes, I should have taken more time to look at the code. I thought you 
were implementing `object.__getitem__`.
In general, Python implements its operators as a simple redirection to a 
special method, with the exception of binary operators which are 
necessarily more complex.


f(...) ->  type(f).__call__(f, ...)
o.a -> type(o).__getattribute__(o, "a")
o[i] -> type(o).__getitem__(o, i)

Which is why I don't like the additional complexity you are adding to 
the dispatching. If we really must have `__class_getitem__` (and I don't 
think that we do) then implementing `type.__getitem__` is a much less 
intrusive way to do it.


In fact, I initially implemented type.__getitem__, but I didn't like it 
for various reasons.


Could you elaborate?



I don't think that any of the above are changes to the language. These 
are rather implementation details. The only unusual thing is that while 
dunders are
searched on class, __class_getitem__ is searched on the object (class 
object in this case) itself. But this is clearly explained in the PEP.


In fact, the addition of `__mro_entries__` makes `__class_getitem__`
unnecessary.


But how would you implement this:

class C(Generic[T]):
 ...

C[int]  # This should work


The issue of type-hinting container classes is a tricky one. The 
definition is defining both the implementation class and the interface 
type. We want the implementation and interface to be distinct. However, 
we want to avoid needless repetition.


In the example you gave, `C` is a class definition that is intended to 
be used as a generic container. In my mind the cleanest way to do this 
is with a class decorator. Something like:


@Generic[T]
class C: ...

or

@implements(Generic[T])
class C: ...

C would then be a type not a class, as the decorator is free to return a 
non-class object.


It allows the implementation and interface to be distinct:

@implements(Sequence[T])
class MySeq(list): ...

@implements(Set[Node])
class SmallNodeSet(list): ...
# For small sets a list is more efficient than a set.

but avoid repetition for the more common case:

class IntStack(List[int]): ...

Given the power and flexibility of the built-in data structures, 
defining custom containers is relatively rare. I'm not saying that it 
should not be considered, but a few minor hurdles are acceptable to keep 
the rest of the language (including more common uses of type-hints) clean.




The name `__mro_entries__` suggests that this method is solely
related method resolution order, but it is really about providing an
instance of `type` where one is expected. This is analogous to
`__int__`, `__float__` and `__index__` which provide an int, float
and int respectively.
This rather suggests (to me at least) the name `__type__` instead of
`__mro_entries__`


This was already discussed during months, and in particular the name 
__type__ was not liked by ... you 


Ha, you have a better memory than I :) I won't make any more naming 
suggestions.
What I should have said is that the name should reflect what it does, 
not the initial reason for including it.



https://github.com/python/typing/issues/432#issuecomment-304070379
So I would propose to stop bikesheding this (also Guido seems to like 
the currently proposed name).


Should `isinstance` and `issubclass` call `__mro_entries__` before
raising an error if the second argument is not a class?
In other words, if `List` implements `__mro_entries__` to return
`list` then should `issubclass(x, List)` act like `issubclass(x, list)`?
(IMO, it shouldn't) The reasoning behind this decision should be
made explicit in the PEP.


I think this is orthogonal to the PEP. There are many situations where a 
class is expected,
and IMO it is clear that all that are not mentioned in the PEP stay 
unchanged.


Indeed, but you do mention issubclass in the PEP. I think a few extra 
words of explanation would be helpful.


Cheers,
Mark.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Make the stable API-ABI usable

2017-11-20 Thread Hrvoje Niksic 

On 11/19/2017 12:50 PM, Serhiy Storchaka wrote:

But if PyTuple_GET_ITEM() is used for getting a reference to a C array
of items it can't be replaced with PyTuple_GetItem(). And actually there
is no replacement for this case in the limited API.

  PyObject **items = &PyTuple_GET_ITEM(tuple, 0);


That use case might be better covered with a new function, e.g. 
PyTuple_GetStorage, which the PyObject ** pointing to the first element 
of the internal array.


This function would serve two purposes:

* provide the performance benefits of PyTuple_GET_ITEM in tight loops, 
but without the drawback of exposing the PyTuple layout to the code that 
invokes the macro;


* allow invocation of APIs that expect a pointer to contiguous storage, 
such as STL algorithms that expect random access iterators.


Something similar is already available as PySequence_Fast_ITEMS, except 
that one is again a macro, and is tied to PySequence_FAST API, which may 
not be appropriate for the kind of performance-critical code where 
PyTuple_GET_ITEM tends to be used. (That kind of code is designed to 
deal specifically with lists or tuples and doesn't benefit from implicit 
conversion of arbitrary sequences to a temporary list; that conversion 
would only serve to mask bugs.)

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python initialization and embedded Python

2017-11-20 Thread Victor Stinner 
To not lost track of the issue, I created this issue on the bpo:
https://bugs.python.org/issue32086

Victor

2017-11-20 7:54 GMT+01:00 Nick Coghlan :
> On 19 November 2017 at 18:52, Victor Stinner  wrote:
>> Maybe we can find a compromise: revert the change on memory allocators. They
>> are too special to require to call PyRuntime_Init().
>>
>> Currently, you cannot call PyMem_SetAllocators() before PyRuntime_Init().
>
> At least the raw allocators, anyway - that way, the developer facing
> documentation/comments can just say that the raw allocators can't have
> any prerequisites that aren't shared by regular
> malloc/calloc/realloc/free calls.
>
> If that's enough to get Py_DecodeLocale working again prior to
> _PyRuntime_Init(), then I'd suggest officially adding that to the
> "must work prior to Py_Initialize" list, otherwise we can re-examine
> it based on whatever's still broken after reverting the raw allocator
> changes.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Show DeprecationWarning in debug mode?

2017-11-20 Thread Victor Stinner 
2017-11-18 18:13 GMT+01:00 Brett Cannon :
> +1 from me as well.

Ok, I created https://bugs.python.org/issue32088 and
https://github.com/python/cpython/pull/4474 to implement the proposed
change.

Victor
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python initialization and embedded Python

2017-11-20 Thread Eric Snow 
On Nov 18, 2017 19:20, "Nick Coghlan"  wrote:


OK, in that case I think the answer to Victor's question is:

1. Breaking calling Py_DecodeLocale() before calling Py_Initialize()
is a compatibility break with the API implied by our own usage
examples, and we'll need to revert the breakage for 3.7,


+1

The break was certainly unintentional. :/  Fortunately, Py_DecodeLocale()
should be the only "Process-wide parameter" needing repair.  I suppose,
PyMem_RawMalloc() and PyMem_RawFree() *could* be considered too, but my
understanding is that they aren't really intended for direct use
(especially pre-init).

and ensure at
least one release's worth of DeprecationWarning before requiring
either the use of an alternative API (where the caller controls the
memory management), or else a new lower level pre-initialization API
(i.e. making `PyRuntime_Initialize` a public API)


There shouldn't be a need to deprecate anything, right?  We just need to
restore the pre-init behavior of Py_DecodeLocale.

2. We should provide a consolidated list of these functions in the C
API initialization docs


+1

PyMem_Raw*() do not belong in that group, right?  Again, my understanding
is that they aren't intended for direct third-party use (are they even a
part of the C-API?), and particularly pre-init.  That Py_DecodeLocale() can
use PyMem_RawMalloc() pre-init is an implementation detail.

3. We should add more test cases to _testembed.c that ensure they all
work correctly prior to Py_Initialize (some of them are already tested
there, but definitely not all of them)


+1

-eric
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python initialization and embedded Python

2017-11-20 Thread Victor Stinner 
2017-11-20 16:31 GMT+01:00 Eric Snow :
> That Py_DecodeLocale() can use PyMem_RawMalloc() pre-init is an 
> implementation detail.

Py_DecodeLocale() uses PyMem_RawMalloc(), and so its result must be
freed by PyMem_RawFree(). It's part the documentation.

I'm not sure that I understood correctly. Do you agree to move "PyMem"
globals back to Objects/obmalloc.c? (to allow to call
PyMem_RawMalloc() before Py_Initialize())

Victor
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Show DeprecationWarning in debug mode?

2017-11-20 Thread Lukasz Langa 
Merged. Thanks! ✨ 🍰 ✨

- Ł

> On Nov 20, 2017, at 7:01 AM, Victor Stinner  wrote:
> 
> 2017-11-18 18:13 GMT+01:00 Brett Cannon :
>> +1 from me as well.
> 
> Ok, I created https://bugs.python.org/issue32088 and
> https://github.com/python/cpython/pull/4474 to implement the proposed
> change.
> 
> Victor
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/lukasz%40langa.pl



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Comments on PEP 563 (Postponed Evaluation of Annotations)

2017-11-20 Thread Lukasz Langa 
I agree with you. The special handling of outermost strings vs. strings 
embedded inside annotations bugged me a lot. Now you convinced me that this 
functionality should be moved to `get_type_hints()` and the __future__ import 
shouldn't try to special-case this one instance, while leaving others as is.

I will be amending the PEP accordingly.

- Ł

> On Nov 19, 2017, at 10:56 AM, Mark Shannon  wrote:
> 
> Hi,
> 
> Overall I am strongly in favour of this PEP. It pretty much cures all the 
> ongoing pain of using PEP 3017 annotations for type hints.
> 
> There is one thing I don't like however, and that is treating strings as if 
> the quotes weren't there.
> While this seems like a superficial simplification to make transition easier, 
> it introduces inconsistency and will ultimately make both implementing and 
> using type hints harder.
> 
> Having the treatment of strings depend on their depth in the AST seems 
> confusing and unnecessary:
> "List[int]" becomes 'List[int]' # quotes removed
> but
> List["int"] becomes 'List["int"]' # quoted retained
> 
> Also,
> 
> T = "My unparseable annotation"
> def f()->T: pass
> 
> would remain legal, but
> 
> def f()->"My unparseable annotation"
> 
> would become illegal.
> 
> The change in behaviour between the above two code snippets is already 
> confusing enough without making one of them a SyntaxError.
> 
> Using annotations for purposes other than type hinting is legal and has been 
> for quite a while.
> Also, PEP 484 type-hints are not the only type system in the Python 
> ecosystem. Cython has a long history of using static type hints.
> 
> For tools other than MyPy, the inconsistent quoting is onerous and will 
> require double-quoting to prevent a parse error.
> For example
> 
> def foo()->"unsigned int": ...
> 
> will become illegal and require the cumbersome
> 
> def foo()->'"unsigned int"': ...
> 
> Cheers,
> Mark.
> 
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/lukasz%40langa.pl



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 563: Postponed Evaluation of Annotations

2017-11-20 Thread Lukasz Langa 
See my response to Mark's e-mail. I agree that special-casing outermost strings 
is not generic enough of a solution and will be removing this special handling 
from the PEP and the implementation.

- Ł

> On Nov 19, 2017, at 2:38 PM, Koos Zevenhoven  wrote:
> 
> On Mon, Nov 13, 2017 at 11:59 PM, Brett Cannon  > wrote:
> ​[..]​
> On Sun, Nov 12, 2017, 10:22 Koos Zevenhoven,  > wrote:​​
> 
> There's two thing I don't understand here:
> 
> * What does it mean to preserve the string verbatim? No matter how I read it, 
> I can't tell if it's with quotes or without.
> 
> Maybe I'm missing some context.
> 
> I believe the string passes through unchanged (i.e. no quotes). Think of the 
> PEP as simply turning all non-string annotations into string ones.
> 
> 
> ​Ok, maybe that was just wishful thinking on my part ;-).
> 
> More info in the other threads, for example:
> 
> https://mail.python.org/pipermail/python-dev/2017-November/150642.html 
> 
> https://mail.python.org/pipermail/python-dev/2017-November/150637.html 
> 
> 
> -- Koos
> 
> 
> --
> + Koos Zevenhoven + http://twitter.com/k7hoven  +
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/lukasz%40langa.pl



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Show DeprecationWarning in debug mode?

2017-11-20 Thread Victor Stinner 
Oh, that was quick. Thanks!

I counted Serhiy, Barry, Nick, you and me in favor of the change, and
nobody against the code. So well, it's ok to merge it :-)

Victor

2017-11-20 18:49 GMT+01:00 Lukasz Langa :
> Merged. Thanks! ✨ 🍰 ✨
>
> - Ł
>
>> On Nov 20, 2017, at 7:01 AM, Victor Stinner  wrote:
>>
>> 2017-11-18 18:13 GMT+01:00 Brett Cannon :
>>> +1 from me as well.
>>
>> Ok, I created https://bugs.python.org/issue32088 and
>> https://github.com/python/cpython/pull/4474 to implement the proposed
>> change.
>>
>> Victor
>> ___
>> Python-Dev mailing list
>> [email protected]
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: 
>> https://mail.python.org/mailman/options/python-dev/lukasz%40langa.pl
>
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Comments on PEP 563 (Postponed Evaluation of Annotations)

2017-11-20 Thread Koos Zevenhoven 
On Mon, Nov 20, 2017 at 7:58 PM, Lukasz Langa  wrote:

> I agree with you. The special handling of outermost strings vs. strings
> embedded inside annotations bugged me a lot. Now you convinced me that this
> functionality should be moved to `get_type_hints()` and the __future__
> import shouldn't try to special-case this one instance, while leaving
> others as is.
>
> ​
> 


That's better. I don't necessarily care if there will be a warning when a
string is given as annotation, but if the idea is to simplify things for
the future and get rid of strings to represent types, then this would be a
good moment to gently "enforce" it.

––Koos


-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Comments on PEP 563 (Postponed Evaluation of Annotations)

2017-11-20 Thread Guido van Rossum 
OK, this is fine. It won't affect mypy (which will continue to treat string
literals the same as before) but it will require an update to typing.py --
hope you are working on that with Ivan.

On Mon, Nov 20, 2017 at 9:58 AM, Lukasz Langa  wrote:

> I agree with you. The special handling of outermost strings vs. strings
> embedded inside annotations bugged me a lot. Now you convinced me that this
> functionality should be moved to `get_type_hints()` and the __future__
> import shouldn't try to special-case this one instance, while leaving
> others as is.
>
> I will be amending the PEP accordingly.
>
> - Ł
>
> > On Nov 19, 2017, at 10:56 AM, Mark Shannon  wrote:
> >
> > Hi,
> >
> > Overall I am strongly in favour of this PEP. It pretty much cures all
> the ongoing pain of using PEP 3017 annotations for type hints.
> >
> > There is one thing I don't like however, and that is treating strings as
> if the quotes weren't there.
> > While this seems like a superficial simplification to make transition
> easier, it introduces inconsistency and will ultimately make both
> implementing and using type hints harder.
> >
> > Having the treatment of strings depend on their depth in the AST seems
> confusing and unnecessary:
> > "List[int]" becomes 'List[int]' # quotes removed
> > but
> > List["int"] becomes 'List["int"]' # quoted retained
> >
> > Also,
> >
> > T = "My unparseable annotation"
> > def f()->T: pass
> >
> > would remain legal, but
> >
> > def f()->"My unparseable annotation"
> >
> > would become illegal.
> >
> > The change in behaviour between the above two code snippets is already
> confusing enough without making one of them a SyntaxError.
> >
> > Using annotations for purposes other than type hinting is legal and has
> been for quite a while.
> > Also, PEP 484 type-hints are not the only type system in the Python
> ecosystem. Cython has a long history of using static type hints.
> >
> > For tools other than MyPy, the inconsistent quoting is onerous and will
> require double-quoting to prevent a parse error.
> > For example
> >
> > def foo()->"unsigned int": ...
> >
> > will become illegal and require the cumbersome
> >
> > def foo()->'"unsigned int"': ...
> >
> > Cheers,
> > Mark.
> >
> > ___
> > Python-Dev mailing list
> > [email protected]
> > https://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> lukasz%40langa.pl
>
>
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Comment on PEP 562 (Module __getattr__ and __dir__)

2017-11-20 Thread Guido van Rossum 
Yeah, I don't think there's an action item here except *maybe* changes to
the wording of the PEP. Ivan?

On Mon, Nov 20, 2017 at 12:33 AM, Serhiy Storchaka 
wrote:

> 20.11.17 03:02, Guido van Rossum пише:
>
>> Serhiy's definition sounds recursive (defining __getattr__ to define the
>> behavior of __getattr__) but Mark's suggestion makes his intention unclear
>> since the error message is still the same.
>>
>
> It is recursive only when the '__dict__' attribute is not defined. I
> assumed that it is defined for simplicity. And if isn't defined
> hasattr(self, '__dict__') will cause a recursion too.
>
> In any case the real C code handles this more carefully and effectively.
>
>
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%
> 40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Comment on PEP 562 (Module __getattr__ and __dir__)

2017-11-20 Thread Ivan Levkivskyi 
On 20 November 2017 at 20:51, Guido van Rossum  wrote:

> Yeah, I don't think there's an action item here except *maybe* changes to
> the wording of the PEP. Ivan?
>

Yes, I will make a small PR with a more detailed description of how
__getattr__ works.

--
Ivan
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Comments on PEP 560 (Core support for typing module and generic types)

2017-11-20 Thread Ivan Levkivskyi 
On 20 November 2017 at 10:22, Mark Shannon  wrote:

> On 19/11/17 22:36, Ivan Levkivskyi wrote:
>
>> Except that there is no such thing as object._getitem__. Probably you
>> mean PyObject_GetItem (which is just what is done by BINARY_SUBSCR opcode).
>>
> In fact, I initially implemented type.__getitem__, but I didn't like it
>> for various reasons.
>
>
> Could you elaborate?
>

I didn't like that although type.__getitem__ would exist, it would almost
always raise, which can be confusing. Also dir(type) is already long,
I don't want to make it even longer. Maybe there were other reasons that I
forgot.

Anyway, I propose to stop here, since this is not about the PEP, this is
about implementation.
This is a topic of a separate discussion.


> Given the power and flexibility of the built-in data structures, defining
> custom containers is relatively rare. I'm not saying that it should not be
> considered,

but a few minor hurdles are acceptable to keep the rest of the language
> (including more common uses of type-hints) clean.
>

This essentially means changing decisions already made in PEP 484 and not a
topic of this PEP.
Also,

@Generic[T]
class C:
...

is currently a syntax error (only names and function calls are allowed in a
decorator).
Finally, it is too late to change how generics are declared, since it will
break
existing code.

Should `isinstance` and `issubclass` call `__mro_entries__` before
>> raising an error if the second argument is not a class?
>> In other words, if `List` implements `__mro_entries__` to return
>> `list` then should `issubclass(x, List)` act like `issubclass(x,
>> list)`?
>> (IMO, it shouldn't) The reasoning behind this decision should be
>> made explicit in the PEP.
>>
>>
>> I think this is orthogonal to the PEP. There are many situations where a
>> class is expected,
>> and IMO it is clear that all that are not mentioned in the PEP stay
>> unchanged.
>>
>
> Indeed, but you do mention issubclass in the PEP. I think a few extra
> words of explanation would be helpful.
>

OK, I will add a comment to the PEP.

--
Ivan
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python initialization and embedded Python

2017-11-20 Thread Eric Snow 
On Mon, Nov 20, 2017 at 8:43 AM, Victor Stinner
 wrote:
> 2017-11-20 16:31 GMT+01:00 Eric Snow :
>> That Py_DecodeLocale() can use PyMem_RawMalloc() pre-init is an 
>> implementation detail.
>
> Py_DecodeLocale() uses PyMem_RawMalloc(), and so its result must be
> freed by PyMem_RawFree(). It's part the documentation.

Ah, I'd missed that.  Thanks for pointing it out.

>
> I'm not sure that I understood correctly. Do you agree to move "PyMem"
> globals back to Objects/obmalloc.c? (to allow to call
> PyMem_RawMalloc() before Py_Initialize())

I'm okay with that if we can't find another way.  However, shouldn't
we be able to statically initialize the raw allocator in _PyRuntime,
much as we were doing before in obmalloc.c?  I have a rough PR up:

https://github.com/python/cpython/pull/4481

Also, I opened https://bugs.python.org/issue32096 for the regression.
Thanks for bringing it up.

-eric
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python initialization and embedded Python

2017-11-20 Thread Victor Stinner 
2017-11-20 22:35 GMT+01:00 Eric Snow :
> I'm okay with that if we can't find another way.  However, shouldn't
> we be able to statically initialize the raw allocator in _PyRuntime,
> much as we were doing before in obmalloc.c?  I have a rough PR up:
>
> https://github.com/python/cpython/pull/4481
>
> Also, I opened https://bugs.python.org/issue32096 for the regression.
> Thanks for bringing it up.

To statically initialize PyMemAllocatorEx fields, you need to export a
lot of allocator functions. I would prefer to not do that.

static void* _PyMem_DebugRawMalloc(void *ctx, size_t size);
static void* _PyMem_DebugRawCalloc(void *ctx, size_t nelem, size_t elsize);
static void* _PyMem_DebugRawRealloc(void *ctx, void *ptr, size_t size);
static void _PyMem_DebugRawFree(void *ctx, void *ptr);

static void* _PyMem_DebugMalloc(void *ctx, size_t size);
static void* _PyMem_DebugCalloc(void *ctx, size_t nelem, size_t elsize);
static void* _PyMem_DebugRealloc(void *ctx, void *ptr, size_t size);
static void _PyMem_DebugFree(void *ctx, void *p);

static void* _PyObject_Malloc(void *ctx, size_t size);
static void* _PyObject_Calloc(void *ctx, size_t nelem, size_t elsize);
static void _PyObject_Free(void *ctx, void *p);
static void* _PyObject_Realloc(void *ctx, void *ptr, size_t size);

The rules to choose the allocator to each domain are also complex
depending if pymalloc is enabled, debug hooks are enabled by default,
etc. The memory allocator is also linked to _PyMem_Debug which is not
currently in Include/internals/ but Objects/obmalloc.c.

I understand that moving global variables to _PyRuntime helps to
clarify how these variables are initialized and then finalized, but
memory allocators are a complex corner case.

main(), Py_Main() and _PyRuntime_Initialize() now have to change
temporary the allocators to make sure that their initialization and
finalization use the same allocator.

I prefer to revert the change on memory allocators, and retry later to
fix it, once other initializations issues are fixed ;-)

Victor
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Comments on PEP 560 (Core support for typing module and generic types)

2017-11-20 Thread Guido van Rossum 
Mark, with that comment now added to the PEP, are you satisfied?
https://github.com/python/peps/pull/479

On Mon, Nov 20, 2017 at 12:32 PM, Ivan Levkivskyi 
wrote:

> On 20 November 2017 at 10:22, Mark Shannon  wrote:
>
>> On 19/11/17 22:36, Ivan Levkivskyi wrote:
>>
>>> Except that there is no such thing as object._getitem__. Probably you
>>> mean PyObject_GetItem (which is just what is done by BINARY_SUBSCR opcode).
>>>
>> In fact, I initially implemented type.__getitem__, but I didn't like it
>>> for various reasons.
>>
>>
>> Could you elaborate?
>>
>
> I didn't like that although type.__getitem__ would exist, it would almost
> always raise, which can be confusing. Also dir(type) is already long,
> I don't want to make it even longer. Maybe there were other reasons that I
> forgot.
>
> Anyway, I propose to stop here, since this is not about the PEP, this is
> about implementation.
> This is a topic of a separate discussion.
>
>
>> Given the power and flexibility of the built-in data structures, defining
>> custom containers is relatively rare. I'm not saying that it should not be
>> considered,
>
> but a few minor hurdles are acceptable to keep the rest of the language
>> (including more common uses of type-hints) clean.
>>
>
> This essentially means changing decisions already made in PEP 484 and not
> a topic of this PEP.
> Also,
>
> @Generic[T]
> class C:
> ...
>
> is currently a syntax error (only names and function calls are allowed in
> a decorator).
> Finally, it is too late to change how generics are declared, since it will
> break
> existing code.
>
> Should `isinstance` and `issubclass` call `__mro_entries__` before
>>> raising an error if the second argument is not a class?
>>> In other words, if `List` implements `__mro_entries__` to return
>>> `list` then should `issubclass(x, List)` act like `issubclass(x,
>>> list)`?
>>> (IMO, it shouldn't) The reasoning behind this decision should be
>>> made explicit in the PEP.
>>>
>>>
>>> I think this is orthogonal to the PEP. There are many situations where a
>>> class is expected,
>>> and IMO it is clear that all that are not mentioned in the PEP stay
>>> unchanged.
>>>
>>
>> Indeed, but you do mention issubclass in the PEP. I think a few extra
>> words of explanation would be helpful.
>>
>
> OK, I will add a comment to the PEP.
>
> --
> Ivan
>
>
>
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python initialization and embedded Python

2017-11-20 Thread Nick Coghlan 
On 21 November 2017 at 01:31, Eric Snow  wrote:
> On Nov 18, 2017 19:20, "Nick Coghlan"  wrote:
>
>
> OK, in that case I think the answer to Victor's question is:
>
> 1. Breaking calling Py_DecodeLocale() before calling Py_Initialize()
> is a compatibility break with the API implied by our own usage
> examples, and we'll need to revert the breakage for 3.7,
>
>
> +1
>
> The break was certainly unintentional. :/  Fortunately, Py_DecodeLocale()
> should be the only "Process-wide parameter" needing repair.  I suppose,
> PyMem_RawMalloc() and PyMem_RawFree() *could* be considered too, but my
> understanding is that they aren't really intended for direct use (especially
> pre-init).

PyMem_RawFree will need to continue working pre-initialize as well,
since it's the specified cleanup function for Py_DecodeLocale.

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com