Re: [Python-Dev] PEP 544: Protocols - second round

2017-06-23 Thread markus.wissinger 
What I try to do is a new workflow:
- 'declare' types of method parameters by usage in a unit test. Therefore 
ensure the interface is obeyed inside.- publish the used types via hints to 
help callers of the method. -Do express all hints in test code. Do not insert 
any hints manually. Any type used in any test will result in a hint (with some 
clever exceptions). This requires some discipline while writing tests, but 
worked quite well already.
I think I just realized pep544 protocols will not break that scheme, just make 
it a little more difficult to handle. I can create intermediate signatures that 
still contain superfluous subtypes and remove those only in the end when 
protocol hints are sure to exist.
Thanks.Markus 


 Ursprüngliche Nachricht Von: Stefan Richthofer 
<stefan.richtho...@gmx.de> Datum: 23.06.17  01:08  (GMT+01:00) An: Ivan 
Levkivskyi <levkivs...@gmail.com> Cc: Markus Wissinger 
<markus.wissin...@gmail.com>, Python-Dev <python-dev@python.org> Betreff: Re: 
[Python-Dev] PEP 544: Protocols - second round 

>> I am currently exploring a type hint generator that produces hints out of 
>> types used in unit tests.

 

Note that pytypes (https://github.com/Stewori/pytypes) already supports this. 
It can dump PEP 484 style stubfiles from runtime type observations (made via 
decorators or profiler hook).

 

>> isinstance(obj, T) and issubclass(Cls, T) already fail if T is a subscripted 
>> generic like List[int], so that again nothing new here. To check runtime 
>> subtyping with such types one can write a third party introspection tool 
>> based on typing_inspect package on PyPI

 

There are public API functions in pytypes for isinstance and issubclass with 
support for most PEP 484 types, see 
https://github.com/Stewori/pytypes#is_of_typeobj-cls and 
https://github.com/Stewori/pytypes#is_subtypesubclass-superclass respectively. 
We also use them internally for pytypes' runtime typechecking features.

 

Unfortunately there is no release finalized yet, but it's mostly cleanup work 
and pip integration left to do. Hope to get a beta release done soon.

 

Best

 

-Stefan

 

Gesendet: Donnerstag, 22. Juni 2017 um 23:53 Uhr

Von: "Ivan Levkivskyi" <levkivs...@gmail.com>

An: "Markus Wissinger" <markus.wissin...@gmail.com>

Cc: Python-Dev <python-dev@python.org>

Betreff: Re: [Python-Dev] PEP 544: Protocols - second round




On 22 June 2017 at 10:44, Markus Wissinger <markus.wissin...@gmail.com> wrote:


I have to admit I am not happy with separating the concepts of 'runtime' and 
'static' types as implied by pep544.


 

This is not something new, already PEP 483 makes a clear distinction between 
types (a static concept) and classes (a runtime concept).

 



Failing isinstance/issubclass calls for protocols would hurt there. I 
understand that any type checker code that could provide isinstance 
functionality for pep544 protocols would rely on method signatures that my hint 
generator is just producing.



 

isinstance(obj, T) and issubclass(Cls, T) already fail if T is a subscripted 
generic like List[int], so that again nothing new here. To check runtime 
subtyping with such types one can write a third party introspection tool based 
on typing_inspect package on PyPI (which potentially might in future become an 
official wrapper for currently internal typing API).



--

Ivan



 



___ Python-Dev mailing list 
Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev 
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/stefan.richthofer%40gmx.de


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 544: Protocols - second round

2017-06-22 Thread Stefan Richthofer 

>> I am currently exploring a type hint generator that produces hints out of types used in unit tests.

 

Note that pytypes (https://github.com/Stewori/pytypes) already supports this. It can dump PEP 484 style stubfiles from runtime type observations (made via decorators or profiler hook).

 

>> isinstance(obj, T) and issubclass(Cls, T) already fail if T is a subscripted generic like List[int], so that again nothing new here. To check runtime subtyping with such types one can write a third party introspection tool based on typing_inspect package on PyPI

 

There are public API functions in pytypes for isinstance and issubclass with support for most PEP 484 types, see https://github.com/Stewori/pytypes#is_of_typeobj-cls and https://github.com/Stewori/pytypes#is_subtypesubclass-superclass respectively. We also use them internally for pytypes' runtime typechecking features.

 

Unfortunately there is no release finalized yet, but it's mostly cleanup work and pip integration left to do. Hope to get a beta release done soon.

 

Best

 

-Stefan

 

Gesendet: Donnerstag, 22. Juni 2017 um 23:53 Uhr
Von: "Ivan Levkivskyi" <levkivs...@gmail.com>
An: "Markus Wissinger" <markus.wissin...@gmail.com>
Cc: Python-Dev <python-dev@python.org>
Betreff: Re: [Python-Dev] PEP 544: Protocols - second round




On 22 June 2017 at 10:44, Markus Wissinger <markus.wissin...@gmail.com> wrote:


I have to admit I am not happy with separating the concepts of 'runtime' and 'static' types as implied by pep544.


 

This is not something new, already PEP 483 makes a clear distinction between types (a static concept) and classes (a runtime concept).

 



Failing isinstance/issubclass calls for protocols would hurt there. I understand that any type checker code that could provide isinstance functionality for pep544 protocols would rely on method signatures that my hint generator is just producing.



 

isinstance(obj, T) and issubclass(Cls, T) already fail if T is a subscripted generic like List[int], so that again nothing new here. To check runtime subtyping with such types one can write a third party introspection tool based on typing_inspect package on PyPI (which potentially might in future become an official wrapper for currently internal typing API).

--

Ivan

 



___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/stefan.richthofer%40gmx.de



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 544: Protocols - second round

2017-06-22 Thread Ivan Levkivskyi 
On 22 June 2017 at 10:44, Markus Wissinger 
wrote:

> I have to admit I am not happy with separating the concepts of 'runtime'
> and 'static' types as implied by pep544.
>

This is not something new, already PEP 483 makes a clear distinction
between types (a static concept) and classes (a runtime concept).


> Failing isinstance/issubclass calls for protocols would hurt there. I
> understand that any type checker code that could provide isinstance
> functionality for pep544 protocols would rely on method signatures that my
> hint generator is just producing.
>

isinstance(obj, T) and issubclass(Cls, T) already fail if T is a
subscripted generic like List[int], so that again nothing new here. To
check runtime subtyping with such types one can write a third party
introspection tool based on typing_inspect package on PyPI (which
potentially might in future become an official wrapper for currently
internal typing API).

--
Ivan
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 544: Protocols - second round

2017-06-22 Thread Stéfane Fermigier 
On Thu, Jun 22, 2017 at 10:44 AM, Markus Wissinger <
markus.wissin...@gmail.com> wrote:

> Hi,
>
> I have to admit I am not happy with separating the concepts of 'runtime'
> and 'static' types as implied by pep544.
>
> I am currently exploring a type hint generator that produces hints out of
> types used in unit tests. It debugs the tests and collects the parameter
> types of call and return events. It ignores a type when a supertype is
> present. Failing isinstance/issubclass calls for protocols would hurt
> there. I understand that any type checker code that could provide
> isinstance functionality for pep544 protocols would rely on method
> signatures that my hint generator is just producing.
>
> proof of concept implementation (writes method docstrings, no pep484 type
> hints yet):
> https://github.com/markuswissinger/ducktestpy
>
> This is currently just some personal project that some of you will
> consider a strange idea.
>

Not a strange idea, I've had a similar idea and played a bit with it ~10
years ago (inspired by a Java project whose name eludes me now).

Also, I think PyCharm is able to do similar things (see
https://blog.jetbrains.com/pycharm/2013/02/dynamic-runtime-type-inference-in-pycharm-2-7/
).

  S.

-- 
Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier -
http://linkedin.com/in/sfermigier
Founder & CEO, Abilian - Enterprise Social Software -
http://www.abilian.com/
Chairman, Free Group / Systematic Cluster -
http://www.gt-logiciel-libre.org/
Co-Chairman, National Council for Free & Open Source Software (CNLL) -
http://cnll.fr/
Founder & Organiser, PyData Paris - http://pydata.fr/
---
“You never change things by fighting the existing reality. To change
something, build a new model that makes the existing model obsolete.” — R.
Buckminster Fuller
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 544: Protocols - second round

2017-06-22 Thread Markus Wissinger 
Hi,

I have to admit I am not happy with separating the concepts of 'runtime'
and 'static' types as implied by pep544.

I am currently exploring a type hint generator that produces hints out of
types used in unit tests. It debugs the tests and collects the parameter
types of call and return events. It ignores a type when a supertype is
present. Failing isinstance/issubclass calls for protocols would hurt
there. I understand that any type checker code that could provide
isinstance functionality for pep544 protocols would rely on method
signatures that my hint generator is just producing.

proof of concept implementation (writes method docstrings, no pep484 type
hints yet):
https://github.com/markuswissinger/ducktestpy

This is currently just some personal project that some of you will consider
a strange idea. I just want to mention that this use case might not play
well together with pep544.

Regards
Markus

2017-06-05 23:59 GMT+02:00 Guido van Rossum :

> On Fri, Jun 2, 2017 at 3:10 PM, Ivan Levkivskyi 
> wrote:
>
>> On 1 June 2017 at 00:10, Guido van Rossum  wrote:
>>
>>>
>>> On Wed, May 31, 2017 at 2:16 AM, Ivan Levkivskyi 
>>> wrote:
>>>
 On 31 May 2017 at 00:58, Guido van Rossum  wrote:
 [...]

 Thank you for very detailed answers! I have practically nothing to add.
 It seems to me that most of the Kevin's questions stem from unnecessary
 focus
 on runtime type checking. Here are two ideas about how to fix this:

 * Add the word "static" somewhere in the PEP title.

>>>
>>> So the title could become "Protocols: Static structural subtyping (duck
>>> typing)" -- long, but not record-setting.
>>>
>>
>> I am thinking about "Protocols: Structural subtyping (static duck
>> typing)". The reason is that subtyping is already a mostly static concept
>> (in contrast to subclassing),
>> while duck typing is typically associated with the runtime behaviour.
>>
>> This might seem minor, but this version of the title sounds much more
>> naturally to me.
>>
>
> +1
>
> --
> --Guido van Rossum (python.org/~guido)
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> markus.wissinger%40gmail.com
>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 544: Protocols - second round

2017-06-05 Thread Guido van Rossum 
On Fri, Jun 2, 2017 at 3:10 PM, Ivan Levkivskyi 
wrote:

> On 1 June 2017 at 00:10, Guido van Rossum  wrote:
>
>>
>> On Wed, May 31, 2017 at 2:16 AM, Ivan Levkivskyi 
>> wrote:
>>
>>> On 31 May 2017 at 00:58, Guido van Rossum  wrote:
>>> [...]
>>>
>>> Thank you for very detailed answers! I have practically nothing to add.
>>> It seems to me that most of the Kevin's questions stem from unnecessary
>>> focus
>>> on runtime type checking. Here are two ideas about how to fix this:
>>>
>>> * Add the word "static" somewhere in the PEP title.
>>>
>>
>> So the title could become "Protocols: Static structural subtyping (duck
>> typing)" -- long, but not record-setting.
>>
>
> I am thinking about "Protocols: Structural subtyping (static duck
> typing)". The reason is that subtyping is already a mostly static concept
> (in contrast to subclassing),
> while duck typing is typically associated with the runtime behaviour.
>
> This might seem minor, but this version of the title sounds much more
> naturally to me.
>

+1

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 544: Protocols - second round

2017-06-02 Thread Ivan Levkivskyi 
On 1 June 2017 at 00:10, Guido van Rossum  wrote:

>
> On Wed, May 31, 2017 at 2:16 AM, Ivan Levkivskyi 
> wrote:
>
>> On 31 May 2017 at 00:58, Guido van Rossum  wrote:
>> [...]
>>
>> Thank you for very detailed answers! I have practically nothing to add.
>> It seems to me that most of the Kevin's questions stem from unnecessary
>> focus
>> on runtime type checking. Here are two ideas about how to fix this:
>>
>> * Add the word "static" somewhere in the PEP title.
>>
>
> So the title could become "Protocols: Static structural subtyping (duck
> typing)" -- long, but not record-setting.
>

I am thinking about "Protocols: Structural subtyping (static duck typing)".
The reason is that subtyping is already a mostly static concept (in
contrast to subclassing),
while duck typing is typically associated with the runtime behaviour.

This might seem minor, but this version of the title sounds much more
naturally to me.

--
Ivan
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 544: Protocols - second round

2017-05-31 Thread Guido van Rossum 
On Wed, May 31, 2017 at 2:16 AM, Ivan Levkivskyi 
wrote:

> On 31 May 2017 at 00:58, Guido van Rossum  wrote:
> [...]
>
> Thank you for very detailed answers! I have practically nothing to add.
> It seems to me that most of the Kevin's questions stem from unnecessary
> focus
> on runtime type checking. Here are two ideas about how to fix this:
>
> * Add the word "static" somewhere in the PEP title.
>

So the title could become "Protocols: Static structural subtyping (duck
typing)" -- long, but not record-setting.

* Add a short note at the start mentioning this is an extension of the type
> system proposed in PEP 484 and recommending to read PEP 484 first.
>

Hm, the Abstract already spells that out. I suspect that many people react
to the discussion without first reading the PEP itself (I do this myself
:-). The only thing that could possibly be confusing about the abstract is
that it claims to specify "static and runtime semantics" -- but that's
reasonable, since the runtime semantics must somehow be specified even if
they're minimal.

-- 
--Guido van Rossum (python.org/~guido )
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 544: Protocols - second round

2017-05-31 Thread INADA Naoki 
Hi,

I'm interested in startup time too, not only execution time.
Here is very rough test:


with open('with_abc.py', 'w') as f:
print("import abc", file=f)
for i in range(1, 1001):
print(f"class A{i}(metaclass=abc.ABCMeta): pass", file=f)

with open('without_abc.py', 'w') as f:
print("import abc", file=f)
for i in range(1, 1001):
print(f"class A{i}: pass", file=f)


$  time python3 -c 'import abc'

real 0m0.051s
user 0m0.035s
sys 0m0.013s

$  time python3 -c 'import with_abc'

real 0m0.083s
user 0m0.063s
sys 0m0.017s

$  time python3 -c 'import without_abc'

real 0m0.055s
user 0m0.042s
sys 0m0.011s


It seems 1000 ABC classes takes less than 30ms but 1000 normal
classes takes less than 10ms.

I don't know this penalty is acceptable or not.
But how about making ABC optional?


I don't want to use ABC so frequently when there is no real requirement of ABC.

ABC implementation is very complex and sometimes ABC cause unexpected
performance issue, like you fixed in https://github.com/python/typing/pull/383

If we start with "Protocol is always ABC" and we face unexpected
performance penalty later, it may be difficult to find and optimize it.

# If we can stop using ABC for io.IOBase, Python startup time will be
few ms faster.
# Maybe, I should implement weakset and abc in  C.

Regards,
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 544: Protocols - second round

2017-05-31 Thread Ivan Levkivskyi 
On 31 May 2017 at 00:58, Guido van Rossum  wrote:
[...]

Thank you for very detailed answers! I have practically nothing to add.
It seems to me that most of the Kevin's questions stem from unnecessary
focus
on runtime type checking. Here are two ideas about how to fix this:

* Add the word "static" somewhere in the PEP title.
* Add a short note at the start mentioning this is an extension of the type
system proposed in PEP 484 and recommending to read PEP 484 first.

What do you think?

--
Ivan
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 544: Protocols - second round

2017-05-30 Thread Guido van Rossum 
[I added some blank lines to separate the PEP quotes from Kevin's
responses.]

On Mon, May 29, 2017 at 7:51 AM, Kevin Conway 
wrote:

> From the PEP:
> > The problem with them is that a class has to be explicitly marked to
> support them, which is unpythonic and unlike what one would normally do in
> idiomatic dynamically typed Python code.
> > The same problem appears with user-defined ABCs: they must be explicitly
> subclassed or registered.
>
> Neither of these statements are entirely true. The semantics of `abc`
> allow for exactly the kind of detached interfaces this PEP is attempting to
> provide. The `abc.ABCMeta` provides the `__subclasshook__` which allows a
> developer to override the default check of internal `abc` registry state
> with virtually any logic that determines the relationship of a class with
> the interface. The prior art I linked to earlier in the thread uses this
> feature to generically support `issubclass` and `isinstance` in such a way
> that the PEPs goal is achieved.
>

But that doesn't help a static type checker. You can't expect a static
checker to understand the implementation of a particular
`__subclasshook__`. In practice, except for a few "one-trick ponies" such
as Hashable, existing ABCs rely on subclassing or registration to make
isinstance() work, and for statically checking code that uses duck typing
those aren't enough.


> > The intention of this PEP is to solve all these problems by allowing
> users to write the above code without explicit base classes in the class
> definition
>
> As I understand this goal, you want to take what some of us in the
> community have been building ourselves and make it canonical via the
> stdlib.
>

Not really. The goal is to suggest implementation a frequently requested
feature in static checkers based on the type system laid out in PEP 484,
namely checks for duck types. To support this the type system from PEP 484
needs to be extended, and that's what PEP 544 is about.


> What strikes me as odd is that the focus is on 3rd party type checkers
> first rather than introducing this as a feature of the language runtime and
> then updating the type checker contract to make use of it.
>

This seems to be a misunderstanding, or at least an attempt to change the
agenda for PEP 544. The primary goal of the PEP is not to support runtime
checking but static checking. This is not new -- PEP 484 and PEP 526 before
it have also focused on features that are useful primarily for static
checkers. (Also, a bit of history: PEP 484 intentionally focused on static
checking support because there was widespread skepticism about the need for
more runtime checking, but there was a subset of the community that was
very interested in static checking.)


> I see a mention of the `isinstance` check support in the
> postponed/rejected ideas, but the only rationale given for it being in that
> category is, generally, "there are edge cases". For example, the PEP lists
> this as an edge case:
>
> >The problem with this is instance checks could be unreliable, except for
> situations where there is a common signature convention such as Iterable
>
> However, the sample given demonstrates precisely the expected behavior of
> checking if a concrete implements the protocol. It's unclear why this
> sample is given as a negative.
>

I assume we're talking about this example:

  class P(Protocol):
  def common_method_name(self, x: int) -> int: ...

  class X:
  
  def common_method_name(self) -> None: ... # Note different signature

  def do_stuff(o: Union[P, X]) -> int:
  if isinstance(o, P):
  return o.common_method_name(1)  # oops, what if it's an X
instance?

The problem there is that the "state of the art" for runtiming checking
isinstance(o, P) boils down to hasattr(o, 'common_method_name') while the
type checker takes the method signatures into account, so it will consider
X objects not to be instances of P.

The other case given is:
>
> > Another potentially problematic case is assignment of attributes after
> instantiation
>
> Can you elaborate on how type checkers would not encounter this same
> issue? If there is a solution to this problem for type checkers, would that
> same solution not work at runtime? Also, it seems odd to use a custom
> initialize function rather than `__init__`. I don't think it was
> intentional, but this makes it seem like a bit of a strawman that doesn't
> represent typical Python code.
>

Lots of code I've seen initializes variables in a separate function
(usually called from `__init__`). Mypy, at least, considers instance
variables assigned through `self` in all methods of a class to be potential
instance variable declarations, otherwise a lot of code could not be
type-checked.

Again, the example is problematic given that the runtime check for
isinstance(c, P) can't do better than hasattr(c, 'x').  (I think there's a
typo in the PEP here, 'c1' should be 'c'.)

The need to use an

Re: [Python-Dev] PEP 544: Protocols - second round

2017-05-29 Thread Kevin Conway 
>From the PEP:
> The problem with them is that a class has to be explicitly marked to
support them, which is unpythonic and unlike what one would normally do in
idiomatic dynamically typed Python code.
> The same problem appears with user-defined ABCs: they must be explicitly
subclassed or registered.
Neither of these statements are entirely true. The semantics of `abc` allow
for exactly the kind of detached interfaces this PEP is attempting to
provide. The `abc.ABCMeta` provides the `__subclasshook__` which allows a
developer to override the default check of internal `abc` registry state
with virtually any logic that determines the relationship of a class with
the interface. The prior art I linked to earlier in the thread uses this
feature to generically support `issubclass` and `isinstance` in such a way
that the PEPs goal is achieved.

> The intention of this PEP is to solve all these problems by allowing
users to write the above code without explicit base classes in the class
definition
As I understand this goal, you want to take what some of us in the
community have been building ourselves and make it canonical via the
stdlib. What strikes me as odd is that the focus is on 3rd party type
checkers first rather than introducing this as a feature of the language
runtime and then updating the type checker contract to make use of it. I
see a mention of the `isinstance` check support in the postponed/rejected
ideas, but the only rationale given for it being in that category is,
generally, "there are edge cases". For example, the PEP lists this as an
edge case:
>The problem with this is instance checks could be unreliable, except for
situations where there is a common signature convention such as Iterable
However, the sample given demonstrates precisely the expected behavior of
checking if a concrete implements the protocol. It's unclear why this
sample is given as a negative. The other case given is:
> Another potentially problematic case is assignment of attributes after
instantiation
Can you elaborate on how type checkers would not encounter this same issue?
If there is a solution to this problem for type checkers, would that same
solution not work at runtime? Also, it seems odd to use a custom initialize
function rather than `__init__`. I don't think it was intentional, but this
makes it seem like a bit of a strawman that doesn't represent typical
Python code.

> Also, extensive use of ABCs might impose additional runtime costs.
I'd love to see some data around this. Given that it's a rationale for the
PEP I'd expect to see some numbers behind it. For example, is memory cost
of directly registering implementations to abc linear or worse? What is the
runtime growth pattern of isinstance or issubclass when used with heavily
registered or deeply registered abc graphs and is it different than those
calls on concrete class hierarchies? Does the cost affect anything more
than the initial evaluation of the code or, in the absence of
isinstance/issubclass checks, does it continue to have an impact on the
runtime?



On Mon, May 29, 2017 at 5:41 AM Ivan Levkivskyi 
wrote:

> On 28 May 2017 at 19:40, Guido van Rossum  wrote:
>
>> On Sun, May 28, 2017 at 8:27 AM, Ivan Levkivskyi 
>> wrote:
>>
> [...]
>
>> Regarding the title, I'd like to keep the word Protocol in the title too,
>> so I'd go with "Protocols: Structural subtyping (duck typing)" -- hope
>> that's not too long to fit in a PEP title field.
>>
>
> OK, this looks reasonable.
>
>
>>
>> > Type-hints should not have runtime semantics, beyond those that they
 have as classes
 >  lots of code uses isinstance(obj, collections.abc.Iterable) and
 similar checks with other ABCs
 Having interfaces defined as something extended from abc doesn't
 necessitate their use at runtime, but it does open up a great deal of
 options for those of us who want to do so. I've been leveraging abc for a
 few years now to implement a lightweight version of what this PEP is
 attempting to achieve

>>>
>>> IIUC this is not the main goal of the PEP, the main goal is to provide
>>> support/standard for _static_ structural subtyping.
>>> Possibility to use protocols in runtime context is rather a minor bonus
>>> that exists mostly to provide a seamless transition
>>> for projects that already use ABCs.
>>>
>>
>> Is something like this already in the PEP? It deserves attention in one
>> of the earlier sections.
>>
>
> Yes, similar discussions appear in "Rationale and Goals", and "Existing
> approaches to structural subtyping". Maybe I need to polish the text there
> adding more focus on static typing.
>
> --
> Ivan
>
>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 544: Protocols - second round

2017-05-29 Thread Ivan Levkivskyi 
On 28 May 2017 at 19:40, Guido van Rossum  wrote:

> On Sun, May 28, 2017 at 8:27 AM, Ivan Levkivskyi 
> wrote:
>
[...]

> Regarding the title, I'd like to keep the word Protocol in the title too,
> so I'd go with "Protocols: Structural subtyping (duck typing)" -- hope
> that's not too long to fit in a PEP title field.
>

OK, this looks reasonable.


>
> > Type-hints should not have runtime semantics, beyond those that they
>>> have as classes
>>> >  lots of code uses isinstance(obj, collections.abc.Iterable) and
>>> similar checks with other ABCs
>>> Having interfaces defined as something extended from abc doesn't
>>> necessitate their use at runtime, but it does open up a great deal of
>>> options for those of us who want to do so. I've been leveraging abc for a
>>> few years now to implement a lightweight version of what this PEP is
>>> attempting to achieve
>>>
>>
>> IIUC this is not the main goal of the PEP, the main goal is to provide
>> support/standard for _static_ structural subtyping.
>> Possibility to use protocols in runtime context is rather a minor bonus
>> that exists mostly to provide a seamless transition
>> for projects that already use ABCs.
>>
>
> Is something like this already in the PEP? It deserves attention in one of
> the earlier sections.
>

Yes, similar discussions appear in "Rationale and Goals", and "Existing
approaches to structural subtyping". Maybe I need to polish the text there
adding more focus on static typing.

--
Ivan
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 544: Protocols - second round

2017-05-28 Thread Guido van Rossum 
On Sun, May 28, 2017 at 8:27 AM, Ivan Levkivskyi 
wrote:

> On 28 May 2017 at 16:13, Kevin Conway  wrote:
>
>> > Some of the possible options for the title are
>> It seems like you're talking about something most other languages would
>> refer to as "Interfaces". What is unique about this proposal that would
>> call for not using the industry standard language?
>>
>
> Well, I would say there is no "industry standard language" about
> structural subtyping. There are interfaces, protocols,
> traits, mixins, typeclasses, roles, and probably some other terms I am not
> aware of - all with subtly different semantics
> in different languages. There are several reasons why we use term protocol
> and not interface.
> Two important reasons for me are:
> * The term protocol is already de-facto standard in Python for things like
> sequence protocol, iterator protocol,
>   descriptor protocol, etc.
> * Protocols are very different from Java interfaces in one important
> aspect: they don't require explicit
>   declaration of implementation, they are mainly oriented on duck-typing.
> Maybe we need to add a short section to rejected ideas?
>

If you feel like it.

Regarding the title, I'd like to keep the word Protocol in the title too,
so I'd go with "Protocols: Structural subtyping (duck typing)" -- hope
that's not too long to fit in a PEP title field.


> > Type-hints should not have runtime semantics, beyond those that they
>> have as classes
>> >  lots of code uses isinstance(obj, collections.abc.Iterable) and
>> similar checks with other ABCs
>> Having interfaces defined as something extended from abc doesn't
>> necessitate their use at runtime, but it does open up a great deal of
>> options for those of us who want to do so. I've been leveraging abc for a
>> few years now to implement a lightweight version of what this PEP is
>> attempting to achieve
>>
>
> IIUC this is not the main goal of the PEP, the main goal is to provide
> support/standard for _static_ structural subtyping.
> Possibility to use protocols in runtime context is rather a minor bonus
> that exists mostly to provide a seamless transition
> for projects that already use ABCs.
>

Is something like this already in the PEP? It deserves attention in one of
the earlier sections.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 544: Protocols - second round

2017-05-28 Thread Ivan Levkivskyi 
On 28 May 2017 at 16:13, Kevin Conway  wrote:

> > Some of the possible options for the title are
> It seems like you're talking about something most other languages would
> refer to as "Interfaces". What is unique about this proposal that would
> call for not using the industry standard language?
>

Well, I would say there is no "industry standard language" about structural
subtyping. There are interfaces, protocols,
traits, mixins, typeclasses, roles, and probably some other terms I am not
aware of - all with subtly different semantics
in different languages. There are several reasons why we use term protocol
and not interface.
Two important reasons for me are:
* The term protocol is already de-facto standard in Python for things like
sequence protocol, iterator protocol,
  descriptor protocol, etc.
* Protocols are very different from Java interfaces in one important
aspect: they don't require explicit
  declaration of implementation, they are mainly oriented on duck-typing.
Maybe we need to add a short section to rejected ideas?

> Type-hints should not have runtime semantics, beyond those that they have
> as classes
> >  lots of code uses isinstance(obj, collections.abc.Iterable) and
> similar checks with other ABCs
> Having interfaces defined as something extended from abc doesn't
> necessitate their use at runtime, but it does open up a great deal of
> options for those of us who want to do so. I've been leveraging abc for a
> few years now to implement a lightweight version of what this PEP is
> attempting to achieve
>

IIUC this is not the main goal of the PEP, the main goal is to provide
support/standard for _static_ structural subtyping.
Possibility to use protocols in runtime context is rather a minor bonus
that exists mostly to provide a seamless transition
for projects that already use ABCs.

--
Ivan
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 544: Protocols - second round

2017-05-28 Thread Kevin Conway 
> Some of the possible options for the title are
It seems like you're talking about something most other languages would
refer to as "Interfaces". What is unique about this proposal that would
call for not using the industry standard language?

> Type-hints should not have runtime semantics, beyond those that they have
as classes
>  lots of code uses isinstance(obj, collections.abc.Iterable) and similar
checks with other ABCs
Having interfaces defined as something extended from abc doesn't
necessitate their use at runtime, but it does open up a great deal of
options for those of us who want to do so. I've been leveraging abc for a
few years now to implement a lightweight version of what this PEP is
attempting to achieve (https://github.com/kevinconway/iface). Once you
start getting into dynamically loaded plugins you often lose the ability to
strictly enforce the shape of the input until runtime. In those cases, I've
found it exceedingly useful to add 'isinstance' and 'issubbclass' as
assertions to input of untrusted types for the tests and non-production
deployments. For a perf boost in prod you can throw the -O flag and strip
out the assertions to remove the runtime checks. I've found that to be a
valuable pattern.

On Sun, May 28, 2017 at 8:21 AM Ivan Levkivskyi 
wrote:

> Thanks everyone for interesting suggestions!
>
> @Antoine @Guido:
> Some of the possible options for the title are:
> * Protocols (structural subtyping)
> * Protocols (static duck typing)
> * Structural subtyping (static duck typing)
> which one do you prefer?
>
> @Nick:
> Yes, explicit imports are not necessary for static type checkers (I will
> add a short comment about this).
>
> @Mark:
> I agree with Guido on all points here. For example,
> collections.abc.Iterable is already a class,
> and lots of code uses isinstance(obj, collections.abc.Iterable) and
> similar checks with other ABCs
> (also in a structural manner, i.e. via __subclasshook__). So that
> disabling this will case many breakages.
> The question of whether typing.Iterable[int] should be a class is
> independent (orthogonal) and
> does not belong to this PEP.
>
> --
> Ivan
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/kevinjacobconway%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 544: Protocols - second round

2017-05-28 Thread Ivan Levkivskyi 
Thanks everyone for interesting suggestions!

@Antoine @Guido:
Some of the possible options for the title are:
* Protocols (structural subtyping)
* Protocols (static duck typing)
* Structural subtyping (static duck typing)
which one do you prefer?

@Nick:
Yes, explicit imports are not necessary for static type checkers (I will
add a short comment about this).

@Mark:
I agree with Guido on all points here. For example,
collections.abc.Iterable is already a class,
and lots of code uses isinstance(obj, collections.abc.Iterable) and similar
checks with other ABCs
(also in a structural manner, i.e. via __subclasshook__). So that disabling
this will case many breakages.
The question of whether typing.Iterable[int] should be a class is
independent (orthogonal) and
does not belong to this PEP.

--
Ivan
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 544: Protocols - second round

2017-05-26 Thread Guido van Rossum 
On Thu, May 25, 2017 at 10:49 AM, Mark Shannon  wrote:

> I really like this PEP in general. I think this brings the type system for
> type-hints closer to Python semantics.
>

Thank you.


> But there are a few points I disagree with.
> I don't think Protocol types should be tied to ABCs. It just makes things
> more complex with no obvious benefit.
>

There are backwards compatibility benefits -- we could make e.g. Sequence a
Protocol in Python 3.7 and it would be possible to write code that inherits
from Sequence and works in Python 3.6 and 3.7. For this to work we need
some support for non-abstract methods in Protocols.


> I also think all references to 'isinstance' and 'issubclass' should be
> removed. Type-hints should not have runtime semantics, beyond those that
> they have as classes.


Again, backwards compatibility.


> In fact, there is no need for protocol types to be classes at all.
>

That's pretty much a separate discussion (see
https://github.com/python/typing/issues/432).

-- 
--Guido van Rossum (python.org/~guido )
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 544: Protocols - second round

2017-05-26 Thread Guido van Rossum 
On Thu, May 25, 2017 at 7:19 AM, Nick Coghlan  wrote:

> Given the abstract, I'd suggest "Structural Subtyping" as a suitable
> title for the PEP.
>

Maybe even "Structural Subtyping (a.k.a. Duck Typing)"



> That said, I think it's fine to use "protocol" throughout the rest of
> the PEP as is currently the case - object protocols and network
> protocols are clearly different things, it's just the bare word
> "Protocols" appearing as a PEP title in the PEP index with no other
> context that's potentially confusing.
>

Agreed.


> I'm +1 on the general idea of the PEP, and only have one question
> regarding the specifics. Given:
>
> import typing
>
> class MyContainer:
> def __len__(self) -> int:
> ...
> def close(self) -> None:
> ...
>
> Would that be enough for a static typechecker to consider MyContainer
> a structural subtype of both typing.Sized and SupportsClose from the
> PEP, even though neither is imported explicitly into the module? I'm
> assuming the answer is "Yes", but I didn't see it explicitly stated
> anywhere.
>

Yes. Imports don't enter the matter. (Things get tied together at the call
sites.) @Ivan: if there isn't an example that makes this clear we should
add one.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 544: Protocols - second round

2017-05-25 Thread Mark Shannon 

On 24/05/17 14:31, Ivan Levkivskyi wrote:

Hi all,

After collecting suggestions in the previous discussion on python-dev
https://mail.python.org/pipermail/python-dev/2017-March/thread.html#147629
and playing with implementation, here is an updated version of PEP 544.

--
Ivan


I really like this PEP in general. I think this brings the type system 
for type-hints closer to Python semantics.


But there are a few points I disagree with.
I don't think Protocol types should be tied to ABCs. It just makes 
things more complex with no obvious benefit.
I also think all references to 'isinstance' and 'issubclass' should be 
removed. Type-hints should not have runtime semantics, beyond those that 
they have as classes. In fact, there is no need for protocol types to be 
classes at all.


Cheers,
Mark.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 544: Protocols - second round

2017-05-25 Thread Jelle Zijlstra 
2017-05-25 7:19 GMT-07:00 Nick Coghlan :

> On 25 May 2017 at 21:26, Antoine Pitrou  wrote:
> > On Wed, 24 May 2017 23:31:47 +0200
> > Ivan Levkivskyi  wrote:
> >
> >> Hi all,
> >>
> >> After collecting suggestions in the previous discussion on python-dev
> >> https://mail.python.org/pipermail/python-dev/2017-
> March/thread.html#147629
> >> and playing with implementation, here is an updated version of PEP 544.
> >>
> >> --
> >> Ivan
> >>
> >>
> >> A link for those who don't like reading long e-mails:
> >> https://www.python.org/dev/peps/pep-0544/
> >>
> >> =
> >>
> >> PEP: 544
> >> Title: Protocols
> >
> > Can you give this PEP a more explicit title? "Protocols" sound like
> > network protocols to me.
>
> Especially given the existing use of the term in an asyncio context:
> https://www.python.org/dev/peps/pep-3156/#transports-and-protocols
>
> Given the abstract, I'd suggest "Structural Subtyping" as a suitable
> title for the PEP.
>
> That said, I think it's fine to use "protocol" throughout the rest of
> the PEP as is currently the case - object protocols and network
> protocols are clearly different things, it's just the bare word
> "Protocols" appearing as a PEP title in the PEP index with no other
> context that's potentially confusing.
>
> I'm +1 on the general idea of the PEP, and only have one question
> regarding the specifics. Given:
>
> import typing
>
> class MyContainer:
> def __len__(self) -> int:
> ...
> def close(self) -> None:
> ...
>
> Would that be enough for a static typechecker to consider MyContainer
> a structural subtype of both typing.Sized and SupportsClose from the
> PEP, even though neither is imported explicitly into the module? I'm
> assuming the answer is "Yes", but I didn't see it explicitly stated
> anywhere.
>
> Yes, that should be the case. Specifically, if you pass a MyContainer
object to a function whose argument is annotated as typing.Sized or
SupportsClose, a type checker should accept that call.


> Cheers,
> Nick.
>
>
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> jelle.zijlstra%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 544: Protocols - second round

2017-05-25 Thread Nick Coghlan 
On 25 May 2017 at 21:26, Antoine Pitrou  wrote:
> On Wed, 24 May 2017 23:31:47 +0200
> Ivan Levkivskyi  wrote:
>
>> Hi all,
>>
>> After collecting suggestions in the previous discussion on python-dev
>> https://mail.python.org/pipermail/python-dev/2017-March/thread.html#147629
>> and playing with implementation, here is an updated version of PEP 544.
>>
>> --
>> Ivan
>>
>>
>> A link for those who don't like reading long e-mails:
>> https://www.python.org/dev/peps/pep-0544/
>>
>> =
>>
>> PEP: 544
>> Title: Protocols
>
> Can you give this PEP a more explicit title? "Protocols" sound like
> network protocols to me.

Especially given the existing use of the term in an asyncio context:
https://www.python.org/dev/peps/pep-3156/#transports-and-protocols

Given the abstract, I'd suggest "Structural Subtyping" as a suitable
title for the PEP.

That said, I think it's fine to use "protocol" throughout the rest of
the PEP as is currently the case - object protocols and network
protocols are clearly different things, it's just the bare word
"Protocols" appearing as a PEP title in the PEP index with no other
context that's potentially confusing.

I'm +1 on the general idea of the PEP, and only have one question
regarding the specifics. Given:

import typing

class MyContainer:
def __len__(self) -> int:
...
def close(self) -> None:
...

Would that be enough for a static typechecker to consider MyContainer
a structural subtype of both typing.Sized and SupportsClose from the
PEP, even though neither is imported explicitly into the module? I'm
assuming the answer is "Yes", but I didn't see it explicitly stated
anywhere.

Cheers,
Nick.



-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 544: Protocols - second round

2017-05-25 Thread Antoine Pitrou 
On Wed, 24 May 2017 23:31:47 +0200
Ivan Levkivskyi  wrote:

> Hi all,
> 
> After collecting suggestions in the previous discussion on python-dev
> https://mail.python.org/pipermail/python-dev/2017-March/thread.html#147629
> and playing with implementation, here is an updated version of PEP 544.
> 
> --
> Ivan
> 
> 
> A link for those who don't like reading long e-mails:
> https://www.python.org/dev/peps/pep-0544/
> 
> =
> 
> PEP: 544
> Title: Protocols

Can you give this PEP a more explicit title? "Protocols" sound like
network protocols to me.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com