[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Sebastian Rittau]

Am 24.04.2021 um 01:26 schrieb Gregory P. Smith:
Practically speaking, one issue I have is how easy it is to write 
isinstance or issubclass checks. It has historically been much more 
difficult to write and maintain a check that something looks like a duck.


 `if hasattr(foo, 'close') and hasattr(foo, 'seek') and hasattr(foo, 
'read'):`


Just does not roll off the figurative tongue and that is a relatively 
simple example of what is required for a duck check.


To prevent isinstance use when a duck check would be better, we're 
missing an easy builtin elevated to the isinstance() availability 
level behaving as lookslikeaduck() that does matches against a (set 
of) declared typing.Protocol shape(s). An implementation of this 
exists - 
https://www.python.org/dev/peps/pep-0544/#runtime-checkable-decorator-and-narrowing-types-by-isinstance 
 
- but it requires the protocols to declare runtime checkability and 
has them work with isinstance similar to ABCs...  technically accurate 
/BUT via isinstance/? Doh!  It promotes the use of isinstance when it 
really isn't about class hierarchy at all...


Edit: Maybe that's okay, isinstance can be read leniently to mean "is 
an instance of something that one of these things over here says it 
matches" rather than meaning "a parent class type is..."?  From a past 
experience user perspective I don't read "isinstance" as "looks like a 
duck" when I read code.  I assume I'm not alone.


I'm using isinstance from time to time, mostly to satisfy the type 
checker in some cases. I think having a "hasshape()" builtin would be a 
huge win. In addition it would be useful for type checkers to better 
support hasattr() for distinguishing between separate types. For 
example, mypy has a problem with the following:


    class A: pass
    class B:
    x: int

    def foo(x: A | B) -> None:
    if hasattr(x, "b"):
    accepts_b(x)
    else:
    accepts_a(x)

As Nathaniel indicated, how deep do we want to go down this rabbit 
hole of checking?  just names? signatures and types on those?  What 
about exceptions (something our type system has no way to declare at 
all)?  and infinite side effects?  At the end of the day we're 
required to trust the result of whatever check we use and any 
implementation may not conform to our desires no matter how much 
checking we do. Unless we solve the halting problem. :P
I think a PEP to discuss these questions would be appropriate. Things 
like also checking types could also be optional.
Not quite.  A Protocol is merely a way to describe a structural type.  
You do not /need/ to have your /implementations/ of anything inherit 
from typing.Protocol.  I'd /personally/ advise people /do not inherit/ 
from Protocol in their implementation.


+1

Luciano notes that it is preferred to define your protocols as narrow 
and define them in places *where they're used*, to follow a golang 
interface practice.  My thinking aligns with that.


My background is with TypeScript, not go. But TypeScript uses 
"interfaces" extensively to describe the shape of expected objects. I 
agree mostly with you and Luciano here, but it can make sense to define 
some protocols in a more visible location. Examples are the protocols in 
collections.abc. In typeshed we collected a few more common protocols in 
the type-check only module _typeshed. 
(https://github.com/python/typeshed/tree/master/stdlib/_typeshed) These 
are a bit experimental, but could eventually find their way into the 
standard library.


A bit off-topic: But intersection types (as discussed here: 
https://github.com/python/typing/issues/213) could also make a nice 
addition to quickly compose ad-hoc protocols from pre-made protocols:


    def read_stuff(f: HasRead & HasSeek) -> None: ...

That inheritance is used in the /declaration/ of the protocol is an 
implementation detail because our language has never had a syntax for 
declaring an interface. 544 fit within our existing language syntax.


Jukka Lehtosalo proposed a new "type" keyword over at typing-sig: 
https://mail.python.org/archives/list/typing-...@python.org/thread/LV22PX454W4VDTXY6NDJV7NZD4LFK464/ 
This could also be used to define protocols a bit more succintly, and 
with additional syntax constraints, making protocols feel a bit more 
"first class" than they feel now.


 - Sebastian
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VJEBP3U37RFN36TJDNLDQTHKCW7D4KHJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Antoine Pitrou]

On Thu, 22 Apr 2021 12:47:42 -0300
Luciano Ramalho  wrote:
> 
> Go is by far the most successful statically typed language created so
> far in the 21st century,

First, it seems gratuitous to restrict your search to "created so
far in the 21st century".  I suppose that allows you to eliminate Java,
which is extremely successful and was created a bit before the 21st
century.

Second, I'll ask for a source supporting your statement.  I looked for
one and the first result I got is this:
https://pypl.github.io/PYPL.html

... which seems to suggest that Go is less popular than Swift,
TypeScript, or even the relatively obscure Kotlin language.  It's also
behind C# and Objective-C, and the former can probably be considered a
21st century language.

My conclusion from this is that Go's popularity is mostly visible in a
few restricted circles of computing, and in those it doesn't seem to
fare that much better than Rust, either.  But focussing on those two
languages shows a certain selection bias.

Regards

Antoine.


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/K5AI6KUMBVG2S6BO3TWP6HPLT45P3IFA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Antoine Pitrou]

On Wed, 21 Apr 2021 12:36:34 -0700
Christopher Barker  wrote:
> 
> But that's not what duck typing is (at least to me :-) ) For a given
> function, I need the passed in object to quack (and yes, I need that quack
> to sound like a duck) -- but I usually don't care whether that object
> waddles like a duck.
> 
> So yes, isinstance(obj, Sequence) is really the only way to know that obj
> is a Sequence in every important way -- but if you only need it to do one
> or two things like a Sequence, then you don't care.

It depends on the context, though.  Sometimes it's better to check
explicitly and raise a nice error message, then raise a cryptic error
much further that seems to bear little relationship to the line of code
the user wrote.  Especially if that error is raised at the end of a
10-minute computation, or after sending 1GB of data to a S3 bucket.

For this reason, when there's no use case for accepting many kinds of
sequences in a user-facing API, I find it useful to do a `isinstance(x,
(list, tuple))` check before proceeding.

Yes, it's not pure duck typing, but who cares?

Regards

Antoine.


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/D2WLZTH7X34Y57DRSQVCMTDFOTOWSIQ5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Stephen J. Turnbull]

Chris Angelico writes:
 > On Sat, Apr 24, 2021 at 10:14 AM Nick Coghlan  wrote:

 > > Duck typing usually falls squarely into the EAFP category.
 > 
 > Isn't it orthogonal?

Not really.  If you ask 'thing' to 'thing[0]', you don't find out
whether it is a list or a dict (or any number of other things, for
that matter).  EAFP can only verify the presence of the facilities you
actually use, not what thing is, so it *implies* duck-typing.

Nick's statement is not even close to a logical implication, as you
point out, but I think it's the state of the art.  The question I have
is whether that's something essential about duck-typing, or if perhaps
as Gregory points out it's that we don't have TOOWDTI for LYBL duck-
typing.

It's interesting in this regard that so much effort goes into strict
typing and pattern matching which (so far) are LYBL concepts, while
EAFP + duck-typing is "No Reason," "Just Do It" and nobody seems to
have a big problem with that once they get used to it. :-)

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3GQ5LJ43HFQAWNQG65UF7RMNK6T7XAGJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Chris Angelico]

On Sat, Apr 24, 2021 at 10:14 AM Nick Coghlan  wrote:
>
>
>
> On Sat, 24 Apr 2021, 10:02 am Skip Montanaro,  
> wrote:
>>
>>
>>> Practically speaking, one issue I have is how easy it is to write 
>>> isinstance or issubclass checks. It has historically been much more 
>>> difficult to write and maintain a check that something looks like a duck.
>>>
>>>  `if hasattr(foo, 'close') and hasattr(foo, 'seek') and hasattr(foo, 
>>> 'read'):`
>>>
>>> Just does not roll off the figurative tongue and that is a relatively 
>>> simple example of what is required for a duck check.
>>>
>>> To prevent isinstance use when a duck check would be better,
>>
>>
>> I'm going to chime in briefly then return to lurking on this topic, trying 
>> to figure out all the changes to typing while I wasn't paying attention. 
>> Back in ancient times I recall "look before you leap" as the description of 
>> either of the above styles of checks, no matter which was easier to type. At 
>> the time, I thought the general recommendation was to document what 
>> attributes you expected objects to provide and just make the relevant 
>> unguarded references. I no longer recall what the tongue-in-cheek 
>> description of that style was (just "leap"?) Is that more simple usage more 
>> akin to classic "duck typing" than always guarding accesses? I assume that 
>> will still have a place in the pantheon of Python type variants.
>
>
> LBYL: Look before you leap (check then use)
> EAFP: Easier to ask forgiveness than permission (use, then handle any 
> exceptions or let them escape)
>
> (for accessing external resources rather than satisfying internal type 
> consistency, you usually need the latter, as the external state may change 
> between checking and usage)
>
> Duck typing usually falls squarely into the EAFP category.

Isn't it orthogonal? LBYL means checking first, EAFP means going ahead
and doing things, and coping with failure. Duck typing means caring
about it having a specific method/attribute, nominal typing means
caring about it "being a list". You can test either form in advance,
and you can certainly duck-type in EAFP style, so there's only one
quadrant that might not be possible (and even that, I'm sure, would be
possible somehow).

ChrisA
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FJA52VPBTOI5JSOYIEIZDRCJGEOQ6V3P/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Nick Coghlan]

On Fri, 23 Apr 2021, 12:34 pm Inada Naoki,  wrote:

>
> I think using ABC to distinguish sequence or mapping is a bad idea.
>
> There are three policies:
>
> a) Use duck-typing; just us it as sequence. No type check at all.
> b) Use strict type checking; isinstance(x, list) / isinstance(x, (list,
> tuple)).
> c) Use ABC.
>
> But (c) is broken by design. It is not fixable.
> IMHO, We should chose (a) or (b) and reject any idea relying on Sequence
> ABC.
>


That ship sailed long ago, since distinguishing sequences from mappings was
one of the original motivating use cases for ABCs (see the last sentence in
https://www.python.org/dev/peps/pep-3119/#abcs-vs-duck-typing ).

One of the important things to remember about ABCs is that passing a
*runtime* isinstance check is a matter of calling
"the_abc.register(my_type)". Hence the comment earlier in the thread that
with ABCs, passing "isinstance(obj, the_abc)" becomes just another
criterion for quacking like a duck.

Cheers,
Nick.




>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6G5EBNTHXMQJAD6FRVRPNDHNZBPVAOIK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Nick Coghlan]

On Sat, 24 Apr 2021, 10:02 am Skip Montanaro, 
wrote:

>
> Practically speaking, one issue I have is how easy it is to write
>> isinstance or issubclass checks. It has historically been much more
>> difficult to write and maintain a check that something looks like a duck.
>>
>>  `if hasattr(foo, 'close') and hasattr(foo, 'seek') and hasattr(foo,
>> 'read'):`
>>
>> Just does not roll off the figurative tongue and that is a relatively
>> simple example of what is required for a duck check.
>>
>> To prevent isinstance use when a duck check would be better,
>>
>
> I'm going to chime in briefly then return to lurking on this topic, trying
> to figure out all the changes to typing while I wasn't paying attention.
> Back in ancient times I recall "look before you leap" as the description of
> either of the above styles of checks, no matter which was easier to type.
> At the time, I thought the general recommendation was to document what
> attributes you expected objects to provide and just make the relevant
> unguarded references. I no longer recall what the tongue-in-cheek
> description of that style was (just "leap"?) Is that more simple usage more
> akin to classic "duck typing" than always guarding accesses? I assume that
> will still have a place in the pantheon of Python type variants.
>

LBYL: Look before you leap (check then use)
EAFP: Easier to ask forgiveness than permission (use, then handle any
exceptions or let them escape)

(for accessing external resources rather than satisfying internal type
consistency, you usually need the latter, as the external state may change
between checking and usage)

Duck typing usually falls squarely into the EAFP category.

Cheers,
Nick.


> Skip
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/QW7NQ5KTOQLV27MKZO5B3TMSTXIR5MC5/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZA6IHKR3DOICMTEILXCBTX3LJXEVCFMX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Skip Montanaro]

> Practically speaking, one issue I have is how easy it is to write
> isinstance or issubclass checks. It has historically been much more
> difficult to write and maintain a check that something looks like a duck.
>
>  `if hasattr(foo, 'close') and hasattr(foo, 'seek') and hasattr(foo,
> 'read'):`
>
> Just does not roll off the figurative tongue and that is a relatively
> simple example of what is required for a duck check.
>
> To prevent isinstance use when a duck check would be better,
>

I'm going to chime in briefly then return to lurking on this topic, trying
to figure out all the changes to typing while I wasn't paying attention.
Back in ancient times I recall "look before you leap" as the description of
either of the above styles of checks, no matter which was easier to type.
At the time, I thought the general recommendation was to document what
attributes you expected objects to provide and just make the relevant
unguarded references. I no longer recall what the tongue-in-cheek
description of that style was (just "leap"?) Is that more simple usage more
akin to classic "duck typing" than always guarding accesses? I assume that
will still have a place in the pantheon of Python type variants.

Skip
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QW7NQ5KTOQLV27MKZO5B3TMSTXIR5MC5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Gregory P. Smith]

When reading this, I wrote most of it early and left a draft to bake
Then deleted a ton of it after other people replied. I'm conscious that my
terminology might be all over the map.  Keep that in mind before hitting
reply.  It'll take me a while to digest and pedantically use Luciano's
terms, they appear to be a great start. :)

On Tue, Apr 20, 2021 at 10:09 AM Mark Shannon  wrote:

> Hi everyone,
>
> Once upon a time Python was a purely duck typed language.
>
> Then came along abstract based classes, and some nominal typing starting
> to creep into the language.
>
> If you guarded your code with `isinstance(foo, Sequence)` then I could
> not use it with my `Foo` even if my `Foo` quacked like a sequence. I was
> forced to use nominal typing; inheriting from Sequence, or explicitly
> registering as a Sequence.
>

True.  Though in practice I haven't run into this often *myself*.  Do you
have practical examples of where this has bitten users such that code they
would've written pre-abc is no longer possible?  This audience can come up
with plenty of theoretical examples, those aren't so interesting to me.
I'm more interested in observances of actual real world fallout due to
something "important" (as defined however each user wants) using isinstance
checks when it ideally wouldn't.

Practically speaking, one issue I have is how easy it is to write
isinstance or issubclass checks. It has historically been much more
difficult to write and maintain a check that something looks like a duck.

 `if hasattr(foo, 'close') and hasattr(foo, 'seek') and hasattr(foo,
'read'):`

Just does not roll off the figurative tongue and that is a relatively
simple example of what is required for a duck check.

To prevent isinstance use when a duck check would be better, we're missing
an easy builtin elevated to the isinstance() availability level behaving
as lookslikeaduck() that does matches against a (set of) declared
typing.Protocol shape(s). An implementation of this exists -
https://www.python.org/dev/peps/pep-0544/#runtime-checkable-decorator-and-narrowing-types-by-isinstance
- but it requires the protocols to declare runtime checkability and has
them work with isinstance similar to ABCs...  technically accurate *BUT via
isinstance*? Doh!  It promotes the use of isinstance when it really isn't
about class hierarchy at all...

Edit: Maybe that's okay, isinstance can be read leniently to mean "is an
instance of something that one of these things over here says it matches"
rather than meaning "a parent class type is..."?  From a past experience
user perspective I don't read "isinstance" as "looks like a duck" when I
read code.  I assume I'm not alone.

I'd prefer something not involving metaclasses and __instancecheck__ type
class methods.  Something direct so that the author and reader both
explicitly see that they're seeing a duck check rather than a type
hierarchy check.  I don't think this ship has sailed, it could be built up
on top of what already exists if we want it.  Was this already covered in
earlier 544 discussions perhaps?

As Nathaniel indicated, how deep do we want to go down this rabbit hole of
checking?  just names?  signatures and types on those?  What about
exceptions (something our type system has no way to declare at all)?  and
infinite side effects?  At the end of the day we're required to trust the
result of whatever check we use and any implementation may not conform to
our desires no matter how much checking we do. Unless we solve the halting
problem. :P

PEP 544 supports structural typing, but to declare a structural type you
> must inherit from Protocol.
> That smells a lot like nominal typing to me.
>

Not quite.  A Protocol is merely a way to describe a structural type.  You
do not *need* to have your *implementations* of anything inherit from
typing.Protocol.  I'd *personally* advise people *do not inherit* from
Protocol in their implementation. Leave that for a structural type
declaration for type description and annotation purposes only, even though
Protocol appears to support direct inheritance. I understand why some don't
like this separate shape declaration concept.

Luciano notes that it is preferred to define your protocols as narrow and
define them in places *where they're used*, to follow a golang interface
practice.  My thinking aligns with that.

That inheritance is used in the *declaration* of the protocol is an
implementation detail because our language has never had a syntax for
declaring an interface.  544 fit within our existing language syntax.

Then came PEP 563 and said that if you wanted to access the annotations
> of an object, you needed to call typing.get_type_hints() to get
> annotations in a meaningful form.
> This smells a bit like enforced static typing to me.
>

I think useful conversations are ongoing here.  Enforced is the wrong
word.  *[rest of comment deleted in light of Larry's work in progress
response]*

Nominal typing in a dynamically typed language makes 

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Caleb Donovick]

We can add .keys() to Mapping to distinguish Mapping and Sequence.
But it is breaking change, of course. We shouldn’t change it.

We could use the presence of .keys in the subclasses hook only after first
checking
explicit cases (i.e. actual subclass or has been registered). Yes this
would break code
that uses issubclass(X, Mapping) where X looks mapping but isn’t a mapping.
But is this really a concern? What if X had to have all 3 keys, values, and
items
to qualify? Are there really a bunch of classes with __getitem__,
__len__, __iter__, keys, values, and items where the class is expected to
not
be considered a subclass of Mapping?

If there really is a classes like this we could add a warning the subclass
hook about saying
that in a feature version that X is going to be considered a mapping.
Additionally we
could add a black list to the ABCs which would function like the inverse of
register.


On Thu, Apr 22, 2021 at 7:32 PM Inada Naoki songofaca...@gmail.com
 wrote:

On Fri, Apr 23, 2021 at 10:33 AM Chris Angelico  wrote:
> >
> > On Fri, Apr 23, 2021 at 11:22 AM Larry Hastings 
> wrote:
> > >
> > >
> > > On 4/20/21 10:03 AM, Mark Shannon wrote:
> > >
> > > If you guarded your code with `isinstance(foo, Sequence)` then I could
> not use it with my `Foo` even if my `Foo` quacked like a sequence. I was
> forced to use nominal typing; inheriting from Sequence, or explicitly
> registering as a Sequence.
> > >
> > >
> > > If I'm reading the library correctly, this is correct--but, perhaps,
> it could be remedied by adding a __subclasshook__ to Sequence that looked
> for an __iter__ attribute.  That technique might also apply to other ABCs
> in collections.abc, Mapping for example.  Would that work, or am I missing
> an critical detail?
> > >
> >
> > How would you distinguish between a Sequence and a Mapping? Both have
> > __iter__ and __len__. Without actually calling those methods, how
> > would the subclass hook tell them apart?
> >
> > ChrisA
>
> We can add .keys() to Mapping to distinguish Mapping and Sequence.
> But it is breaking change, of course. We shouldn't change it.
>
> I think using ABC to distinguish sequence or mapping is a bad idea.
>
> There are three policies:
>
> a) Use duck-typing; just us it as sequence. No type check at all.
> b) Use strict type checking; isinstance(x, list) / isinstance(x, (list,
> tuple)).
> c) Use ABC.
>
> But (c) is broken by design. It is not fixable.
> IMHO, We should chose (a) or (b) and reject any idea relying on Sequence
> ABC.
>
> Regards,
>
> --
> Inada Naoki  
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/ESLOPO4GLC2QZW4ZDBYEQDPPGB4ZYDWM/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZJQMGI7345TC4AU4MQTWZZA2ENK4NWAL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Inada Naoki]

On Fri, Apr 23, 2021 at 10:33 AM Chris Angelico  wrote:
>
> On Fri, Apr 23, 2021 at 11:22 AM Larry Hastings  wrote:
> >
> >
> > On 4/20/21 10:03 AM, Mark Shannon wrote:
> >
> > If you guarded your code with `isinstance(foo, Sequence)` then I could not 
> > use it with my `Foo` even if my `Foo` quacked like a sequence. I was forced 
> > to use nominal typing; inheriting from Sequence, or explicitly registering 
> > as a Sequence.
> >
> >
> > If I'm reading the library correctly, this is correct--but, perhaps, it 
> > could be remedied by adding a __subclasshook__ to Sequence that looked for 
> > an __iter__ attribute.  That technique might also apply to other ABCs in 
> > collections.abc, Mapping for example.  Would that work, or am I missing an 
> > critical detail?
> >
>
> How would you distinguish between a Sequence and a Mapping? Both have
> __iter__ and __len__. Without actually calling those methods, how
> would the subclass hook tell them apart?
>
> ChrisA

We can add .keys() to Mapping to distinguish Mapping and Sequence.
But it is breaking change, of course. We shouldn't change it.

I think using ABC to distinguish sequence or mapping is a bad idea.

There are three policies:

a) Use duck-typing; just us it as sequence. No type check at all.
b) Use strict type checking; isinstance(x, list) / isinstance(x, (list, tuple)).
c) Use ABC.

But (c) is broken by design. It is not fixable.
IMHO, We should chose (a) or (b) and reject any idea relying on Sequence ABC.

Regards,

-- 
Inada Naoki  
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ESLOPO4GLC2QZW4ZDBYEQDPPGB4ZYDWM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Chris Angelico]

On Fri, Apr 23, 2021 at 11:22 AM Larry Hastings  wrote:
>
>
> On 4/20/21 10:03 AM, Mark Shannon wrote:
>
> If you guarded your code with `isinstance(foo, Sequence)` then I could not 
> use it with my `Foo` even if my `Foo` quacked like a sequence. I was forced 
> to use nominal typing; inheriting from Sequence, or explicitly registering as 
> a Sequence.
>
>
> If I'm reading the library correctly, this is correct--but, perhaps, it could 
> be remedied by adding a __subclasshook__ to Sequence that looked for an 
> __iter__ attribute.  That technique might also apply to other ABCs in 
> collections.abc, Mapping for example.  Would that work, or am I missing an 
> critical detail?
>

How would you distinguish between a Sequence and a Mapping? Both have
__iter__ and __len__. Without actually calling those methods, how
would the subclass hook tell them apart?

ChrisA
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FJZAJTGCCKJD4MYC3OGRFCGJGBIP5EVT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Larry Hastings]

On 4/20/21 10:03 AM, Mark Shannon wrote:
If you guarded your code with `isinstance(foo, Sequence)` then I could 
not use it with my `Foo` even if my `Foo` quacked like a sequence. I 
was forced to use nominal typing; inheriting from Sequence, or 
explicitly registering as a Sequence.



If I'm reading the library correctly, this is correct--but, perhaps, it 
could be remedied by adding a __subclasshook__ to Sequence that looked 
for an __iter__ attribute.  That technique might also apply to other 
ABCs in collections.abc, Mapping for example.  Would that work, or am I 
missing an critical detail?


Cheers,


//arry/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QRS7JWBP3KX3TYSYDVKILSYOXTLOFUY3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Paul Moore]

On Thu, 22 Apr 2021 at 21:40, Adrian Freund  wrote:
> If I understand correctly your concerns with inferring return types for
> inferred protocols are that it might be to restrictive and prevent
> gradual typing. Here are some examples to show how gradual typing would
> still work.

OK, I have no idea what's going on here any more. I have *no* concerns
with inferring the return type. It was you who said that that
inferring would be difficult - the exact quote is "You could
statically type t as Union[Tuple[Literal['version'], int],
Tuple[Literal['name'], str]], but inferring a Protocol for this would
be either very hard or even impossible, especially with even more
complex conditions."

I don't know why you think I have a problem with inferring return
types. All I've ever said is that I thought it might be an interesting
idea if typing an argument as "DuckTyped" could result in type
checkers automatically generated a suitable protocol type, based on
the actual usage of the argument in the function (so that the
programmer doesn't have to explicitly write and maintain a protocol
class in parallel with the code).

> If you have any concrete examples where inferring the return
> type would break gradual typing let me know and I'll have a look at them.

I don't, and I never have. As I say, it seemed to be you who was
claiming that inferring would be too hard.

I don't see much point in continuing this. You seem to be arguing
against points I never made, or maybe I'm completely misunderstanding
you. Either way, we're getting nowhere.

Thanks for taking the time to try to explain, but I think all this has
accomplished is to convince me that there's a "typing mindset" that
embraces a level of strictness that I want nothing to do with. That's
fine, we can agree to differ, but I'm a bit saddened at the thought
that a certain proportion of the information available about typing
might be hard for me to follow because its underlying assumptions are
too different from mine.

Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KADXCO56OFVLILMWWT5GDRGZERBJUQPB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Adrian Freund]

On 4/22/21 5:00 PM, Paul Moore wrote:
> On Thu, 22 Apr 2021 at 15:22, Adrian Freund  wrote:
>> On April 22, 2021 3:15:27 PM GMT+02:00, Paul Moore  
>> wrote:
>>> but that's *absolutely* as far as I'd want to go. Note in particular
>>> that I don't want to constrain the return value
>> The problem is that this isn't enough to have a type safe program. You 
need to also constrain the return type to make sure the returned value can be 
safely passed to other functions.
> But I don't want a type safe program. At least not in an absolute
> sense. All I want is for mypy to catch the occasional error I make
> where I pass the wrong parameter. For me, that's the "gradual" in
> "gradual typing" - it's not a lifestyle, just a convenience. You seem
> to be implying that it's "all or nothing".

I don't think that inferring the required return type breaks gradual
typing, but it is required for people who want type safety.

If I understand correctly your concerns with inferring return types for
inferred protocols are that it might be to restrictive and prevent
gradual typing. Here are some examples to show how gradual typing would
still work. If you have any concrete examples where inferring the return
type would break gradual typing let me know and I'll have a look at them.


def foo(x: DuckType):  # x has to have a .bar(self) method.
# The return type of which is inferred as Any, as it isn't used
    x.bar()



def bar(x):
    x.bar()

def foo(x: DuckType):  # x has to have a .read(self) method.
# The return type of which ist inferred as  Any, as the parameter to 
bar
isn't typed.
    bar(x.read())


Contrast that with


def bar(x: DuckType):  # x has to have a .bar(self) method.
# The return type of which is inferred as Any.
    x.bar()

def foo(x: DuckType):  # x has to have a .read(self) method that returns
something with a .bar(self) method.
# If we don't infer the return type our call to bar() might be unsafe
despite both foo and bar being typed.

    bar(x.read())

> I repeat, all I'm proposing is that
>
> def f(x: int): ...
> def g(x: str): ...
>
> def main(t: DuckTyped) -> None:
> if t[0] == 'version':
> f(t[1])
> elif t[0] == 'name':
> g(t[1])
>
> gets interpreted *exactly* the same as if I'd written
>
> class TType(Protocol):
> def __getitem__(self, int): ...
>
> def f(x: int): ...
> def g(x: str): ...
>
> def main(t: TType) -> None:
> if t[0] == 'version':
> f(t[1])
> elif t[0] == 'name':
> g(t[1])
>
> How can you claim that the second example requires that " large parts
> of your codebase will either need explicit annotations or will be
> unchecked"? And if the second example doesn't require that, nor does
> the first because it's equivalent.

Both examples don't check the calls to f and g despite f and g both
being typed functions
and being called from typed functions.

In a real codebase this will lead to a lot more instances of this
happening. It would happen every time you do anything with something
returned from a method on an inferred protocol

>
> Honestly, this conversation is just reinforcing my suspicion that
> people invested in type annotations have a blind spot when it comes to
> dealing with people and use cases that don't need to go "all in" with
> typing :-(

I don't think this is an all in or nothing. You can infer return types
of inferred protocols and still use gradual typing.  It's just that not
inferring return types causes problems for both full and gradual typing.


Adrian Freund



___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KVB33BV6T6H7PFXN574CMFZUJOVSVUBV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Luciano Ramalho]

On Thu, Apr 22, 2021 at 5:43 AM Chris Angelico  wrote:
> File-like objects are used VERY frequently in the stdlib, and actual
> open file objects have quite a large interface. The use-case is a
> pretty real one: "if I were to create a simulant file object to pass
> to json.load(), what methods do I need?".

My experience with so-called "file-like objects" is that the interface
required most of the time consists of a single method: read()

> Maybe in some cases, the "smaller protocols" option is practical, but
> it would need to have a useful name.

The authors of the typing module already came up with an excellent
convention. For the narrow protocol I mentioned, the conventional name
would be "SupportsRead". Maybe "SupportsRead[str]" and
"SupportsRead[bytes]".

> For instance, if it needs to be
> readable, iterable, closeable, and autocloseable via
> __enter__/__exit__, that's ... uhh a readable, iterable, closeable
> context manager? Not an improvement over "file-like object".

Yes, file-like objects can and do have lots of methods. Often you
don't need more than read()

Cheers,

Luciano

On Thu, Apr 22, 2021 at 7:04 AM Chris Angelico  wrote:
>
> On Thu, Apr 22, 2021 at 7:53 PM Paul Moore  wrote:
> > I wonder whether type checkers could handle a "magic" type (let's call
> > it DuckTyped for now :-)) which basically means "infer a protocol
> > based on usage in this function". So if I do:
> >
> > def my_fn(f: DuckTyped):
> > with f:
> > data = f.read()
> > for line in f:
> > print(line)
> > f.close()
> >
> > then the type checker would automatically build a protocol type like
> > the one I defined above and use that as the type of f? That would make
> > it much easier to include duck typed arguments in function signatures
> > while keeping the benefits of static type checking.
> >
>
> Someone will likely correct me if this is inaccurate, but my
> understanding is that that's exactly what you get if you just don't
> give a type hint. The point of type hints is to give more information
> to the type checker when it's unable to simply infer from usage and
> context.
>
> ChrisA
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/RW5ACSLJP2RLBZWDGQRGBD6ZAVRUQWMG/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Luciano Ramalho
|  Author of Fluent Python (O'Reilly, 2015)
| http://shop.oreilly.com/product/0636920032519.do
|  Technical Principal at ThoughtWorks
|  Twitter: @ramalhoorg
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HP25YCWUQVGPGWDFFFSNOLQQOHGRKEVG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Luciano Ramalho]

On Thu, Apr 22, 2021 at 6:57 AM Paul Moore  wrote:
>
> On Thu, 22 Apr 2021 at 09:46, Chris Angelico  wrote:
> > Maybe in some cases, the "smaller protocols" option is practical, but
> > it would need to have a useful name. For instance, if it needs to be
> > readable, iterable, closeable, and autocloseable via
> > __enter__/__exit__, that's ... uhh a readable, iterable, closeable
> > context manager? Not an improvement over "file-like object".
>
> Note: I've not used protocols myself, so this is speculation.
>
> Is the name of the protocol important? Specifically, if I do, in my code
>
> class X(Protocol):
> def read(self): ...
> def __iter__(self): ...
> def close(self): ...
> def __enter__(self): ...
> def __exit__(self, exc_type, exc_val, exc_tb): ...
>
> def my_fn(fileobj: X) -> None:
> # my stuff
>

That is not a very good example of a Protocol. If you google for best
practices for interfaces in Go (#golang), you'll find they advocate
for very narrow protocols—what they call "interfaces" we decided to
call "protocols".

Many (perhaps most) protocols in the Go standard library define a single method.

I highly recommend reading up on how "interfaces" are used in Go to
reason about how "protocols" should be used in Python (*)

Cheers,

Luciano

(*) That reminded me of how I found Python. In 1998 I was using Perl,
which had just started to support classes. So in the Perl mailing
lists there were quite a few messages then about how classes were used
in Python. After a few mentions, I read the Python tutorial and never
looked back.

On Thu, Apr 22, 2021 at 6:57 AM Paul Moore  wrote:
>
> On Thu, 22 Apr 2021 at 09:46, Chris Angelico  wrote:
> > Maybe in some cases, the "smaller protocols" option is practical, but
> > it would need to have a useful name. For instance, if it needs to be
> > readable, iterable, closeable, and autocloseable via
> > __enter__/__exit__, that's ... uhh a readable, iterable, closeable
> > context manager? Not an improvement over "file-like object".
>
> Note: I've not used protocols myself, so this is speculation.
>
> Is the name of the protocol important? Specifically, if I do, in my code
>
> class X(Protocol):
> def read(self): ...
> def __iter__(self): ...
> def close(self): ...
> def __enter__(self): ...
> def __exit__(self, exc_type, exc_val, exc_tb): ...
>
> def my_fn(fileobj: X) -> None:
> # my stuff
>
> would that not work? An argument is checked to see if it conforms with
> a protocol by confirming it has the right methods, not by name,
> inheritance or registration. And if you want, you can just call X
> "FileLike", it's only a local name so it won't clash with whatever
> other people (or you, in a different module) want to say is
> "file-like". Of course, that's incredibly verbose and messy, and it
> would result in a huge proliferation of throw-away protocol classes,
> which is probably not a good practice that we'd want to encourage, but
> it works.
>
> IMO, the problem isn't that *technically* static typing excludes the
> more traditional duck typing, but rather that the design, approach and
> as a result the emerging "best practices" are focused around
> inheritance based (is that what people mean by "nominal"?) models, to
> the point where duck typing feels like an afterthought that you have
> to work to include.
>
> I wonder whether type checkers could handle a "magic" type (let's call
> it DuckTyped for now :-)) which basically means "infer a protocol
> based on usage in this function". So if I do:
>
> def my_fn(f: DuckTyped):
> with f:
> data = f.read()
> for line in f:
> print(line)
> f.close()
>
> then the type checker would automatically build a protocol type like
> the one I defined above and use that as the type of f? That would make
> it much easier to include duck typed arguments in function signatures
> while keeping the benefits of static type checking.
>
> I will say that at the moment, this doesn't bother me much personally.
> On the larger projects where I've used typing, we've been fine with
> class-based typing and haven't really needed anything more complex
> like protocols. On smaller projects, I just don't use typing at all.
> Whether this will change if I decide to introduce typing in more
> places, I don't know at the moment. I've also not really used typing
> for public APIs, where an over-restrictive type would be more of an
> issue.
>
> Paul
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/EQDLTZDXEE7RRFCAVCGLR5OTJOWFVXH5/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Luciano Ramalho
|  Author of Fluent Python (O'Reilly, 2015)
| 

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Luciano Ramalho]

On Wed, Apr 21, 2021 at 7:31 PM Paul Bryan  wrote:
> As demonstrated, protocols don't get us there because duck typing isn't a 
> matter of having an object exhibit all of the attributes of a duck, but 
> rather some subset of attributes to be used by the consumer. I want this duck 
> to quack; someone else will want it to waddle.

A HUGE insight I learned studying Go is that Protocols should be
defined near the code that CONSUMES it, and not near the code that
PROVIDES it. That's exactly the opposite of how we use ABCs, or Java
folks use interfaces (most of the time).

Cheers,

Luciano


On Wed, Apr 21, 2021 at 7:31 PM Paul Bryan  wrote:
>
> As demonstrated, protocols don't get us there because duck typing isn't a 
> matter of having an object exhibit all of the attributes of a duck, but 
> rather some subset of attributes to be used by the consumer. I want this duck 
> to quack; someone else will want it to waddle. I don't see how type hints 
> could reasonably support "file like object" in the duck type sense (unless 
> the consumer were to specify the exact attributes of the duck it's interested 
> in, which I fear would become a tedious type writing style).
>
> I too have sensed static typing driving the typing development agenda in 
> Python recently, causing other typing methods to take a back seat, so to 
> speak. I add my voice to those requesting Python handle other typing methods.
>
> Barring an innovation to allow a "subset" of a type to be declared in a type 
> hint, I would conclude that static typing and duck typing are diametrically 
> opposed. If we agree that both are valuable, developers could build consensus 
> on that point, and work to ensure that one does not move forward at the 
> expense of the other.
>
> Paul
>
> On Wed, 2021-04-21 at 12:36 -0700, Christopher Barker wrote:
>
> Thanks Mark for posting this. I know some of us are uneasy about the pace of 
> the typing train 
>
> On Tue, Apr 20, 2021 at 11:20 AM Nathaniel Smith  wrote:
>
> > If you guarded your code with `isinstance(foo, Sequence)` then I could
> > not use it with my `Foo` even if my `Foo` quacked like a sequence. I was
> > forced to use nominal typing; inheriting from Sequence, or explicitly
> > registering as a Sequence.
>
> You say this like it's a bad thing, but how is this avoidable, even in
> principle? Structural typing lets you check whether Foo is duck-shaped
> -- has appropriate attribute names, etc. But quacking like a duck is
> harder: you also have to implement the Sequence behavioral contract,
> and realistically the only way to know that is if the author of Foo
> tells you.
>
>
> But that's not what duck typing is (at least to me :-) ) For a given 
> function, I need the passed in object to quack (and yes, I need that quack to 
> sound like a duck) -- but I usually don't care whether that object waddles 
> like a duck.
>
> So yes, isinstance(obj, Sequence) is really the only way to know that obj is 
> a Sequence in every important way -- but if you only need it to do one or two 
> things like a Sequence, then you don't care.
>
> And this is not uncommon -- I suspect it's very rare for a single function to 
> use most of the methods of a given ABC (or protocol, or whatever).
>
> And a lot of the standard library works exactly this way. Two examples 
> (chosen arbitrarily, I just happen to have thought about how they work):
>
> json.load() simply calls ``fp.read()``, and passes the result on down to 
> json.loads(). That's it -- no checking of anything.
>
> If fp does not have a read() method, you get an AttributeError. If fp has a 
> read() method, but it returns something other than a string, then you get 
> some other Exception. And if it returns a string, but that string isn't valid 
> JSON, you get yet another kind of error.
>
> In short, json.load(fp, ...) requires fp to have a read() method that returns 
> a valid JSON string. But it doesn't check, nor does it need to, if it's 
> getting an actual io.TextIOBase object. Is that the right one? I'm not 
> totally sure, which I kind of think makes my point -- I've been using 
> "file-like" objects for years (decades) without worrying about it.
>
> Example 2:
>
> The str.translate method takes:
>
> "a mapping of Unicode ordinals to Unicode ordinals, strings, or None"
>
> Ok, then you need to pass in a Mapping, yes? Well, no you don't. The docs go 
> on to say:
>
> The table must implement lookup/indexing via __getitem__, for instance a
> dictionary or list.
>
> Ah -- so we don't need a Mapping -- we need anything indexable by an integer 
> that contains "ordinals, strings, or None". What the heck ABC could we use 
> for that?
>
> The ABCs do have an already complex hierarchy of containers, but there is no 
> "Indexable", (quacks) and certainly no "indexable and returns these 
> particular things. (quacks a certain way). (maybe there's something in the 
> typing module that would work for static typing -- I have no idea).
>
> I'm pretty sure this 

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Luciano Ramalho]

On Wed, Apr 21, 2021 at 4:44 PM Christopher Barker  wrote:
>> You say this like it's a bad thing, but how is this avoidable, even in
>> principle? Structural typing lets you check whether Foo is duck-shaped
>> -- has appropriate attribute names, etc. But quacking like a duck is
>> harder: you also have to implement the Sequence behavioral contract,
>> and realistically the only way to know that is if the author of Foo
>> tells you.
>
>
> But that's not what duck typing is (at least to me :-) ) For a given 
> function, I need the passed in object to quack (and yes, I need that quack to 
> sound like a duck) -- but I usually don't care whether that object waddles 
> like a duck.

I agree with Christopher here. For duck typing, all we care is that
the method(s) we need are implemented in a sensible way.

> And this is not uncommon -- I suspect it's very rare for a single function to 
> use most of the methods of a given ABC (or protocol, or whatever).

Also agree, and very often the function cares only about one single method.

> And a lot of the standard library works exactly this way.

Totally. I found it astonishing that PEP 484 did not allow to
correctly annotate LOTS of functions in the standard library,
including built-ins people use every day like `sorted()`. Fortunately,
that was corrected with PEP 544.

> But there is the "culture" of Python -- and it has been very much shifting 
> toward more typing

I disagree because Python is too widespread these days to talk about
"the culture". We'll just need to learn to coexist as groups that use
the language differently.

-- A recent publication (sorry can't find it now -- my google fu is
failing me) examined code on PyPi and found a lot of type hints --
many of which were apparently not being used with a static type
checker. So why were they there?

Maybe you mean this one:

"Python 3 types in the wild: a tale of two type systems"
Metadata:
https://dl.acm.org/doi/10.1145/3426422.3426981
PDF:
http://hirzels.com/martin/papers/dls20-python-types.pdf

> And I've seen a lot more isinstance(Some_ABC) code lately as well.

I believe a lot of that is people coming from statically typed
languages and not feeling comfortable with duck typing, while for some
other reason not adopting type hints.

> From looking at the work of my beginning students, I can tell that they are 
> seeing examples out there that use more typing, to the point that they think 
> it's a best practice (or maybe even required?).

It is considered best practice in some places. I hope we as a
community never adopt it as the canonical way.

> Maybe it is -- but if the community is moving that way, we should be honest 
> about it.

Clearly parts of the community are moving that way. The paper I linked
shows the use of type hints is still very limited in their sample.

>> I'm not even sure that this *is* nominal typing. You could just as
>> well argue that "the operation `isinstance(..., Sequence)` returns
>> `True`" is just another of the behavioral constraints that are
>> required to quack like a sequence.

It is not just nominal typing because of the subclass hook.

> I'm not sure of the definition of "nominal" typing -- but it absolutely is 
> NOT duck typing (As Luciano pointed out, Alex Martelli coined the term Goose 
> Typing for this).

Yes. I wish Alex joined this conversation.

Cheers,

Luciano

On Wed, Apr 21, 2021 at 8:28 PM Luciano Ramalho  wrote:
>
> On Wed, Apr 21, 2021 at 4:44 PM Christopher Barker  
> wrote:
> >> You say this like it's a bad thing, but how is this avoidable, even in
> >> principle? Structural typing lets you check whether Foo is duck-shaped
> >> -- has appropriate attribute names, etc. But quacking like a duck is
> >> harder: you also have to implement the Sequence behavioral contract,
> >> and realistically the only way to know that is if the author of Foo
> >> tells you.
> >
> >
> > But that's not what duck typing is (at least to me :-) ) For a given 
> > function, I need the passed in object to quack (and yes, I need that quack 
> > to sound like a duck) -- but I usually don't care whether that object 
> > waddles like a duck.
>
> I agree with Christopher here. For duck typing, all we care is that
> the method(s) we need are implemented in a sensible way.
>
> > And this is not uncommon -- I suspect it's very rare for a single function 
> > to use most of the methods of a given ABC (or protocol, or whatever).
>
> Also agree, and very often the function cares only about one single method.
>
> > And a lot of the standard library works exactly this way.
>
> Totally. I found it astonishing that PEP 484 did not allow to
> correctly annotate LOTS of functions in the standard library,
> including built-ins people use every day like `sorted()`. Fortunately,
> that was corrected with PEP 544.
>
> > But there is the "culture" of Python -- and it has been very much shifting 
> > toward more typing
>
> I disagree because Python is too widespread these days to talk about
> 

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Terry Reedy]

On 4/22/2021 9:15 AM, Paul Moore wrote:


Absolutely, I see no problem with "use duck typing for this argument"
being opt-in.


As in x: 'duck'? or x: '!', where '!' means
 'infer it!', or

from typing import Infer
... x: Infer
?

Ditto for ->  ?

--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NSALGTJQIITQO3ZHI7E5G2J77JQERAKZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Christopher Barker]

On Thu, Apr 22, 2021 at 10:47 AM Matthew Einhorn  wrote:
> In PyCharm, the above code will result in it highlighting the number `12`
with the following warning: "Type 'int' doesn't have expected attribute
'close'"

Which gives yet another use for type hints: helping out IDEs.


> If instead you add an `elif isinstance(t, str):`, under that condition
it'll auto-complete `t.` with all string properties/methods.

now this makes me nervous -- if folks start adding isinstance checks to
make their IDE more helpful , we are rally getting away from Duck Typing.

However, you could have presumably typed it as Sequence[int] and gotten all
the benefits of duck typing and IDE completion.

However, if code were to really use a duck-typed "str-like" i would produce
a failure in the type checker even if it was perfectly functional.

NOTE: str is not a great example, as it's one type that we often do need to
explicitly check -- to make the distinction between a Sequence of strings,
which a str is, and a str itself.  And str is so fundamental and complex a
type it's rarely duck-typed anyway.

-CHB


___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/EGBSQALPGCTLAPM6FLIQLDV2YD2OLVAB/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PTVPRVZDZLXGLOM5WREHWQX3ZWUKO5TS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Matthew Einhorn]

On Thu, Apr 22, 2021, at 1:01 PM, Brett Cannon wrote:
> 
> 
> On Thu, Apr 22, 2021 at 4:11 AM Paul Moore  wrote:
>> On Thu, 22 Apr 2021 at 11:21, Paul Moore  wrote:
>> >
>> > On Thu, 22 Apr 2021 at 11:06, Chris Angelico  wrote:
>> > >
>> > > Someone will likely correct me if this is inaccurate, but my
>> > > understanding is that that's exactly what you get if you just don't
>> > > give a type hint. The point of type hints is to give more information
>> > > to the type checker when it's unable to simply infer from usage and
>> > > context.
>> >
>> > Hmm, I sort of wondered about that as I wrote it. But in which case,
>> > what's the problem here? My understanding was that people were
>> > concerned that static typing was somehow in conflict with duck typing,
>> > but if the static checkers enforce the inferred duck type on untyped
>> > arguments, then that doesn't seem to be the case. Having said that, I
>> > thought that untyped arguments were treated as if they had a type of
>> > "Any", which means "don't type check".
>> 
>> Looks like it doesn't:
>> 
>> > cat .\test.py
>> def example(f) -> None:
>> f.close()
>> 
>> import sys
>> example(12)
>> example(sys.stdin)
>> PS 12:00 00:00.009 C:\Work\Scratch\typing
>> > mypy .\test.py
>> Success: no issues found in 1 source file
>> 
>> What I was after was something that gave an error on the first call,
>> but not on the second. Compare this:
>>  

In PyCharm, the above code will result in it highlighting the number `12` with 
the following warning: "Type 'int' doesn't have expected attribute 'close'"

Similarly, for:

def f(x: int): ...
def g(x: str): ...

def main(t: DuckTyped) -> None:
if t[0] == 'version':
f(t[1])
elif t[0] == 'name':
g(t[1])


If you replace `f(t[1])` or `g(t[1])` with just `t.` and activate 
auto-completion, it'll show `__getitem__` as an option (but it won't show any 
additional int/str methods).

If instead you add an `elif isinstance(t, str):`, under that condition it'll 
auto-complete `t.` with all string properties/methods.___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EGBSQALPGCTLAPM6FLIQLDV2YD2OLVAB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Brett Cannon]

On Thu, Apr 22, 2021 at 4:11 AM Paul Moore  wrote:

> On Thu, 22 Apr 2021 at 11:21, Paul Moore  wrote:
> >
> > On Thu, 22 Apr 2021 at 11:06, Chris Angelico  wrote:
> > >
> > > Someone will likely correct me if this is inaccurate, but my
> > > understanding is that that's exactly what you get if you just don't
> > > give a type hint. The point of type hints is to give more information
> > > to the type checker when it's unable to simply infer from usage and
> > > context.
> >
> > Hmm, I sort of wondered about that as I wrote it. But in which case,
> > what's the problem here? My understanding was that people were
> > concerned that static typing was somehow in conflict with duck typing,
> > but if the static checkers enforce the inferred duck type on untyped
> > arguments, then that doesn't seem to be the case. Having said that, I
> > thought that untyped arguments were treated as if they had a type of
> > "Any", which means "don't type check".
>
> Looks like it doesn't:
>
> > cat .\test.py
> def example(f) -> None:
> f.close()
>
> import sys
> example(12)
> example(sys.stdin)
> PS 12:00 00:00.009 C:\Work\Scratch\typing
> > mypy .\test.py
> Success: no issues found in 1 source file
>
> What I was after was something that gave an error on the first call,
> but not on the second. Compare this:
>
> > cat .\test.py
> from typing import Protocol
>
> class X(Protocol):
> def close(self): ...
>
> def example(f: X) -> None:
> f.close()
>
> import sys
> example(12)
> example(sys.stdin)
> PS 12:03 00:00.015 C:\Work\Scratch\typing
> > mypy .\test.py
> test.py:10: error: Argument 1 to "example" has incompatible type
> "int"; expected "X"
> Found 1 error in 1 file (checked 1 source file)
>

Do note that this is only based on what mypy does, not necessarily what all
Python type checkers do. I.e. it's quite possible pytype, pyre, or pyright
infer more (especially https://pypi.org/project/pytype/ since they
specifically say they infer types).
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/Y42GJAJH3BSW5ZDLZQKDZDWNAGRMXDQ3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Adrian Freund]

On April 22, 2021 3:15:27 PM GMT+02:00, Paul Moore  wrote:
>On Thu, 22 Apr 2021 at 13:23, Adrian Freund  wrote:
>>
>> According to PEP 484 all missing annotations in checked functions should be 
>> handled as Any. Any is compatible with all types.
>
>Yep, that's what I understood to be the case.
>
>> I think from a technical standpoint it should be possible to infer protocols 
>> for arguments for most functions, but there are some edge cases where this 
>> would not be possible, making it impractical to make this the default 
>> behavior. Having an annotation to make a type checker infer a protocol would 
>> be interesting though.
>
>Absolutely, I see no problem with "use duck typing for this argument"
>being opt-in.
>
>> For example:
>>
>> def f(x: int): ...
>> def g(x: str): ...
>>
>> def main(t):
>> if t[0] == 'version':
>> f(t[1])
>> elif t[0] == 'name':
>> g(t[1])
>>
>>
>> You could statically type t as Union[Tuple[Literal['version'], int], 
>> Tuple[Literal['name'], str]], but inferring a Protocol for this would be 
>> either very hard or even impossible, especially with even more complex 
>> conditions.
>
>Yes, but that's inferred static typing which is *not* what I was
>proposing.
I think I understood what you were proposing, but my example might have been 
less than ideal. Sorry for that.
 I mixed some static types in there to simplify it. The union wasn't meant at 
what it should infer but was meant as a comparison to what we would to 
currently, with static, nominal typing.

Let me try again without static types.

def file(x):
print(x.read())  # x has to have .read(): object

def string(x):
print(str(x))  # x has to have .__str__(self): object

def main(t):
if t[0] == 'file':
file(t[1])
elif t[0] == 'string':
string(t[1])

Here we can infer that t has to have a __getitem__(self, idx: int), but we 
can't infer it's return type

>I was suggesting that the checker could easily infer that t
>must have a __getitem__ method, and nothing more. So the protocol to
>infer is
>
>class TypeOfT(Protocol):
>def __getitem__(self, idx): ...
>
>It would be nice to go one step further and infer
>
>class TypeOfT(Protocol):
>def __getitem__(self, idx: int): ...
>
>but that's *absolutely* as far as I'd want to go. Note in particular
>that I don't want to constrain the return value 
The problem is that this isn't enough to have a type safe program. You need to 
also constrain the return type to make sure the returned value can be safely 
passed to other functions.
If you don't do this large parts of your codebase will either need explicit 
annotations or will be unchecked.
>- we've no way to know
>what type it might have in the general case. IMO, inferring anything
>else would over-constrain t - there's nothing in the available
>information, for example, that says t must be a tuple, or a list, or
>that t[3] should have any particular type, or anything like that.
You can infer the return type of a function by looking at all the returns it 
contains, and inferring the types of the returned expressions. That isn't too 
hard and pytype for example already does it.

You can infer the return type a protocol function should have by looking at all 
the places it's result are used.
If you have inferred return types then constraining return types using inferred 
protocols would be practical in my opinion.
>
>My instinct is that working out that t needs to have a __getitem__
>that takes an int is pretty straightforward, as all you have to do is
>look at where t is used in the function. Four places, all followed by
>[] with a literal integer in the brackets. That's it. I fully
>appreciate that writing *code* to do that can be a lot harder than it
>looks, but that's an implementation question, not a matter of whether
>it's reasonable as a proposal in theory.
>
>This feels like *precisely* where there seems to be a failure of
>communication between the static typing and the duck typing worlds. I
>have no idea what I said that would make you think that I wanted
>anything like that Union type you quoted above. And yet obviously, you
>somehow got that message from what I did say.
Like I said above the Union. Was just meant as an example of that we would do 
with static, nominal typing, not what we want with duck typing. Sorry for the 
misunderstanding.
>
>Anyway, as I said this is just an interesting idea as far as I'm
>concerned. I've no actual need for it right now, so I'm happy to leave
>it to the mypy developers whether they want to do anything with it.
>
>Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/AG6LWGMUNGP5O3PXJZUVDRRXO5VTJ7FW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Adrian Freund]

According to PEP 484 all missing annotations in checked functions should be 
handled as Any. Any is compatible with all types.

I think from a technical standpoint it should be possible to infer protocols 
for arguments for most functions, but there are some edge cases where this 
would not be possible, making it impractical to make this the default behavior. 
Having an annotation to make a type checker infer a protocol would be 
interesting though.

For example:

def f(x: int): ...
def g(x: str): ...

def main(t):
if t[0] == 'version':
f(t[1])
elif t[0] == 'name':
g(t[1])


You could statically type t as Union[Tuple[Literal['version'], int], 
Tuple[Literal['name'], str]], but inferring a Protocol for this would be either 
very hard or even impossible, especially with even more complex conditions.


Adrian Freund

On April 22, 2021 1:04:11 PM GMT+02:00, Paul Moore  wrote:
>On Thu, 22 Apr 2021 at 11:21, Paul Moore  wrote:
>>
>> On Thu, 22 Apr 2021 at 11:06, Chris Angelico  wrote:
>> >
>> > Someone will likely correct me if this is inaccurate, but my
>> > understanding is that that's exactly what you get if you just don't
>> > give a type hint. The point of type hints is to give more information
>> > to the type checker when it's unable to simply infer from usage and
>> > context.
>>
>> Hmm, I sort of wondered about that as I wrote it. But in which case,
>> what's the problem here? My understanding was that people were
>> concerned that static typing was somehow in conflict with duck typing,
>> but if the static checkers enforce the inferred duck type on untyped
>> arguments, then that doesn't seem to be the case. Having said that, I
>> thought that untyped arguments were treated as if they had a type of
>> "Any", which means "don't type check".
>
>Looks like it doesn't:
>
>> cat .\test.py
>def example(f) -> None:
>f.close()
>
>import sys
>example(12)
>example(sys.stdin)
>PS 12:00 00:00.009 C:\Work\Scratch\typing
>> mypy .\test.py
>Success: no issues found in 1 source file
>
>What I was after was something that gave an error on the first call,
>but not on the second. Compare this:
>
>> cat .\test.py
>from typing import Protocol
>
>class X(Protocol):
>def close(self): ...
>
>def example(f: X) -> None:
>f.close()
>
>import sys
>example(12)
>example(sys.stdin)
>PS 12:03 00:00.015 C:\Work\Scratch\typing
>> mypy .\test.py
>test.py:10: error: Argument 1 to "example" has incompatible type
>"int"; expected "X"
>Found 1 error in 1 file (checked 1 source file)
>
>Paul
>___
>Python-Dev mailing list -- python-dev@python.org
>To unsubscribe send an email to python-dev-le...@python.org
>https://mail.python.org/mailman3/lists/python-dev.python.org/
>Message archived at 
>https://mail.python.org/archives/list/python-dev@python.org/message/54C6G2JLYYD6B37J5KVKPCKSQDCGLRKA/
>Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/M4XDZUPWKPGRO3NF5VONG22YHOHAYZCM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Sebastian Rittau]

Am 22.04.21 um 10:42 schrieb Chris Angelico:

File-like objects are used VERY frequently in the stdlib, and actual
open file objects have quite a large interface. The use-case is a
pretty real one: "if I were to create a simulant file object to pass
to json.load(), what methods do I need?".

Maybe in some cases, the "smaller protocols" option is practical, but
it would need to have a useful name. For instance, if it needs to be
readable, iterable, closeable, and autocloseable via
__enter__/__exit__, that's ... uhh a readable, iterable, closeable
context manager? Not an improvement over "file-like object".


Experience from typeshed shows that many functions in the stdlib and 
third-party libraries only use one or very few methods, very often just 
read() or write(). From a practical standpoint, small protocols seem 
quite feasible. These are quite an improvement over "file-like" objects, 
where no one knows what that actually means.


 - Sebastian


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/H33OH6GGIGHR7SVE6NL32I7OBO7B7F7E/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Luciano Ramalho]

Please let's not try to make Python a "typesafe" language. The success
of Python owes a lot to the fact that duck typing is approachable,
flexible and powerful.

Even if you advocate static typing, I think it's a very good idea to
limit the ambition of the type system if you want to keep most users
happy—despite the opinion of language lawyers.

Go is by far the most successful statically typed language created so
far in the 21st century, and I believe a lot of that success is due to
the simplicity of its type system. It's way more successful than Rust,
possibly for this very reason.

I find it admirable that Go was released without generics, which were
seriously considered only after its runaway success. Generics will
appear in Go 1.17, scheduled for August, 2021—9 years after Go 1.0.

Please let us heed the wise words of Brian W. Kernighan and Alan A. A.
Donovan in the introduction of "The Go Programming Language" book, a
masterpiece (even if you don't like the language, the book is one of
the best introduction to any language that I've ever read):

"""
Go has enough of a type system to avoid most of the careless mistakes
that plague programmers in dynamic languages, but it has a simpler
type system than comparable typed languages. This approach can
sometimes lead to isolated pockets of ‘‘untyped’’ programming within a
broader framework of types, and Go programmers do not go to the
lengths that C++ or Haskell programmers do to express safety
properties as type-based proofs. But in practice Go gives programmers
much of the safety and run-time performance benefits of a relatively
strong type system without the burden of a complex one.
"""

I also consider the support of static duck typing and goose typing
(via runtime type assertions and type switches) key factors that make
Go an excellent language to "get stuff done"—the best compliment Dave
Beazley traditionally has for Python.

Cheers,

Luciano

On Thu, Apr 22, 2021 at 12:05 PM Paul Moore  wrote:
>
> On Thu, 22 Apr 2021 at 15:22, Adrian Freund  wrote:
> >
> > On April 22, 2021 3:15:27 PM GMT+02:00, Paul Moore  
> > wrote:
>
> > >but that's *absolutely* as far as I'd want to go. Note in particular
> > >that I don't want to constrain the return value
> > The problem is that this isn't enough to have a type safe program. You need 
> > to also constrain the return type to make sure the returned value can be 
> > safely passed to other functions.
>
> But I don't want a type safe program. At least not in an absolute
> sense. All I want is for mypy to catch the occasional error I make
> where I pass the wrong parameter. For me, that's the "gradual" in
> "gradual typing" - it's not a lifestyle, just a convenience. You seem
> to be implying that it's "all or nothing".
>
> > If you don't do this large parts of your codebase will either need explicit 
> > annotations or will be unchecked.
>
> That's just not true. (And even if it were true, you're assuming that
> I care - I've already said that my goal is much more relaxed than
> complete type safety).
>
> I repeat, all I'm proposing is that
>
> def f(x: int): ...
> def g(x: str): ...
>
> def main(t: DuckTyped) -> None:
> if t[0] == 'version':
> f(t[1])
> elif t[0] == 'name':
> g(t[1])
>
> gets interpreted *exactly* the same as if I'd written
>
> class TType(Protocol):
> def __getitem__(self, int): ...
>
> def f(x: int): ...
> def g(x: str): ...
>
> def main(t: TType) -> None:
> if t[0] == 'version':
> f(t[1])
> elif t[0] == 'name':
> g(t[1])
>
> How can you claim that the second example requires that " large parts
> of your codebase will either need explicit annotations or will be
> unchecked"? And if the second example doesn't require that, nor does
> the first because it's equivalent.
>
> Honestly, this conversation is just reinforcing my suspicion that
> people invested in type annotations have a blind spot when it comes to
> dealing with people and use cases that don't need to go "all in" with
> typing :-(
>
> Paul
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/CUZNKEA357N3672CCRA7DZ6C4WABN6FK/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Luciano Ramalho
|  Author of Fluent Python (O'Reilly, 2015)
| http://shop.oreilly.com/product/0636920032519.do
|  Technical Principal at ThoughtWorks
|  Twitter: @ramalhoorg
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/STAULSS6WI7DLSHEXYXHFLHQFVWAIAKY/
Code of Conduct: 

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Paul Moore]

On Thu, 22 Apr 2021 at 15:22, Adrian Freund  wrote:
>
> On April 22, 2021 3:15:27 PM GMT+02:00, Paul Moore  
> wrote:

> >but that's *absolutely* as far as I'd want to go. Note in particular
> >that I don't want to constrain the return value
> The problem is that this isn't enough to have a type safe program. You need 
> to also constrain the return type to make sure the returned value can be 
> safely passed to other functions.

But I don't want a type safe program. At least not in an absolute
sense. All I want is for mypy to catch the occasional error I make
where I pass the wrong parameter. For me, that's the "gradual" in
"gradual typing" - it's not a lifestyle, just a convenience. You seem
to be implying that it's "all or nothing".

> If you don't do this large parts of your codebase will either need explicit 
> annotations or will be unchecked.

That's just not true. (And even if it were true, you're assuming that
I care - I've already said that my goal is much more relaxed than
complete type safety).

I repeat, all I'm proposing is that

def f(x: int): ...
def g(x: str): ...

def main(t: DuckTyped) -> None:
if t[0] == 'version':
f(t[1])
elif t[0] == 'name':
g(t[1])

gets interpreted *exactly* the same as if I'd written

class TType(Protocol):
def __getitem__(self, int): ...

def f(x: int): ...
def g(x: str): ...

def main(t: TType) -> None:
if t[0] == 'version':
f(t[1])
elif t[0] == 'name':
g(t[1])

How can you claim that the second example requires that " large parts
of your codebase will either need explicit annotations or will be
unchecked"? And if the second example doesn't require that, nor does
the first because it's equivalent.

Honestly, this conversation is just reinforcing my suspicion that
people invested in type annotations have a blind spot when it comes to
dealing with people and use cases that don't need to go "all in" with
typing :-(

Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CUZNKEA357N3672CCRA7DZ6C4WABN6FK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Paul Moore]

On Thu, 22 Apr 2021 at 13:23, Adrian Freund  wrote:
>
> According to PEP 484 all missing annotations in checked functions should be 
> handled as Any. Any is compatible with all types.

Yep, that's what I understood to be the case.

> I think from a technical standpoint it should be possible to infer protocols 
> for arguments for most functions, but there are some edge cases where this 
> would not be possible, making it impractical to make this the default 
> behavior. Having an annotation to make a type checker infer a protocol would 
> be interesting though.

Absolutely, I see no problem with "use duck typing for this argument"
being opt-in.

> For example:
>
> def f(x: int): ...
> def g(x: str): ...
>
> def main(t):
> if t[0] == 'version':
> f(t[1])
> elif t[0] == 'name':
> g(t[1])
>
>
> You could statically type t as Union[Tuple[Literal['version'], int], 
> Tuple[Literal['name'], str]], but inferring a Protocol for this would be 
> either very hard or even impossible, especially with even more complex 
> conditions.

Yes, but that's inferred static typing which is *not* what I was
proposing. I was suggesting that the checker could easily infer that t
must have a __getitem__ method, and nothing more. So the protocol to
infer is

class TypeOfT(Protocol):
def __getitem__(self, idx): ...

It would be nice to go one step further and infer

class TypeOfT(Protocol):
def __getitem__(self, idx: int): ...

but that's *absolutely* as far as I'd want to go. Note in particular
that I don't want to constrain the return value - we've no way to know
what type it might have in the general case. IMO, inferring anything
else would over-constrain t - there's nothing in the available
information, for example, that says t must be a tuple, or a list, or
that t[3] should have any particular type, or anything like that.

My instinct is that working out that t needs to have a __getitem__
that takes an int is pretty straightforward, as all you have to do is
look at where t is used in the function. Four places, all followed by
[] with a literal integer in the brackets. That's it. I fully
appreciate that writing *code* to do that can be a lot harder than it
looks, but that's an implementation question, not a matter of whether
it's reasonable as a proposal in theory.

This feels like *precisely* where there seems to be a failure of
communication between the static typing and the duck typing worlds. I
have no idea what I said that would make you think that I wanted
anything like that Union type you quoted above. And yet obviously, you
somehow got that message from what I did say.

Anyway, as I said this is just an interesting idea as far as I'm
concerned. I've no actual need for it right now, so I'm happy to leave
it to the mypy developers whether they want to do anything with it.

Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CU7OES4P3KUCQMD3U5766XVOESL43DRF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Paul Moore]

On Thu, 22 Apr 2021 at 11:21, Paul Moore  wrote:
>
> On Thu, 22 Apr 2021 at 11:06, Chris Angelico  wrote:
> >
> > Someone will likely correct me if this is inaccurate, but my
> > understanding is that that's exactly what you get if you just don't
> > give a type hint. The point of type hints is to give more information
> > to the type checker when it's unable to simply infer from usage and
> > context.
>
> Hmm, I sort of wondered about that as I wrote it. But in which case,
> what's the problem here? My understanding was that people were
> concerned that static typing was somehow in conflict with duck typing,
> but if the static checkers enforce the inferred duck type on untyped
> arguments, then that doesn't seem to be the case. Having said that, I
> thought that untyped arguments were treated as if they had a type of
> "Any", which means "don't type check".

Looks like it doesn't:

> cat .\test.py
def example(f) -> None:
f.close()

import sys
example(12)
example(sys.stdin)
PS 12:00 00:00.009 C:\Work\Scratch\typing
> mypy .\test.py
Success: no issues found in 1 source file

What I was after was something that gave an error on the first call,
but not on the second. Compare this:

> cat .\test.py
from typing import Protocol

class X(Protocol):
def close(self): ...

def example(f: X) -> None:
f.close()

import sys
example(12)
example(sys.stdin)
PS 12:03 00:00.015 C:\Work\Scratch\typing
> mypy .\test.py
test.py:10: error: Argument 1 to "example" has incompatible type
"int"; expected "X"
Found 1 error in 1 file (checked 1 source file)

Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/54C6G2JLYYD6B37J5KVKPCKSQDCGLRKA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Paul Moore]

On Thu, 22 Apr 2021 at 11:06, Chris Angelico  wrote:
>
> On Thu, Apr 22, 2021 at 7:53 PM Paul Moore  wrote:
> > I wonder whether type checkers could handle a "magic" type (let's call
> > it DuckTyped for now :-)) which basically means "infer a protocol
> > based on usage in this function". So if I do:
> >
> > def my_fn(f: DuckTyped):
> > with f:
> > data = f.read()
> > for line in f:
> > print(line)
> > f.close()
> >
> > then the type checker would automatically build a protocol type like
> > the one I defined above and use that as the type of f? That would make
> > it much easier to include duck typed arguments in function signatures
> > while keeping the benefits of static type checking.
> >
>
> Someone will likely correct me if this is inaccurate, but my
> understanding is that that's exactly what you get if you just don't
> give a type hint. The point of type hints is to give more information
> to the type checker when it's unable to simply infer from usage and
> context.

Hmm, I sort of wondered about that as I wrote it. But in which case,
what's the problem here? My understanding was that people were
concerned that static typing was somehow in conflict with duck typing,
but if the static checkers enforce the inferred duck type on untyped
arguments, then that doesn't seem to be the case. Having said that, I
thought that untyped arguments were treated as if they had a type of
"Any", which means "don't type check".

So I guess the point here is that either the typing
community/documentation isn't doing a very good job of explaining how
duck types and static types work together, or that people who like
duck typed interfaces¹ aren't reading the available documentation in
type checkers explaining how to do that with static typing :-)

Paul

¹ And I include myself in that - maybe I need to go and read the mypy
docs properly rather than just learning what I need by following
examples in existing code...
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/IRSXXYHMYCRBBA5HKPNZ7OJDRTEGUAG7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Chris Angelico]

On Thu, Apr 22, 2021 at 7:53 PM Paul Moore  wrote:
> I wonder whether type checkers could handle a "magic" type (let's call
> it DuckTyped for now :-)) which basically means "infer a protocol
> based on usage in this function". So if I do:
>
> def my_fn(f: DuckTyped):
> with f:
> data = f.read()
> for line in f:
> print(line)
> f.close()
>
> then the type checker would automatically build a protocol type like
> the one I defined above and use that as the type of f? That would make
> it much easier to include duck typed arguments in function signatures
> while keeping the benefits of static type checking.
>

Someone will likely correct me if this is inaccurate, but my
understanding is that that's exactly what you get if you just don't
give a type hint. The point of type hints is to give more information
to the type checker when it's unable to simply infer from usage and
context.

ChrisA
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RW5ACSLJP2RLBZWDGQRGBD6ZAVRUQWMG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Paul Moore]

On Thu, 22 Apr 2021 at 09:46, Chris Angelico  wrote:
> Maybe in some cases, the "smaller protocols" option is practical, but
> it would need to have a useful name. For instance, if it needs to be
> readable, iterable, closeable, and autocloseable via
> __enter__/__exit__, that's ... uhh a readable, iterable, closeable
> context manager? Not an improvement over "file-like object".

Note: I've not used protocols myself, so this is speculation.

Is the name of the protocol important? Specifically, if I do, in my code

class X(Protocol):
def read(self): ...
def __iter__(self): ...
def close(self): ...
def __enter__(self): ...
def __exit__(self, exc_type, exc_val, exc_tb): ...

def my_fn(fileobj: X) -> None:
# my stuff

would that not work? An argument is checked to see if it conforms with
a protocol by confirming it has the right methods, not by name,
inheritance or registration. And if you want, you can just call X
"FileLike", it's only a local name so it won't clash with whatever
other people (or you, in a different module) want to say is
"file-like". Of course, that's incredibly verbose and messy, and it
would result in a huge proliferation of throw-away protocol classes,
which is probably not a good practice that we'd want to encourage, but
it works.

IMO, the problem isn't that *technically* static typing excludes the
more traditional duck typing, but rather that the design, approach and
as a result the emerging "best practices" are focused around
inheritance based (is that what people mean by "nominal"?) models, to
the point where duck typing feels like an afterthought that you have
to work to include.

I wonder whether type checkers could handle a "magic" type (let's call
it DuckTyped for now :-)) which basically means "infer a protocol
based on usage in this function". So if I do:

def my_fn(f: DuckTyped):
with f:
data = f.read()
for line in f:
print(line)
f.close()

then the type checker would automatically build a protocol type like
the one I defined above and use that as the type of f? That would make
it much easier to include duck typed arguments in function signatures
while keeping the benefits of static type checking.

I will say that at the moment, this doesn't bother me much personally.
On the larger projects where I've used typing, we've been fine with
class-based typing and haven't really needed anything more complex
like protocols. On smaller projects, I just don't use typing at all.
Whether this will change if I decide to introduce typing in more
places, I don't know at the moment. I've also not really used typing
for public APIs, where an over-restrictive type would be more of an
issue.

Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EQDLTZDXEE7RRFCAVCGLR5OTJOWFVXH5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Chris Angelico]

On Thu, Apr 22, 2021 at 5:03 PM Ryan Gonzalez  wrote:
>
> On Apr 21, 2021, 5:29 PM -0500, Paul Bryan , wrote:
>
> As demonstrated, protocols don't get us there because duck typing isn't a 
> matter of having an object exhibit all of the attributes of a duck, but 
> rather some subset of attributes to be used by the consumer. I want this duck 
> to quack; someone else will want it to waddle. I don't see how type hints 
> could reasonably support "file like object" in the duck type sense (unless 
> the consumer were to specify the exact attributes of the duck it's interested 
> in, which I fear would become a tedious type writing style).
>
>
> I'd argue that, if you frequently have cases where functions use a relatively 
> small subset of a much larger interface, it's simply that your interfaces are 
> too large. You're always free to define your own, smaller protocols that just 
> implement the subset of the interfaces you need.
>

File-like objects are used VERY frequently in the stdlib, and actual
open file objects have quite a large interface. The use-case is a
pretty real one: "if I were to create a simulant file object to pass
to json.load(), what methods do I need?".

Maybe in some cases, the "smaller protocols" option is practical, but
it would need to have a useful name. For instance, if it needs to be
readable, iterable, closeable, and autocloseable via
__enter__/__exit__, that's ... uhh a readable, iterable, closeable
context manager? Not an improvement over "file-like object".

ChrisA
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2LQQXLETT6TFB4UATJBEN5HDI47KAZME/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Ryan Gonzalez]

On Apr 21, 2021, 5:29 PM -0500, Paul Bryan , wrote:
> As demonstrated, protocols don't get us there because duck typing isn't a 
> matter of having an object exhibit all of the attributes of a duck, but 
> rather some subset of attributes to be used by the consumer. I want this duck 
> to quack; someone else will want it to waddle. I don't see how type hints 
> could reasonably support "file like object" in the duck type sense (unless 
> the consumer were to specify the exact attributes of the duck it's interested 
> in, which I fear would become a tedious type writing style).

I'd argue that, if you frequently have cases where functions use a relatively 
small subset of a much larger interface, it's simply that your interfaces are 
too large. You're always free to define your own, smaller protocols that just 
implement the subset of the interfaces you need.
>
> I too have sensed static typing driving the typing development agenda in 
> Python recently, causing other typing methods to take a back seat, so to 
> speak. I add my voice to those requesting Python handle other typing methods.
>
> Barring an innovation to allow a "subset" of a type to be declared in a type 
> hint,

I mentioned above that defining smaller protocols is an option. If the main 
concern is due to wanting to define them in the type signature, or wanting to 
ensure the types match the base interface, then that's not new: TypeScript does 
them both (well, at least you could accomplish the latter roughly using 
intersection types).
>
> Paul
>
> On Wed, 2021-04-21 at 12:36 -0700, Christopher Barker wrote:
> > Thanks Mark for posting this. I know some of us are uneasy about the pace 
> > of the typing train 
> >
> > On Tue, Apr 20, 2021 at 11:20 AM Nathaniel Smith  wrote:
> > > > If you guarded your code with `isinstance(foo, Sequence)` then I could
> > > > not use it with my `Foo` even if my `Foo` quacked like a sequence. I was
> > > > forced to use nominal typing; inheriting from Sequence, or explicitly
> > > > registering as a Sequence.
> > >
> > > You say this like it's a bad thing, but how is this avoidable, even in
> > > principle? Structural typing lets you check whether Foo is duck-shaped
> > > -- has appropriate attribute names, etc. But quacking like a duck is
> > > harder: you also have to implement the Sequence behavioral contract,
> > > and realistically the only way to know that is if the author of Foo
> > > tells you.
> > >
> >
> > But that's not what duck typing is (at least to me :-) ) For a given 
> > function, I need the passed in object to quack (and yes, I need that quack 
> > to sound like a duck) -- but I usually don't care whether that object 
> > waddles like a duck.
> >
> > So yes, isinstance(obj, Sequence) is really the only way to know that obj 
> > is a Sequence in every important way -- but if you only need it to do one 
> > or two things like a Sequence, then you don't care.
> >
> > And this is not uncommon -- I suspect it's very rare for a single function 
> > to use most of the methods of a given ABC (or protocol, or whatever).
> >
> > And a lot of the standard library works exactly this way. Two examples 
> > (chosen arbitrarily, I just happen to have thought about how they work):
> >
> > json.load() simply calls ``fp.read()``, and passes the result on down to 
> > json.loads(). That's it -- no checking of anything.
> >
> > If fp does not have a read() method, you get an AttributeError. If fp has a 
> > read() method, but it returns something other than a string, then you get 
> > some other Exception. And if it returns a string, but that string isn't 
> > valid JSON, you get yet another kind of error.
> >
> > In short, json.load(fp, ...) requires fp to have a read() method that 
> > returns a valid JSON string. But it doesn't check, nor does it need to, if 
> > it's getting an actual io.TextIOBase object. Is that the right one? I'm not 
> > totally sure, which I kind of think makes my point -- I've been using 
> > "file-like" objects for years (decades) without worrying about it.
> >
> > Example 2:
> >
> > The str.translate method takes:
> >
> > "a mapping of Unicode ordinals to Unicode ordinals, strings, or None"
> >
> > Ok, then you need to pass in a Mapping, yes? Well, no you don't. The docs 
> > go on to say:
> >
> > The table must implement lookup/indexing via __getitem__, for instance a
> > dictionary or list.
> >
> > Ah -- so we don't need a Mapping -- we need anything indexable by an 
> > integer that contains "ordinals, strings, or None". What the heck ABC could 
> > we use for that?
> >
> > The ABCs do have an already complex hierarchy of containers, but there is 
> > no "Indexable", (quacks) and certainly no "indexable and returns these 
> > particular things. (quacks a certain way). (maybe there's something in the 
> > typing module that would work for static typing -- I have no idea).
> >
> > I'm pretty sure this particular API was designed to accommodate the old py2 
> > str.translate, 

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Carol Willing]

Luciano,

Thank you for such a thoughtful and eloquent response. Your wisdom is
definitely appreciated, and I agree this is an opportunity to go forward
with more clarity. I'm so proud of the Python dev community for their
thoughtful and respectful conversation.

All, Without the efforts of Larry, Guido, Mark, Lukasz, Luciano,
the pydantic/FastAPI folks, the Steering Council and their willingness
listen and problem solve, the outcome would have been far less appealing
and useful.

Thank you!

Carol

On Wed, Apr 21, 2021 at 4:26 PM Paul Bryan  wrote:

> I agree, that's duck typing with a protocol, and precisely the tedious
> type style I would want to avoid.
>
> I don't know what would be a good suggestion. Something where you
> reference a "fully equipped" type and cherry pick the attributes you want?
> Something like `Protocol[Duck, ("quack", "waddle")]`?
>
> Paul
>
>
> On Wed, 2021-04-21 at 16:13 -0700, Jelle Zijlstra wrote:
>
>
>
> El mié, 21 abr 2021 a las 15:30, Paul Bryan () escribió:
>
> As demonstrated, protocols don't get us there because duck typing isn't a
> matter of having an object exhibit all of the attributes of a duck, but
> rather some subset of attributes to be used by the consumer. I want this
> duck to quack; someone else will want it to waddle. I don't see how type
> hints could reasonably support "file like object" in the duck type sense
> (unless the consumer were to specify the exact attributes of the duck it's
> interested in, which I fear would become a tedious type writing style). '
>
>
> This approach (a Protocol that declares exactly what you need the
> file-like object to do) is in fact what we've been doing in typeshed, the
> repository that provides type hints for the standard library. For those
> unfamiliar, it would look something like this:
>
> from typing import Protocol
>
> class SupportsRead(Protocol):
> def read(self) -> bytes: ...
>
> def uses_a_file(f: SupportsRead) -> None:
>  f.read()
>
> That's somewhat tedious, to be sure, but it is working duck typing.
>
>
>
> I too have sensed static typing driving the typing development agenda in
> Python recently, causing other typing methods to take a back seat, so to
> speak. I add my voice to those requesting Python handle other typing
> methods.
>
> Barring an innovation to allow a "subset" of a type to be declared in a
> type hint, I would conclude that static typing and duck typing are
> diametrically opposed. If we agree that both are valuable, developers could
> build consensus on that point, and work to ensure that one does not move
> forward at the expense of the other.
>
> What would you suggest? Should the syntax for declaring Protocols be more
> concise?
>
>
>
> Paul
>
> On Wed, 2021-04-21 at 12:36 -0700, Christopher Barker wrote:
>
> Thanks Mark for posting this. I know some of us are uneasy about the pace
> of the typing train 
>
> On Tue, Apr 20, 2021 at 11:20 AM Nathaniel Smith  wrote:
>
> > If you guarded your code with `isinstance(foo, Sequence)` then I could
> > not use it with my `Foo` even if my `Foo` quacked like a sequence. I was
> > forced to use nominal typing; inheriting from Sequence, or explicitly
> > registering as a Sequence.
>
> You say this like it's a bad thing, but how is this avoidable, even in
> principle? Structural typing lets you check whether Foo is duck-shaped
> -- has appropriate attribute names, etc. But quacking like a duck is
> harder: you also have to implement the Sequence behavioral contract,
> and realistically the only way to know that is if the author of Foo
> tells you.
>
>
> But that's not what duck typing is (at least to me :-) ) For a given
> function, I need the passed in object to quack (and yes, I need that quack
> to sound like a duck) -- but I usually don't care whether that object
> waddles like a duck.
>
> So yes, isinstance(obj, Sequence) is really the only way to know that obj
> is a Sequence in every important way -- but if you only need it to do one
> or two things like a Sequence, then you don't care.
>
> And this is not uncommon -- I suspect it's very rare for a single function
> to use most of the methods of a given ABC (or protocol, or whatever).
>
> And a lot of the standard library works exactly this way. Two examples
> (chosen arbitrarily, I just happen to have thought about how they work):
>
> json.load() simply calls ``fp.read()``, and passes the result on down to
> json.loads(). That's it -- no checking of anything.
>
> If fp does not have a read() method, you get an AttributeError. If fp has
> a read() method, but it returns something other than a string, then you get
> some other Exception. And if it returns a string, but that string isn't
> valid JSON, you get yet another kind of error.
>
> In short, json.load(fp, ...) requires fp to have a read() method that
> returns a valid JSON string. But it doesn't check, nor does it need to, if
> it's getting an actual io.TextIOBase object. Is that the right one? I'm not
> totally sure, which 

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Caleb Donovick]

My personal motivating example for PEP 637 was shorthand for protocols.
`x: Protocol[foo=int, bar=Callable[str, int]]` could say x has attribute
foo which is an int and method bar from str to int.



On Wed, Apr 21, 2021 at 4:23 PM Paul Bryan  wrote:

> I agree, that's duck typing with a protocol, and precisely the tedious
> type style I would want to avoid.
>
> I don't know what would be a good suggestion. Something where you
> reference a "fully equipped" type and cherry pick the attributes you want?
> Something like `Protocol[Duck, ("quack", "waddle")]`?
>
> Paul
>
>
> On Wed, 2021-04-21 at 16:13 -0700, Jelle Zijlstra wrote:
>
>
>
> El mié, 21 abr 2021 a las 15:30, Paul Bryan () escribió:
>
> As demonstrated, protocols don't get us there because duck typing isn't a
> matter of having an object exhibit all of the attributes of a duck, but
> rather some subset of attributes to be used by the consumer. I want this
> duck to quack; someone else will want it to waddle. I don't see how type
> hints could reasonably support "file like object" in the duck type sense
> (unless the consumer were to specify the exact attributes of the duck it's
> interested in, which I fear would become a tedious type writing style). '
>
>
> This approach (a Protocol that declares exactly what you need the
> file-like object to do) is in fact what we've been doing in typeshed, the
> repository that provides type hints for the standard library. For those
> unfamiliar, it would look something like this:
>
> from typing import Protocol
>
> class SupportsRead(Protocol):
> def read(self) -> bytes: ...
>
> def uses_a_file(f: SupportsRead) -> None:
>  f.read()
>
> That's somewhat tedious, to be sure, but it is working duck typing.
>
>
>
> I too have sensed static typing driving the typing development agenda in
> Python recently, causing other typing methods to take a back seat, so to
> speak. I add my voice to those requesting Python handle other typing
> methods.
>
> Barring an innovation to allow a "subset" of a type to be declared in a
> type hint, I would conclude that static typing and duck typing are
> diametrically opposed. If we agree that both are valuable, developers could
> build consensus on that point, and work to ensure that one does not move
> forward at the expense of the other.
>
> What would you suggest? Should the syntax for declaring Protocols be more
> concise?
>
>
>
> Paul
>
> On Wed, 2021-04-21 at 12:36 -0700, Christopher Barker wrote:
>
> Thanks Mark for posting this. I know some of us are uneasy about the pace
> of the typing train 
>
> On Tue, Apr 20, 2021 at 11:20 AM Nathaniel Smith  wrote:
>
> > If you guarded your code with `isinstance(foo, Sequence)` then I could
> > not use it with my `Foo` even if my `Foo` quacked like a sequence. I was
> > forced to use nominal typing; inheriting from Sequence, or explicitly
> > registering as a Sequence.
>
> You say this like it's a bad thing, but how is this avoidable, even in
> principle? Structural typing lets you check whether Foo is duck-shaped
> -- has appropriate attribute names, etc. But quacking like a duck is
> harder: you also have to implement the Sequence behavioral contract,
> and realistically the only way to know that is if the author of Foo
> tells you.
>
>
> But that's not what duck typing is (at least to me :-) ) For a given
> function, I need the passed in object to quack (and yes, I need that quack
> to sound like a duck) -- but I usually don't care whether that object
> waddles like a duck.
>
> So yes, isinstance(obj, Sequence) is really the only way to know that obj
> is a Sequence in every important way -- but if you only need it to do one
> or two things like a Sequence, then you don't care.
>
> And this is not uncommon -- I suspect it's very rare for a single function
> to use most of the methods of a given ABC (or protocol, or whatever).
>
> And a lot of the standard library works exactly this way. Two examples
> (chosen arbitrarily, I just happen to have thought about how they work):
>
> json.load() simply calls ``fp.read()``, and passes the result on down to
> json.loads(). That's it -- no checking of anything.
>
> If fp does not have a read() method, you get an AttributeError. If fp has
> a read() method, but it returns something other than a string, then you get
> some other Exception. And if it returns a string, but that string isn't
> valid JSON, you get yet another kind of error.
>
> In short, json.load(fp, ...) requires fp to have a read() method that
> returns a valid JSON string. But it doesn't check, nor does it need to, if
> it's getting an actual io.TextIOBase object. Is that the right one? I'm not
> totally sure, which I kind of think makes my point -- I've been using
> "file-like" objects for years (decades) without worrying about it.
>
> Example 2:
>
> The str.translate method takes:
>
> "a mapping of Unicode ordinals to Unicode ordinals, strings, or None"
>
> Ok, then you need to pass in a Mapping, yes? Well, no 

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Paul Bryan]

I agree, that's duck typing with a protocol, and precisely the tedious
type style I would want to avoid.

I don't know what would be a good suggestion. Something where you
reference a "fully equipped" type and cherry pick the attributes you
want? Something like `Protocol[Duck, ("quack", "waddle")]`?

Paul
 

On Wed, 2021-04-21 at 16:13 -0700, Jelle Zijlstra wrote:
> 
> 
> El mié, 21 abr 2021 a las 15:30, Paul Bryan ()
> escribió:
> > As demonstrated, protocols don't get us there because duck typing
> > isn't a matter of having an object exhibit all of the attributes of
> > a duck, but rather some subset of attributes to be used by the
> > consumer. I want this duck to quack; someone else will want it to
> > waddle. I don't see how type hints could reasonably support "file
> > like object" in the duck type sense (unless the consumer were to
> > specify the exact attributes of the duck it's interested in, which
> > I fear would become a tedious type writing style). '
> > 
> 
> 
> This approach (a Protocol that declares exactly what you need the
> file-like object to do) is in fact what we've been doing in typeshed,
> the repository that provides type hints for the standard library. For
> those unfamiliar, it would look something like this:
> 
> from typing import Protocol
> 
> class SupportsRead(Protocol):
>     def read(self) -> bytes: ...
> 
> def uses_a_file(f: SupportsRead) -> None:
>      f.read()
> 
> That's somewhat tedious, to be sure, but it is working duck typing.
>  
> > 
> > I too have sensed static typing driving the typing development
> > agenda in Python recently, causing other typing methods to take a
> > back seat, so to speak. I add my voice to those requesting Python
> > handle other typing methods.
> > 
> > Barring an innovation to allow a "subset" of a type to be declared
> > in a type hint, I would conclude that static typing and duck typing
> > are diametrically opposed. If we agree that both are valuable,
> > developers could build consensus on that point, and work to ensure
> > that one does not move forward at the expense of the other.
> > 
> 
> What would you suggest? Should the syntax for declaring Protocols be
> more concise?
>  
> > 
> > Paul
> > 
> > On Wed, 2021-04-21 at 12:36 -0700, Christopher Barker wrote:
> > > Thanks Mark for posting this. I know some of us are uneasy about
> > > the pace of the typing train 
> > > 
> > > On Tue, Apr 20, 2021 at 11:20 AM Nathaniel Smith 
> > > wrote:
> > > > > If you guarded your code with `isinstance(foo, Sequence)`
> > > > then I could
> > > > > not use it with my `Foo` even if my `Foo` quacked like a
> > > > sequence. I was
> > > > > forced to use nominal typing; inheriting from Sequence, or
> > > > explicitly
> > > > > registering as a Sequence.
> > > > 
> > > > You say this like it's a bad thing, but how is this avoidable,
> > > > even in
> > > > principle? Structural typing lets you check whether Foo is
> > > > duck-shaped
> > > > -- has appropriate attribute names, etc. But quacking like a
> > > > duck is
> > > > harder: you also have to implement the Sequence behavioral
> > > > contract,
> > > > and realistically the only way to know that is if the author of
> > > > Foo
> > > > tells you.
> > > > 
> > > 
> > > 
> > > But that's not what duck typing is (at least to me :-) ) For a
> > > given function, I need the passed in object to quack (and yes, I
> > > need that quack to sound like a duck) -- but I usually don't care
> > > whether that object waddles like a duck.
> > > 
> > > So yes, isinstance(obj, Sequence) is really the only way to know
> > > that obj is a Sequence in every important way -- but if you only
> > > need it to do one or two things like a Sequence, then you don't
> > > care.
> > > 
> > > And this is not uncommon -- I suspect it's very rare for a single
> > > function to use most of the methods of a given ABC (or protocol,
> > > or whatever).
> > > 
> > > And a lot of the standard library works exactly this way. Two
> > > examples (chosen arbitrarily, I just happen to have thought about
> > > how they work):
> > > 
> > > json.load() simply calls ``fp.read()``, and passes the result on
> > > down to json.loads(). That's it -- no checking of anything.
> > > 
> > > If fp does not have a read() method, you get an AttributeError.
> > > If fp has a read() method, but it returns something other than a
> > > string, then you get some other Exception. And if it returns a
> > > string, but that string isn't valid JSON, you get yet another
> > > kind of error.
> > > 
> > > In short, json.load(fp, ...) requires fp to have a read() method
> > > that returns a valid JSON string. But it doesn't check, nor does
> > > it need to, if it's getting an actual io.TextIOBase object. Is
> > > that the right one? I'm not totally sure, which I kind of think
> > > makes my point -- I've been using "file-like" objects for years
> > > (decades) without worrying about it.
> > > 
> > > Example 2:
> > > 
> > > The str.translate method takes:

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Jelle Zijlstra]

El mié, 21 abr 2021 a las 15:30, Paul Bryan () escribió:

> As demonstrated, protocols don't get us there because duck typing isn't a
> matter of having an object exhibit all of the attributes of a duck, but
> rather some subset of attributes to be used by the consumer. I want this
> duck to quack; someone else will want it to waddle. I don't see how type
> hints could reasonably support "file like object" in the duck type sense
> (unless the consumer were to specify the exact attributes of the duck it's
> interested in, which I fear would become a tedious type writing style). '
>

This approach (a Protocol that declares exactly what you need the file-like
object to do) is in fact what we've been doing in typeshed, the repository
that provides type hints for the standard library. For those unfamiliar, it
would look something like this:

from typing import Protocol

class SupportsRead(Protocol):
def read(self) -> bytes: ...

def uses_a_file(f: SupportsRead) -> None:
 f.read()

That's somewhat tedious, to be sure, but it is working duck typing.


>
> I too have sensed static typing driving the typing development agenda in
> Python recently, causing other typing methods to take a back seat, so to
> speak. I add my voice to those requesting Python handle other typing
> methods.
>
> Barring an innovation to allow a "subset" of a type to be declared in a
> type hint, I would conclude that static typing and duck typing are
> diametrically opposed. If we agree that both are valuable, developers could
> build consensus on that point, and work to ensure that one does not move
> forward at the expense of the other.
>
What would you suggest? Should the syntax for declaring Protocols be more
concise?


>
> Paul
>
> On Wed, 2021-04-21 at 12:36 -0700, Christopher Barker wrote:
>
> Thanks Mark for posting this. I know some of us are uneasy about the pace
> of the typing train 
>
> On Tue, Apr 20, 2021 at 11:20 AM Nathaniel Smith  wrote:
>
> > If you guarded your code with `isinstance(foo, Sequence)` then I could
> > not use it with my `Foo` even if my `Foo` quacked like a sequence. I was
> > forced to use nominal typing; inheriting from Sequence, or explicitly
> > registering as a Sequence.
>
> You say this like it's a bad thing, but how is this avoidable, even in
> principle? Structural typing lets you check whether Foo is duck-shaped
> -- has appropriate attribute names, etc. But quacking like a duck is
> harder: you also have to implement the Sequence behavioral contract,
> and realistically the only way to know that is if the author of Foo
> tells you.
>
>
> But that's not what duck typing is (at least to me :-) ) For a given
> function, I need the passed in object to quack (and yes, I need that quack
> to sound like a duck) -- but I usually don't care whether that object
> waddles like a duck.
>
> So yes, isinstance(obj, Sequence) is really the only way to know that obj
> is a Sequence in every important way -- but if you only need it to do one
> or two things like a Sequence, then you don't care.
>
> And this is not uncommon -- I suspect it's very rare for a single function
> to use most of the methods of a given ABC (or protocol, or whatever).
>
> And a lot of the standard library works exactly this way. Two examples
> (chosen arbitrarily, I just happen to have thought about how they work):
>
> json.load() simply calls ``fp.read()``, and passes the result on down to
> json.loads(). That's it -- no checking of anything.
>
> If fp does not have a read() method, you get an AttributeError. If fp has
> a read() method, but it returns something other than a string, then you get
> some other Exception. And if it returns a string, but that string isn't
> valid JSON, you get yet another kind of error.
>
> In short, json.load(fp, ...) requires fp to have a read() method that
> returns a valid JSON string. But it doesn't check, nor does it need to, if
> it's getting an actual io.TextIOBase object. Is that the right one? I'm not
> totally sure, which I kind of think makes my point -- I've been using
> "file-like" objects for years (decades) without worrying about it.
>
> Example 2:
>
> The str.translate method takes:
>
> "a mapping of Unicode ordinals to Unicode ordinals, strings, or None"
>
> Ok, then you need to pass in a Mapping, yes? Well, no you don't. The docs
> go on to say:
>
> The table must implement lookup/indexing via __getitem__, for instance a
> dictionary or list.
>
> Ah -- so we don't need a Mapping -- we need anything indexable by an
> integer that contains "ordinals, strings, or None". What the heck ABC could
> we use for that?
>
> The ABCs do have an already complex hierarchy of containers, but there is
> no "Indexable", (quacks) and certainly no "indexable and returns these
> particular things. (quacks a certain way). (maybe there's something in the
> typing module that would work for static typing -- I have no idea).
>
> I'm pretty sure this particular API was designed to accommodate the old
> 

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Paul Bryan]

As demonstrated, protocols don't get us there because duck typing isn't
a matter of having an object exhibit all of the attributes of a duck,
but rather some subset of attributes to be used by the consumer. I want
this duck to quack; someone else will want it to waddle. I don't see
how type hints could reasonably support "file like object" in the duck
type sense (unless the consumer were to specify the exact attributes of
the duck it's interested in, which I fear would become a tedious type
writing style). 

I too have sensed static typing driving the typing development agenda
in Python recently, causing other typing methods to take a back seat,
so to speak. I add my voice to those requesting Python handle other
typing methods.

Barring an innovation to allow a "subset" of a type to be declared in a
type hint, I would conclude that static typing and duck typing are
diametrically opposed. If we agree that both are valuable, developers
could build consensus on that point, and work to ensure that one does
not move forward at the expense of the other.

Paul

On Wed, 2021-04-21 at 12:36 -0700, Christopher Barker wrote:
> Thanks Mark for posting this. I know some of us are uneasy about the
> pace of the typing train 
> 
> On Tue, Apr 20, 2021 at 11:20 AM Nathaniel Smith 
> wrote:
> > > If you guarded your code with `isinstance(foo, Sequence)` then I
> > could
> > > not use it with my `Foo` even if my `Foo` quacked like a
> > sequence. I was
> > > forced to use nominal typing; inheriting from Sequence, or
> > explicitly
> > > registering as a Sequence.
> > 
> > You say this like it's a bad thing, but how is this avoidable, even
> > in
> > principle? Structural typing lets you check whether Foo is duck-
> > shaped
> > -- has appropriate attribute names, etc. But quacking like a duck
> > is
> > harder: you also have to implement the Sequence behavioral
> > contract,
> > and realistically the only way to know that is if the author of Foo
> > tells you.
> > 
> 
> 
> But that's not what duck typing is (at least to me :-) ) For a given
> function, I need the passed in object to quack (and yes, I need that
> quack to sound like a duck) -- but I usually don't care whether that
> object waddles like a duck.
> 
> So yes, isinstance(obj, Sequence) is really the only way to know that
> obj is a Sequence in every important way -- but if you only need it
> to do one or two things like a Sequence, then you don't care.
> 
> And this is not uncommon -- I suspect it's very rare for a single
> function to use most of the methods of a given ABC (or protocol, or
> whatever).
> 
> And a lot of the standard library works exactly this way. Two
> examples (chosen arbitrarily, I just happen to have thought about how
> they work):
> 
> json.load() simply calls ``fp.read()``, and passes the result on down
> to json.loads(). That's it -- no checking of anything.
> 
> If fp does not have a read() method, you get an AttributeError. If fp
> has a read() method, but it returns something other than a string,
> then you get some other Exception. And if it returns a string, but
> that string isn't valid JSON, you get yet another kind of error.
> 
> In short, json.load(fp, ...) requires fp to have a read() method that
> returns a valid JSON string. But it doesn't check, nor does it need
> to, if it's getting an actual io.TextIOBase object. Is that the right
> one? I'm not totally sure, which I kind of think makes my point --
> I've been using "file-like" objects for years (decades) without
> worrying about it.
> 
> Example 2:
> 
> The str.translate method takes:
> 
> "a mapping of Unicode ordinals to Unicode ordinals, strings, or None"
> 
> Ok, then you need to pass in a Mapping, yes? Well, no you don't. The
> docs go on to say:
> 
> The table must implement lookup/indexing via __getitem__, for
> instance a
> dictionary or list.
> 
> Ah -- so we don't need a Mapping -- we need anything indexable by an
> integer that contains "ordinals, strings, or None". What the heck ABC
> could we use for that?
> 
> The ABCs do have an already complex hierarchy of containers, but
> there is no "Indexable", (quacks) and certainly no "indexable and
> returns these particular things. (quacks a certain way). (maybe
> there's something in the typing module that would work for static
> typing -- I have no idea).
> 
> I'm pretty sure this particular API was designed to accommodate the
> old py2 str.translate, which took a length-256 sequence, while also
> accommodating full Unicode, which would have required a 2^32 length
> sequence to do the same thing :-)
> 
> But again -- this is duck typing, built into the stdlib, and it works
> just fine.
> 
> Granted, until PEP 563 (kind of) , there has been nothing that
> weakens or disallows such duck typing -- those of us that want to
> write fully duck-typed code can continue to do so.
> 
> But there is the "culture" of Python -- and it has been very much
> shifting toward more typing -- A recent publication (sorry can't 

[Python-Dev] Re: Keeping Python a Duck Typed Language

[David Antonini]

021 11:58:23 -0700
From: Yury Selivanov 
Subject: [Python-Dev] Re: PEP 654: Exception Groups and except*
[REPOST]
To: "srku...@mail.de" 
Cc: Irit Katriel , Irit Katriel
, python-dev , Yury
Selivanov 
Message-ID:

Content-Type: text/plain; charset="UTF-8"

On Wed, Apr 21, 2021 at 11:50 AM srku...@mail.de  wrote:
>
> Removing two concepts and preserving semantics simplifies the matter for 
> users. People need less to memorize and less to learn.
>
> Or am I missing something here? Couldn’t we achieve our goal without these 
> two new classes?

No, we can't. What you are proposing would make it very hard for users
to understand at a glance if what you have in an innocently looking
`except Exception` is correct or not.  In my async/await code I'd have
to always check the `__group__` attribute to make sure it's not an
exception group in disguise.

So while you're "simplifying" the proposal by removing a couple of
types, you're complicating it in all other places. Besides, I don't
think that adding the ExceptionGroup type is a controversial idea that
needs any simplification.

Yury

--

Message: 3
Date: Wed, 21 Apr 2021 12:36:34 -0700
From: Christopher Barker 
Subject: [Python-Dev] Re: Keeping Python a Duck Typed Language.
To: Nathaniel Smith 
Cc: Python Dev 
Message-ID:

Content-Type: multipart/alternative;
boundary="23a40905c080af6d"

Thanks Mark for posting this. I know some of us are uneasy about the pace
of the typing train 

On Tue, Apr 20, 2021 at 11:20 AM Nathaniel Smith  wrote:

> > If you guarded your code with `isinstance(foo, Sequence)` then I could
> > not use it with my `Foo` even if my `Foo` quacked like a sequence. I was
> > forced to use nominal typing; inheriting from Sequence, or explicitly
> > registering as a Sequence.
>
> You say this like it's a bad thing, but how is this avoidable, even in
> principle? Structural typing lets you check whether Foo is duck-shaped
> -- has appropriate attribute names, etc. But quacking like a duck is
> harder: you also have to implement the Sequence behavioral contract,
> and realistically the only way to know that is if the author of Foo
> tells you.
>

But that's not what duck typing is (at least to me :-) ) For a given
function, I need the passed in object to quack (and yes, I need that quack
to sound like a duck) -- but I usually don't care whether that object
waddles like a duck.

So yes, isinstance(obj, Sequence) is really the only way to know that obj
is a Sequence in every important way -- but if you only need it to do one
or two things like a Sequence, then you don't care.

And this is not uncommon -- I suspect it's very rare for a single function
to use most of the methods of a given ABC (or protocol, or whatever).

And a lot of the standard library works exactly this way. Two examples
(chosen arbitrarily, I just happen to have thought about how they work):

json.load() simply calls ``fp.read()``, and passes the result on down to
json.loads(). That's it -- no checking of anything.

If fp does not have a read() method, you get an AttributeError. If fp has a
read() method, but it returns something other than a string, then you get
some other Exception. And if it returns a string, but that string isn't
valid JSON, you get yet another kind of error.

In short, json.load(fp, ...) requires fp to have a read() method that
returns a valid JSON string. But it doesn't check, nor does it need to, if
it's getting an actual io.TextIOBase object. Is that the right one? I'm not
totally sure, which I kind of think makes my point -- I've been using
"file-like" objects for years (decades) without worrying about it.

Example 2:

The str.translate method takes:

"a mapping of Unicode ordinals to Unicode ordinals, strings, or None"

Ok, then you need to pass in a Mapping, yes? Well, no you don't. The docs
go on to say:

The table must implement lookup/indexing via __getitem__, for instance a
dictionary or list.

Ah -- so we don't need a Mapping -- we need anything indexable by an
integer that contains "ordinals, strings, or None". What the heck ABC could
we use for that?

The ABCs do have an already complex hierarchy of containers, but there is
no "Indexable", (quacks) and certainly no "indexable and returns these
particular things. (quacks a certain way). (maybe there's something in the
typing module that would work for static typing -- I have no idea).

I'm pretty sure this particular API was designed to accommodate the old py2
str.translate, which took a length-256 sequence, while also
accommodating full Unicode, which would have required a 2^32 length
sequence to do the same thing :-)

But again -- this is duck typing, built into the stdlib, and it works
just fine.

Granted, until PEP 563 (kind of) , there has been nothing

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Christopher Barker]

Thanks Mark for posting this. I know some of us are uneasy about the pace
of the typing train 

On Tue, Apr 20, 2021 at 11:20 AM Nathaniel Smith  wrote:

> > If you guarded your code with `isinstance(foo, Sequence)` then I could
> > not use it with my `Foo` even if my `Foo` quacked like a sequence. I was
> > forced to use nominal typing; inheriting from Sequence, or explicitly
> > registering as a Sequence.
>
> You say this like it's a bad thing, but how is this avoidable, even in
> principle? Structural typing lets you check whether Foo is duck-shaped
> -- has appropriate attribute names, etc. But quacking like a duck is
> harder: you also have to implement the Sequence behavioral contract,
> and realistically the only way to know that is if the author of Foo
> tells you.
>

But that's not what duck typing is (at least to me :-) ) For a given
function, I need the passed in object to quack (and yes, I need that quack
to sound like a duck) -- but I usually don't care whether that object
waddles like a duck.

So yes, isinstance(obj, Sequence) is really the only way to know that obj
is a Sequence in every important way -- but if you only need it to do one
or two things like a Sequence, then you don't care.

And this is not uncommon -- I suspect it's very rare for a single function
to use most of the methods of a given ABC (or protocol, or whatever).

And a lot of the standard library works exactly this way. Two examples
(chosen arbitrarily, I just happen to have thought about how they work):

json.load() simply calls ``fp.read()``, and passes the result on down to
json.loads(). That's it -- no checking of anything.

If fp does not have a read() method, you get an AttributeError. If fp has a
read() method, but it returns something other than a string, then you get
some other Exception. And if it returns a string, but that string isn't
valid JSON, you get yet another kind of error.

In short, json.load(fp, ...) requires fp to have a read() method that
returns a valid JSON string. But it doesn't check, nor does it need to, if
it's getting an actual io.TextIOBase object. Is that the right one? I'm not
totally sure, which I kind of think makes my point -- I've been using
"file-like" objects for years (decades) without worrying about it.

Example 2:

The str.translate method takes:

"a mapping of Unicode ordinals to Unicode ordinals, strings, or None"

Ok, then you need to pass in a Mapping, yes? Well, no you don't. The docs
go on to say:

The table must implement lookup/indexing via __getitem__, for instance a
dictionary or list.

Ah -- so we don't need a Mapping -- we need anything indexable by an
integer that contains "ordinals, strings, or None". What the heck ABC could
we use for that?

The ABCs do have an already complex hierarchy of containers, but there is
no "Indexable", (quacks) and certainly no "indexable and returns these
particular things. (quacks a certain way). (maybe there's something in the
typing module that would work for static typing -- I have no idea).

I'm pretty sure this particular API was designed to accommodate the old py2
str.translate, which took a length-256 sequence, while also
accommodating full Unicode, which would have required a 2^32 length
sequence to do the same thing :-)

But again -- this is duck typing, built into the stdlib, and it works
just fine.

Granted, until PEP 563 (kind of) , there has been nothing that weakens or
disallows such duck typing -- those of us that want to write fully
duck-typed code can continue to do so.

But there is the "culture" of Python -- and it has been very much shifting
toward more typing -- A recent publication (sorry can't find it now -- my
google fu is failing me) examined code on PyPi and found a lot of type
hints -- many of which were apparently not being used with a static type
checker. So why were they there?

And I've seen a lot more isinstance(Some_ABC) code lately as well.

>From looking at the work of my beginning students, I can tell that they are
seeing examples out there that use more typing, to the point that they
think it's a best practice (or maybe even required?). Maybe it is -- but if
the community is moving that way, we should be honest about it.


> I'm not even sure that this *is* nominal typing. You could just as
> well argue that "the operation `isinstance(..., Sequence)` returns
> `True`" is just another of the behavioral constraints that are
> required to quack like a sequence.
>

I'm not sure of the definition of "nominal" typing -- but it absolutely is
NOT duck typing (As Luciano pointed out, Alex Martelli coined the term
Goose Typing for this).

The big distinction is whether we want to know if the object is a duck, or
if we only need it to do one or two things like a duck.

-CHB


-- 
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Sebastian Rittau]

Am 20.04.2021 um 19:03 schrieb Mark Shannon:
PEP 544 supports structural typing, but to declare a structural type 
you must inherit from Protocol.

That smells a lot like nominal typing to me.


I'm not sure what inheriting from Protocol has to do with nominal 
typing, even though I would personally prefer to have a separate keyword 
for declaring protocols. But current typing philosophy is to prefer 
structural over nominal typing:


 * Use abstract types like "Iterable" or "Sequence" over concrete types
   list "list".
 * Use protocols instead of instances where it makes sense (although
   pragmatism very often means using an existing concrete class instead
   of defining a protocol).
 * We are slowly replacing typing.IO et al. with more restricted protocols.

Personally I think that the typing infrastructure should move even more 
towards structural typing than it does at the moment and there is 
certainly a lot of room to grow. But overall I feel that typing is 
moving towards allowing more duck typing than it does at the moment.


 - Sebastian

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4SDOHDJENVLNDZVJYZN47JSHWJMN4SHV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Luciano Ramalho]

Now, given the terms I used in the last e-mail, it does worry me that
the push towards supporting STATIC TYPING in Python sometimes ignores
the needs of people who use the other typing disciplines. For example,
Pydantic is an example of using type hints for GOOSE TYPING.

And it alwo worries me that many of those not concerned with STATIC
TYPING are not following closely the decisions being made in the 19
(and counting) typing PEPs., some of which affect everyone who uses
Python.

Here are two examples:


PEP 484 DISMISSES THE NUMBERS ABCs

Direct quote from section "The Numeric Tower" [0]:

"There are some issues with these ABCs".

[0] https://www.python.org/dev/peps/pep-0484/#the-numeric-tower

No further explanation is given. The PEP then proposes ad-hoc rules to
support int, float and complex, leaving users of the Numbers ABCs,
Fraction, Decimal, and the huge NumPy community in the dark as to how
to handle other numeric types.

Now, I understand that the Numeric tower, as defined, cannot be used
for STATIC TYPING because the root class—Number—has no methods. Also,
there are deeper arguments regarding the soundness of that hierarchy.
See for example "Bad Ideas In Type Theory" [1].

[1] https://www.yodaiken.com/2017/09/15/bad-ideas-in-type-theory/

However, I think a package in the standard library, and more
importantly, the users of that package, deserve more attention.

If PEP 484 et. al. wants to serve all the Python community, it must be
part of its mission to address the problem of the Numbers ABCs, either
fixing it, or deprecating it while proposing something better, but not
creating an ad-hoc rule for three built-in types, ignoring all others.


PEP 563 EMBRACES THEN DEPRECATES ALL OTHER USES OF ANNOTATIONS

Section "Non-typing usage of annotations" [2] of PEP 563 starts with:

"While annotations are still available for arbitrary use besides type
checking..."

but ends in a very different tone:

"uses for annotations incompatible with the aforementioned PEPs should
be considered deprecated."

[2] https://www.python.org/dev/peps/pep-0563/#non-typing-usage-of-annotations

I think if more eyes beyond the STATIC TYPING community were looking
at the PEP, that inconsistency would have been flagged and dealt with
in some way.


OUR PROBLEM

There is a rift that need to work to close in the Python community.

Some people invested in STATIC TYPING are involved with million-line
codebases where performance is paramount and the cost of bugs may be
very high, and that creates a big motivation to use Python in a more
constrained way. They are not paying attention to other uses of type
hints—which are no deviations at all, because PEP 3107 explicitly
encouraged people to experiment with the annotations. If that is
deprecated, more people should be part of the conversation—as we are
seeing now.

On the other hand, most people who were attracted to Python *because*
of the flexibility of DUCK TYPING and GOOSE TYPING perhaps don't
follow the typing PEPs so closely—there are many, and most are pretty
hard to read, by the nature of the subject matter.


A BEAUTIFUL BRIDGE

PEP 544 was a *huge* step forward to bridge the gap between static
typing and duck typing in the language. Thank you very much for all
who proposed and implemented it. That's the kind of bridge we need!

(The `SupportsFloat` and similar protocols introduced with PEP 544 are
a way to test number types when they are not limited to float, int and
complex. But they expose some inconsistencies, and we still should do
something about the Numbers ABCs, IMHO)


GOING FORWARD

I think the PEP 563 v. PyDantic et. al. case is a very good
opportunity for us to think of ways to engage in dialog with the
different parts of the Python community on how they use types in the
language, considering STATIC TYPING, DUCK TYPING, STATIC DUCK TYPING,
GOOSE TYPING, without dismissing any of these styles.

If typing.get_type_hints is offered as a bridge for runtime uses of
type hints, it must work well and its shortcomings must be documented
as part of the official Python documentation, so that everyone knows
how reliable they are.

And in the future, if people come up with some new way of handling
types, every new addition must be considered in light of existing
typing styles—after a good, broad conversation involving different
user bases.


Cheers,

Luciano



On Tue, Apr 20, 2021 at 7:35 PM Luciano Ramalho  wrote:
>
> I am not taking sides now, but I want to share with you a useful
> diagram to reason about typing support in Python.
>
> I struggled to explain what Python offers until I came up with this diagram:
>
> https://standupdev.com/wiki/doku.php?id=python_protocols#the_typing_map
>
> The Typing Map has two orthogonal axis:
>
> - when are types checked:
> -- runtime checking
> -- static checking
>
> - how are type checked:
> -- structural types
> -- nominal types
>
> The quadrants are informally labeled with the terms in ALL CAPS below.
>
> Traditionally, 

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Mark Shannon]

Hi Luciano,

On 20/04/2021 11:35 pm, Luciano Ramalho wrote:

I am not taking sides now, but I want to share with you a useful
diagram to reason about typing support in Python.

I struggled to explain what Python offers until I came up with this diagram:

https://standupdev.com/wiki/doku.php?id=python_protocols#the_typing_map


That's really nice, thanks.



The Typing Map has two orthogonal axis:

- when are types checked:
-- runtime checking
-- static checking

- how are type checked:
-- structural types
-- nominal types

The quadrants are informally labeled with the terms in ALL CAPS below.

Traditionally, mainstream languages supported one of two diagonally
opposite quadrants: STATIC TYPING and DUCK TYPING.

Now the situation is more complicated.

- Java supports only STATIC TYPING: static checking of nominal types;
Python started supporting nominal types with PEP 484

- Before ABCs, Python supported only DUCK TYPING: runtime checking of
structural types;

- With ABCs, Python started supporting GOOSE TYPING (a term invented
by Alex Martelli, in cc because I just quoted him): runtime checking
of nominal types (with subclass hook which is a backdoor to support
explicit checks on structural types as well);

- With PEP 544, Python started supporting STATIC DUCK TYPING: static
checking of structural types;

There are languages that support multiple quadrants:

- TypeScript, like Python, supports all four quadrants.

- Go supports STATIC TYPING, but it also famously popularized STATIC
DUCK TYPING, and even supports GOOSE TYPING with features like type
assertions and type switches [1] designed for explicit runtime
checking of nominal or structural types.

[1] https://tour.golang.org/methods/16

The Typing Map will be featured in my upcoming PyCon US talk [2]

[2] https://us.pycon.org/2021/schedule/presentation/80/

Cheers,

Luciano


PS. If you are aware of other languages that support more than one of
these quadrants, please let me know!




On Tue, Apr 20, 2021 at 6:53 PM Eric Casteleijn  wrote:




On Tue, Apr 20, 2021 at 10:03 AM Mark Shannon  wrote:


...
PEP 544 supports structural typing, but to declare a structural type you
must inherit from Protocol.
That smells a lot like nominal typing to me.



Note that to implement a protocol you do not have to inherit from anything. You 
create a structural type that subclasses Protocol, but then any object that 
satisfies that protocol can be passed where that type is expected, without 
having to inherit anything, so I would argue that this really is structural 
typing.


--
- eric casteleijn (he/him)
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/R3VP4KORAWI6KK4CNFL6JNYCATWR47EV/
Code of Conduct: http://python.org/psf/codeofconduct/





___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UXBP7AII3Z4Z7COK7P4OETAR7J5F5XP5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Luciano Ramalho]

I am not taking sides now, but I want to share with you a useful
diagram to reason about typing support in Python.

I struggled to explain what Python offers until I came up with this diagram:

https://standupdev.com/wiki/doku.php?id=python_protocols#the_typing_map

The Typing Map has two orthogonal axis:

- when are types checked:
-- runtime checking
-- static checking

- how are type checked:
-- structural types
-- nominal types

The quadrants are informally labeled with the terms in ALL CAPS below.

Traditionally, mainstream languages supported one of two diagonally
opposite quadrants: STATIC TYPING and DUCK TYPING.

Now the situation is more complicated.

- Java supports only STATIC TYPING: static checking of nominal types;
Python started supporting nominal types with PEP 484

- Before ABCs, Python supported only DUCK TYPING: runtime checking of
structural types;

- With ABCs, Python started supporting GOOSE TYPING (a term invented
by Alex Martelli, in cc because I just quoted him): runtime checking
of nominal types (with subclass hook which is a backdoor to support
explicit checks on structural types as well);

- With PEP 544, Python started supporting STATIC DUCK TYPING: static
checking of structural types;

There are languages that support multiple quadrants:

- TypeScript, like Python, supports all four quadrants.

- Go supports STATIC TYPING, but it also famously popularized STATIC
DUCK TYPING, and even supports GOOSE TYPING with features like type
assertions and type switches [1] designed for explicit runtime
checking of nominal or structural types.

[1] https://tour.golang.org/methods/16

The Typing Map will be featured in my upcoming PyCon US talk [2]

[2] https://us.pycon.org/2021/schedule/presentation/80/

Cheers,

Luciano


PS. If you are aware of other languages that support more than one of
these quadrants, please let me know!




On Tue, Apr 20, 2021 at 6:53 PM Eric Casteleijn  wrote:
>
>
>
> On Tue, Apr 20, 2021 at 10:03 AM Mark Shannon  wrote:
>>
>> ...
>> PEP 544 supports structural typing, but to declare a structural type you
>> must inherit from Protocol.
>> That smells a lot like nominal typing to me.
>
>
> Note that to implement a protocol you do not have to inherit from anything. 
> You create a structural type that subclasses Protocol, but then any object 
> that satisfies that protocol can be passed where that type is expected, 
> without having to inherit anything, so I would argue that this really is 
> structural typing.
>
>
> --
> - eric casteleijn (he/him)
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/R3VP4KORAWI6KK4CNFL6JNYCATWR47EV/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Luciano Ramalho
|  Author of Fluent Python (O'Reilly, 2015)
| http://shop.oreilly.com/product/0636920032519.do
|  Technical Principal at ThoughtWorks
|  Twitter: @ramalhoorg
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DYFZORZKWAB7BFCXMDPNTGSWH6IPPIRL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Eric Casteleijn]

On Tue, Apr 20, 2021 at 10:03 AM Mark Shannon  wrote:

> ...
> PEP 544 supports structural typing, but to declare a structural type you
> must inherit from Protocol.
> That smells a lot like nominal typing to me.
>

Note that to implement a protocol you do not have to inherit from anything.
You create a structural type that subclasses Protocol, but then any object
that satisfies that protocol can be passed where that type is expected,
without having to inherit anything, so I would argue that this really is
structural typing.


-- 
- eric casteleijn (he/him)
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/R3VP4KORAWI6KK4CNFL6JNYCATWR47EV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Larry Hastings]

On 4/20/21 10:03 AM, Mark Shannon wrote:
Then came PEP 563 and said that if you wanted to access the 
annotations of an object, you needed to call typing.get_type_hints() 
to get annotations in a meaningful form.

This smells a bit like enforced static typing to me.


I'm working to address this.  We're adding a new function to the 
standard library called inspect.get_annotations().  It's a lot less 
opinionated than typing.get_type_hints()--it simply returns the 
un-stringized annotations.  It also papers over some other awkward 
annotations behaviors, such as classes inheriting annotations from base 
classes.  I propose that it be the official new Best Practice for 
accessing annotations (as opposed to type hints, you should still use 
typing.get_type_hints() for that).


   https://bugs.python.org/issue43817

This will get checked in in time for Python 3.10b1.

Cheers,


//arry/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7TUTEDQ6MDKDV5VQCRLFUYI2XP3DIPDS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Nathaniel Smith]

On Tue, Apr 20, 2021 at 10:07 AM Mark Shannon  wrote:
>
> Hi everyone,
>
> Once upon a time Python was a purely duck typed language.
>
> Then came along abstract based classes, and some nominal typing starting
> to creep into the language.
>
> If you guarded your code with `isinstance(foo, Sequence)` then I could
> not use it with my `Foo` even if my `Foo` quacked like a sequence. I was
> forced to use nominal typing; inheriting from Sequence, or explicitly
> registering as a Sequence.

You say this like it's a bad thing, but how is this avoidable, even in
principle? Structural typing lets you check whether Foo is duck-shaped
-- has appropriate attribute names, etc. But quacking like a duck is
harder: you also have to implement the Sequence behavioral contract,
and realistically the only way to know that is if the author of Foo
tells you.

I'm not even sure that this *is* nominal typing. You could just as
well argue that "the operation `isinstance(..., Sequence)` returns
`True`" is just another of the behavioral constraints that are
required to quack like a sequence.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GKQV6MXBIDL3AISZE6TFPZBUSMAGPSKD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Chris Angelico]

On Wed, Apr 21, 2021 at 3:04 AM Mark Shannon  wrote:
> Then came type hints. PEP 484 explicitly said that type hints were
> optional and would *always* be optional.
>
> Then came along many typing PEPs that assumed that type hints would only
> used for static typing, making static typing a bit less optional.

How do they make type hints less optional?

> Then came PEP 563 and said that if you wanted to access the annotations
> of an object, you needed to call typing.get_type_hints() to get
> annotations in a meaningful form.
> This smells a bit like enforced static typing to me.

How?

> An extreme example of that is this:
>
> # Some magic code to mess with collections.abc.Sequence
>  >>> match {}:
> ... case []:
> ...print("WTF!")
> ...
> WTF!
>
> With duck typing this would be impossible (unless you use ctypes to mess
> with the dict object).

Why would you want to? A dict is not a sequence.

> So, lets stick to our promise that type hints will always be optional,
> and restore duck typing.
>
>
> I'm not suggesting that we get rid type hints and abstract base classes.
> They are popular for a reason.
> But let's treat them as useful tools, not warp the rest of the language
> to fit them.

So if you're not asking for them to be removed, then what ARE you
asking for? Is there any evidence that type hints will, in the future,
become mandatory?

I don't understand.

ChrisA
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5JW456T773BU5ZKKWBWIUELLIYDSR36J/
Code of Conduct: http://python.org/psf/codeofconduct/