Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Sun, Apr 26, 2015 at 2:01 AM, Ronan Lamy wrote: >>> * PEP484 hints are too high-level. Replacing an 'int' object with a >>> single machine word would be useful, but an 'int' annotation gives no >>> guarantee that it's correct (because Python 3 ints can have arbitrary >>> size and because subclasses of 'int' can override any operation to >>> invoke arbitrary code). >> >> >> Then create your own int16, uint64 etc types. > > > The PEP doesn't explain how to do that. And even if it's possible, such > types wouldn't be very useful as they're not stable under any arithmetic > operation (e.g. + doesn't necessarily fit in int16). If you define a function that adds two integers and want it to be optimized, it's going to have to check for overflow anyway. That said, though, I'd much rather a machine-word optimization be a feature of "int" than a special adornment that you give to some of your data. CPython 3.x doesn't do it, CPython 2.x kinda did it (an int would turn into a long if it got too big, but a long would never turn into an int), but MicroPython would be most welcome to do the job fully. (I would guess that PyPy already does this kind of thing.) Yes, that means integer addition can't become a single machine language instruction, but on the other hand, it means the Python code doesn't need to concern itself with any boundaries. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Le 25/04/15 04:07, Steven D'Aprano a écrit : On Sat, Apr 25, 2015 at 02:05:15AM +0100, Ronan Lamy wrote: * Hints have no run-time effect. The interpreter cannot assume that they are obeyed. I know what you mean, but just for the record, annotations are runtime inspectable, so people can (and probably have already started) to write runtime argument checking decorators or frameworks which rely on the type hints. * PEP484 hints are too high-level. Replacing an 'int' object with a single machine word would be useful, but an 'int' annotation gives no guarantee that it's correct (because Python 3 ints can have arbitrary size and because subclasses of 'int' can override any operation to invoke arbitrary code). Then create your own int16, uint64 etc types. The PEP doesn't explain how to do that. And even if it's possible, such types wouldn't be very useful as they're not stable under any arithmetic operation (e.g. + doesn't necessarily fit in int16). * A lot more information is needed to produce good code (e.g. “this f() called here really means this function there, and will never be monkey-patched” – same with len() or list(), btw). * Most of this information cannot easily be expressed as a type * If the interpreter gathers all that information, it'll probably have gathered a superset of what PEP484 can provide anyway. All this is a red herring. If type hints are useful to PyPy, that's a bonus. Cython uses its own system of type hints, a future version may be able to use PEP 484 hints instead. But any performance benefit is a bonus. PEP 484 is for increasing correctness, not speed. Yes, talking about performance in the context of PEP 484 is a red herring, that's what I'm saying. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 22 April 2015 at 03:03, Carol Willing wrote: > 2. Clearly, great thought has been put into this PEP. If anyone has a good > analysis of the potential impact on Python 3 adoption, please do pass along. > I would be interested in reading the information. I don't have hard data, but I do get to see quite a few of the challenges that dynamically typed languages (including Python) have scaling to large development teams. When you look at the way folks actually *use* languages like Java and C# in practice, you find that they're almost universally "tool mavens" (in Oliver Steele's sense: http://blog.osteele.com/posts/2004/11/ides/) where automated tools are taking care of most of the boilerplate for them. Find me a C# developer, and I'll bet you they're a Visual Studio user, find me a Java developer, and I'll bet you they're an Eclipse or IntelliJ user. This approach actually offers a lot of benefits in putting a "skill floor" under a development team - while you can't make people think, you can at least automatically rule out broad categories of mundane errors, and focus on the business logic of the problem you're aiming to solve. As a result, my main reaction to PEP 484 in a Python 3 adoption context is that "Python 3 offers all the agility and flexibility of Python 2, with all the structural assurances of Java or C#" is actually a huge selling point for anyone in an institutional context attempting to persuade their management chain to back a migration effort from Python 2 to Python 3. Another factor to consider here is that my understanding is that one of the *reasons* folks want better structural analysis (by annotating the Python 2 stdlib and key third part libraries in typeshed) is to help automate Python 2 -> Python 3 conversions in the absence of comprehensive test coverage. While to some degree this works against the previous point, there's a difference between having this as an addon vs having it as a standard feature (and unlike the network security changes and bundling pip, this is one where I'm entirely happy leaving it as the kind of carrot that can take pride of place in a corporate business case). The legitimate concerns that arise are around what happens to *community* code bases, including the standard library itself, as well as what folks are likely to see if they run "inspect.getsource()" on standard library components. For that, I think there's a lot of value in continuing to have explicit type hints be the exception rather than the rule in the upstream community, so the idea of the typeshed project is enormously appealing to me. If anyone doesn't want to deal with type hints themselves, but has a contributor that really wants to annotate their library, then "take it to typeshed" will hopefully become a recurring refrain :) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Sat, Apr 25, 2015 at 02:05:15AM +0100, Ronan Lamy wrote: > * Hints have no run-time effect. The interpreter cannot assume that they > are obeyed. I know what you mean, but just for the record, annotations are runtime inspectable, so people can (and probably have already started) to write runtime argument checking decorators or frameworks which rely on the type hints. > * PEP484 hints are too high-level. Replacing an 'int' object with a > single machine word would be useful, but an 'int' annotation gives no > guarantee that it's correct (because Python 3 ints can have arbitrary > size and because subclasses of 'int' can override any operation to > invoke arbitrary code). Then create your own int16, uint64 etc types. > * A lot more information is needed to produce good code (e.g. “this f() > called here really means this function there, and will never be > monkey-patched” – same with len() or list(), btw). > * Most of this information cannot easily be expressed as a type > * If the interpreter gathers all that information, it'll probably have > gathered a superset of what PEP484 can provide anyway. All this is a red herring. If type hints are useful to PyPy, that's a bonus. Cython uses its own system of type hints, a future version may be able to use PEP 484 hints instead. But any performance benefit is a bonus. PEP 484 is for increasing correctness, not speed. -- Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Fri, Apr 24, 2015 at 6:05 PM, Ronan Lamy wrote:
> Le 24/04/15 19:45, Paul Sokolovsky a écrit :
>
>> Hello,
>>
>> On Fri, 24 Apr 2015 18:27:29 +0100
>> Ronan Lamy wrote:
>>
>> PyPy's FAQ
> has an explanation of why type hints are not for performance.
>
> http://pypy.readthedocs.org/en/latest/faq.html#would-type-annotations-help-pypy-s-performance
>
You probably intended to write "why type hints are not for *PyPy's*
performance". There're many other language implementations and
modules for which it may be useful, please don't limit your
imagination by a single case.
>>>
>>> Those points apply to basically any compliant implementation of
>>> Python relying on speculative optimisation. Python is simply too
>>> dynamic for PEP484-style hints to provide any useful performance
>>> improvement targets.
>>>
>>
>> What's your point - saying that type annotations alone not enough to
>> achieve the best ("C-like") performance, which is true, or saying that
>> if they are alone not enough, then they are not needed at all, which
>> is ... strange ?
>>
>
> My point is that the arguments in the PyPy FAQ aren't actually specific to
> PyPy, and therefore that the conclusion, that hints are almost entirely
> useless if you’re looking at performance, holds in general.
> So let me restate these arguments in terms of a generic,
> performance-minded implementation of the full Python language spec:
>
> * Hints have no run-time effect. The interpreter cannot assume that they
> are obeyed.
> * PEP484 hints are too high-level. Replacing an 'int' object with a single
> machine word would be useful, but an 'int' annotation gives no guarantee
> that it's correct (because Python 3 ints can have arbitrary size and
> because subclasses of 'int' can override any operation to invoke arbitrary
> code).
> * A lot more information is needed to produce good code (e.g. “this f()
> called here really means this function there, and will never be
> monkey-patched” – same with len() or list(), btw).
> * Most of this information cannot easily be expressed as a type
> * If the interpreter gathers all that information, it'll probably have
> gathered a superset of what PEP484 can provide anyway.
I'm with the PyPy folks here -- I don't see any use for PEP 484 type hints
from a code generation perspective. Even if the hints were guaranteed to
be correct, the PEP 484 type system doesn't follow substitutability. I
don't mean that as a critique, I think it's a decision that makes it more
useful by keeping it in line with the majority of type usage in Python, but
it means that even if the hints are correct they don't really end up
providing any guarantees to the JIT.
>
>
> And speaking of PyPy, it really should think how to improve its
performance - not of generated programs, but of generation itself.
If compilation of a trivial program on a pumpy hardware takes 5
minutes and gigabytes of RAM and diskspace, few people will use it
for other purposes beyond curiosity. There's something very
un-Pythonic in waiting 5 mins just to run 10-line script. Type
hints can help here too ;-) (by not wasting resources propagating
types thru the same old standard library for example).
>>>
>>> Sorry, but that's nonsense. PyPy would be a seriously useless
>>> interpreter if running a 10-line script required such a lengthy
>>> compilation, so, obviously, that's not what happens.
>>>
>>> You seem to misunderstand what PyPy is: it's an interpreter with a
>>> just-in-time compiler, not a static compiler. It doesn't generate
>>> programs in any meaningful sense. Instead, it interprets the program,
>>> and when it detects a hot code path, it compiles it to machine code
>>> based on the precise types it sees. No resources are wasted on code
>>> that isn't actually executed.
>>>
>>
>> Regardless of whether I understood that meta-meta stuff, I just
>> followed couple of tutorials, each of them warning of memory and disk
>> space issues, and both running long to get results. Everyone else
>> following tutorials will get the same message I did - PyPy is a
>> slow-to-work-with bloat.
>>
>
> Ah, I suppose you're talking about the RPython tool chain, which is used
> to build PyPy. Though it's an interesting topic in itself (and is pretty
> much comparable to Cython wrt. type hints), it has about as much relevance
> to PyPy users as the inner workings of GCC have to CPython users.
>
> Well, the thing is that people don't seem to want to write PyPy tutorials,
> because it's boring. However, I can give you the definitive 3-line version:
> 1. Download and install PyPy [http://pypy.org/download.html]
> 2. Launch the 'pypy' executable.
> 3. Go read https://docs.python.org/2/tutorial/
>
> As for uber-meta stuff PyPy offers - I'm glad that's all done in
>> my favorite language, leaving all other languages behind. I'm saddened
>> there's no mundane JIT or static compiler usable and accepted by all
>>
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Le 24/04/15 19:45, Paul Sokolovsky a écrit :
Hello,
On Fri, 24 Apr 2015 18:27:29 +0100
Ronan Lamy wrote:
PyPy's FAQ
has an explanation of why type hints are not for performance.
http://pypy.readthedocs.org/en/latest/faq.html#would-type-annotations-help-pypy-s-performance
You probably intended to write "why type hints are not for *PyPy's*
performance". There're many other language implementations and
modules for which it may be useful, please don't limit your
imagination by a single case.
Those points apply to basically any compliant implementation of
Python relying on speculative optimisation. Python is simply too
dynamic for PEP484-style hints to provide any useful performance
improvement targets.
What's your point - saying that type annotations alone not enough to
achieve the best ("C-like") performance, which is true, or saying that
if they are alone not enough, then they are not needed at all, which
is ... strange ?
My point is that the arguments in the PyPy FAQ aren't actually specific
to PyPy, and therefore that the conclusion, that hints are almost
entirely useless if you’re looking at performance, holds in general.
So let me restate these arguments in terms of a generic,
performance-minded implementation of the full Python language spec:
* Hints have no run-time effect. The interpreter cannot assume that they
are obeyed.
* PEP484 hints are too high-level. Replacing an 'int' object with a
single machine word would be useful, but an 'int' annotation gives no
guarantee that it's correct (because Python 3 ints can have arbitrary
size and because subclasses of 'int' can override any operation to
invoke arbitrary code).
* A lot more information is needed to produce good code (e.g. “this f()
called here really means this function there, and will never be
monkey-patched” – same with len() or list(), btw).
* Most of this information cannot easily be expressed as a type
* If the interpreter gathers all that information, it'll probably have
gathered a superset of what PEP484 can provide anyway.
And speaking of PyPy, it really should think how to improve its
performance - not of generated programs, but of generation itself.
If compilation of a trivial program on a pumpy hardware takes 5
minutes and gigabytes of RAM and diskspace, few people will use it
for other purposes beyond curiosity. There's something very
un-Pythonic in waiting 5 mins just to run 10-line script. Type
hints can help here too ;-) (by not wasting resources propagating
types thru the same old standard library for example).
Sorry, but that's nonsense. PyPy would be a seriously useless
interpreter if running a 10-line script required such a lengthy
compilation, so, obviously, that's not what happens.
You seem to misunderstand what PyPy is: it's an interpreter with a
just-in-time compiler, not a static compiler. It doesn't generate
programs in any meaningful sense. Instead, it interprets the program,
and when it detects a hot code path, it compiles it to machine code
based on the precise types it sees. No resources are wasted on code
that isn't actually executed.
Regardless of whether I understood that meta-meta stuff, I just
followed couple of tutorials, each of them warning of memory and disk
space issues, and both running long to get results. Everyone else
following tutorials will get the same message I did - PyPy is a
slow-to-work-with bloat.
Ah, I suppose you're talking about the RPython tool chain, which is used
to build PyPy. Though it's an interesting topic in itself (and is pretty
much comparable to Cython wrt. type hints), it has about as much
relevance to PyPy users as the inner workings of GCC have to CPython users.
Well, the thing is that people don't seem to want to write PyPy
tutorials, because it's boring. However, I can give you the definitive
3-line version:
1. Download and install PyPy [http://pypy.org/download.html]
2. Launch the 'pypy' executable.
3. Go read https://docs.python.org/2/tutorial/
As for uber-meta stuff PyPy offers - I'm glad that's all done in
my favorite language, leaving all other languages behind. I'm saddened
there's no mundane JIT or static compiler usable and accepted by all
community - many other languages have that.
This all goes pretty offtopic wrt to the original discussion, so again,
what's your point - you say that all these things can't be done in
Python, or there's no need for it to be done? That people should look
somewhere else? I submitted a bug to jinja2 project and posted message
on its mailing list - I didn't get reply for 3 months. Why? Because its
maintainer went hacking another language, how was it called, alGOl, or
something. You want me and other folks to go too? Sorry, I'm staying so
far, and keep dreaming of better Python's future (where for example if
I need to get more performance from existing app, I can gradually
optimize it based on need, not rewrite it in another language or be
hitting "not implemented" in uber-meta stuff).
"If
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Hello,
On Fri, 24 Apr 2015 18:27:29 +0100
Ronan Lamy wrote:
> Also ask why no one used type specifier, they are possible since
> Python 3.0 ?
> Because it is the wrong way for Python.
> >>>
> >>> That's an example of how perceptions differ. In my list,
> >>> everyone(*) uses them - MyPy, MicroPython, etc. Even more should
> >>> use them (any JIT module, which are many), but sit in the bushes,
> >>> waiting for a kick, like PEP484 provides.
> >>
> >> It's OK that type hints are only to assist the programmer.
> >
> > Yes, it's OK to have a situation where type hints assist only a
> > programmer. It's not OK to think that type hints may be useful only
> > for programmer, instead of bunch more purposes, several of which
> > were already shown in the long previous discussion.
> >
> >> PyPy's FAQ
> >> has an explanation of why type hints are not for performance.
> >> http://pypy.readthedocs.org/en/latest/faq.html#would-type-annotations-help-pypy-s-performance
> >
> > You probably intended to write "why type hints are not for *PyPy's*
> > performance". There're many other language implementations and
> > modules for which it may be useful, please don't limit your
> > imagination by a single case.
>
> Those points apply to basically any compliant implementation of
> Python relying on speculative optimisation. Python is simply too
> dynamic for PEP484-style hints to provide any useful performance
> improvement targets.
What's your point - saying that type annotations alone not enough to
achieve the best ("C-like") performance, which is true, or saying that
if they are alone not enough, then they are not needed at all, which
is ... strange ?
> > And speaking of PyPy, it really should think how to improve its
> > performance - not of generated programs, but of generation itself.
> > If compilation of a trivial program on a pumpy hardware takes 5
> > minutes and gigabytes of RAM and diskspace, few people will use it
> > for other purposes beyond curiosity. There's something very
> > un-Pythonic in waiting 5 mins just to run 10-line script. Type
> > hints can help here too ;-) (by not wasting resources propagating
> > types thru the same old standard library for example).
>
> Sorry, but that's nonsense. PyPy would be a seriously useless
> interpreter if running a 10-line script required such a lengthy
> compilation, so, obviously, that's not what happens.
>
> You seem to misunderstand what PyPy is: it's an interpreter with a
> just-in-time compiler, not a static compiler. It doesn't generate
> programs in any meaningful sense. Instead, it interprets the program,
> and when it detects a hot code path, it compiles it to machine code
> based on the precise types it sees. No resources are wasted on code
> that isn't actually executed.
Regardless of whether I understood that meta-meta stuff, I just
followed couple of tutorials, each of them warning of memory and disk
space issues, and both running long to get results. Everyone else
following tutorials will get the same message I did - PyPy is a
slow-to-work-with bloat.
As for uber-meta stuff PyPy offers - I'm glad that's all done in
my favorite language, leaving all other languages behind. I'm saddened
there's no mundane JIT or static compiler usable and accepted by all
community - many other languages have that.
This all goes pretty offtopic wrt to the original discussion, so again,
what's your point - you say that all these things can't be done in
Python, or there's no need for it to be done? That people should look
somewhere else? I submitted a bug to jinja2 project and posted message
on its mailing list - I didn't get reply for 3 months. Why? Because its
maintainer went hacking another language, how was it called, alGOl, or
something. You want me and other folks to go too? Sorry, I'm staying so
far, and keep dreaming of better Python's future (where for example if
I need to get more performance from existing app, I can gradually
optimize it based on need, not rewrite it in another language or be
hitting "not implemented" in uber-meta stuff).
--
Best regards,
Paul mailto:pmis...@gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Le 23/04/15 14:55, Paul Sokolovsky a écrit : Hello, On Thu, 23 Apr 2015 09:15:44 -0400 Daniel Holth wrote: [] Also ask why no one used type specifier, they are possible since Python 3.0 ? Because it is the wrong way for Python. That's an example of how perceptions differ. In my list, everyone(*) uses them - MyPy, MicroPython, etc. Even more should use them (any JIT module, which are many), but sit in the bushes, waiting for a kick, like PEP484 provides. It's OK that type hints are only to assist the programmer. Yes, it's OK to have a situation where type hints assist only a programmer. It's not OK to think that type hints may be useful only for programmer, instead of bunch more purposes, several of which were already shown in the long previous discussion. PyPy's FAQ has an explanation of why type hints are not for performance. http://pypy.readthedocs.org/en/latest/faq.html#would-type-annotations-help-pypy-s-performance You probably intended to write "why type hints are not for *PyPy's* performance". There're many other language implementations and modules for which it may be useful, please don't limit your imagination by a single case. Those points apply to basically any compliant implementation of Python relying on speculative optimisation. Python is simply too dynamic for PEP484-style hints to provide any useful performance improvement targets. And speaking of PyPy, it really should think how to improve its performance - not of generated programs, but of generation itself. If compilation of a trivial program on a pumpy hardware takes 5 minutes and gigabytes of RAM and diskspace, few people will use it for other purposes beyond curiosity. There's something very un-Pythonic in waiting 5 mins just to run 10-line script. Type hints can help here too ;-) (by not wasting resources propagating types thru the same old standard library for example). Sorry, but that's nonsense. PyPy would be a seriously useless interpreter if running a 10-line script required such a lengthy compilation, so, obviously, that's not what happens. You seem to misunderstand what PyPy is: it's an interpreter with a just-in-time compiler, not a static compiler. It doesn't generate programs in any meaningful sense. Instead, it interprets the program, and when it detects a hot code path, it compiles it to machine code based on the precise types it sees. No resources are wasted on code that isn't actually executed. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Thu, Apr 23, 2015 at 03:25:30PM +0100, Harry Percival wrote: > lol @ the fact that the type hints are breaking github's syntax highlighter > :) That just tells us that Github's syntax highlighter has been broken for over five years. Function annotations go back to Python 3.0, more than five years ago. The only thing which is new about type hinting is that we're adding a standard *use* for those annotations. I just tested a version of kwrite from 2005, ten years old, and it highlights the following annotated function perfectly: def func(a:str='hello', b:int=int(x+1)) -> None: print(a + b) Of course, I'm hoping that any decent type checker won't need the type hints. It should be able to infer from the default values that a is a string and b an int, and only require a type hint if you want to accept other types as well. (It should also highlight that a+b cannot succeed.) -- Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Wed, Apr 22, 2015 at 5:45 PM, Guido van Rossum wrote: > Given that even if Difference existed, and even if we had a predefined > type alias for Difference[Iterable[str], str], you' still have to remember > to mark up all those functions with that annotation. It almost sounds > simpler to just predefine this function: > > def make_string_list(a: Union[str, Iterable[str]]) -> Iterable[str]: > if isinstance(a, str): > return [a] > else: > return a > fair enough -- and I do indeed have that code in various places already. Somehow, I've always been uncomfortable with checking specifically for the str type -- guess I want everything to be fully duck-typable. But then I wouldn't be doing type hints, either, would I? -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Hello, On Thu, 23 Apr 2015 15:25:30 +0100 Harry Percival wrote: > lol @ the fact that the type hints are breaking github's syntax > highlighter :) What one can expect from software written in Ruby? ;-) -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
lol @ the fact that the type hints are breaking github's syntax highlighter :) On 23 April 2015 at 14:44, Paul Sokolovsky wrote: > Hello, > > On Thu, 23 Apr 2015 14:48:58 +0200 > Wolfgang Langner wrote: > > > Hello, > > > > On Thu, Apr 23, 2015 at 11:59 AM, Paul Sokolovsky > > wrote: > > > > > Hello, > > > > > > On Thu, 23 Apr 2015 10:43:52 +0200 > > > Wolfgang Langner wrote: > > > > > > [] > > > > > > > Also ask why no one used type specifier, they are possible since > > > > Python 3.0 ? > > > > Because it is the wrong way for Python. > > > > > > That's an example of how perceptions differ. In my list, everyone(*) > > > uses them - MyPy, MicroPython, etc. Even more should use them (any > > > JIT module, which are many), but sit in the bushes, waiting for a > > > kick, like PEP484 provides. > > > > > > > > > (*) Everyone of those who needs them. Otherwise, let's throw out > > > metaclasses - noone uses them. > > > > > > > > They are there to be used and won't go away. > > But for most Libraries out there no one used it. > > > > JIT (Numba), Cython and other compilers/tools doing optimization have > > their own syntax and needs. > > That's exactly what needs to change, and where this PEP helps, as was > already pointed out: > http://code.activestate.com/lists/python-dev/135659/ > > > They need even more information like i32, i64, different floats and > > so on. > > That's poor excuse for not trying to be a good member of Python > community (and work on standard type annotation syntax, instead of > digging own hole). > > > MyPy is really new and in development. And the docstring type > > spec way is still possible. > > Anything is possible. The talk is about what makes most of sense. > Docstrings were always arbitrary string designated at human's > consumption, how (ab)using them for type annotations is better than > having clean language-grammar syntax? > > > MicroPython uses one annotation for their inline assembler stuff not > > type hints. > > Here's how MicroPython uses type annotations: > > https://github.com/micropython/micropython/blob/master/tests/micropython/viper_ptr8_load.py > > > For the library there are no type hints for function definitions. > > > > > > Remark: I use Metaclasses, seldom but if needed very useful. :-) > > > > > > -- > > bye by Wolfgang > > > > -- > Best regards, > Paul mailto:pmis...@gmail.com > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/hjwp2%40cantab.net > -- -- Harry J.W. Percival -- Twitter: @hjwp Mobile: +44 (0) 78877 02511 Skype: harry.percival ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Thu, Apr 23, 2015 at 9:55 AM, Paul Sokolovsky wrote: > Hello, > > On Thu, 23 Apr 2015 09:15:44 -0400 > Daniel Holth wrote: > > [] > >> >> Also ask why no one used type specifier, they are possible since >> >> Python 3.0 ? >> >> Because it is the wrong way for Python. >> > >> > That's an example of how perceptions differ. In my list, everyone(*) >> > uses them - MyPy, MicroPython, etc. Even more should use them (any >> > JIT module, which are many), but sit in the bushes, waiting for a >> > kick, like PEP484 provides. >> >> It's OK that type hints are only to assist the programmer. > > Yes, it's OK to have a situation where type hints assist only a > programmer. It's not OK to think that type hints may be useful only for > programmer, instead of bunch more purposes, several of which > were already shown in the long previous discussion. > >> PyPy's FAQ >> has an explanation of why type hints are not for performance. >> http://pypy.readthedocs.org/en/latest/faq.html#would-type-annotations-help-pypy-s-performance > > You probably intended to write "why type hints are not for *PyPy's* > performance". There're many other language implementations and modules > for which it may be useful, please don't limit your imagination by a > single case. > > And speaking of PyPy, it really should think how to improve its > performance - not of generated programs, but of generation itself. If > compilation of a trivial program on a pumpy hardware takes 5 minutes > and gigabytes of RAM and diskspace, few people will use it for other > purposes beyond curiosity. There's something very un-Pythonic in > waiting 5 mins just to run 10-line script. Type hints can help here > too ;-) (by not wasting resources propagating types thru the same old > standard library for example). Naturally, PyPy is very controversial. Type annotations can help to compile Python into a subset of Python. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Hello, On Thu, 23 Apr 2015 09:15:44 -0400 Daniel Holth wrote: [] > >> Also ask why no one used type specifier, they are possible since > >> Python 3.0 ? > >> Because it is the wrong way for Python. > > > > That's an example of how perceptions differ. In my list, everyone(*) > > uses them - MyPy, MicroPython, etc. Even more should use them (any > > JIT module, which are many), but sit in the bushes, waiting for a > > kick, like PEP484 provides. > > It's OK that type hints are only to assist the programmer. Yes, it's OK to have a situation where type hints assist only a programmer. It's not OK to think that type hints may be useful only for programmer, instead of bunch more purposes, several of which were already shown in the long previous discussion. > PyPy's FAQ > has an explanation of why type hints are not for performance. > http://pypy.readthedocs.org/en/latest/faq.html#would-type-annotations-help-pypy-s-performance You probably intended to write "why type hints are not for *PyPy's* performance". There're many other language implementations and modules for which it may be useful, please don't limit your imagination by a single case. And speaking of PyPy, it really should think how to improve its performance - not of generated programs, but of generation itself. If compilation of a trivial program on a pumpy hardware takes 5 minutes and gigabytes of RAM and diskspace, few people will use it for other purposes beyond curiosity. There's something very un-Pythonic in waiting 5 mins just to run 10-line script. Type hints can help here too ;-) (by not wasting resources propagating types thru the same old standard library for example). -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Hello, On Thu, 23 Apr 2015 14:48:58 +0200 Wolfgang Langner wrote: > Hello, > > On Thu, Apr 23, 2015 at 11:59 AM, Paul Sokolovsky > wrote: > > > Hello, > > > > On Thu, 23 Apr 2015 10:43:52 +0200 > > Wolfgang Langner wrote: > > > > [] > > > > > Also ask why no one used type specifier, they are possible since > > > Python 3.0 ? > > > Because it is the wrong way for Python. > > > > That's an example of how perceptions differ. In my list, everyone(*) > > uses them - MyPy, MicroPython, etc. Even more should use them (any > > JIT module, which are many), but sit in the bushes, waiting for a > > kick, like PEP484 provides. > > > > > > (*) Everyone of those who needs them. Otherwise, let's throw out > > metaclasses - noone uses them. > > > > > They are there to be used and won't go away. > But for most Libraries out there no one used it. > > JIT (Numba), Cython and other compilers/tools doing optimization have > their own syntax and needs. That's exactly what needs to change, and where this PEP helps, as was already pointed out: http://code.activestate.com/lists/python-dev/135659/ > They need even more information like i32, i64, different floats and > so on. That's poor excuse for not trying to be a good member of Python community (and work on standard type annotation syntax, instead of digging own hole). > MyPy is really new and in development. And the docstring type > spec way is still possible. Anything is possible. The talk is about what makes most of sense. Docstrings were always arbitrary string designated at human's consumption, how (ab)using them for type annotations is better than having clean language-grammar syntax? > MicroPython uses one annotation for their inline assembler stuff not > type hints. Here's how MicroPython uses type annotations: https://github.com/micropython/micropython/blob/master/tests/micropython/viper_ptr8_load.py > For the library there are no type hints for function definitions. > > > Remark: I use Metaclasses, seldom but if needed very useful. :-) > > > -- > bye by Wolfgang -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Thu, Apr 23, 2015 at 5:59 AM, Paul Sokolovsky wrote: > Hello, > > On Thu, 23 Apr 2015 10:43:52 +0200 > Wolfgang Langner wrote: > > [] > >> Also ask why no one used type specifier, they are possible since >> Python 3.0 ? >> Because it is the wrong way for Python. > > That's an example of how perceptions differ. In my list, everyone(*) > uses them - MyPy, MicroPython, etc. Even more should use them (any JIT > module, which are many), but sit in the bushes, waiting for a kick, like > PEP484 provides. It's OK that type hints are only to assist the programmer. PyPy's FAQ has an explanation of why type hints are not for performance. http://pypy.readthedocs.org/en/latest/faq.html#would-type-annotations-help-pypy-s-performance ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
* Wolfgang Langner [2015-04-23 10:43:52 +0200]: > 2. Using it in the language as part of the function signature, my first > thought was oh good, then I changed my mind >to: oh it can be very ugly and unreadable, it is the wrong place. >Now I am against it, best is, if I have to specify type signatures, do > it in one place, keep them up to date. >Most of the time this is the documentation. Why not use the docstring > with a standard type specifier for this. >Suggested here: > http://pydev.blogspot.de/2015/04/type-hinting-on-python.html While I happen to agree with you (but I'm happy with both variants really), I think that's a thing which has definitely been decided already. The idea is also listed in the PEP: https://www.python.org/dev/peps/pep-0484/#other-backwards-compatible-conventions Docstrings. There is an existing convention for docstrings, based on the Sphinx notation ( :type arg1: description ). This is pretty verbose (an extra line per parameter), and not very elegant. We could also make up something new, but the annotation syntax is hard to beat (because it was designed for this very purpose). > For nearly every function I have written, there is a docstring and most of > the time also a type specified. > But if I must provide all this in a second place it is not the right way to > go. Over time normally one place misses some changes and is wrong. It seems there's an extension for Sphinx already to use type annotations: https://pypi.python.org/pypi/sphinx-autodoc-annotation It seems to be older than PEP 484 (December 2013), so I hope it'll be updated or already work well with the ideas in the PEP. > Also ask why no one used type specifier, they are possible since Python 3.0 > ? > Because it is the wrong way for Python. Well, except that Sphinx extension, and MyPy, and MicroPython, and a thing which already exists for run-time type checking[1] and probably a whole lot more. The whole *reason* for PEP 484 (at least from my perspective) is to have a common base for the existing usages of type annotations. Florian [1] https://github.com/ceronman/typeannotations -- http://www.the-compiler.org | m...@the-compiler.org (Mail/XMPP) GPG: 916E B0C8 FD55 A072 | http://the-compiler.org/pubkey.asc I love long mails! | http://email.is-not-s.ms/ pgp0dRiwqbI7j.pgp Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Hello, On Thu, 23 Apr 2015 10:43:52 +0200 Wolfgang Langner wrote: [] > Also ask why no one used type specifier, they are possible since > Python 3.0 ? > Because it is the wrong way for Python. That's an example of how perceptions differ. In my list, everyone(*) uses them - MyPy, MicroPython, etc. Even more should use them (any JIT module, which are many), but sit in the bushes, waiting for a kick, like PEP484 provides. (*) Everyone of those who needs them. Otherwise, let's throw out metaclasses - noone uses them. > Regards, > > Wolfgang -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Hello, On Thu, Apr 23, 2015 at 11:59 AM, Paul Sokolovsky wrote: > Hello, > > On Thu, 23 Apr 2015 10:43:52 +0200 > Wolfgang Langner wrote: > > [] > > > Also ask why no one used type specifier, they are possible since > > Python 3.0 ? > > Because it is the wrong way for Python. > > That's an example of how perceptions differ. In my list, everyone(*) > uses them - MyPy, MicroPython, etc. Even more should use them (any JIT > module, which are many), but sit in the bushes, waiting for a kick, like > PEP484 provides. > > > (*) Everyone of those who needs them. Otherwise, let's throw out > metaclasses - noone uses them. > > They are there to be used and won't go away. But for most Libraries out there no one used it. JIT (Numba), Cython and other compilers/tools doing optimization have their own syntax and needs. They need even more information like i32, i64, different floats and so on. MyPy is really new and in development. And the docstring type spec way is still possible. MicroPython uses one annotation for their inline assembler stuff not type hints. For the library there are no type hints for function definitions. Remark: I use Metaclasses, seldom but if needed very useful. :-) -- bye by Wolfgang ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Hi, having a lot experience with Python beginners and people programming Java/Python I have also an opinion about this. ;-) First reaction was, oh good. Then I read every thread and comment about it, looked at a lot internal code give all some time and the result is: I found a lot of code written like Java with assert and isinstance for type checks. Not very Pythonic but someone from Java thinks this is good. Because Python has no type checks. A lot of people think types help in the IDE and for tools to do automatic checks, find some errors earlier. My conclusion: 1. typing.py is good to make a standard how to specify type hints. Can also be used by tools and IDE's 2. Using it in the language as part of the function signature, my first thought was oh good, then I changed my mind to: oh it can be very ugly and unreadable, it is the wrong place. Now I am against it, best is, if I have to specify type signatures, do it in one place, keep them up to date. Most of the time this is the documentation. Why not use the docstring with a standard type specifier for this. Suggested here: http://pydev.blogspot.de/2015/04/type-hinting-on-python.html 3. Stub files are good if someone needs this feature, it is complete separate and if someone don't want to write them it is no problem. But it is good to have a way to name them and a place to find them and know this. IDE's parse the docstring already, it is optional to have one and every Editor has folding support to disable it if it is to much of information. Checkers like mypy can use the type info from there to do checks. If there is a standard way to specify types in docstrings. It is also possible to specify a type hint for a variable, this is good for documentations and can also be used for type checks. For stub files the same can be used. So no different syntax. Also If someone wants it and a module has no documentation it can be added from there. For nearly every function I have written, there is a docstring and most of the time also a type specified. But if I must provide all this in a second place it is not the right way to go. Over time normally one place misses some changes and is wrong. I am convinced something like: def myfunction(a, b): """ My special function. :param a: ... :type a: int :param b: second value :type b: int :rtype: None """ or shorter form also possible in Sphinx if only one type for parameter is specified: def myfunction(a, b): """ My special function. :param int a: ... :param int b: second value :rtype: None """ Is easier to maintain good to read, changes not the world how to write Python functions and is useful for IDE's and checkers. And the best, most editors and IDE's support folding of docstrings and if someone don't want to see them it can be done now. The docstring is a good place for type specification, because as stated Python never will do type checks on execution. So no need to make it part of the language syntax. All this can also be applied over time to the standard library. If it makes sense so specify a type it is good to have it in the documentation. And sphinx is the standard documentation tool for Python now. Also ask why no one used type specifier, they are possible since Python 3.0 ? Because it is the wrong way for Python. Regards, Wolfgang ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 4/22/2015 8:45 PM, Guido van Rossum wrote: On Wed, Apr 22, 2015 at 2:38 PM, Chris Barker mailto:chris.bar...@noaa.gov>> wrote: On Tue, Apr 21, 2015 at 11:32 PM, Terry Reedy mailto:tjre...@udel.edu>> wrote: I was just thinking today that for this, typing needs a subtraction (difference) operation in addition to an addition (union) operation: Difference(Iterable(str), str) Anyway, the point is that being able to say "all these types, except this one" would solve this particular problem -- but would it solve any others? Do we want this to work around a quirk in Pythons string type? More seriously, I doubt there are other important use cases for Difference. I thought about Difference(numbers.Number, complex) to get ordered numbers, but numbers.Real should probably work. I agree more real uses are needed before adding Difference. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Wed, Apr 22, 2015 at 2:38 PM, Chris Barker wrote: > > > Oh wait, maybe it won't -- a string IS a sequence of strings. That's why >> this is an insidious bug in the first place. > > On Tue, Apr 21, 2015 at 11:32 PM, Terry Reedy wrote: > > >> I was just thinking today that for this, typing needs a subtraction >> (difference) operation in addition to an addition (union) operation: >> Difference(Iterable(str), str) >> > > Yup -- that might solve, it, but it feels a bit odd -- I can take any > Iterable of string, except a string. -- but what if there are others that > won't work??? But I guess that's the core of putting type hints on a > dynamic language. > > Still, I tend to think that this particular issue is really a limitation > with Python's type system -- nothing to do with type hinting. > > I can see that a character type seems useless in Python, but there are > lessons from other places: a numpy array is a collection of (usually) > numbers that can be treated as a single entity -- much like a string is a > collection of characters that is treated as a single entity -- in both > cases, it's core to convenience and performance to do that. But with numpy, > when you index an array, you get something back with one less dimension: > > index into a 3-d array, you get a 2-d array > index into a 2-d array, you get a 1-d array > index into a 1-d array, you get a scalar -- NOT a length-one 1-d array > > Sometimes this is a pain for generic code, but more often than not it's > critical to writing dynamic code -- not because you couldn't do the > operations you want, but because it's important to distinguish between a > scalar and an array that happens to have only one value. > > Anyway, the point is that being able to say "all these types, except this > one" would solve this particular problem -- but would it solve any others? > Do we want this to work around a quirk in Pythons string type? > > NOTE: I know full well that adding a character type to Python is not worth > it. > If you switch to bytes the problem goes away. :-P More seriously, I doubt there are other important use cases for Difference. Given that even if Difference existed, and even if we had a predefined type alias for Difference[Iterable[str], str], you' still have to remember to mark up all those functions with that annotation. It almost sounds simpler to just predefine this function: def make_string_list(a: Union[str, Iterable[str]]) -> Iterable[str]: if isinstance(a, str): return [a] else: return a and call this in those functions that have an Interable[str] argument. Now instead of getting errors for all the places where a caller mistakenly passes a single str, you've *fixed* all those call sites. Isn't that more Pythonic? :-) -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
> Oh wait, maybe it won't -- a string IS a sequence of strings. That's why > this is an insidious bug in the first place. On Tue, Apr 21, 2015 at 11:32 PM, Terry Reedy wrote: > I was just thinking today that for this, typing needs a subtraction > (difference) operation in addition to an addition (union) operation: > Difference(Iterable(str), str) > Yup -- that might solve, it, but it feels a bit odd -- I can take any Iterable of string, except a string. -- but what if there are others that won't work??? But I guess that's the core of putting type hints on a dynamic language. Still, I tend to think that this particular issue is really a limitation with Python's type system -- nothing to do with type hinting. I can see that a character type seems useless in Python, but there are lessons from other places: a numpy array is a collection of (usually) numbers that can be treated as a single entity -- much like a string is a collection of characters that is treated as a single entity -- in both cases, it's core to convenience and performance to do that. But with numpy, when you index an array, you get something back with one less dimension: index into a 3-d array, you get a 2-d array index into a 2-d array, you get a 1-d array index into a 1-d array, you get a scalar -- NOT a length-one 1-d array Sometimes this is a pain for generic code, but more often than not it's critical to writing dynamic code -- not because you couldn't do the operations you want, but because it's important to distinguish between a scalar and an array that happens to have only one value. Anyway, the point is that being able to say "all these types, except this one" would solve this particular problem -- but would it solve any others? Do we want this to work around a quirk in Pythons string type? NOTE: I know full well that adding a character type to Python is not worth it. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 21 April 2015 at 17:59, Guido van Rossum wrote: > For me, PEP 484 is a stepping stone. Among the authors of PEP 484 there was > much discussion about duck typing, and mypy even has some limited support > for duck typing (I think you can still find it by searching the mypy code > for "protocol"). But we ran out of time getting all the details written up > and agreed upon, so we decided to punt -- for now. But duck typing still > needs to have a way to talk about things like "seek method with this type > signature" (something like `def seek(self, offset: int, whence: > int=SEEK_SET) -> int`) so the current proposal gets us part of the way > there. > > The hope is that once 3.5 is out (with PEP 484's typing.py included > *provisional* mode) we can start working on the duck typing specification. > The alternative would have been to wait until 3.6, but we didn't think that > there would be much of an advantage to postponing the more basic type > hinting syntax (it would be like refusing to include "import" until you've > sorted out packages). During the run of 3.5 we'll hopefully get feedback on > where duck typing is most needed and how to specify it -- valuable input > that would be much harder to obtain of *no* part of the type hints notation > were standardized. This makes a lot of sense. If PEP 484 is intended to be a stepping stone (or compromise, or beta, or whatever word one wants to use), then it is easy to forgive it its limitations, and I'm looking forward to seeing it improve. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 21 April 2015 at 18:12, Steven D'Aprano wrote: > I expect that dealing with duck typing will be very high on the list > of priorities for the future. In the meantime, for this specific use-case, > you're probably not going to be able to statically check this type hint. > Your choices would be: > > - don't type check anything; To be clear, for the moment this is what requests will do, unless the other maintainers strongly disagree with me (they don't). I am quite convinced that PEP 484 is insufficiently powerful to make it worthwhile for requests to provide 'official' type hints. I suspect someone will provide hints to typeshed, and I certainly hope they're good, because if they're bad we'll definitely field bug reports about them (more on this in a different thread I think). ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 4/21/2015 6:41 PM, Chris Barker wrote: Well, it'll catch passing in a string instead of a sequence of strings -- one of teh common and semi-insidious type errors I see a lot (at least with newbies). Oh wait, maybe it won't -- a string IS a sequence of strings. That's why this is an insidioous bug in teh first place: List[string] will work, yes -- but now I'm locked down the user to an actual, list, yes? So I try: Iterable[string] ah yes -- now users can use tuples of strings, or a custom class, or -- but wait, darn it, a string IS an iterable of strings.. SIGH. I was just thinking today that for this, typing needs a subtraction (difference) operation in addition to an addition (union) operation: Difference(Iterable(str), str) -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 4/21/2015 6:50 PM, Chris Barker wrote: On Tue, Apr 21, 2015 at 11:58 AM, Paul Sokolovsky mailto:pmis...@gmail.com>> wrote: It does, and hope people won't be caught in "static typechecking" loop and consider other usages too. I an interested is using type hints for automatic or at least semi-automatic generation of tests, in particular for parameters hinted as generic and abstract. A parameter "ints: Iterable(int)" should be work with empty, length 1, and longer tuples, lists, sets, dicts (with int keys) and a custom class. Unless there is supplementary doc otherwise, it should handle negative, 0, and positive ints, small and large, in regular and random patterns. (Thinking about this more, some of this could be done without the PEP being approved, but there has to be *some* syntax for test generator input information, and using the standard vocabulary of typing instead of inventing something would be easier.) I"m confused -- from the bit I've been skimming the discussion, over on python-ideas, and now here, is that this is all about "static typechecking". The PEP is about a) defining type hints as the primary use of annotations* and b) adding a new module to standardize type hints instead of having multiple groups define multiple variations on the theme. The urgency of developing a standard now is that there *are* at least two or three independent efforts. * http://www.artima.com/weblogs/viewpost.jsp?thread=85551 (2004) makes it clear that Guido thought of static typing first and annotations second, in the form added to 3.0 years later, as an implementation thereof. The primary motivation of the people working on the new module is static typechecking by 3rd party programs. This seems to be Guido's primary interest and what he believes will be the primary use. It is also the primary promotional argument since type checking already exists. However, this does not stop others from having other motivations and making other uses of the information once available. It's not about run-time type checking. The PEP does not prevent that, and will make it more feasible by standardizing on one system. It's not about type-base performance optimization. Nothing is planned for CPython, but type hints are already being used by MicroPython for this purpose. (To me, this is one of the pluses of it having targeted 3.3.) It's not about any use of annotations other than types. Right, but there are multiple possible uses for type information. What is it about other than static typechecking? There is also documentation. Type hints are partially an alternative to stylized entries in doc strings. A program might combine the two or check consistency. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 22 April 2015 at 08:26, Guido van Rossum wrote: > In the end this should be up to you and the reviewers, but for such a > venerable module like unittest I'd be hesitant to be an early adopter. I'd > also expect that much of unittest is too dynamic in nature to benefit from > type hints. But maybe you should just try to use them for testtools and see > for yourself how beneficial or cumbersome they are in that particular case? Exactly yes. I've been experimenting recently with mypy to see. So far I've regressed backthrough 4 repos (unittest2, testtools, traceback2, linecache2) to get something small enough to work and experiment with. -Rob -- Robert Collins Distinguished Technologist HP Converged Cloud ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Cory Benfield "python-dev@python.org" On Tue, Apr 21, 2015 at 8:47 AM, Cory Benfield wrote: > I'm talking from the position of being a library author, where supporting > versions of Python lower than 3.5 will be a reality for at least 5 more years. > I will not be able to inline my type hints, so they'll have to go in > stub files, and now we've got the same problem: type hints can go out > of step just as easily as documentation can. I imagine authors interested in both type hinting and backwards compatibility could write their code using the 3.0+ annotation syntax, while using the various backwards compatibility packages and schemes, then "split" the source file using some inevitably-available tool to generate both annotation-free source and associated stub files. The author maintains (directly, at least) files on a 1-to-1 ratio -- the fact that they are split prior to distribution is transparent (much like generating documentation from source / comments). Running such a preprocessing step may not be ideal, but it is certainly an option that the motivated package author would seem to have, if they were so inclined. -- FWIW, my gut reaction was the same as many on the list (the syntax can get verbose and negatively impact readability), but I think we're only seeing one side of the eventual coin, looking at individual, sometimes specifically chosen examples. In theory, if this "takes off", IDEs could do a lot to help reduce the burden. Maybe they provide a "read-only" view of functions/methods where annotations are hidden until you put the caret inside the block, at which point the hints are revealed. Maybe even inside the block they're hidden, until you attempt to edit the arguments. Maybe they're just always gathered up by the IDE and accessible through some panel widget. Some of these approaches may sound helpful to some, others not so much, but the point is that there are a lot of creative ways that IDEs *could* help ease the transition, if type hinting becomes of significant interest to the community. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Thank you Jack. Jack: "I hate code and I want as little of it as possible in our product" I love that quote -- and I ALWAYS use it when I teach newbies Python. It's kind of the point of Python -- you can get a lot done by writing very little code. I'm still confused about what all this type annotation is about -- yes, I certainly see lots of advantages to static typing -- but also many disadvantages. And one of them is a bunch more stuff to read and write. But key is: If you like typing, use a statically typed language. If you want the advantages of dynamic typing -- then why do you want all the extra clutter and "safety"? And it seems to me that optional/partial typing buys you much of the hassle and restrictions, and essentially NONE of the safety (type casting pointers in C, anyone?). And I've had this debate with proponents of JAVA and the like a bunch -- sure, type checking will catch bugs in your code -- but they are almost always shallow bugs -- and bugs that if your tests didn't catch them, you have really lame tests! On the other had, you just can't get good performance doing low-level things with a dynamic language. So I use Cython, which, in a way, is Python with optional static typing. It gives you down to the metal performance, where, and only where, you need it, but not really much in the way of type checking. However, this is exactly the opposite of what everyone seem to be talking about using optional typing in Python for -- which is type checking ,but not any run-time optimizations -- in fact, AFAICT, not even run-time type checking. So a whole lot of clutter for very little gain. :-( NOTE: MyPy is out in the wild -- I'd be really interested to see how it all is really working out -- even on those "enterprise" code bases -- other than managers feeling better, are developers finding out that: "WOW -- this baby caught some nasty, bugs that I would not have found with testing -- and would have been hard to debug after the fact!" But in the end, I agree with the OP here -- stub files let pre-run-time static type checking happen without cluttering up Python for the rest of us -- so a nice compromise. So I guess we'll see. - Chris On Mon, Apr 20, 2015 at 4:41 PM, Jack Diederich wrote: > Twelve years ago a wise man said to me "I suggest that you also propose a > new name for the resulting language" > > I talked with many of you at PyCon about the costs of PEP 484. There are > plenty of people who have done a fine job promoting the benefits. > > * It is not optional. Please stop saying that. The people promoting it > would prefer that everyone use it. If it is approved it will be optional in > the way that PEP8 is optional. If I'm reading your annotated code it is > certainly /not/ optional that I understand the annotations. > > * Uploading stubs for other people's code is a terrible idea. Who do I > contact when I update the interface to my library? The random Joe who > "helped" by uploading annotations three months ago and then quit the > internet? I don't even want to think about people maliciously adding stubs > to PyPI. > > * The cognitive load is very high. The average function signature will > double in length. This is not a small cost and telling me it is "optional" > to pretend that every other word on the line doesn't exist is a farce. > > * Every company's style guide is about to get much longer. That in itself > is an indicator that this is a MAJOR language change and not just some > "optional" add-on. > > * People will screw it up. The same people who can't be trusted to program > without type annotations are also going to be *writing* those type > annotations. > > * Teaching python is about to get much less attractive. It will not be > optional for teachers to say "just pretend all this stuff over here doesn't > exist" > > * "No new syntax" is a lie. Or rather a red herring. There are lots of new > things it will be required to know and just because the compiler doesn't > have to change doesn't mean the language isn't undergoing a major change. > > If this wasn't in a PEP and it wasn't going to ship in the stdlib very few > people would use it. If you told everyone they had to install a different > python implementation they wouldn't. This is much worse than that - it is > Python4 hidden away inside a PEP. > > There are many fine languages that have sophisticated type systems. And > many bondage & discipline languages that make you type things three times > to make really really sure you meant to type that. If you find those other > languages appealing I invite you to go use them instead. > > -Jack > > https://mail.python.org/pipermail/python-dev/2003-February/033291.html > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/chris.barker%40noaa.gov > > -- Christopher Barker, Ph.D. Oceanographer
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 11:58 AM, Paul Sokolovsky wrote: > It does, and hope people won't be caught in "static typechecking" > loop and consider other usages too. > I"m confused -- from the bit I've been skimming the discussion, over on python-ideas, and now here, is that this is all about "static typechecking". It's not about run-time type checking. It's not about type-base performance optimization. It's not about any use of annotations other than types. What is it about other than static typechecking? -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 2:33 AM, Cory Benfield wrote: > It seems like the only place the type annotations will get used is in > relatively trivial cases where the types are obvious anyway. I don't > deny that *some* bugs will be caught, but I suspect they'll > overwhelmingly be crass ones that would have been caught quickly > anyway. Well, it'll catch passing in a string instead of a sequence of strings -- one of teh common and semi-insidious type errors I see a lot (at least with newbies). Oh wait, maybe it won't -- a string IS a sequence of strings. That's why this is an insidioous bug in teh first place: List[string] will work, yes -- but now I'm locked down the user to an actual, list, yes? So I try: Iterable[string] ah yes -- now users can use tuples of strings, or a custom class, or -- but wait, darn it, a string IS an iterable of strings.. SIGH. NOTE: I've thought for years that Python should have a character type for this reason, but that's a side note. Oh, and I'm a heavy numpy user. And, in fact, I write a lot of functions that are essentially statically typed -- i.e. they will only work with a Nx3 array of float64 for instance. However, in this case, I want run-time type checking, and I DON'T want the API to be statically typed, so I use something like: the_arr = np.asarray( input_object, dypte-np.float64).reshape(-1, 3) and in my docstring: :param input_object: the input data to compute something from :type input_object: (Nx3) numpy array or floats or somethign that can be turned into one. Is there any way for type hinting to help here??? - Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 1:18 PM, Robert Collins wrote: > On 22 April 2015 at 04:28, Guido van Rossum wrote: > > Until some point in a possible but distant future when we're all thinking > > back fondly about the argument we're currently having, it will be the > choice > > of the author of new (and *only* new) stdlib modules whether and how to > use > > type hints. Such a hypothetical author would also be reviewing updates to > > "their" module and point out lack of type hints just like you might point > > out an incomplete docstring, an outdated comment, or a missing test. (The > > type checker would be responsible for pointing out bugs. :-P ) > > What about major changes to existing modules? I have a backlog of > intended feature uplifts from testtools into unittest - if the type > hints thing works out I am likely to put them into testtools. Whats > your view on type hints to such *new code* in existing modules? > In the end this should be up to you and the reviewers, but for such a venerable module like unittest I'd be hesitant to be an early adopter. I'd also expect that much of unittest is too dynamic in nature to benefit from type hints. But maybe you should just try to use them for testtools and see for yourself how beneficial or cumbersome they are in that particular case? -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 22 April 2015 at 04:28, Guido van Rossum wrote: > On Tue, Apr 21, 2015 at 12:49 AM, Antoine Pitrou > wrote: >> >> On Mon, 20 Apr 2015 20:43:38 -0400 >> "R. David Murray" wrote: >> > +1 to this from me too. I'm afraid that means I'm -1 on the PEP. >> > >> > I didn't write this in my earlier email because I wasn't sure about it, >> > but my gut reaction after reading Harry's email was "if type annotations >> > are used in the stdlib, I'll probably stop contributing". That doesn't >> > mean that's *true*, but that's the first time I've ever had that >> > thought, so it is probably worth sharing. >> >> I think it would be nice to know what the PEP means for daily stdlib >> development. If patches have to carry typing information each time they >> add/enhance an API that's an addition burden. If typing is done >> separately by interested people then it sounds like it wouldn't have >> much of an impact on everyone else's workflow. > > > This point will be moot until new code appears in the stdlib whose author > likes type hints. As I said, we won't be converting existing code to add > type hints (I'm actively against that for the stdlib, for reasons I've > explained already). > > *If* type hints prove useful, I expect that adding type hints **to code that > deserves them** is treated no different in the workflow than adding tests or > docs. I.e. something that is the right thing to do because it has obvious > benefits for users and/or future maintainers. If at some point running a > type checker over the stdlib as part of continuous integration become > routine, type hints can also replace certain silly tests. > > Until some point in a possible but distant future when we're all thinking > back fondly about the argument we're currently having, it will be the choice > of the author of new (and *only* new) stdlib modules whether and how to use > type hints. Such a hypothetical author would also be reviewing updates to > "their" module and point out lack of type hints just like you might point > out an incomplete docstring, an outdated comment, or a missing test. (The > type checker would be responsible for pointing out bugs. :-P ) What about major changes to existing modules? I have a backlog of intended feature uplifts from testtools into unittest - if the type hints thing works out I am likely to put them into testtools. Whats your view on type hints to such *new code* in existing modules? -Rob -- Robert Collins Distinguished Technologist HP Converged Cloud ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
(Gmail messed up the attributions - apologies if I didn't fix them up correctly). 21 April 2015 at 19:55, Łukasz Langa wrote: >> >> On Apr 21, 2015, at 11:23 AM, Guido van Rossum wrote: >> >>> 2. Clearly, great thought has been put into this PEP. If anyone has a good >>> analysis of the potential impact on Python 3 adoption, please do pass along. >>> I would be interested in reading the information. >> >> I wish I had a crystal ball, but this is hard to predict. Anecdotally, some >> people believe this will be catnip, while others believe it to be poison. >> The truth will surely be somewhere in the middle. At this point we don't >> know what drives Python 3 adoption except time -- it's definitely going up. >> :-) >> > > Anecdotal evidence shows that some organizations perceive this feature as > one that justifies migration. Some of those organizations are pretty serious > about open-sourcing things. That makes me believe that by sheer volume of > the code they’re producing, Python 3 adoption will continue to increase. > > As Gregory Smith rightfully pointed out, nobody wants ugly code. I > understand why people are afraid of that and it warms my heart that they > are. The community cares so much about aesthetics and readability, it’s > great! > > We will evolve this over time. This will be a learning process for everybody > but we can’t learn to swim by only theorizing about it. We thought of the > evolving nature of the solution from Day 1, hence the *provisional* nature > of it. The wordy syntax is another example of that. Not requiring changes to > the interpreter and the standard library was very high on the list of > priorities. Once the *concept* proves itself, then we can improve on the > syntax. > > Acknowledging PEP 484 being just the second step^ in a long journey is why > some “obvious” parts are left out for now (hello, duck typing; hello, > multiple dispatch; hello, runtime type checks in unit tests; etc. etc.). > Those are often big enough to warrant their own PEPs. > > ^ PEP 3107 being the first. Thank you for this response. For some reason, it's reassured me a lot (I've no idea really why it struck a chord with me more than any of the other responses in the thread, just one of those things, I guess). Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, 21 Apr 2015 21:31:49 +0300, Paul Sokolovsky wrote: > On Tue, 21 Apr 2015 09:50:59 -0700 Ethan Furman wrote: > > > On 04/21, Paul Sokolovsky wrote: > > > > > > And for example yesterday's big theme was people blackmailing that > > > they stop contributing to stdlib if annotations are in [...] > > > > A volunteer's honest reaction is not blackmail, and categorizing it > > as such is not helpful to the discussion. > > Sure, that was rather humoresque note. Still, one may wonder why > "honest reaction" is like that, if from reading PEP484 it's clear that > it doesn't change status quo: https://www.python.org/dev/peps/pep-3107 > added annotations long ago, and PEP484 just provides default But what concerned me as a Python core developer was the perception that as a core developer I would have to deal with type hints and type checking in the stdlib, because there was talk of including type hints for the stdlib (as stub files) in 3.6. *That* was the source of my concern, which is reduced by the statement that we'll only need to think about type checking in the stdlib once the tools are *clearly* adding value, in *our* (the stdlib module maintainers') opinion, so that we *want* to use the type hints. The discussion of stub files being maintained by interested parties in a separate repository is also helpful. Again, that means that wider adoption should mostly only happen if the developers see real benefit. I still dislike the idea of having to read type hints (I agree with whoever it was said a well written docstring is more helpful than abstract type hints when reading Python code), but perhaps I will get used to it if/when it shows up in libraries I use. I think it would serve your goal better if instead of dismissing concerns you were to strive to understand them and then figure out how to allay them. Overall, I suspect that those who are doubtful are put off by the "rah rah this is great you are fools for not wanting to use it" tone of (some) of the proponents (which makes us think this is going to get pushed on us willy-nilly), whereas hearing a consistent message that "this will *enable* interested parties to collaborate on type checking without impacting the way the rest of you code *unless it proves its value to you*" would be better received. Guido has done the latter, IMO, as have some others. --David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 21 April 2015 at 17:55, Gregory P. Smith wrote: > I view most of this thread as FUD. The fear is understandable, I'm trying to > tell people to stop panicing. I think (hope!) everyone is clear that what's being expressed in this thread is honest (emotional) reactions. There's a negative connotation to the term FUD that's uncomfortable in this context, although it's understandable that the authors and supporters of the PEP feel frustrated about the feedback, so merely using terms with negative connotations is a pretty measured response, actually :-) We've got people expressing fear and others telling them not to panic. Realistically, just being told not to panic won't help. What will help is feeling that the fears are understood and being considered, even if ultimately the response is "we think it'll be alright - wait and see". Most people contributing to this thread have probably not been involved in the earlier discussions (I know I tuned out of the threads on python-ideas) so are now reacting at short notice to what looks like a big change. Things should calm down in due course, but "what the heck is all this?" reactions are to be expected. And honestly, the PEP is pretty difficult to read. That's probably the nature of the subject (my eyes glaze over when reading about type theory) but it doesn't help. When I read the PEP, after a few sections I found myself losing focus because I kept thinking "but what about X?" Result - I didn't manage to read the whole PEP and came away with a feeling that all those things I thought of "couldn't be handled". Not a positive reaction, unfortunately. That's the fault of the reader rather than the PEP, but maybe the PEP could include a "common concerns/FAQ" section, with discussions of the issues people are worried about? One final point. The overwhelming feeling I'm getting about the debate is that the main response to people with concerns is "you don't have to use it" and "it's optional". But that's not the point - we're talking past each other to an extent. I (and I presume most others) understand the fact that type hints are optional. We may or may not use them ourselves (I'm not actually ruling out that I might end up liking them!) But that's not the concern - what we're trying to understand is how we, as a community of programmers, and an ecosystem of interdependent codebases, deal with the possibility that there could be two differing philosophies (to hint or not to hint) trying to work together. Maybe it'll be a non-event, and we carry on as usual. Maybe nobody will use types in public code for years, because we all end up having to support 2.x whether we want to or not. Maybe we shouldn't try to solve big social issues like that and we should just trust in the fact that Python has a great community who *will* find a way to work together. I don't know, but that's my real question, and "it's optional" isn't really the response I'm looking for. (The response I probably deserve is likely "don't take it all so seriously", and it's probably fair. I apologise - it's been that sort of day :-)) Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Hello, On Tue, 21 Apr 2015 12:17:01 -0400 "R. David Murray" wrote: > On Tue, 21 Apr 2015 18:27:50 +0300, Paul Sokolovsky > wrote: > > > I was replying to Steven's message. Did you read it? > > > > Yes. And I try to follow general course of discussion, as its hard > > to follow individual sub-threads. And for example yesterday's big > > theme was people blackmailing that they stop contributing to stdlib > > if annotations are in, and today's theme appear to be people > > telling that static type checking won't be useful. And just your > > reply to Steven was a final straw which prompted me to point out > > that static type checking is not a crux of it, but just one (not > > the biggest IMHO) usage. > > Please be respectful rather than inflammatory. If you read what I > wrote, I did not say that I was going to stop contributing, I > specifically talked about that gut reaction being both emotional and > illogical. Sure, and I didn't mean you personally - your original message conveyed that it was "gut feeling" very well. More like following to your message, where people gave "+1". To a gut feeling? To the thought of stopping contribution? I dunno, and that "blackmail" note wasn't intended to be more serious than "burn the witch" and other funny stylistic devices which already appeared and make the discussion a bit more lively ;-). > That doesn't make the reaction any less real, and the fact > that such reactions exist is a data point you should consider in > conducting your PR campaign for this issue. (I don't mean that last > as a negative: this issue *requires* an honest PR campaign.) It does, and hope people won't be caught in "static typechecking" loop and consider other usages too. > > --David -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
> On Apr 21, 2015, at 11:23 AM, Guido van Rossum wrote: > > 2. Clearly, great thought has been put into this PEP. If anyone has a good > analysis of the potential impact on Python 3 adoption, please do pass along. > I would be interested in reading the information. > > I wish I had a crystal ball, but this is hard to predict. Anecdotally, some > people believe this will be catnip, while others believe it to be poison. The > truth will surely be somewhere in the middle. At this point we don't know > what drives Python 3 adoption except time -- it's definitely going up. :-) Anecdotal evidence shows that some organizations perceive this feature as one that justifies migration. Some of those organizations are pretty serious about open-sourcing things. That makes me believe that by sheer volume of the code they’re producing, Python 3 adoption will continue to increase. As Gregory Smith rightfully pointed out, nobody wants ugly code. I understand why people are afraid of that and it warms my heart that they are. The community cares so much about aesthetics and readability, it’s great! We will evolve this over time. This will be a learning process for everybody but we can’t learn to swim by only theorizing about it. We thought of the evolving nature of the solution from Day 1, hence the *provisional* nature of it. The wordy syntax is another example of that. Not requiring changes to the interpreter and the standard library was very high on the list of priorities. Once the *concept* proves itself, then we can improve on the syntax. Acknowledging PEP 484 being just the second step^ in a long journey is why some “obvious” parts are left out for now (hello, duck typing; hello, multiple dispatch; hello, runtime type checks in unit tests; etc. etc.). Those are often big enough to warrant their own PEPs. ^ PEP 3107 being the first. -- Lukasz Langa | Facebook Production Engineer | Global Consistency (+1) 650-681-7811___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 2:23 PM, Guido van Rossum wrote: > At least nobody will be writing type hints in Cyrillic. :-) Why not? It works just fine: >>> Список = list >>> def sum(x: Список): ... pass ... >>> (See https://en.wikipedia.org/wiki/Rapira for some prior art.) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Hello, On Tue, 21 Apr 2015 09:50:59 -0700 Ethan Furman wrote: > On 04/21, Paul Sokolovsky wrote: > > > > And for example yesterday's big theme was people blackmailing that > > they stop contributing to stdlib if annotations are in [...] > > A volunteer's honest reaction is not blackmail, and categorizing it > as such is not helpful to the discussion. Sure, that was rather humoresque note. Still, one may wonder why "honest reaction" is like that, if from reading PEP484 it's clear that it doesn't change status quo: https://www.python.org/dev/peps/pep-3107 added annotations long ago, and PEP484 just provides default semantics for them. Note that *default*, not the "only". PEP484 is full of conditions and long transition windows. Did PEP3107 shutter everyone's world? No. And PEP484 is nothing but a logical continuation of PEP3107, coming forward with real use for annotations, but not intended to shutter everyone's world. Well, hopefully Guido now clarified what was already written PEP484 (or not written, like nowhere it says "we add requirement of mandatory typehints in version 3.x [of stdlib or otherwise]"). But now people try to come up with *anti*-patterns on how to use type annotation and argue that these anti-patterns are not useful. Surely, annotations are useful in some places and not useful in other. requests doesn't need them? Good. But it's quite useful to annotate FFT routines and subroutines as taking arrays of floats, we can't get "faster than C"(tm) without that, like some other languages did (or claim to have done, and now look attractive). (You don't do FFT in Python? OMG, that's old.) -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 10:03 AM, Carol Willing < willi...@willingconsulting.com> wrote: > > Two areas of clarification would be helpful for me: > > 1. Optional: What does this really mean in practice? Am I opting in to > static type checking and type hints? Or must I opt out of type hints? > Having to opt out would probably put more burden on the educational use > cases than opting in would for a large corporate project. > You're definitely opting in. You must do two things to opt in: (a) add type hints to some of your signatures; (b) run a type checker over your code. You will only get the benefits of type hints of you do both, and there is no pressure to opt in. If someone else adds type hints to their code, which you import, but you don't run the type checker, nothing changes for you when you run your code (you can still get away with things that happen to work even the type checker would reject them). When you are reading their source code, you will see their type hints -- like every other aspect of writing good code, some people's type hints will be readable, while others' less so. At least nobody will be writing type hints in Cyrillic. :-) > > 2. Clearly, great thought has been put into this PEP. If anyone has a good > analysis of the potential impact on Python 3 adoption, please do pass > along. I would be interested in reading the information. > I wish I had a crystal ball, but this is hard to predict. Anecdotally, some people believe this will be catnip, while others believe it to be poison. The truth will surely be somewhere in the middle. At this point we don't know what drives Python 3 adoption except time -- it's definitely going up. :-) -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 4/21/15 9:17 AM, R. David Murray wrote: Please be respectful rather than inflammatory. Thank you David. If you read what I wrote, I did not say that I was going to stop contributing, I specifically talked about that gut reaction being both emotional and illogical. That doesn't make the reaction any less real, and the fact that such reactions exist is a data point you should consider in conducting your PR campaign for this issue. (I don't mean that last as a negative: this issue *requires* an honest PR campaign.) As David stated, gut reactions are real. These reactions have the potential, if listened to and respected, to lead toward an optimal (not ideal) solution. Likely, the implementation of optional static type checking will evolve from reasoned, respectful debate of the issues not inflammatory quotes. Quite frankly, belittling someone's understanding or knowledge does not serve the PEP or technical issues at hand. I like the option of static type checking for critical high availability and high reliability applications (i.e. air traffic control, financial transactions). I'm less interested in static type checking of inputs when prototyping or developing less critical applications. There have been good technical points made by many on this thread especially given the different use cases. These use cases, and likely a few more, are important to an honest, continued technical refinement of the PEP. Two areas of clarification would be helpful for me: 1. Optional: What does this really mean in practice? Am I opting in to static type checking and type hints? Or must I opt out of type hints? Having to opt out would probably put more burden on the educational use cases than opting in would for a large corporate project. 2. Clearly, great thought has been put into this PEP. If anyone has a good analysis of the potential impact on Python 3 adoption, please do pass along. I would be interested in reading the information. Warmly, Carol P.S. I do member a time when tools to easily check for memory leaks in C were new and magical. Looking at the Coverity scans, I'm glad that the old magic is reaping real benefits. -- *Carol Willing* Developer | Willing Consulting https://willingconsulting.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, 21 Apr 2015 10:10:06 -0700, Guido van Rossum wrote: > On Tue, Apr 21, 2015 at 9:17 AM, R. David Murray > wrote: > > > Please be respectful rather than inflammatory. If you read what I > > wrote, I did not say that I was going to stop contributing, I > > specifically talked about that gut reaction being both emotional and > > illogical. That doesn't make the reaction any less real, and the fact > > that such reactions exist is a data point you should consider in > > conducting your PR campaign for this issue. (I don't mean that last as > > a negative: this issue *requires* an honest PR campaign.) > > > > Well, my own reactions at this point in the flame war are also quite > emotional. :-( > > I have done my best in being honest in my PR campaign. But I feel like the I believe you have. My inclusion of the word 'honest' was meant to contrast the kind of PR we need with the kind of PR people typically think about, which is often not particularly honest. > opposition (not you, but definitely some others -- have you seen Twitter?) > are spreading FUD based on an irrational conviction that this will destroy No, I tend only to peek in there occasionally. > Python. It will not. It may not prove the solution to all Python's problems > -- there's always 3.6. (Oh wait, Python 2.7 is perfect. I've heard that > before -- Paul Everitt famously said the same of Python 1.5.2. Aren't you > glad I didn't take him literally? :-P ) Yes. But somewhere there or not long before was my introduction to Python, so I remember it fondly :) --David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, 21 Apr 2015 16:55:49 -, "Gregory P. Smith" wrote: > We will not be putting type annotations anywhere in the stdlib or expecting > anyone else to maintain them there. That would never happen until tools > that are convincing enough in their utility for developers to _want_ to use > are available and accepted. That'll be a developer workflow thing we could > address with a later PEP. IF it happens at all. This is the most reassuring statement I've heard so far ;) --David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 1:10 PM, Guido van Rossum wrote: > On Tue, Apr 21, 2015 at 9:17 AM, R. David Murray > wrote: >> >> Please be respectful rather than inflammatory. If you read what I >> wrote, I did not say that I was going to stop contributing, I >> specifically talked about that gut reaction being both emotional and >> illogical. That doesn't make the reaction any less real, and the fact >> that such reactions exist is a data point you should consider in >> conducting your PR campaign for this issue. (I don't mean that last as >> a negative: this issue *requires* an honest PR campaign.) > > > Well, my own reactions at this point in the flame war are also quite > emotional. :-( > > I have done my best in being honest in my PR campaign. But I feel like the > opposition (not you, but definitely some others -- have you seen Twitter?) > are spreading FUD based on an irrational conviction that this will destroy > Python. It will not. It may not prove the solution to all Python's problems > -- there's always 3.6. (Oh wait, Python 2.7 is perfect. I've heard that > before -- Paul Everitt famously said the same of Python 1.5.2. Aren't you > glad I didn't take him literally? :-P ) > > -- > --Guido van Rossum (python.org/~guido) I am looking forward to using type annotations. I am not very good at remembering the operations that are available on the objects passed around in my code, but I am very good at typing CTRL-space. To get that I am happy to modify my code with weird docstrings or any other notation. Good support for completion, aided by standard annotations, eliminates a huge amount of cross-referencing while coding. I'm also hopeful that static type checking, aided with annotations, will help with unicode porting. Duck typing does not work very well when you are trying to differentiate between bytes and str. Also, Python 1.5.2 was pretty good :-) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 9:47 AM, Antoine Pitrou wrote: > On Tue, 21 Apr 2015 09:28:45 -0700 > Guido van Rossum wrote: > > On Tue, Apr 21, 2015 at 12:49 AM, Antoine Pitrou > > wrote: > > > > > On Mon, 20 Apr 2015 20:43:38 -0400 > > > "R. David Murray" wrote: > > > > +1 to this from me too. I'm afraid that means I'm -1 on the PEP. > > > > > > > > I didn't write this in my earlier email because I wasn't sure about > it, > > > > but my gut reaction after reading Harry's email was "if type > annotations > > > > are used in the stdlib, I'll probably stop contributing". That > doesn't > > > > mean that's *true*, but that's the first time I've ever had that > > > > thought, so it is probably worth sharing. > > > > > > I think it would be nice to know what the PEP means for daily stdlib > > > development. If patches have to carry typing information each time they > > > add/enhance an API that's an addition burden. If typing is done > > > separately by interested people then it sounds like it wouldn't have > > > much of an impact on everyone else's workflow. > > > > This point will be moot until new code appears in the stdlib whose author > > likes type hints. As I said, we won't be converting existing code to add > > type hints (I'm actively against that for the stdlib, for reasons I've > > explained already). > > I was thinking of potential stub files. Or are you also putting a ban > on those for existing stdlib code? Sorry if that has already been > answered... > No ban, but for now the plan is to collect and manage those separately as part of typeshed. I imagine the typical failure case is where the stubs are more restrictive than the implementation -- after all if the implementation becomes *more* restrictive it's breaking backward compatibility. I think it's okay for the stubs to lag behind like that, so I don't expect pressure on stdlib contributors. If anything the pressure will be on typeshed. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, 21 Apr 2015 18:27:50 +0300 Paul Sokolovsky wrote: > Let me try: MicroPython already uses type annotations for statically > typed functions. E.g. > > def add(x:int, y:int): > return x + y > > will translate the function to just 2 machine instructions. That's quite nice. > Oh really, you care to support single-precisions in Numba? Numba is quite specialized. In our area, single-precision arithmetic can sometimes give double the speed on modern CPUs (especially when LLVM is able to vectorize the code). The speedup can be even larger on a GPU (where double-precision execution resources are scarce). I agree it doesn't make sense for a general-purpose Python compiler. > Anyway, back to your example, it would be done like: > > SFloat = float > DFloat = float > > For a random tool out there, "SFloat" and "DFloat" would be just > aliases to floats, but Numba will know they have additional semantics > behind them. (That assumes that typedefs like SFloat can be accessed in > symbolic form - that's certainly possible if you have your own > parser/VM, but might worth to think how to do it on "CPython" level). Fortunately, we don't have our own parser :-) We work from the CPython bytecode. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 03:51:05PM +0100, Cory Benfield wrote:
> On 21 April 2015 at 15:31, Chris Angelico wrote:
> > Granted, there are some
> > vague areas - how many functions take a "file-like object", and are
> > they all the same? - but between MyPy types and the abstract base
> > types that already exist, there are plenty of ways to formalize duck
> > typing.
>
> Are there? Can I have a link or an example, please? I feel like I
> don't know how I'm supposed to do this, and I'd like to see how that
> works. I'll even give a concrete use-case: I want to be able to take a
> file-like object that has a .read() method and a .seek() method.
I've never done this before, so I might not quite have done it
correctly, but this appears to work just fine:
py> import abc
py> class SeekableReadable(metaclass=abc.ABCMeta):
... @classmethod
... def __subclasshook__(cls, C):
... if hasattr(C, 'seek') and hasattr(C, 'read'):
... return True
... return NotImplemented
...
py> f = open('/tmp/foo')
py> isinstance(f, SeekableReadable)
True
py> from io import StringIO
py> issubclass(StringIO, SeekableReadable)
True
py> issubclass(int, SeekableReadable)
False
That gives you your runtime check for an object with seek() and read()
methods. For compile-time checking, I expect you would define
SeekableReadable as above, then make the declaration:
def read_from_start(f:SeekableReadable, size:int):
f.seek(0)
return f.read(size)
So now you have runtime interface checking via an ABC, plus
documentation for the function parameter type via annotation.
But will the static checker understand that annotation? My guess is,
probably not as it stands. According to the docs, MyPy currently
doesn't support this sort of duck typing, but will:
[quote]
There are also plans to support more Python-style “duck typing”
in the type system. The details are still open.
[end quote]
http://mypy.readthedocs.org/en/latest/class_basics.html#abstract-base-classes-and-multiple-inheritance
I expect that dealing with duck typing will be very high on the list
of priorities for the future. In the meantime, for this specific use-case,
you're probably not going to be able to statically check this type hint.
Your choices would be:
- don't type check anything;
- don't type check the read_from_start() function, but type check
everything else;
- don't type check the f parameter (remove the SeekableReadable
annotation, or replace it with Any, but leave the size:int
annotation);
- possibly some type checkers will infer from the function body that f
must have seek() and read() methods, and you don't have to declare
anything (structural typing instead of nominal?);
- (a bad idea, but just for the sake of completeness) leave the
annotation in, and ignore false negatives.
Remember that there is no built-in Python type checker. If you have no
checker, the annotations are just documentation and nothing else will
have changed. If you don't like the checker you have, you'll be able to
replace it with another.
--
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 9:17 AM, R. David Murray wrote: > Please be respectful rather than inflammatory. If you read what I > wrote, I did not say that I was going to stop contributing, I > specifically talked about that gut reaction being both emotional and > illogical. That doesn't make the reaction any less real, and the fact > that such reactions exist is a data point you should consider in > conducting your PR campaign for this issue. (I don't mean that last as > a negative: this issue *requires* an honest PR campaign.) > Well, my own reactions at this point in the flame war are also quite emotional. :-( I have done my best in being honest in my PR campaign. But I feel like the opposition (not you, but definitely some others -- have you seen Twitter?) are spreading FUD based on an irrational conviction that this will destroy Python. It will not. It may not prove the solution to all Python's problems -- there's always 3.6. (Oh wait, Python 2.7 is perfect. I've heard that before -- Paul Everitt famously said the same of Python 1.5.2. Aren't you glad I didn't take him literally? :-P ) -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Apr 21 2015, Paul Sokolovsky wrote: > Hello, > > On Tue, 21 Apr 2015 08:05:59 -0700 > Nikolaus Rath wrote: > >> On Apr 20 2015, Chris Angelico wrote: >> > Maybe it'd be of value to have a quick "code stripper" that takes >> > away all the annotations, plus any other junk/framing that you're >> > not interested in, and gives you something you can browse in a text >> > editor? >> >> If you need to preprocess your source code to make it suitable for >> human consumption something is very wrong with your language design. >> I can't believe you're seriously suggesting this. > > I'm sure that was irony, d'oh. That'd be a relief. It didn't sound ironic to me. > The proposed type annotations are very readable. [..] I don't have an informed opinion about that yet. I was just commenting on the general idea of stripping them away if they're not readable. Best, -Nikolaus -- GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F »Time flies like an arrow, fruit flies like a Banana.« ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 7:51 AM, Cory Benfield wrote: > The correct specification is "read method with this type signature" > and "seek method with this type signature". I would even be prepared > to waive the type signatures on read and seek, given that enforcing > the type hinting on others is not a good idea. > > I suspect I have a mismatch with several others in this discussion. My > position is that if I'm going to have a type system, I'd like to have > a powerful one: Haskell, not Java. Otherwise I'll get by with the duck > typing that has worked just fine for us over the last 20 years. I > suspect, however, that many others in this conversation want any type > system at all, so long as they can have one. > For me, PEP 484 is a stepping stone. Among the authors of PEP 484 there was much discussion about duck typing, and mypy even has some limited support for duck typing (I think you can still find it by searching the mypy code for "protocol"). But we ran out of time getting all the details written up and agreed upon, so we decided to punt -- for now. But duck typing still needs to have a way to talk about things like "seek method with this type signature" (something like `def seek(self, offset: int, whence: int=SEEK_SET) -> int`) so the current proposal gets us part of the way there. The hope is that once 3.5 is out (with PEP 484's typing.py included *provisional* mode) we can start working on the duck typing specification. The alternative would have been to wait until 3.6, but we didn't think that there would be much of an advantage to postponing the more basic type hinting syntax (it would be like refusing to include "import" until you've sorted out packages). During the run of 3.5 we'll hopefully get feedback on where duck typing is most needed and how to specify it -- valuable input that would be much harder to obtain of *no* part of the type hints notation were standardized. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 12:50 AM Antoine Pitrou wrote: > On Mon, 20 Apr 2015 20:43:38 -0400 > "R. David Murray" wrote: > > +1 to this from me too. I'm afraid that means I'm -1 on the PEP. > > > > I didn't write this in my earlier email because I wasn't sure about it, > > but my gut reaction after reading Harry's email was "if type annotations > > are used in the stdlib, I'll probably stop contributing". That doesn't > > mean that's *true*, but that's the first time I've ever had that > > thought, so it is probably worth sharing. > > I think it would be nice to know what the PEP means for daily stdlib > development. If patches have to carry typing information each time they > add/enhance an API that's an addition burden. If typing is done > separately by interested people then it sounds like it wouldn't have > much of an impact on everyone else's workflow. > Separately by interested people. That won't change until tools appear and mature that help maintain the types for us. (if ever) Nobody wants unreadable code. Nobody is proposing to make unreadable code happen or encourage its creation. One thing I feel is often overlooked in the discussion on this PEP: It is about creating a unified type expression syntax for everyone working on Python typing to centralize around. Regardless of if the PEPs version falls short for some purposes. It allows for sharing work. There are multiple ongoing projects that are trying to make use of type information with Python, this allows them to all speak the same language. (MyPy, MicroPython, Cython, the static analyzer we are trying to create at Google, several others not on the top of my head I'm sure, etc.) We will not be putting type annotations anywhere in the stdlib or expecting anyone else to maintain them there. That would never happen until tools that are convincing enough in their utility for developers to _want_ to use are available and accepted. That'll be a developer workflow thing we could address with a later PEP. IF it happens at all. I view most of this thread as FUD. The fear is understandable, I'm trying to tell people to stop panicing. This PEP does not mean that Python is suddenly going to become unreadable. Just that a set of people working on a common goal have a way to communicate with one another. If that work bears fruit, great, it'll be shared and provided as tools that people want to use. If not, it won't matter in the slightest and the typing module and this PEP will be relegated to history. This is a 100% non-invasive PEP. No new keywords! Motivation behind static analysis and type checkers comes directly from the success of the type annotations and checking done to Javascript in Google's javascript Closure compiler that has been available for years. Steven mentioned Facebook's Flow which does a similar thing. These are both opt-in and by and large we've found that developers love using them in any decent sized code base. That model is the goal for any of our Python typing related projects to get to. If developers don't _want_ to use it in the end, we have failed and they can happily continue not using it because it was never required. The reason this PEP exists is for tool developers to be able to do their thing and prove to everyone that it is (a) possible and (b) genuinely useful. IF that proves successful, we can consider if we need a saner syntax for anyone to want to use it. For now we've got this PEP which is a bit of a hack using the Python 3 annotations and a typing module but at the same time doesn't involve any language changes we might regret. I call that a win! -gps ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 04/21, Paul Sokolovsky wrote: > > And for example yesterday's big theme was people blackmailing that they > stop contributing to stdlib if annotations are in [...] A volunteer's honest reaction is not blackmail, and categorizing it as such is not helpful to the discussion. -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, 21 Apr 2015 09:28:45 -0700 Guido van Rossum wrote: > On Tue, Apr 21, 2015 at 12:49 AM, Antoine Pitrou > wrote: > > > On Mon, 20 Apr 2015 20:43:38 -0400 > > "R. David Murray" wrote: > > > +1 to this from me too. I'm afraid that means I'm -1 on the PEP. > > > > > > I didn't write this in my earlier email because I wasn't sure about it, > > > but my gut reaction after reading Harry's email was "if type annotations > > > are used in the stdlib, I'll probably stop contributing". That doesn't > > > mean that's *true*, but that's the first time I've ever had that > > > thought, so it is probably worth sharing. > > > > I think it would be nice to know what the PEP means for daily stdlib > > development. If patches have to carry typing information each time they > > add/enhance an API that's an addition burden. If typing is done > > separately by interested people then it sounds like it wouldn't have > > much of an impact on everyone else's workflow. > > > > This point will be moot until new code appears in the stdlib whose author > likes type hints. As I said, we won't be converting existing code to add > type hints (I'm actively against that for the stdlib, for reasons I've > explained already). I was thinking of potential stub files. Or are you also putting a ban on those for existing stdlib code? Sorry if that has already been answered... Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 12:49 AM, Antoine Pitrou wrote: > On Mon, 20 Apr 2015 20:43:38 -0400 > "R. David Murray" wrote: > > +1 to this from me too. I'm afraid that means I'm -1 on the PEP. > > > > I didn't write this in my earlier email because I wasn't sure about it, > > but my gut reaction after reading Harry's email was "if type annotations > > are used in the stdlib, I'll probably stop contributing". That doesn't > > mean that's *true*, but that's the first time I've ever had that > > thought, so it is probably worth sharing. > > I think it would be nice to know what the PEP means for daily stdlib > development. If patches have to carry typing information each time they > add/enhance an API that's an addition burden. If typing is done > separately by interested people then it sounds like it wouldn't have > much of an impact on everyone else's workflow. > This point will be moot until new code appears in the stdlib whose author likes type hints. As I said, we won't be converting existing code to add type hints (I'm actively against that for the stdlib, for reasons I've explained already). *If* type hints prove useful, I expect that adding type hints **to code that deserves them** is treated no different in the workflow than adding tests or docs. I.e. something that is the right thing to do because it has obvious benefits for users and/or future maintainers. If at some point running a type checker over the stdlib as part of continuous integration become routine, type hints can also replace certain silly tests. Until some point in a possible but distant future when we're all thinking back fondly about the argument we're currently having, it will be the choice of the author of new (and *only* new) stdlib modules whether and how to use type hints. Such a hypothetical author would also be reviewing updates to "their" module and point out lack of type hints just like you might point out an incomplete docstring, an outdated comment, or a missing test. (The type checker would be responsible for pointing out bugs. :-P ) -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, 21 Apr 2015 18:27:50 +0300, Paul Sokolovsky wrote: > > I was replying to Steven's message. Did you read it? > > Yes. And I try to follow general course of discussion, as its hard to > follow individual sub-threads. And for example yesterday's big theme > was people blackmailing that they stop contributing to stdlib if > annotations are in, and today's theme appear to be people telling that > static type checking won't be useful. And just your reply to Steven > was a final straw which prompted me to point out that static type > checking is not a crux of it, but just one (not the biggest IMHO) usage. Please be respectful rather than inflammatory. If you read what I wrote, I did not say that I was going to stop contributing, I specifically talked about that gut reaction being both emotional and illogical. That doesn't make the reaction any less real, and the fact that such reactions exist is a data point you should consider in conducting your PR campaign for this issue. (I don't mean that last as a negative: this issue *requires* an honest PR campaign.) --David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Wed, 22 Apr 2015 01:09:52 +1000, Chris Angelico wrote: > def incremental_parser(input: FileLike) -> List[Token]: > tokens = [] > data = "" > while True: > if not data: > data = input.read(64) > token = Token(data[0]); data = data[1:] > while token.wants_more(): > token.give_more(data[0]); data = data[1:] > tokens.append(token) > if token.is_end_of_stream(): break > input.seek(-len(data), 1) > return tokens > > If you were to exhaustively stipulate the requirements on the > file-like object, you'd have to say: > > * Provides a read() method which takes an integer and returns up to > that many bytes > * Provides a seek() method which takes two integers > * Is capable of relative seeking by at least 63 bytes backward > * Is open for reading > * Etcetera > > That's not the type system's job. Not in Python. Maybe in Haskell, but Just a note that if I'm reading the high level description right, this kind of analysis is exactly the kind of thing that Flow does for javascript. --David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 9:09 AM, Chris Angelico wrote: > ... > > Pretty accurate, yeah. Here's how I see it: > > def incremental_parser(input: FileLike) -> List[Token]: > tokens = [] > data = "" > while True: > if not data: > data = input.read(64) > token = Token(data[0]); data = data[1:] > while token.wants_more(): > token.give_more(data[0]); data = data[1:] > tokens.append(token) > if token.is_end_of_stream(): break > input.seek(-len(data), 1) > return tokens > > If you were to exhaustively stipulate the requirements on the > file-like object, you'd have to say: > > * Provides a read() method which takes an integer and returns up to > that many bytes > * Provides a seek() method which takes two integers > * Is capable of relative seeking by at least 63 bytes backward > * Is open for reading > * Etcetera > Potentially you could use io.RawIOBase as the ABC for the type you need for FileLike, including read and seek. See the mixins in https://docs.python.org/3/library/io.html#class-hierarchy > > > That's not the type system's job. Not in Python. Maybe in Haskell, Not in Haskell either FWIW in terms of what its type system can prove > but > not in Python. So how much _should_ go into the type hint? I'm happy > with "FileLike" or however it's to be spelled; maybe separate readable > files from writable ones, as those are two fairly clear variants, but > that's about all you really need. RawIOBase is also potential overkill... maybe you just wanted something that duck typed for a few methods you implemented. Any and dynamic typing is a good choice then. > If you provide incremental_parser() > with an input file that's non-seekable, it's going to have problems - > and your editor may or may not even be able to detect that (some files > are seekable but only in the forward direction, but they'll have the > exact same seek() method). > With what has been proposed, there are no static guarantees about how many bytes can be read, or that input is even seekable (does seekable() return True?) or it is open for reading. Instead we can only *prove* a limited amount in static type systems about the runtime dynamic behavior of code, and PEP 484 is weaker than other approaches (for very good reasons IMHO). Still useful however :) - Jim ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 21 April 2015 at 16:09, Chris Angelico wrote: > Pretty accurate, yeah. Here's how I see it: > > def incremental_parser(input: FileLike) -> List[Token]: > tokens = [] > data = "" > while True: > if not data: > data = input.read(64) > token = Token(data[0]); data = data[1:] > while token.wants_more(): > token.give_more(data[0]); data = data[1:] > tokens.append(token) > if token.is_end_of_stream(): break > input.seek(-len(data), 1) > return tokens > > If you were to exhaustively stipulate the requirements on the > file-like object, you'd have to say: > > * Provides a read() method which takes an integer and returns up to > that many bytes > * Provides a seek() method which takes two integers > * Is capable of relative seeking by at least 63 bytes backward > * Is open for reading > * Etcetera > > That's not the type system's job. Not in Python. Maybe in Haskell, but > not in Python. So how much _should_ go into the type hint? I'm happy > with "FileLike" or however it's to be spelled; maybe separate readable > files from writable ones, as those are two fairly clear variants, but > that's about all you really need. If you provide incremental_parser() > with an input file that's non-seekable, it's going to have problems - > and your editor may or may not even be able to detect that (some files > are seekable but only in the forward direction, but they'll have the > exact same seek() method). Ok, that makes sense to me. =) Looking at mypy's source code, I see shutil.copyfileobj has the following signature: def copyfileobj(fsrc: IO[AnyStr], fdst: IO[AnyStr], length: int = 16*1024) -> None: so it seems that mypy defines a general IO datatype here. I continue to be worried about the lack of a *general* solution to this problem, but I'm glad that some specific solutions exist. I still think library authors will not do much to take up this proposal. =) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Hello, On Tue, 21 Apr 2015 08:05:59 -0700 Nikolaus Rath wrote: > On Apr 20 2015, Chris Angelico wrote: > > Maybe it'd be of value to have a quick "code stripper" that takes > > away all the annotations, plus any other junk/framing that you're > > not interested in, and gives you something you can browse in a text > > editor? > > If you need to preprocess your source code to make it suitable for > human consumption something is very wrong with your language design. > I can't believe you're seriously suggesting this. I'm sure that was irony, d'oh. Just as my suggestion to have stickers "This system runs free of type annotations" for the most zealous anti-annotations folks. The proposed type annotations are very readable. Not as readable as C's type syntax, but ok. Sorry, irony again. I remember myself as a 10-years old kid, trying to wrap my head about C's types syntax, and not getting anything of it. So, please contrast it to C - PEP484 gives very readable syntax, and even both human- and machine-readable. If you feel like writing 3-story type annotation, just resist the temptation, "Any" is your best friend. -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Hello, On Tue, 21 Apr 2015 16:11:51 +0200 Antoine Pitrou wrote: [] > >> You can't at the same time point out that type checking has no > >> power or control over runtime behaviour, and then claim that type > >> checking makes runtime behaviour (for example, ability to accept or > >> reject certain types) saner. It is a trivial contradiction. > > > > I suspected there's either short-sightedness, or just a word play > > for for a purpose of word play. > > > > Type annotation are NOT introduced for the purpose of static type > > checking. > > I was replying to Steven's message. Did you read it? Yes. And I try to follow general course of discussion, as its hard to follow individual sub-threads. And for example yesterday's big theme was people blackmailing that they stop contributing to stdlib if annotations are in, and today's theme appear to be people telling that static type checking won't be useful. And just your reply to Steven was a final straw which prompted me to point out that static type checking is not a crux of it, but just one (not the biggest IMHO) usage. > > Runtime type checking (run as via "make test", not > > as "production") is much more interesting to catch errors. > > Obviously you're giving the word "runtime" a different meaning than I > do. The type checker isn't supposed to actually execute the user's > functions (it's not that it's forbidden, simply that it's not how it > will work in all likelihood): therefore, it doesn't have any handle on > what *actually* happens at runtime, vs. what is declared in the typing > declarations. Well, maybe "typechecker" is the wrong word then. I'm talking about instrumented VM which actually interprets type annotation while running bytecode - for example, on entry to a function, it takes argument annotations, and executes sequence of equivalent isinstance() checks, etc., etc. One won't use such instrumented VM at "production" runtime, but it will be useful to enable while running testsuite (or integration tests), to catch more issues. > > Even more interesting usage is to allow ahead-of-time, and thus > > unbloated, optimization. There're bunch of JITters and AOTters for > > Python language, each of which uses own syntax (via decorators, > > etc.) to annotate functions. > > As a developer of one of those tools, I've already said that I find it > unlikely for the PEP to be useful for that purpose. The issue is that > the vocabulary created in the PEP is not extensible enough. How so, if it essentially allows you make typedefs? Syntax of such typedef follow basic conventions (so *any* tool should understand its *basic* semantics), but you're free to assign (implicit, particular for your tool, but of course documented for it) semantics. > Note I'm not saying it's impossible. I'm just skeptical that in its > current form it will help us. And apparently none of our "competitors" That's my biggest fear - that "JIT" Python community is not yet ready to receive and value this feature. > seems very enthusiastic either (feel free to prove me wrong: I might > have missed something :-)). Let me try: MicroPython already uses type annotations for statically typed functions. E.g. def add(x:int, y:int): return x + y will translate the function to just 2 machine instructions. And we'd prefer to use standard language syntax, instead of having our own conventions (e.g. above you see that return type "is inferred"). > > Having language-specified type annotations > > allows for portable syntax for such optimized code. > > Only if the annotations allow expressing the subtleties required by > the specific optimizer. For example, "Float" is too vague for Numba: > we would like to know if that is meant to be a single- or > double-precision float. Oh really, you care to support single-precisions in Numba? We have a lot of problems with sfloats in MicroPython, because whole Python stdlib API was written with implicit assumption that float is (at least) double. E.g., we cannot have time.time() which both returns fractional seconds and is based of Jan 1, 1970 - there's simply not enough bits in single-precision floats! It's on my backlog to bring up this and related issues before wider Python development community. Anyway, back to your example, it would be done like: SFloat = float DFloat = float For a random tool out there, "SFloat" and "DFloat" would be just aliases to floats, but Numba will know they have additional semantics behind them. (That assumes that typedefs like SFloat can be accessed in symbolic form - that's certainly possible if you have your own parser/VM, but might worth to think how to do it on "CPython" level). > > Don't block the language if you're stuck > > with an unimaginative implementation, there's much more to Python > > than that. > > The Python language doesn't really have anything to do with that. It's > just an additional library with a set of conventions. Which is also > why a PEP wouldn't be required to make it alive
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Wed, Apr 22, 2015 at 12:51 AM, Cory Benfield wrote: > On 21 April 2015 at 15:31, Chris Angelico wrote: >> Granted, there are some >> vague areas - how many functions take a "file-like object", and are >> they all the same? - but between MyPy types and the abstract base >> types that already exist, there are plenty of ways to formalize duck >> typing. > > Are there? Can I have a link or an example, please? I feel like I > don't know how I'm supposed to do this, and I'd like to see how that > works. I'll even give a concrete use-case: I want to be able to take a > file-like object that has a .read() method and a .seek() method. Someone who's been more involved in the details of MyPy can probably say more specifically what ought to be done about file-like objects. >> And frankly, even with the uncertainties, I'd still rather >> have a function declared as taking a "file-like object" than "an >> object with a .read() method that takes an integer and returns up to >> that many bytes of data, and a .seek() method that blah blah blah >> blah". Sometimes, the precision is completely useless. > > It is completely useless. Happily, this is a strawman, and no-one was > asking for it, so we can all live happily ever after! > > The correct specification is "read method with this type signature" > and "seek method with this type signature". I would even be prepared > to waive the type signatures on read and seek, given that enforcing > the type hinting on others is not a good idea. > > I suspect I have a mismatch with several others in this discussion. My > position is that if I'm going to have a type system, I'd like to have > a powerful one: Haskell, not Java. Otherwise I'll get by with the duck > typing that has worked just fine for us over the last 20 years. I > suspect, however, that many others in this conversation want any type > system at all, so long as they can have one. > > Is that an accurate characterisation of your position, Chris? Pretty accurate, yeah. Here's how I see it: def incremental_parser(input: FileLike) -> List[Token]: tokens = [] data = "" while True: if not data: data = input.read(64) token = Token(data[0]); data = data[1:] while token.wants_more(): token.give_more(data[0]); data = data[1:] tokens.append(token) if token.is_end_of_stream(): break input.seek(-len(data), 1) return tokens If you were to exhaustively stipulate the requirements on the file-like object, you'd have to say: * Provides a read() method which takes an integer and returns up to that many bytes * Provides a seek() method which takes two integers * Is capable of relative seeking by at least 63 bytes backward * Is open for reading * Etcetera That's not the type system's job. Not in Python. Maybe in Haskell, but not in Python. So how much _should_ go into the type hint? I'm happy with "FileLike" or however it's to be spelled; maybe separate readable files from writable ones, as those are two fairly clear variants, but that's about all you really need. If you provide incremental_parser() with an input file that's non-seekable, it's going to have problems - and your editor may or may not even be able to detect that (some files are seekable but only in the forward direction, but they'll have the exact same seek() method). I might be wrong about where MyPy is trying to go with this, but doubtless someone will correct me on any inaccuracies above. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Apr 20 2015, Chris Angelico wrote: > Maybe it'd be of value to have a quick "code stripper" that takes away > all the annotations, plus any other junk/framing that you're not > interested in, and gives you something you can browse in a text > editor? If you need to preprocess your source code to make it suitable for human consumption something is very wrong with your language design. I can't believe you're seriously suggesting this. Best, -Nikolaus -- GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F »Time flies like an arrow, fruit flies like a Banana.« ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 21 April 2015 at 15:31, Chris Angelico wrote: > Granted, there are some > vague areas - how many functions take a "file-like object", and are > they all the same? - but between MyPy types and the abstract base > types that already exist, there are plenty of ways to formalize duck > typing. Are there? Can I have a link or an example, please? I feel like I don't know how I'm supposed to do this, and I'd like to see how that works. I'll even give a concrete use-case: I want to be able to take a file-like object that has a .read() method and a .seek() method. > And frankly, even with the uncertainties, I'd still rather > have a function declared as taking a "file-like object" than "an > object with a .read() method that takes an integer and returns up to > that many bytes of data, and a .seek() method that blah blah blah > blah". Sometimes, the precision is completely useless. It is completely useless. Happily, this is a strawman, and no-one was asking for it, so we can all live happily ever after! The correct specification is "read method with this type signature" and "seek method with this type signature". I would even be prepared to waive the type signatures on read and seek, given that enforcing the type hinting on others is not a good idea. I suspect I have a mismatch with several others in this discussion. My position is that if I'm going to have a type system, I'd like to have a powerful one: Haskell, not Java. Otherwise I'll get by with the duck typing that has worked just fine for us over the last 20 years. I suspect, however, that many others in this conversation want any type system at all, so long as they can have one. Is that an accurate characterisation of your position, Chris? ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
So. This is how you try and get me to care about Python 3. Can't speak for others, but this does the opposite for me. This makes me ecstatic that Python 2 has a nearly-frozen api. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Hello, On Tue, 21 Apr 2015 15:08:27 +0200 Antoine Pitrou wrote: [] > Because the user might not run the type checker, obviously. To quote > you: """When we say that type checking is optional, we mean it.""" > > You can't at the same time point out that type checking has no > power or control over runtime behaviour, and then claim that type > checking makes runtime behaviour (for example, ability to accept or > reject certain types) saner. It is a trivial contradiction. I suspected there's either short-sightedness, or just a word play for for a purpose of word play. Type annotation are NOT introduced for the purpose of static type checking. The current PEP just gives an example of their usage for static type checking as an example backed by the immediate availability of a tool which does that, MyPy. But granted, static type checking is the most boring of possible usages for type annotation. Runtime type checking (run as via "make test", not as "production") is much more interesting to catch errors. The tooling for that is yet to be written though. Even more interesting usage is to allow ahead-of-time, and thus unbloated, optimization. There're bunch of JITters and AOTters for Python language, each of which uses own syntax (via decorators, etc.) to annotate functions. Having language-specified type annotations allows for portable syntax for such optimized code. Granted, the most juicy usages described above won't be available in CPython out of the box. But there should be that motto: "CPython is the most boring of all Pythons". Don't block the language if you're stuck with an unimaginative implementation, there's much more to Python than that. -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 7:56 PM, Arnaud Delobelle wrote: > If people constantly get told by their editor / IDE that they are calling > function with the wrong argument types, what are they going to do? They may > start adopting the same approach as in Java / C++ etc... where interfaces > must be explicitly defined and the practice of duck typing may become > forgotten because discouraged by the tools programmers use. Style guides should generally be recommending the use of concrete types for return values, but abstract types for parameters - you might well have a function that returns a list, but most functions that accept lists are really accepting sequences. Granted, there are some vague areas - how many functions take a "file-like object", and are they all the same? - but between MyPy types and the abstract base types that already exist, there are plenty of ways to formalize duck typing. And frankly, even with the uncertainties, I'd still rather have a function declared as taking a "file-like object" than "an object with a .read() method that takes an integer and returns up to that many bytes of data, and a .seek() method that blah blah blah blah". Sometimes, the precision is completely useless. What an editor would do with the hint that a function takes a file-like object I'm not sure, but if it issues a complaint because I'm passing it something that doesn't have a writelines() method, it's the fault of the editor, not the type hint. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Le 21/04/2015 15:50, Paul Sokolovsky a écrit : > Hello, > > On Tue, 21 Apr 2015 15:08:27 +0200 > Antoine Pitrou wrote: > > [] > >> Because the user might not run the type checker, obviously. To quote >> you: """When we say that type checking is optional, we mean it.""" >> >> You can't at the same time point out that type checking has no >> power or control over runtime behaviour, and then claim that type >> checking makes runtime behaviour (for example, ability to accept or >> reject certain types) saner. It is a trivial contradiction. > > I suspected there's either short-sightedness, or just a word play for > for a purpose of word play. > > Type annotation are NOT introduced for the purpose of static type > checking. I was replying to Steven's message. Did you read it? > Runtime type checking (run as via "make test", not > as "production") is much more interesting to catch errors. Obviously you're giving the word "runtime" a different meaning than I do. The type checker isn't supposed to actually execute the user's functions (it's not that it's forbidden, simply that it's not how it will work in all likelihood): therefore, it doesn't have any handle on what *actually* happens at runtime, vs. what is declared in the typing declarations. But if by "runtime" you mean any action that happens *out-of-band* during development (such as running pip or a syntax-checking linter), then sure, fine :-) > Even more interesting usage is to allow ahead-of-time, and thus > unbloated, optimization. There're bunch of JITters and AOTters for > Python language, each of which uses own syntax (via decorators, etc.) > to annotate functions. As a developer of one of those tools, I've already said that I find it unlikely for the PEP to be useful for that purpose. The issue is that the vocabulary created in the PEP is not extensible enough. Note I'm not saying it's impossible. I'm just skeptical that in its current form it will help us. And apparently none of our "competitors" seems very enthusiastic either (feel free to prove me wrong: I might have missed something :-)). > Having language-specified type annotations > allows for portable syntax for such optimized code. Only if the annotations allow expressing the subtleties required by the specific optimizer. For example, "Float" is too vague for Numba: we would like to know if that is meant to be a single- or double-precision float. > Don't block the language if you're stuck > with an unimaginative implementation, there's much more to Python than > that. The Python language doesn't really have anything to do with that. It's just an additional library with a set of conventions. Which is also why a PEP wouldn't be required to make it alive, it's just there to make it an official standard. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 03:08:27PM +0200, Antoine Pitrou wrote: > On Tue, 21 Apr 2015 22:47:23 +1000 > Steven D'Aprano wrote: > > > > Ironically, type hinting will *reduce* the need for intrusive, > > anti-duck-testing explicit calls to isinstance() at runtime: > > It won't, since as you pointed out yourself, type checks are purely > optional and entirely separate from compilation and runtime evaluation. Perhaps you are thinking of libraries, where the library function has to deal with whatever junk people throw at it. To such libraries, I believe that the major benefit of type hints is not so much in proving the library's correctness in the face of random arguments, but as documentation. In any case, of course you are correct that public library functions and methods will continue to need to check their arguments. (Private functions, perhaps not.) But for applications, the situation is different. If my application talks to a database and extracts a string which it passes on to its own function spam(), then it will be a string. Not a string-like object. Not something that quacks like a string. A string. Once the type checker is satisfied that spam() always receives a string, then further isinstance checks inside spam() is a waste of time. If spam()'s caller changes and might return something which is not a string, then the type checker will flag that. Obviously to get this benefit you need to actually use a type checker. I didn't think I needed to mention that. -- Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Apr 21, 2015, at 01:34 PM, Steven D'Aprano wrote: >Putting the type information in a stub file is an exponentially more distant >fourth best, or to put it another way, *the worst* solution for where to put >type hints. Not only do you Repeat Yourself with the name of the parameter, >but also the name of the function (or method and class) AND module. The type >information *isn't even in the same file*, which increases the chance of it >being lost, forgotten, deleted, out of date, unmaintained, etc. All true, but the trade-off is the agility and ease of working on, reading, and understanding the stdlib, all of which IMHO will suffer if type hints are inlined there. What I don't want to have happen is for type hints to slowly infiltrate the stdlib to the point where no patch will be accepted unless it also has hints. I have the same gut reaction to this as RDM expressed a few posts back. One of the thing I love most about Python is its dynamic typing. I'm all for giving linter developers a hook for experimenting with their tools, I just don't care and I don't want to *have* to care. Maybe some day they will make it so compelling that I will care, but I want to be convinced first. So I think stub files in the stdlib are the right compromise today. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Hello, On Tue, 21 Apr 2015 11:56:15 +0100 Rob Cliffe wrote: > On 21/04/2015 10:33, Cory Benfield wrote: > > On 21 April 2015 at 10:10, Chris Angelico wrote: > >> At this point, you may want to just stop caring about the exact > >> type. Part of the point of gradual typing is that you can > >> short-cut a lot of this. And quite frankly, this isn't really > >> helping anything. Just skip it and say that it's Union[Mapping, > >> Iterable, None]. > > I think this is my problem with the proposal. This change doesn't > > catch any of the bugs anyone was going to hit (no-one is passing a > > callable to this parameter), and it doesn't catch any of the bugs > > someone might actually hit (getting the tuple elements in the wrong > > order, for example). At this point the type signature is worse than > > useless. > Exactly. At this point putting the type signatures in seems to be a > pointless exercise in masochism (for the author) and sadism (for the > reader). Or interesting and productive exercise, which will lead to further improvements and benefits - for other kinds of authors and readers. Those kinds left your kind alone by letting to keep not writing annotations. Why you feel so embarrassed to let the other kind progress? Python becomes multi-paradigm language, what's so bad with that? You're afraid that you will have discomfort reading other's code? My god, I love Python, and hate 50% of code written in it, most of it needs to be rewritten to be satisfactory! Why it bothers you that there will be direction on how other people may (re)write it? > > The refreshing thing about Python is that it is a fantastic, > *concise*, dynamically typed language, at the opposite end of the > spectrum from C++ (ugh!). C++ is fantastic, concise, statically typed language. You've just used to see 90% of code in it which needs to be rewritten. [] > Apologies, Guido, but: [] > You see where I'm going with this - adding type hints to Python feels > a bit like painting feet on the snake. Or at least turning it into a > different language. I'm sure type annotations never could be discussed for inclusion if Guido didn't feel that they fit with Python's way. Sometimes it makes sense to trust BDFL. It's much less obtrusive and qustionable change than forcing Unicode by default, making print a function, or keeping 3-headed sys.exc_info() ugliness instead throwing it out of Python3. Because well, type annotation *are optional*. And it's by now clear that there will be --strip-type-annotations flag to pip and stickers "This system runs type annotations free". > > Best wishes > Rob Cliffe -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, 21 Apr 2015 at 09:59 Cory Benfield wrote: [...] > Further, Python's type system is not sufficiently flexible to allow > library authors to adequately specify the types their code actually > works on. I need to be able to talk about interfaces, because > interfaces are the contract around which APIs are build in Python, but > I cannot do that with this system in a way that makes any sense at > all. To even begin to be useful for library authors this PEP would > need to allow some kind of type hierarchy that is *not* defined by > inheritance, but instead by interfaces. We've been discouraging use of > 'type' and 'isinstance' for years because they break duck typing, but > that has *nothing* on what this PEP appears to do to duck typing. > > This is why I feel like this PEP may be a real threat to duck typing. If people constantly get told by their editor / IDE that they are calling function with the wrong argument types, what are they going to do? They may start adopting the same approach as in Java / C++ etc... where interfaces must be explicitly defined and the practice of duck typing may become forgotten because discouraged by the tools programmers use. I guess what I'm saying is that this could encourage a very significant cultural change in the way Python code is written towards a less flexible mindset. -- Arnaud ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Hey, I just wanted to say to everyone, thanks for being so patient and willing to engage with this discussion, despite my not having done my research and read the (substantial) prior discussion on the topic. Here it is (or at least, some of it!) for any other newcomers: https://mail.python.org/pipermail/python-list/2015-January/697202.html https://mail.python.org/pipermail/python-list/2015-January/697315.html https://github.com/ambv/typehinting/issues/55 I'm not sure any of it will radically change anyone's mind. Everyone (I think) agrees that type hints carry some cost in terms of legibility and simplicity of the language. Everyone agrees they have some benefits. The disagreement remains over whether it will be worth it, and whether stub files vs inline vs something-else would be the best place to hide them. Is "let's wait and see" an acceptable position? Probably the only real way to resolve the disagreement is to start seeing the type hints in use, and form opinions based on real-world practice. Some people will like them inline. Lots of people will use stub files, since they're the only way to maintain 2+3 compatibility, so it's my home that the community will standardise around that. Still other people will want to have another crack at using docstrings, but the standardisation on notation of types will still help them. And still other others will want no part in them, and will argue against them in their communities. Hopefully a best practice will evolve. And hopefully it'll be the stub file one, and then we really can deprecate function annotations in 3.6 ;-) On 21 April 2015 at 11:56, Rob Cliffe wrote: > On 21/04/2015 10:33, Cory Benfield wrote: > > On 21 April 2015 at 10:10, Chris Angelico > wrote: > > At this point, you may want to just stop caring about the exact type. > Part of the point of gradual typing is that you can short-cut a lot of > this. And quite frankly, this isn't really helping anything. Just skip > it and say that it's Union[Mapping, Iterable, None]. > > I think this is my problem with the proposal. This change doesn't > catch any of the bugs anyone was going to hit (no-one is passing a > callable to this parameter), and it doesn't catch any of the bugs > someone might actually hit (getting the tuple elements in the wrong > order, for example). At this point the type signature is worse than > useless. > > Exactly. At this point putting the type signatures in seems to be a > pointless exercise in masochism (for the author) and sadism (for the > reader). > > The refreshing thing about Python is that it is a fantastic, *concise*, > dynamically typed language, at the opposite end of the spectrum from C++ > (ugh!). > We have got used to the consequences (good and bad) of this: > Duck typing, e.g. a function to sort numbers (sorry Tim Peters bad > example) turns out to support any kind of object (e.g. strings) that > supports comparison. > Not to mention objects of some class that will be written in 5 > years time. > (Adding a type hint that restricted the argument to say a sequence > of numbers turns out to be a mistake. And what is a number? > Is Fraction? What about complex numbers, which can't be sorted? > What if the function were written before the Decimal class?) > Errors are often not caught until run time that would be caught at > compile time in other languages (though static code checkers help). > (Not much of a disadvantage because of Python's superb error > diagnostics.) > Python code typically says what it is doing, with the minimum of > syntactic guff. (Well, apart from colons after if/while/try etc. :-) ) > Which makes it easy to read. > Now it seems as if this proposal wants to start turning Python in the C++ > direction, encouraging adding ugly boilerplate code. (This may only be > tangentially relevant, but I want to scream when I see some combination of > public/private/protected/static/extern etc., most of which I don't > understand.) > > Chris A makes the valid point (if I understand correctly) that > Authors of libraries should make it as easy as possible to > (i) know what object types can be passed to functions > (ii) diagnose when the wrong type of object is passed > Authors of apps are not under such obligation, they can basically do > what they want. > > Well, > (i) can be done with good documentation (docstrings etc.). > (ii) can be done with appropriate runtime checks and good error > messages. > E.g. (Cory's example) I'm sure it is possible to test somehow if an object > is file-like (if only by trying to access it like a file). > Is thorough argument checking and provision of good diagnostics going to > be easy for the library author? No. Is it going to be a lot of work to do > thoroughly? Almost certainly yes. > But what the hell, writing a production-quality library is not an exercise > for newbies. > > It seems to me that type hints
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 20/04/2015 19:30, Harry Percival wrote: Hi all, tldr; type hints in python source are scary. Would reserving them for stub files be better? I was trying to find Jack's original post as I think his summary is excellent and aligns well with where I think I'm coming from on this: https://mail.python.org/pipermail/python-dev/2015-April/139253.html However, Harry makes just as many good points so I'll reply here, with a note that while switching to (type/header/stub - my ordered preference for how to describe .pyi files) is preferable to type hints inside files, it's still a massive change to the language, not one I can say I've missed over the past 15 years, and one if asked to vote on (I know that's not the case ;-)) that I would choose to vote against. Anyway, I've not posted much to python-dev in quite a while, but this is a topic that I would be kicking myself in 5-10 years time when I've had to move to Javascript or because everyone else has drifted away from Python as it had become ugly... Chris ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 20/04/2015 20:09, Paul Moore wrote: On 20 April 2015 at 19:41, Barry Warsaw wrote: tldr; type hints in python source are scary. Would reserving them for stub files be better? I think so. I think PEP 8 should require stub files for stdlib modules and strongly encourage them for 3rd party code. Agreed. I have many of the same concerns as Harry, but I wouldn't have expressed them quite as well. I'm not too worried about actually removing annotations from the core language, but I agree that we should create a strong culture of "type hints go in stub files" to keep source files readable and clean. On that note, I'm not sure "stub" files is a particularly good name. Maybe "type files" would be better? Something that emphasises that they are the correct place to put type hints, not a workaround. Or we could just be honest and admit that we're choosing to add header files to Python. It's a shame, as it's more complexity, and it's being inflicted on those who might be writing a library for the first time, or those becoming core committers for the first time, or those just trying to "do the right thing". Currently, the burden is a heavier one (type inference, rather than reading it from a file) but borne by people best placed to handle it (authors of IDEs, type checking software, etc). Chris ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, 21 Apr 2015 23:16:19 +1000 Steven D'Aprano wrote: > > I could keep going, but I hope I've made my point. I don't think so. Just because other languages are looking at it doesn't mean it will end up successful. It means it's an interesting idea, that's all. A litmus test for this PEP would be to apply it to a sizable code base and evaluate the results. Looking at isolated function examples doesn't tell us how it will scale (and I'm obviously not talking about performance, but the ability to still give useful diagnostics in the presence of typing "holes" - i.e. untyped functions - or imprecisions, when factored in a complex graph of call dependencies). > Whatever language you > are using in 5-10 years time, it will almost certainly be either mostly > static with some dynamic features like Java, or dynamic with optional > and gradual typing. Anybody can make predictions. As a data point, 6 years ago people were predicting that the average desktop CPU would have 16 cores nowadays... (we don't hear much from them anymore :-)) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 21 April 2015 at 13:47, Steven D'Aprano wrote: > On Tue, Apr 21, 2015 at 11:56:15AM +0100, Rob Cliffe wrote: > >> (Adding a type hint that restricted the argument to say a >> sequence of numbers turns out to be a mistake. > > Let's find out how big a mistake it is with an test run. > > py> def sorter(alist: List[int]) -> List[int]: > ... return sorted(alist) > ... > py> data = (chr(i) + 'ay' for i in range(97, 107)) > py> type(data) > > py> sorter(data) > ['aay', 'bay', 'cay', 'day', 'eay', 'fay', 'gay', 'hay', 'iay', 'jay'] > > > When we say that type checking is optional, we mean it. That's a very good point - although I have to say that I sort of feel like I'm cheating doing that. It doesn't matter how many times you say "optional", it does still feel like it's wrong, in basically the same way as using undocumented methods on a class, etc. Conceded, type checking is optional. But that message seems to be failing to get through. It's not a technical thing, it's a PR issue. Which is *very* close to meaning that the problem is FUD, but less in the sense of "people deliberately spreading FUD" as "people being worried, not understanding the proposal, and not being sure how type hints will work in practice". I'm reasonably willing to just accept that the PEP will be getting implemented, and to wait and see how things turn out. But my fear, uncertainty and doubt remain. I'll try not to spread them, though. >> You see where I'm going with this - adding type hints to Python feels a >> bit like painting feet on the snake. > > Pythons are one of the few snakes which have vestigal legs: > > http://en.wikipedia.org/wiki/Pelvic_spur LOL, instant win :-) Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 01:25:34PM +0100, Chris Withers wrote: > Anyway, I've not posted much to python-dev in quite a while, but this is > a topic that I would be kicking myself in 5-10 years time when I've had > to move to Javascript or because everyone > else has drifted away from Python as it had become ugly... Facebook released Flow, a static typechecker for Javascript, to a very positive reaction. From their announcement: Flow’s type checking is opt-in — you do not need to type check all your code at once. However, underlying the design of Flow is the assumption that most JavaScript code is implicitly statically typed; even though types may not appear anywhere in the code, they are in the developer’s mind as a way to reason about the correctness of the code. Flow infers those types automatically wherever possible, which means that it can find type errors without needing any changes to the code at all. On the other hand, some JavaScript code, especially frameworks, make heavy use of reflection that is often hard to reason about statically. For such inherently dynamic code, type checking would be too imprecise, so Flow provides a simple way to explicitly trust such code and move on. This design is validated by our huge JavaScript codebase at Facebook: Most of our code falls in the implicitly statically typed category, where developers can check their code for type errors without having to explicitly annotate that code with types. Quoted here: http://blog.jooq.org/2014/12/11/the-inconvenient-truth-about-dynamic-vs-static-typing/ More about flow: http://flowtype.org/ Matz is interested in the same sort of gradual type checking for Ruby as Guido wants to add to Python: https://www.omniref.com/blog/blog/2014/11/17/matz-at-rubyconf-2014-will-ruby-3-dot-0-be-statically-typed/ Julia already includes this sort of hybrid dynamic+static type checking: http://julia.readthedocs.org/en/latest/manual/types/ I could keep going, but I hope I've made my point. Whatever language you are using in 5-10 years time, it will almost certainly be either mostly static with some dynamic features like Java, or dynamic with optional and gradual typing. -- Steven ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, 21 Apr 2015 22:47:23 +1000 Steven D'Aprano wrote: > > Ironically, type hinting will *reduce* the need for intrusive, > anti-duck-testing explicit calls to isinstance() at runtime: It won't, since as you pointed out yourself, type checks are purely optional and entirely separate from compilation and runtime evaluation. > Why bother making that expensive isinstance call every single time the > function is called, if the type checker can prove that x is always a > float? Because the user might not run the type checker, obviously. To quote you: """When we say that type checking is optional, we mean it.""" You can't at the same time point out that type checking has no power or control over runtime behaviour, and then claim that type checking makes runtime behaviour (for example, ability to accept or reject certain types) saner. It is a trivial contradiction. (and because of the former property, type hints can be entirely wrong - for example incomplete, or too strict, or too lax - without anyone noticing as long as they are self-consistent, since runtime behaviour is unaffected by them) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 21 April 2015 at 12:23, Gustavo Carneiro wrote: > Documentation is not checked. It often loses sync with the actual code. > Docs say one thing, code does another. Agreed. I don't think anyone would disagree here. I'm talking from the position of being a library author, where supporting versions of Python lower than 3.5 will be a reality for at least 5 more years. I will not be able to inline my type hints, so they'll have to go in stub files, and now we've got the same problem: type hints can go out of step just as easily as documentation can. > Documenting types in docstrings forces you to repeat yourself. You have to > write the parameter names twice, once in the function definition line, > another in the docstring. And you need to understand the syntax of whatever > docstring format will be used to check the code. Yes, but I'm repeating myself O(small integer) lines away from where I first wrote it. The maintenance burden of keeping those things in step is not very high. As for your last point, my code is not checked against my docstrings, primarily for the reasons I mentioned in my original email. My docstrings are merely rendered into documentation. > I'm sure I'm not the only one that thinks that type hints actually *improve > readability*. In some cases, sure. No contest. > but I am certain that most of the time the type hints make the > code easier to read because you don't have to spend so much mental effort > trying to figure out "gee, I wonder if this parameter is supposed to be > bool, string, or int?" Yes. If you spend a lot of your time doing this, then type hints will help you, *assuming* the programmer who originally wrote the ambiguous code diligently maintains the type hints. I have no reason to believe that a programmer whose code is that ambiguous is going to do that, or even write type hints at all. The bigger problem is that, previously, Python functions never took bool, string, or int. They took 'object that can be evaluated in a boolean context', 'object that behaves like strings in some undefined way', and 'objects that behave like ints in some undefined way'. The type hint you provide is *not helpful*. What I *need* is a type hint that says 'object that has the .endswith() method' or 'object that can be safely cast to a string', so I can tell whether I can pass you a custom URL class that happens to render into a textual URL or whether it definitely has to be a string object. But you can't express that relationship in this model, so you will just say 'string', and now I will turn off type checking because the type hinter keeps shouting at me for using my URLBuilder object. If you only work with primitive types and only want your users to use primitive types, this proposal almost certainly works brilliantly. If you *don't*, this proposal begins to become extremely uncomfortable extremely fast. > Even if it only helps in the simplest cases, it saves the reader of the code > a lot of effort if you add up all those little cases together. > > My suggestion, wherever an annotated type is too complex, don't annotate it. > If you do /only/ the simple cases, the end result is easier to read. If that's genuinely the bar for success for this PEP, then fine, I wish it the best. I'd *like* a PEP that can deal with the complex cases too, but I freely admit to having no idea how such a thing may be implemented. All I can say is that I think this PEP will have limited uptake amongst many third-party libraries, particularly the complex ones. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 11:56:15AM +0100, Rob Cliffe wrote:
> (Adding a type hint that restricted the argument to say a
> sequence of numbers turns out to be a mistake.
Let's find out how big a mistake it is with an test run.
py> def sorter(alist: List[int]) -> List[int]:
... return sorted(alist)
...
py> data = (chr(i) + 'ay' for i in range(97, 107))
py> type(data)
py> sorter(data)
['aay', 'bay', 'cay', 'day', 'eay', 'fay', 'gay', 'hay', 'iay', 'jay']
When we say that type checking is optional, we mean it.
[Disclaimer: I had to fake the List object, since I don't have the
typing module, but everything else is exactly as you see it.]
Annotations will be available for type checking. If you don't want to
type check, don't type check. If you want to go against the type hints,
you can go against the type hints, and get exactly the same runtime
errors as you have now:
py> sorter(None)
Traceback (most recent call last):
File "", line 1, in
File "", line 2, in sorter
TypeError: 'NoneType' object is not iterable
There is no compile time checking unless you choose to run a type
checker or linter -- and even if the type checker flags errors, you can
ignore it and run the program regardless. Just like today with linters
like PyFlakes, PyLint and similar.
For those who choose not to run a type checker, the annotations will be
nothing more than introspectable documentation.
> And what is a number?
> Is Fraction? What about complex numbers, which can't be
> sorted? What if the function were written before the Decimal class?)
I know that you are intending these as rhetorical questions, but Python
has had a proper numeric tower since version 2.5 or 2.6. So:
py> from numbers import Number
py> from decimal import Decimal
py> isinstance(Decimal("1.25"), Number)
True
py> isinstance(2+3j, Number)
True
> Errors are often not caught until run time that would be caught at
> compile time in other languages (though static code checkers help).
Yes they do help, which is exactly the point.
> (Not much of a disadvantage because of Python's superb error
> diagnostics.)
That's certainly very optimistic of you.
If I had to pick just one out of compile time type checking versus run
time unit tests, I'd pick run time tests. But it is naive to deny the
benefits of compile time checks in catching errors that you otherwise
might not have found even with extensive unit tests (and lets face it,
we never have enough unit tests).
Ironically, type hinting will *reduce* the need for intrusive,
anti-duck-testing explicit calls to isinstance() at runtime:
def func(x:float):
if isinstance(x, float): ...
else: raise TypeError
Why bother making that expensive isinstance call every single time the
function is called, if the type checker can prove that x is always a
float?
> Python code typically says what it is doing, with the minimum of
> syntactic guff. (Well, apart from colons after if/while/try etc. :-) )
> Which makes it easy to read.
> Now it seems as if this proposal wants to start turning Python in the
> C++ direction, encouraging adding ugly boilerplate code. (This may only
> be tangentially relevant, but I want to scream when I see some
> combination of public/private/protected/static/extern etc., most of
> which I don't understand.)
Perhaps if you understood it you would be less inclined to scream.
> Chris A makes the valid point (if I understand correctly) that
> Authors of libraries should make it as easy as possible to
> (i) know what object types can be passed to functions
> (ii) diagnose when the wrong type of object is passed
> Authors of apps are not under such obligation, they can basically
> do what they want.
>
> Well,
> (i) can be done with good documentation (docstrings etc.).
> (ii) can be done with appropriate runtime checks and good error
> messages.
How ironic. After singing the praises of duck-typing, now you are
recommending runtime type checks.
As far as good error messages go, they don't help you one bit when the
application suddenly falls over in a totally unexpected place due to a
bug in your code.
I can't go into too many details due to commercial confidentiality, but
we experienced something similar recently. A situation nobody foresaw,
that wasn't guarded against, and wasn't tested for, came up after
deployment. There was a traceback, of course, but a failure in the field
200km away with a stressed customer and hundreds of angry users is not
as useful as a compile-time failure during development.
> You see where I'm going with this - adding type hints to Python feels a
> bit like painting feet on the snake.
Pythons are one of the few snakes which have vestigal legs:
http://en.wikipedia.org/wiki/Pelvic_spur
--
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 21/04/2015 12:23, Gustavo Carneiro wrote: Well, (i) can be done with good documentation (docstrings etc.). Documentation is not checked. It often loses sync with the actual code. Docs say one thing, code does another. That certainly something that could be fixed by formalising the documentation specification and having a checker check that rather than changing the syntax of the language... Chris ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 20/04/2015 19:30, Harry Percival wrote: Hi all, tldr; type hints in python source are scary. Would reserving them for stub files be better? I think Jack's summary of this is excellent and aligns well with where I think I'm coming from on this: https://mail.python.org/pipermail/python-dev/2015-April/139253.html Harry Also makes just as many good points so I'll reply here, with a note that while switching to (type/header/stub - my ordered preference for how to describe .pyi files) is preferable to type hints inside files, it's still a massive change to the language, not one I can say I've missed over the past 15 years, and one if asked to vote on (I know that's not the case ;-)) that I would choose to vote against. Anyway, I've not posted much to python-dev in quite a while, but this is a topic that I would be kicking myself in 5-10 years time when I've had to move to Javascript or because everyone else has drifted away from Python as it had become ugly... Chris ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 20/04/2015 20:09, Paul Moore wrote: On 20 April 2015 at 19:41, Barry Warsaw wrote: tldr; type hints in python source are scary. Would reserving them for stub files be better? I think so. I think PEP 8 should require stub files for stdlib modules and strongly encourage them for 3rd party code. Agreed. I have many of the same concerns as Harry, but I wouldn't have expressed them quite as well. I'm not too worried about actually removing annotations from the core language, but I agree that we should create a strong culture of "type hints go in stub files" to keep source files readable and clean. On that note, I'm not sure "stub" files is a particularly good name. Maybe "type files" would be better? Something that emphasises that they are the correct place to put type hints, not a workaround. Or we could just be honest and admit that we're choosing to add header files to Python. It's a shame, as it's more complexity, and it's being inflicted on those who might be writing a library for the first time, or those becoming core committers for the first time, or those just trying to "do the right thing". Currently, the burden is a heavier one (type inference, rather than reading it from a file) but borne by people best placed to handle it (authors of IDEs, type checking software, etc). Chris ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 21 April 2015 at 11:56, Rob Cliffe wrote: > On 21/04/2015 10:33, Cory Benfield wrote: > > On 21 April 2015 at 10:10, Chris Angelico > wrote: > > At this point, you may want to just stop caring about the exact type. > Part of the point of gradual typing is that you can short-cut a lot of > this. And quite frankly, this isn't really helping anything. Just skip > it and say that it's Union[Mapping, Iterable, None]. > > I think this is my problem with the proposal. This change doesn't > catch any of the bugs anyone was going to hit (no-one is passing a > callable to this parameter), and it doesn't catch any of the bugs > someone might actually hit (getting the tuple elements in the wrong > order, for example). At this point the type signature is worse than > useless. > > Exactly. At this point putting the type signatures in seems to be a > pointless exercise in masochism (for the author) and sadism (for the > reader). > > The refreshing thing about Python is that it is a fantastic, *concise*, > dynamically typed language, at the opposite end of the spectrum from C++ > (ugh!). > We have got used to the consequences (good and bad) of this: > Duck typing, e.g. a function to sort numbers (sorry Tim Peters bad > example) turns out to support any kind of object (e.g. strings) that > supports comparison. > Not to mention objects of some class that will be written in 5 > years time. > (Adding a type hint that restricted the argument to say a sequence > of numbers turns out to be a mistake. And what is a number? > Is Fraction? What about complex numbers, which can't be sorted? > What if the function were written before the Decimal class?) > Errors are often not caught until run time that would be caught at > compile time in other languages (though static code checkers help). > (Not much of a disadvantage because of Python's superb error > diagnostics.) > Python code typically says what it is doing, with the minimum of > syntactic guff. (Well, apart from colons after if/while/try etc. :-) ) > Which makes it easy to read. > Now it seems as if this proposal wants to start turning Python in the C++ > direction, encouraging adding ugly boilerplate code. (This may only be > tangentially relevant, but I want to scream when I see some combination of > public/private/protected/static/extern etc., most of which I don't > understand.) > > Chris A makes the valid point (if I understand correctly) that > Authors of libraries should make it as easy as possible to > (i) know what object types can be passed to functions > (ii) diagnose when the wrong type of object is passed > Authors of apps are not under such obligation, they can basically do > what they want. > > Well, > (i) can be done with good documentation (docstrings etc.). > Documentation is not checked. It often loses sync with the actual code. Docs say one thing, code does another. Documenting types in docstrings forces you to repeat yourself. You have to write the parameter names twice, once in the function definition line, another in the docstring. And you need to understand the syntax of whatever docstring format will be used to check the code. I'm sure I'm not the only one that thinks that type hints actually *improve readability*. I will not argue whether it makes the code uglier or prettier, but I am certain that most of the time the type hints make the code easier to read because you don't have to spend so much mental effort trying to figure out "gee, I wonder if this parameter is supposed to be bool, string, or int?" The type (or types) that a parameter is expected to have becomes explicit, otherwise you have to read the entire function body to understand. > (ii) can be done with appropriate runtime checks and good error > messages. > E.g. (Cory's example) I'm sure it is possible to test somehow if an object > is file-like (if only by trying to access it like a file). > Is thorough argument checking and provision of good diagnostics going to > be easy for the library author? No. Is it going to be a lot of work to do > thoroughly? Almost certainly yes. > But what the hell, writing a production-quality library is not an exercise > for newbies. > > It seems to me that type hints are attempting to be a silver bullet and to > capture in a simple formula what is often, in practice, *not simple at > all*, viz. "Is this passed object suitable?". Attempting - and failing, > except in the simplest cases. > Even if it only helps in the simplest cases, it saves the reader of the code a lot of effort if you add up all those little cases together. My suggestion, wherever an annotated type is too complex, don't annotate it. If you do /only/ the simple cases, the end result is easier to read. > > Apologies, Guido, but: > There was once a Chinese student who was a marvellous painter. He painted > a perfect life-like picture of a snake. > But he was unable to sto
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 21/04/2015 10:33, Cory Benfield wrote: On 21 April 2015 at 10:10, Chris Angelico wrote: At this point, you may want to just stop caring about the exact type. Part of the point of gradual typing is that you can short-cut a lot of this. And quite frankly, this isn't really helping anything. Just skip it and say that it's Union[Mapping, Iterable, None]. I think this is my problem with the proposal. This change doesn't catch any of the bugs anyone was going to hit (no-one is passing a callable to this parameter), and it doesn't catch any of the bugs someone might actually hit (getting the tuple elements in the wrong order, for example). At this point the type signature is worse than useless. Exactly. At this point putting the type signatures in seems to be a pointless exercise in masochism (for the author) and sadism (for the reader). The refreshing thing about Python is that it is a fantastic, *concise*, dynamically typed language, at the opposite end of the spectrum from C++ (ugh!). We have got used to the consequences (good and bad) of this: Duck typing, e.g. a function to sort numbers (sorry Tim Peters bad example) turns out to support any kind of object (e.g. strings) that supports comparison. Not to mention objects of some class that will be written in 5 years time. (Adding a type hint that restricted the argument to say a sequence of numbers turns out to be a mistake. And what is a number? Is Fraction? What about complex numbers, which can't be sorted? What if the function were written before the Decimal class?) Errors are often not caught until run time that would be caught at compile time in other languages (though static code checkers help). (Not much of a disadvantage because of Python's superb error diagnostics.) Python code typically says what it is doing, with the minimum of syntactic guff. (Well, apart from colons after if/while/try etc. :-) ) Which makes it easy to read. Now it seems as if this proposal wants to start turning Python in the C++ direction, encouraging adding ugly boilerplate code. (This may only be tangentially relevant, but I want to scream when I see some combination of public/private/protected/static/extern etc., most of which I don't understand.) Chris A makes the valid point (if I understand correctly) that Authors of libraries should make it as easy as possible to (i) know what object types can be passed to functions (ii) diagnose when the wrong type of object is passed Authors of apps are not under such obligation, they can basically do what they want. Well, (i) can be done with good documentation (docstrings etc.). (ii) can be done with appropriate runtime checks and good error messages. E.g. (Cory's example) I'm sure it is possible to test somehow if an object is file-like (if only by trying to access it like a file). Is thorough argument checking and provision of good diagnostics going to be easy for the library author? No. Is it going to be a lot of work to do thoroughly? Almost certainly yes. But what the hell, writing a production-quality library is not an exercise for newbies. It seems to me that type hints are attempting to be a silver bullet and to capture in a simple formula what is often, in practice, *not simple at all*, viz. "Is this passed object suitable?". Attempting - and failing, except in the simplest cases. Apologies, Guido, but: There was once a Chinese student who was a marvellous painter. He painted a perfect life-like picture of a snake. But he was unable to stop and leave it alone. In his zeal he went on to paint feet on the snake. Which of course completely spoiled the picture, as snakes don't have feet. Hence "to paint feet on the snake": to be unable to resist tinkering with something that is already good. (I suppose "If it ain't broke, don't fix it" is an approximate Western equivalent.) You see where I'm going with this - adding type hints to Python feels a bit like painting feet on the snake. Or at least turning it into a different language. Best wishes Rob Cliffe ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Mon, Apr 20, 2015 at 08:37:28PM -0700, Guido van Rossum wrote: > On Mon, Apr 20, 2015 at 4:41 PM, Jack Diederich wrote: > > > Twelve years ago a wise man said to me "I suggest that you also propose a > > new name for the resulting language" > > > > The barrage of FUD makes me feel like the woman who asked her doctor for a > second opinion and was told "you're ugly too." Don't worry Guido, some of us are very excited to see this coming to fruition :-) It's been over ten years since your first blog post on optional typing for Python. At least nobody can accuse you of rushing into this. http://www.artima.com/weblogs/viewpost.jsp?thread=85551 -- Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 21 April 2015 at 10:10, Chris Angelico wrote: > At this point, you may want to just stop caring about the exact type. > Part of the point of gradual typing is that you can short-cut a lot of > this. And quite frankly, this isn't really helping anything. Just skip > it and say that it's Union[Mapping, Iterable, None]. I think this is my problem with the proposal. This change doesn't catch any of the bugs anyone was going to hit (no-one is passing a callable to this parameter), and it doesn't catch any of the bugs someone might actually hit (getting the tuple elements in the wrong order, for example). At this point the type signature is worse than useless. It seems like the only place the type annotations will get used is in relatively trivial cases where the types are obvious anyway. I don't deny that *some* bugs will be caught, but I suspect they'll overwhelmingly be crass ones that would have been caught quickly anyway. The minute a type signature might actually help prevent a subtle bug we can't use them anymore because the toolchain isn't powerful enough to do it so we just put 'Any' and call it a day. And even then we haven't helped the trivial case much because anyone who wants to provide a duck-typed object that should be perfectly suitable in this case no longer can. What this proposal *needs* to be generally useful, IMO, is the ability to specify types that actually match the way we have encouraged people to write code: duck typed, favouring composition over inheritance, etc. At the very least that means 'type constraints' (see Haskell, Rust, really any language with a powerful and robust type system: hell, even see Go's interfaces) need to be something we can specify. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 6:58 PM, Cory Benfield wrote: > On 21 April 2015 at 01:45, Chris Angelico wrote: >> When you're writing a library, it can be a great help to provide type >> annotations, because every application that uses your library can >> benefit. > > It can be a great help to whom? Not to me (the library author), > because I can't use them in my library code, because I have to support > 2.7. That's by no means a bad thing (after all, most libraries are > written to help others), but I found it really unclear who was being > advantaged here. Mainly application users get the benefit, I expect. But I may be wrong. > ... That's got *nothing* on the type of the `files` > argument, which is the most incredibly polymorphic argument I've ever > seen: the best I can work out it would be: > > Optional[ > Union[ > Mapping[ > basestring, > Union[ > Tuple[basestring, Optional[Union[basestring, file]]], > Tuple[basestring, Optional[Union[basestring, file]], > Optional[basestring]], > Tuple[basestring, Optional[Union[basestring, file]], > Optional[basestring], Optional[Headers]] > ] > ], > Iterable[ > Tuple[ > basestring, > Union[ > Tuple[basestring, Optional[Union[basestring, file]]], > Tuple[basestring, Optional[Union[basestring, > file]], Optional[basestring]], > Tuple[basestring, Optional[Union[basestring, > file]], Optional[basestring], Optional[Headers]] > ] > ] > ] > ] At this point, you may want to just stop caring about the exact type. Part of the point of gradual typing is that you can short-cut a lot of this. And quite frankly, this isn't really helping anything. Just skip it and say that it's Union[Mapping, Iterable, None]. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 21 April 2015 at 01:45, Chris Angelico wrote: > When you're writing a library, it can be a great help to provide type > annotations, because every application that uses your library can > benefit. It can be a great help to whom? Not to me (the library author), because I can't use them in my library code, because I have to support 2.7. That's by no means a bad thing (after all, most libraries are written to help others), but I found it really unclear who was being advantaged here. To show the downside, I decided to annotate requests' API a little bit. Then I stopped, because I really couldn't work out how to do it. For example, requests.get() takes a URL parameter. This can be an instance of basestring, or anything that provides a __str__ or __unicode__ method that provides us a string that looks like a URL (such as a URL abstraction class). We don't know ahead of time what those objects may be, so there's no type signature I can provide here that is of any use beyond 'Any'. We also provide headers/params options, whose type signature would be (as far as I can tell) Union[Iterable[Tuple[basestring, basestring]], Mapping[basestring, basestring]], which is a fairly unpleasant type (and also limiting, as we totally accept two-element lists here as well as two-tuples. That's got *nothing* on the type of the `files` argument, which is the most incredibly polymorphic argument I've ever seen: the best I can work out it would be: Optional[ Union[ Mapping[ basestring, Union[ Tuple[basestring, Optional[Union[basestring, file]]], Tuple[basestring, Optional[Union[basestring, file]], Optional[basestring]], Tuple[basestring, Optional[Union[basestring, file]], Optional[basestring], Optional[Headers]] ] ], Iterable[ Tuple[ basestring, Union[ Tuple[basestring, Optional[Union[basestring, file]]], Tuple[basestring, Optional[Union[basestring, file]], Optional[basestring]], Tuple[basestring, Optional[Union[basestring, file]], Optional[basestring], Optional[Headers]] ] ] ] ] Headers = Union[ Mapping[basestring, basestring], Iterable[Tuple[basestring, basestring]], ] This includes me factoring out the Headers type for my own sanity, and does not include the fact that in practice almost all instances of 'basestring' above actually mean 'anything that can be safely cast to basestring', and 'file' means 'any file-like object including BytesIO and StringIO' (speaking of which, the PEP doesn't even begin to go into how to handle that kind of requirement: do we have some kind of Readable interface that can be implemented? Or do I have to manually union together all types someone might want to use?). Now, this can be somewhat improved: the three types of tuple (again, not just tuples, often lists!) can be factored out, as can the Union[basestring, file]. However, there's still really no way around the fact that this is a seriously complex type signature! Python has had years of people writing APIs that feel natural, even if that means weird contortions around 'types'. Writing an argument as 'clever' as the 'files' argument in requests in a statically typed language is an absolute nightmare, but Python makes it an easy and natural thing to do. Further, Python's type system is not sufficiently flexible to allow library authors to adequately specify the types their code actually works on. I need to be able to talk about interfaces, because interfaces are the contract around which APIs are build in Python, but I cannot do that with this system in a way that makes any sense at all. To even begin to be useful for library authors this PEP would need to allow some kind of type hierarchy that is *not* defined by inheritance, but instead by interfaces. We've been discouraging use of 'type' and 'isinstance' for years because they break duck typing, but that has *nothing* on what this PEP appears to do to duck typing. I suppose the TLDR of this email is that I think that libraries with 'pythonic' APIs are the least likely to take up this typing system because it will provide the least value to them. Maintaining signatures will be a pain (stub files are necessary), writing clear signatures will be a pain (see above), and writing signatures that are both sufficiently open to be back-compatible and sufficiently restrictive to be able to catch bugs seems like it might be very nearly impossible. I'd love for someone to tell me I'm wrong, though. I want to like this PEP, I really do. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Mon, 20 Apr 2015 20:43:38 -0400 "R. David Murray" wrote: > +1 to this from me too. I'm afraid that means I'm -1 on the PEP. > > I didn't write this in my earlier email because I wasn't sure about it, > but my gut reaction after reading Harry's email was "if type annotations > are used in the stdlib, I'll probably stop contributing". That doesn't > mean that's *true*, but that's the first time I've ever had that > thought, so it is probably worth sharing. I think it would be nice to know what the PEP means for daily stdlib development. If patches have to carry typing information each time they add/enhance an API that's an addition burden. If typing is done separately by interested people then it sounds like it wouldn't have much of an impact on everyone else's workflow. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 4/20/2015 9:07 PM, Chris Angelico wrote: On Tue, Apr 21, 2015 at 10:52 AM, Ben Finney wrote: Jack is not complaining only about *writing* code. He's complaining about the effect this will have on code that we all are expected to *read*. For reading, good function and parameter names and a good summary as the first line of the docstring are probably more useful *to humans* than formal type annotations. Ahh. Yes, that's a concern. When you go digging into that library to find out how it works, yes, you'd be face-to-face with their type annotations. Maybe it'd be of value to have a quick "code stripper" that takes away all the annotations, I was already thinking about that for Idle. The file should then be either semi read-only or renamed. Also options to merge annotations in from stub files for editing and to split back out when writing. plus any other junk/framing that you're not interested in, and gives you something you can browse in a text editor? It could take away all the Sphinx adornments from docstrings, any source control versioning markers, all that kind of thing. Interesting idea. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Mon, Apr 20, 2015 at 4:41 PM, Jack Diederich wrote: > Twelve years ago a wise man said to me "I suggest that you also propose a > new name for the resulting language" > The barrage of FUD makes me feel like the woman who asked her doctor for a second opinion and was told "you're ugly too." > I talked with many of you at PyCon about the costs of PEP 484. There are > plenty of people who have done a fine job promoting the benefits. > So excuse me if I also defend the PEP against your attack. > * It is not optional. Please stop saying that. The people promoting it > would prefer that everyone use it. If it is approved it will be optional in > the way that PEP8 is optional. If I'm reading your annotated code it is > certainly /not/ optional that I understand the annotations. > The PEP 8 analogy is actually great -- that PEP's style advice *is* optional and plenty of people ignore it happily. The benefits come out when you're sharing code but violations are no big deal. And as to requiring what the annotations mean: the CPython interpreter itself doesn't even use them when executing your code, so you can certainly pretend that the annotations aren't there and read the body of the function or the docstring instead. > * Uploading stubs for other people's code is a terrible idea. Who do I > contact when I update the interface to my library? The random Joe who > "helped" by uploading annotations three months ago and then quit the > internet? I don't even want to think about people maliciously adding stubs > to PyPI. > We're certainly not planning to let arbitrary people upload stubs for arbitrary code to PyPI that will automatically be used by the type checkers. (We can't stop people from publishing their stubs, just as you can't stop people from writing blog posts or stackoverflow answers with examples for your library.) The actual plan is to have a common repository of stubs (a prototype exists at https://github.com/JukkaL/typeshed) but we also plan some kind of submission review. I've proposed that when submitting stubs for a package you don't own, the typeshed owners ask the package owner what their position is, and we expect the answers to fall on the following spectrum: - I don't want stubs uploaded for my package - I'll write the stubs myself - I want to review all stubs that are uploaded for my package before they are accepted - Please go ahead and add stubs for my package and let me know when they're ready - Go ahead, I trust you This seems a reasonable due diligence policy that avoids the scenarios you're worried about. (Of course if you refuse stubs a black market for stubs might spring into existence. That sounds kind of exciting... :-) > * The cognitive load is very high. The average function signature will > double in length. This is not a small cost and telling me it is "optional" > to pretend that every other word on the line doesn't exist is a farce. > Have you never found yourself trying to read someone else's code and being stopped in your tracks because you couldn't find out what kind of argument was expected somewhere? I'm personally quite tired of seeing that an argument is a "file" and not knowing whether it's meant to be an IO stream or a filename (and was that bytes or unicode? :-). Or seeing something documented as "creation time" without knowing whether that's a datetime or a a POSIX timestamp (or maybe milliseconds or a string encoding a date). Etc., etc. And yes, you can put the info in docstrings or comments (though it won't be less verbose, and it will be more likely wrong). One way to see type hints is as a way to save on docstrings and comments by using a standard notation that ties type info to the argument without the need to repeat the argument name. Yes, you need to use your brain. But don't tell my that all the Python code you read or write is self-explanatory, because it isn't. > * Every company's style guide is about to get much longer. That in itself > is an indicator that this is a MAJOR language change and not just some > "optional" add-on. > Actually, standardization *reduces* the length of a company style guide, like PEP 8 did before. Would you rather have instructions for the specific linter your company uses, instead of the single phrase "use type hints (PEP 484)"? > * People will screw it up. The same people who can't be trusted to program > without type annotations are also going to be *writing* those type > annotations. > I don't follow this argument. So what? People will screw up anything you give them. introspection, metaclasses, import hooks, reduce(), you name it. > * Teaching python is about to get much less attractive. It will not be > optional for teachers to say "just pretend all this stuff over here doesn't > exist" > This argument is particularly weak. Teachers are actually *very* effective teaching only the parts of Python they need -- some start with just functions and variables. Others with just classes and methods. Few
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Mon, Apr 20, 2015 at 02:41:06PM -0400, Barry Warsaw wrote: > On Apr 20, 2015, at 07:30 PM, Harry Percival wrote: > > >tldr; type hints in python source are scary. Would reserving them for stub > >files be better? > > I think so. I think PEP 8 should require stub files for stdlib modules and > strongly encourage them for 3rd party code. A very, very strong -1 to that. Stub files are a necessary evil. Except where absolutely necessary, they should be strongly discouraged. A quote from the Go FAQs: Dependency management is a big part of software development today but the “header files” of languages in the C tradition are antithetical to clean dependency analysis—and fast compilation. http://golang.org/doc/faq#What_is_the_purpose_of_the_project Things that go together should be together. A function parameter and its type information (if any) go together: the type is as much a part of the parameter declaration as the name and the default. Putting them together is the best situation: def func(n: Integer): ... and should strongly be prefered as best practice for when you choose to use type hinting at all. Alternatives are not as good. Second best is to put them close by, as in a decorator: @typing(n=Integer) # Don't Repeat Yourself violation def func(n): ... A distant third best is a docstring. Not only does it also violate DRY, but it also increases the likelyhood of errors: def func(n): """Blah blah blah blah blah blah Arguments: m: Integer """ Keeping documentation and code in synch is hard, and such mistakes are not uncommon. Putting the type information in a stub file is an exponentially more distant fourth best, or to put it another way, *the worst* solution for where to put type hints. Not only do you Repeat Yourself with the name of the parameter, but also the name of the function (or method and class) AND module. The type information *isn't even in the same file*, which increases the chance of it being lost, forgotten, deleted, out of date, unmaintained, etc. -- Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Mon, Apr 20, 2015 at 07:30:39PM +0100, Harry Percival wrote: > Hi all, > > tldr; type hints in python source are scary. Would reserving them for stub > files be better? No no no, a thousand times no it would not! Please excuse my extreme reaction, but over on the python-list mailing list (comp.lang.python if you prefer Usenet) we already had this discussion back in January. Anyone wishing to read those conversations should start here: https://mail.python.org/pipermail/python-list/2015-January/697202.html https://mail.python.org/pipermail/python-list/2015-January/697315.html Be prepared for a long, long, long read. Nothing in your post hasn't already been discussed (except for your proposal to deprecate annotations altogether). So if you feel that I'm giving any of your ideas or concerns short-shrift, I'm not, it's just that I've already given them more time than I can afford. And now I get to do it all again, yay. (When reading those threads, please excuse my occasional snark towards Rick, he is a notorious troll on the list and sometimes I let myself be goaded into somewhat less than professional responses.) While I sympathise with your thought that "it's scary", I think it is misguided and wrong. As a thought-experiment, let us say that we roll back the clock to 1993 or thereabouts, just as Python 1.0 (or so) was about to be released, and Guido proposed adding *default values* to function declarations [assuming they weren't there from the start]. If we were used to Python's clean syntax: def zipmap(f, xx, yy): the thought of having to deal with default values: def zipmap(f=None, xx=(), yy=()): might be scary. Especially since those defaults could be arbitrarily complex expressions. Twenty years on, what should we think about such fears? Type hinting or declarations are extremely common in programming languages, and I'm not just talking about older languages like C and Java. New languages, both dynamic and static, like Cobra, Julia, Haskell, Go, Boo, D, F#, Fantom, Kotlin, Rust and many more include optional or mandatory type declarations. You cannot be a programmer without expecting to deal with type hints/declarations somewhere. As soon as you read code written in other languages (and surely you do that, don't you? you practically cannot escape Java and C code on the internet) and in my opinion Python cannot be a modern language without them. I'm going to respond to your recommendation to use stub files in another post (replying to Barry Warsaw), here I will discuss your concerns first. > My first reaction to type hints was "yuck", and I'm sure I'm not the only > one to think that. viz (from some pycon slides): > > def zipmap(f: Callable[[int, int], int], xx: List[int], >yy: List[int]) -> List[Tuple[int, int, int]]: > > arg. and imagine it with default arguments. You've picked a complex example and written it poorly. I'd say yuck too, but let's use something closer to PEP-8 formatting: def zipmap(f: Callable[[int, int], int], xx: List[int], yy: List[int] ) -> List[Tuple[int, int, int]]: Not quite so bad with each parameter on its own line. It's actually quite readable, once you learn what the annotations mean. Like all new syntax, of course you need to learn it. But the type hints are just regular Python expressions. > Of course, part of this reaction is just a knee-jerk reaction to the new > and unfamiliar, and should be dismissed, entirely justifiably, as mere > irrationality. But I'm sure sensible people agree that they do make our > function definitions longer, more complex, and harder to read. Everything has a cost and all features, or lack of features, are a trade-off. Function definitions could be even shorter and simpler and easier to read if we didn't have default values. [...] > I'm not so sure. My worry is that once type hinting gets standardised, > then they will become a "best practice", and there's a particular > personality type out there that's going to start wanting to add type hints > to every function they write. Similarly to mindlessly obeying PEP8 while > ignoring its intentions, hobgoblin-of-little-minds style, I think we're > very likely to see type hints appearing in a lot of python source, or a lot > of pre-commit-hook checkers. Pretty soon it will be hard to find any open > source library code that doesn't have type hints, or any project style > guide that doesn't require them. I doubt that very much. I'm not a betting man, but if I were, I would put money on it. Firstly: libraries tend to be multi-version, and these days they are often hybrid Python 2 & 3 code. Since annotations are 3 only, libraries cannot use these type hints until they drop support for Python 2.7, which will surely be *no less* than five years away. Probably more like ten. So "annotations everywhere" are, at best, many years away. Secondly, more importantl
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Mon, Apr 20, 2015 at 11:34:51PM +0100, Harry Percival wrote: > exactly. yay stub files! we all agree! everyone loves them! Not even close. -- Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On 20 Apr 2015 14:44, "Barry Warsaw" wrote: > > On Apr 20, 2015, at 07:30 PM, Harry Percival wrote: > > >tldr; type hints in python source are scary. Would reserving them for stub > >files be better? > > I think so. I think PEP 8 should require stub files for stdlib modules and > strongly encourage them for 3rd party code. +1 Having stub files take precedence over inline annotations would also neatly deal with the annotation compatibility problem (you can use a stub file to annotate code using annotations for another purpose). Another point in favour of that approach: stub files could be evaluated holistically rather than statement at a time, permitting implicit forward references. Cheers, Nick. > > Cheers, > -Barry > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Tue, Apr 21, 2015 at 10:52 AM, Ben Finney wrote: > Chris Angelico writes: > >> On Tue, Apr 21, 2015 at 9:41 AM, Jack Diederich wrote: >> > * It is not optional. Please stop saying that. The people promoting >> > it would prefer that everyone use it. If it is approved it will be >> > optional in the way that PEP8 is optional. If I'm reading your >> > annotated code it is certainly /not/ optional that I understand the >> > annotations. > […] >> >> Maybe I'm completely misreading everything here […] > > I think you've misunderstood the complaint. > >> When you're writing a library, it can be a great help to provide type >> annotations, because every application that uses your library can >> benefit. When you're writing an application, you can completely ignore >> them, but still get the benefit of everyone else's. > > Jack is not complaining only about *writing* code. He's complaining > about the effect this will have on code that we all are expected to > *read*. Ahh. Yes, that's a concern. When you go digging into that library to find out how it works, yes, you'd be face-to-face with their type annotations. I doubt the worst-case complex ones are going to come up all that often, though. Sure, you might declare that something returns a list of dictionaries mapping tuples of integers and strings to list of sets of atoms, but that's hardly common. (And chances are you can just declare that it returns List[Dict] and have done with it.) Maybe it'd be of value to have a quick "code stripper" that takes away all the annotations, plus any other junk/framing that you're not interested in, and gives you something you can browse in a text editor? It could take away all the Sphinx adornments from docstrings, any source control versioning markers, all that kind of thing. Then you could read through the code in a simpler form, while still having type annotations there for you if you need them. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
Chris Angelico writes: > On Tue, Apr 21, 2015 at 9:41 AM, Jack Diederich wrote: > > * It is not optional. Please stop saying that. The people promoting > > it would prefer that everyone use it. If it is approved it will be > > optional in the way that PEP8 is optional. If I'm reading your > > annotated code it is certainly /not/ optional that I understand the > > annotations. […] > > Maybe I'm completely misreading everything here […] I think you've misunderstood the complaint. > When you're writing a library, it can be a great help to provide type > annotations, because every application that uses your library can > benefit. When you're writing an application, you can completely ignore > them, but still get the benefit of everyone else's. Jack is not complaining only about *writing* code. He's complaining about the effect this will have on code that we all are expected to *read*. Programmers spend a great deal of time reading code written by other people. The costs of this proposal are only partly on the writers of the code; they are significantly borne by the people *reading* that code. For them, it is not optional. > I have no fears for my own code. Are you afraid for yours? Jack, if I understand correctly, fears for the code that will be written by others in conformance with this proposal, that he will then have to read and understand. -- \ “The opposite of a correct statement is a false statement. But | `\ the opposite of a profound truth may well be another profound | _o__) truth.” —Niels Bohr | Ben Finney ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
