date:20150424


Barry Warsaw wrote:

Sure, tools can be updated but it is it *necessary*
to choose a syntax that breaks tools?

def async useful():

seems okay to me.


That will break any tool that assumes the word following
'def' is the name of the function being defined.

Putting it at the end would seem least likely to
cause breakage:

   def useful() async:

--
Greg
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] async/await in Python; v2


Yury Selivanov wrote:


If you have a future object 'fut', it's not intuitive
or pythonic to write 'cocall fut()'.


Another way to approach that would be to provide
a cofunction await() used like this:

   cocall await(fut)

That would read more naturally and wouldn't require
modifying fut at all.

--
Greg
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] async/await in Python; v2


Yury Selivanov wrote:

In a PEP 3152 aware version of asyncio, it's just *not
possible to write*

cocall gather(coro1(1,2), coro(2,3))

you just have to use your 'costart' built-in:

cocall gather(costart(coro1, 1, 2), costart(coro, 2,3)).


Another way to write that would be

  cocall gather(Task(coro1, 1, 2), Task(coro, 2, 3))

I think that actually reads quite nicely, and makes it
very clear that parallel tasks are being spawned, rather
than invoked sequentially. With the current way, that's
not clear at all.

It's not quite as convenient, because you don't get
currying for free the way you do with generators. But
I feel that such implicit currying is detrimental to
readability. It looks like you're passing the results
returned by coro1 and coro2 to gather, rather than
coro1 and coro2 themselves.

Yes, it will require some code to be changed, but
if you're turning all your coroutines into cofunctions
or async defs, you're changing quite a lot of things
already.


PEP 3152 was created in pre-asyncio era, and it shows.


I would say that asyncio was created in a pre-PEP-3152
world. Or at least it was developed without allowing
for the possibility of adopting something like PEP 3152
in the future.

Asyncio was based on generators and yield-from because
it was the best thing we had at the time. I'll be
disappointed if we've raced so far ahead with those
ideas that it's now impossible to replace them with
anything better.

PEP 3152 is designed to present a *simpler* model of
coroutine programming, by having only one concept, the
suspendable function, instead of two -- generator
functions on the one hand, and iterators/futures/awaitables/
whatever you want to call them on the other.

PEP 492 doesn't do that. It adds some things and
changes some things, but it doesn't simplify anything.


Your idea of syntaticaly forcing to use 'cocall' with
parens is cute,


You say that as though "forcing" the use of parens were
a goal in itself. It's not -- it's a *consequence* of
what a cocall is.

--
Greg
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3152 and yield from Future()


Guido van Rossum wrote:

I think this is the nail in PEP 3152's coffin.


Seems more like a small tack to me. :-)
I've addressed all the issues raised there in
earlier posts.

--
Greg
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3152 and yield from Future()


Victor Stinner wrote:

Oh, I missed something in the PEP 3152: a obj__cocall__() method can
be an iterator/generator, it can be something different than a
cofunction.


In fact, it *can't* be cofunction. It's part of the
machinery for implementing cofunctions.


It's not easy to understand the whole puzzle. IMO the PEP 492 better
explains how pieces are put together ;-)


Yes, it's written in a rather minimal style, sorry
about that.

--
Greg
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3152 and yield from Future()


Yury Selivanov wrote:


It's a common pattern in asyncio when functions
return futures.  It's OK later to refactor those
functions to coroutines *and* vice-versa.  This
is a fundamental problem for PEP 3152 approach.


Hmmm. So you have an ordinary function that returns
a future, and you want to turn it into a coroutine
function, but still have it return a future in
order to keep the API the same, is that right?

Turning it into a coroutine means you're going
to have to change every site that calls it, so
its API has already changed. Given that, I'm not
sure what advantage there is in keeping the future-
returning part of the API.

However, if we use the await()-cofunction idea,
then a call to the initial version looks like

   cocall await(f(x))

and after the refactoring it becomes

   cocall await(cocall f(x))

That doesn't look so bad to me.

--
Greg
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] async/await in Python; v2

Stephen J. Turnbull wrote:

Yury Selivanov writes:

 > I also read "for async item in iter:" as "I'm iterating iter
 > with async item".

I thought that was precisely the intended semantics: item is available
asynchronously.

The async-at-the-end idea could be used here as well.

   for item in iter async:
  ...

   with something as x async:
  ...

--
Greg
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3152 and yield from Future()

2015-04-24 Thread Paul Moore

On 24 April 2015 at 09:34, Greg Ewing  wrote:
> and after the refactoring it becomes
>
>cocall await(cocall f(x))
>
> That doesn't look so bad to me.

I've not been following this discussion (and coroutines make my head
hurt) but this idiom looks like it's bound to result in people getting
the idea that you scatter "cocall" throughout an expression until you
get it to work.

Paul
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3152 and yield from Future()


Paul Moore wrote:

On 24 April 2015 at 09:34, Greg Ewing  wrote:


  cocall await(cocall f(x))

That doesn't look so bad to me.


I've not been following this discussion (and coroutines make my head
hurt) but this idiom looks like it's bound to result in people getting
the idea that you scatter "cocall" throughout an expression until you
get it to work.


They won't need to do that, because they'll get told
exactly where they've left one out, or put one in that
they shouldn't have.

Also, the places you need to put cocall are exactly
the same as the places you need yield-from currently,
or await under PEP 492.

--
Greg
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] async/await in Python; v2

2015-04-24 Thread Andrew Svetlov

On Fri, Apr 24, 2015 at 3:14 AM, Greg Ewing  wrote:
> Andrew Svetlov wrote:
>
>> But we already have asyncio and code based on asyncio coroutines.
>> To make it work I should always use costart() in places where asyncio
>> requires coroutine.
>
>
> As I understand it, asyncio would require changes to
> make it work seamlessly with PEP 492 as well, since
> an object needs to have either a special flag or
> an __await__ method before it can have 'await'
> applied to it.
>
PEP 492 requires a change of asyncio.Future only.
PEP 3152 requires of change in any asyncio-based library, this is the
difference.
>
> --
> Greg
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com



-- 
Thanks,
Andrew Svetlov
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3152 and yield from Future()

2015-04-24 Thread Andrew Svetlov

On Fri, Apr 24, 2015 at 11:34 AM, Greg Ewing
 wrote:
> Yury Selivanov wrote:
>
>> It's a common pattern in asyncio when functions
>> return futures.  It's OK later to refactor those
>> functions to coroutines *and* vice-versa.  This
>> is a fundamental problem for PEP 3152 approach.
>
>
> Hmmm. So you have an ordinary function that returns
> a future, and you want to turn it into a coroutine
> function, but still have it return a future in
> order to keep the API the same, is that right?

No. In asyncio there is no difference between coroutine and regular
function returning future.
>From caller site next both are equal:

@asyncio.coroutine
def f():
return 1

def g():
fut = asyncio.Future()
fut.set_result(1)
return fut

Both may be called via `yield from`:
ret1 = yield from f()
ret2 = yield from g()

>
> Turning it into a coroutine means you're going
> to have to change every site that calls it, so
> its API has already changed. Given that, I'm not
> sure what advantage there is in keeping the future-
> returning part of the API.
>
> However, if we use the await()-cofunction idea,
> then a call to the initial version looks like
>
>cocall await(f(x))
>
> and after the refactoring it becomes
>
>cocall await(cocall f(x))
>
> That doesn't look so bad to me.
>
> --
> Greg
>
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com



-- 
Thanks,
Andrew Svetlov
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] async/await in Python; v2

On Thu, Apr 23, 2015 at 01:51:52PM -0400, Barry Warsaw wrote:

> Why "async def" and not "def async"?
> 
> My concern is about existing tools that already know that "def" as the first
> non-whitespace on the line starts a function/method definition.  Think of a
> regexp in an IDE that searches backwards from the current line to find the
> function its defined on.  Sure, tools can be updated but it is it *necessary*
> to choose a syntax that breaks tools?

Surely its the other way? If I'm searching for the definition of a 
function manually, I search for "def spam". `async def spam` will still 
be found, while `def async spam` will not.

It seems to me that tools that search for r"^\s*def\s+spam\s*\(" are 
going to break whichever choice is made, while a less pedantic search 
like r"def\s+spam\s*\(" will work only if async comes first.

-- 
Steve
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] async/await in Python; v2

2015-04-24 Thread Barry Warsaw

On Apr 24, 2015, at 11:17 PM, Steven D'Aprano wrote:

>It seems to me that tools that search for r"^\s*def\s+spam\s*\(" are

They would likely search for something like r"^\s*def\s+[a-zA-Z0-9_]+" which
will hit "def async spam" but not "async def".

Cheers,
-Barry
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] async/await in Python; v2

2015-04-24 Thread Barry Warsaw

On Apr 24, 2015, at 08:03 PM, Greg Ewing wrote:

>Putting it at the end would seem least likely to
>cause breakage:
>
>def useful() async:

That's not bad IMHO.  I wonder how crazy it is in the face of, ahem, function
annotations.

Cheers,
-Barry
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3152 and yield from Future()


Greg,

On 2015-04-24 4:13 AM, Greg Ewing wrote:

Guido van Rossum wrote:

I think this is the nail in PEP 3152's coffin.


Seems more like a small tack to me. :-)
I've addressed all the issues raised there in
earlier posts.



I'm sorry, but this is ridiculous.

You haven't addressed the issues. We raise issues,
saying that your PEP isn't backwards compatible and
is breaking existing idioms.  You say - there is a
workaround for that; or that we can rewrite that and
make it like that; or something else that doesn't
make any sense for existing asyncio developers.

Your PEP isn't backwards compatible. Period.

It *will* be harder for people to get, as it *does*
introduce a new calling grammar that isn't obvious
for at least some people.

We, asyncio developers, who write asyncio code,
*don't* want to write 'cocall fut()'.  I don't
understand *why* I'm required to put parentheses
there (besides someone just requiring me to do so,
because they failed to solve some problem in
backwards compatible way).  You avoid confusion
in one place, but you introduce it in other places.

I'm sorry, but your current way of handling the
discussion isn't really productive.  You don't
listen to arguments by Victor Stinner, Andrew
Svetlov, and me.  At this point, this whole PEP
3152 related discussion isn't helping anyone.

Yury

P.S. I'm sorry if this sounded harsh, this wasn't
my intent.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] typeshed for 3rd party packages

On Wed, Apr 22, 2015 at 11:26:14AM -0500, Ian Cordasco wrote:

> On a separate thread Cory provided an example of what the hints would look
> like for *part* of one function in the requests public functional API.
> While our API is outwardly simple, the values we accept in certain cases
> are actually non-trivially represented. Getting the hints *exactly* correct
> would be extraordinarily difficult.

I don't think you need to get them exactly correct. The type-checker 
does two things:

(1) catch type errors involving types which should not be allowed;

(2) allow code which involves types which should be allowed.

If the type hints are wrong, there are two errors: false positives, when 
code which should be allowed is flagged as a type error; and false 
negatives, when code which should be flagged as an error is not.
Ideally, there should be no false positives. But false negatives are not 
so important, since you will still be doing runtime checks. All that 
means is that the static type-checker will be a little less capable of 
picking up type errors at compile time.

-- 
Steve
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] typeshed for 3rd party packages

2015-04-24 Thread Cory Benfield

On 24 April 2015 at 15:21, Steven D'Aprano  wrote:

> If the type hints are wrong, there are two errors: false positives, when
> code which should be allowed is flagged as a type error; and false
> negatives, when code which should be flagged as an error is not.
> Ideally, there should be no false positives. But false negatives are not
> so important, since you will still be doing runtime checks. All that
> means is that the static type-checker will be a little less capable of
> picking up type errors at compile time.

I think that's a rational view that will not be shared as widely as I'd like.

Given that the purpose of a type checker is to catch bugs caused by
passing incorrectly typed objects to a function, it seems entirely
reasonable to me to raise a bug against a type hint that allows code
that was of an incorrect type where that incorrectness *could* have
been caught by the type hint. Extending from that into the general
ratio of "reports that are actually bugs" versus "reports that are
errors on the part of the reporter", I can assume that plenty of
people will raise bug reports for incorrect cases as well.

>From the perspective of sustainable long-term maintenance, I think the
only way to do type hints is to have them be sufficiently exhaustive
that a user would have to actively *try* to hit an edge case false
negative. I believe that requests' API is too dynamically-typed to fit
into that category at this time.

PS: I should mention that, as Gary Bernhardt pointed out at PyCon,
people often believe (incorrectly) that types are a replacement for
tests. For that reason I feel like underspecified type hints are
something of an attractive nuisance. Again, I really think this is a
case of do it properly or not at all.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Summary of Python tracker Issues

2015-04-24 Thread Python tracker


ACTIVITY SUMMARY (2015-04-17 - 2015-04-24)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open4814 (+22)
  closed 31000 (+43)
  total  35814 (+65)

Open issues with patches: 2249 


Issues opened (46)
==

#15582: Enhance inspect.getdoc to follow inheritance chains
http://bugs.python.org/issue15582  reopened by serhiy.storchaka

#23991: ZipFile sanity checks
http://bugs.python.org/issue23991  opened by Antony.Lee

#23992: multiprocessing: MapResult shouldn't fail fast upon exception
http://bugs.python.org/issue23992  opened by neologix

#23993: Use surrogateescape error handler by default in open() if the 
http://bugs.python.org/issue23993  opened by haypo

#23994: argparse fails to detect program name when there is a slash at
http://bugs.python.org/issue23994  opened by boramalper

#23995: msvcrt could not be imported
http://bugs.python.org/issue23995  opened by petrikas

#23996: _PyGen_FetchStopIterationValue() crashes on unnormalised excep
http://bugs.python.org/issue23996  opened by scoder

#23997: unicodedata_UCD_lookup() has theoretical buffer overflow
http://bugs.python.org/issue23997  opened by christian.heimes

#23999: Undefined behavior in dtoa.c (rshift 32 of 32bit data type)
http://bugs.python.org/issue23999  opened by christian.heimes

#24000: More fixes for the Clinic mapping of converters to format unit
http://bugs.python.org/issue24000  opened by larry

#24001: Clinic: use raw types in types= set
http://bugs.python.org/issue24001  opened by larry

#24004: avoid explicit generator type check in asyncio
http://bugs.python.org/issue24004  opened by scoder

#24009: Get rid of rare format units in PyArg_Parse*
http://bugs.python.org/issue24009  opened by serhiy.storchaka

#24010: Add error checks to PyInit__locale()
http://bugs.python.org/issue24010  opened by christian.heimes

#24011: Add error checks to PyInit_signal()
http://bugs.python.org/issue24011  opened by christian.heimes

#24012: Add error checks to PyInit_pyexpat()
http://bugs.python.org/issue24012  opened by christian.heimes

#24013: Improve os.scandir() and DirEntry documentation
http://bugs.python.org/issue24013  opened by benhoyt

#24015: timeit should start with 1 loop, not 10
http://bugs.python.org/issue24015  opened by nomeata

#24016: Add a Sprints organization/preparation section to devguide
http://bugs.python.org/issue24016  opened by willingc

#24017: Implemenation of the PEP 492 - Coroutines with async and await
http://bugs.python.org/issue24017  opened by haypo

#24018: add a Generator ABC
http://bugs.python.org/issue24018  opened by scoder

#24021: document urllib.urlretrieve
http://bugs.python.org/issue24021  opened by krichter

#24022: Python heap corruption issue
http://bugs.python.org/issue24022  opened by benjamin.peterson

#24024: str.__doc__ needs an update
http://bugs.python.org/issue24024  opened by lemburg

#24026: Python hangs forever in wait() of threading.py
http://bugs.python.org/issue24026  opened by appidman

#24027: IMAP library lacks documentation about expected parameter type
http://bugs.python.org/issue24027  opened by pmoleri

#24028: Idle: add doc subsection on calltips
http://bugs.python.org/issue24028  opened by terry.reedy

#24030: IMAP library encoding enhancement
http://bugs.python.org/issue24030  opened by pmoleri

#24032: urlparse.urljoin does not add query part
http://bugs.python.org/issue24032  opened by albertsmuktupavels

#24033: Update _test_multiprocessing.py to use script helpers
http://bugs.python.org/issue24033  opened by bobcatfish

#24034: Make fails Objects/typeslots.inc
http://bugs.python.org/issue24034  opened by masamoto

#24035: When Caps Locked,  + alpha-character still displayed as
http://bugs.python.org/issue24035  opened by principia1687

#24036: GB2312 codec is using a wrong covert table
http://bugs.python.org/issue24036  opened by Ma Lin

#24037: Argument Clinic: add the boolint converter
http://bugs.python.org/issue24037  opened by serhiy.storchaka

#24039: Minimize option doesn't work on Search Dialog box for idle
http://bugs.python.org/issue24039  opened by prince09cs

#24040: plistlib assumes dict_type is descendent of dict
http://bugs.python.org/issue24040  opened by Behdad.Esfahbod

#24041: Implement Mac East Asian encodings properly
http://bugs.python.org/issue24041  opened by Behdad.Esfahbod

#24042: Convert os._getfullpathname() and os._isdir() to Argument Clin
http://bugs.python.org/issue24042  opened by serhiy.storchaka

#24043: Implement mac_romanian and mac_croatian encodings
http://bugs.python.org/issue24043  opened by Behdad.Esfahbod

#24045: Behavior of large returncodes  (sys.exit(nn))
http://bugs.python.org/issue24045  opened by ethan.furman

#24046: Incomplete build on AIX
http://bugs.python.org/issue24046  opened by [email protected]

#24048: remove_module() needs to save/restore exception state
http:

[Python-Dev] PEP 492 vs. PEP 3152, new round

2015-04-24 Thread Guido van Rossum

I've tried to catch up with the previous threads. A summary of issues
brought up:

1. precise syntax of `async def` (or do we need it at all)
2. do we need `async for` and `async with` (and how to spell them)
3. syntactic priority of `await`
4. `cocall` vs. `await`
5. do we really need `__aiter__` and friends
6. StopAsyncException
7. compatibility with asyncio and existing users of it

(I've added a few myself.)

I'll try to take them one by one.


*1. precise syntax of `async def`*

Of all the places to put `async` I still like *before* the `def` the best.
I often do "imprecise search" for e.g. /def foo/ and would be unhappy if
this didn't find async defs. Putting it towards the end (`def foo async()`
or `def foo() async`) makes it easier to miss. A decorator makes it hard to
make the syntactic distinctions required to reject `await` outside an async
function. So I still prefer *`async def`*.

*2. do we need `async for` and `async with`*

Yes we do. Most of you are too young to remember, but once upon a time you
couldn't loop over the lines of a file with a `for` loop like you do now.
The amount of code that was devoted to efficiently iterate over files was
tremendous. We're back in that stone age with the asyncio `StreamReader`
class -- it supports `read()`, `readline()` and so on, but you can't use it
with `for`, so you have to write a `while True` loop. `asyncio for` makes
it possible to add a simple `__anext__` to the `StreamReader` class, as
follows:
```
async def __anext__(self):
line = await self.readline()
if not line:
   raise StopAsyncIteration
return line
```
A similar argument can be made for `async with`; the transaction commit is
pretty convincing, but it also helps to be able to wait e.g. for a
transport to drain upon closing a write stream. As for how to spell these,
I think having `async` at the front makes it most clear that this is a
special form.

(Though maybe we should consider `await for` and `await with`? That would
have the advantage of making it easy to scan for all suspension points by
searching for /await/. But being a verb it doesn't read very well.)

*3. syntactic priority of `await`*

Yury, could you tweak the syntax for `await` so that we can write the most
common usages without parentheses? In particular I'd like to be able to
write
```
return await foo()
with await foo() as bar: ...
foo(await bar(), await bletch())
```
(I don't care about `await foo() + await bar()` but it would be okay.)
```
I think this is reasonable with some tweaks of the grammar (similar to what
Greg did for cocall, but without requiring call syntax at the end).

*4. `cocall` vs. `await`*

Python evolves. We couldn't have PEP 380 (`yield from`) without prior
experience with using generators as coroutines (PEP 342), which in turn
required basic generators (PEP 255), and those were a natural evolution of
Python's earlier `for` loop.

We couldn't PEP 3156 (asyncio) without PEP 380 and all that came before.
The asyncio library is getting plenty of adoption and it has the concept of
separating the *getting* of a future[1] from *waiting* for it.  IIUC this
is also how `await` works in C# (it just requires something with an async
type). This has enabled a variety of operations that take futures and
produce more futures.

[1] I write `future` with a lowercase 'f' to include concepts like
coroutine generator objects.

*I just can't get used to this aspect of PEP 3152, so I'm rejecting it.*
Sorry Greg, but that's the end. We must see `await` as a refinement of
`yield from`, not as an alternative. (Yury: PEP 492 is not accepted yet,
but you're getting closer.)

One more thing: this separation is "Pythonic" in the sense that it's
similar to the way *getting* a callable object is a separate act from
*calling* it. While this is a cause for newbie bugs (forgetting to call an
argument-less function) it has also enabled the concept of "callable" as
more general and more powerful in Python: any time you need to pass a
callable, you can pass e.g. a bound method or a class or something you got
from `functools.partial`, and that's a useful thing (other languages
require you to introduce something like a lambda in such cases, which can
be painful if the thing you wrap has a complex signature -- or they don't
support function parameters at all, like Java).

I know that Greg defends it by explaining that `cocal f(args)` is not a
`cocall` operator applied to `f(args)`, it is the *single* operator `cocall
...(args)` applied to `f`. But this is too subtle, and it just doesn't jive
with the long tradition of using `yield from f` where f is some previously
obtained future.

*5. do we really need `__aiter__` and friends*

There's a lot of added complexity, but I think it's worth it. I don't think
we need to make the names longer, the 'a' prefix is fine for these methods.
I think it's all in the protocols: regular `with` uses `__enter__` and
`__exit__`; `async with` uses `__aenter__` and `__aexit__` (which must
ret

Re: [Python-Dev] typeshed for 3rd party packages

On Fri, Apr 24, 2015 at 03:44:45PM +0100, Cory Benfield wrote:
> On 24 April 2015 at 15:21, Steven D'Aprano  wrote:
> 
> > If the type hints are wrong, there are two errors: false positives, when
> > code which should be allowed is flagged as a type error; and false
> > negatives, when code which should be flagged as an error is not.
> > Ideally, there should be no false positives. But false negatives are not
> > so important, since you will still be doing runtime checks. All that
> > means is that the static type-checker will be a little less capable of
> > picking up type errors at compile time.
> 
> I think that's a rational view that will not be shared as widely as I'd like.

I can't tell if you are agreeing with me, or disagreeing. The above 
sentence seems to be agreeing with me, but you later end your message 
with "do it properly or not at all" which disagrees. So I'm confused.

> Given that the purpose of a type checker is to catch bugs caused by
> passing incorrectly typed objects to a function, it seems entirely
> reasonable to me to raise a bug against a type hint that allows code
> that was of an incorrect type where that incorrectness *could* have
> been caught by the type hint.

Of course it is reasonable for people to submit bug reports to do with 
the type hints. And it is also reasonable for the package maintainer to 
reject the bug report as "Won't Fix" if it makes the type hint too 
complex.

The beauty of gradual typing is that unlike Java or Haskell, you can 
choose to have as little or as much type checking as works for you. You 
don't have to satisfy the type checker over the entire program before 
the code will run, you only need check the parts you want to check.

> Extending from that into the general
> ratio of "reports that are actually bugs" versus "reports that are
> errors on the part of the reporter", I can assume that plenty of
> people will raise bug reports for incorrect cases as well.

Okay. Do you get many false positive bug reports for your tests too?

> From the perspective of sustainable long-term maintenance, I think the
> only way to do type hints is to have them be sufficiently exhaustive
> that a user would have to actively *try* to hit an edge case false
> negative. I believe that requests' API is too dynamically-typed to fit
> into that category at this time.

I think we agree that, static type checks or no static type checks, 
requests is going to need to do runtime type checks. So why does it 
matter if it misses a few type errors at compile time?

I think we're all in agreement that for extremely dynamic code like 
requests, you may not get as much value from static type checks as some 
other libraries or applications. You might even decide that you get no 
value at all. Okay, that's fine. I'm just suggesting that you don't have 
just two choices, "all or nothing". The whole point of gradual typing is 
to give developers more options.

> PS: I should mention that, as Gary Bernhardt pointed out at PyCon,
> people often believe (incorrectly) that types are a replacement for
> tests.

They *can* be a replacement for tests. You don't see Java or Haskell
programmers writing unit tests to check that their code never tries to 
add a string to a float. Even if they could write such as test, they 
don't bother because the type checker will catch that sort of error.

The situation in Python is a bit different, and as Antoine points out, 
libraries cannot rely on their callers obeying the type restrictions of 
the public API. (Private functions are different -- if you call my 
private function with the wrong type and blow up your computer, it's 
your own fault.) For libraries, I see type checks as complementing 
tests, not replacing them.

But for application code, type checks may replace unit tests, provided 
that nobody checks in production code until both the type checker and 
the unit tests pass. If you work under that rule, there's no point in 
having the unit tests check what the type checker already tested.

> For that reason I feel like underspecified type hints are
> something of an attractive nuisance. Again, I really think this is a
> case of do it properly or not at all.

In my opinion, underspecified type hints are no more of an attractive 
nuisance than a test suite which doesn't test enough. Full coverage is 
great, but 10% coverage is better than 5% coverage, which is better than 
nothing. That applies whether we are talking about tests, type checks, 
or documentation.

-- 
Steve
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Type hints -- a mediocre programmer's reaction

2015-04-24 Thread Ronan Lamy

Le 23/04/15 14:55, Paul Sokolovsky a écrit :

Hello,

On Thu, 23 Apr 2015 09:15:44 -0400
Daniel Holth wrote:

[]

Also ask why no one used type specifier, they are possible since
Python 3.0 ?
Because it is the wrong way for Python.

That's an example of how perceptions differ. In my list, everyone(*)
uses them - MyPy, MicroPython, etc. Even more should use them (any
JIT module, which are many), but sit in the bushes, waiting for a
kick, like PEP484 provides.

It's OK that type hints are only to assist the programmer.

Yes, it's OK to have a situation where type hints assist only a
programmer. It's not OK to think that type hints may be useful only for
programmer, instead of bunch more purposes, several of which
were already shown in the long previous discussion.

PyPy's FAQ
has an explanation of why type hints are not for performance.
http://pypy.readthedocs.org/en/latest/faq.html#would-type-annotations-help-pypy-s-performance

You probably intended to write "why type hints are not for *PyPy's*
performance". There're many other language implementations and modules
for which it may be useful, please don't limit your imagination by a
single case.

Those points apply to basically any compliant implementation of Python
relying on speculative optimisation. Python is simply too dynamic for
PEP484-style hints to provide any useful performance improvement targets.

And speaking of PyPy, it really should think how to improve its
performance - not of generated programs, but of generation itself. If
compilation of a trivial program on a pumpy hardware takes 5 minutes
and gigabytes of RAM and diskspace, few people will use it for other
purposes beyond curiosity. There's something very un-Pythonic in
waiting 5 mins just to run 10-line script. Type hints can help here
too ;-) (by not wasting resources propagating types thru the same old
standard library for example).

Sorry, but that's nonsense. PyPy would be a seriously useless
interpreter if running a 10-line script required such a lengthy
compilation, so, obviously, that's not what happens.

You seem to misunderstand what PyPy is: it's an interpreter with a
just-in-time compiler, not a static compiler. It doesn't generate
programs in any meaningful sense. Instead, it interprets the program,
and when it detects a hot code path, it compiles it to machine code
based on the precise types it sees. No resources are wasted on code that
isn't actually executed.

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 492 vs. PEP 3152, new round


Guido,

On 2015-04-24 1:03 PM, Guido van Rossum wrote:


*3. syntactic priority of `await`*

Yury, could you tweak the syntax for `await` so that we can write the most
common usages without parentheses? In particular I'd like to be able to
write
```
return await foo()
with await foo() as bar: ...
foo(await bar(), await bletch())
```
(I don't care about `await foo() + await bar()` but it would be okay.)
```
I think this is reasonable with some tweaks of the grammar (similar to what
Greg did for cocall, but without requiring call syntax at the end).

I don't remember the reason why yield requires parentheses
in expressions, hopefully it's not something fundamental.
This has always annoyed me, so let's try to fix that for
await.  I'll experiment.



Ditto for `__aiter__` and `__anext__`. I guess this means that the async
equivalent to obtaining an iterator through `it = iter(xs)` followed by
`for x over it` will have to look like `ait = await aiter(xs)` followed by
`for x over ait`, where an iterator is required to have an `__aiter__`
method that's an async function and returns self immediately. But what if
you left out the `await` from the first call? I.e. can this work?
```
ait = aiter(xs)
async for x in ait:
 print(x)


With the current semantics that PEP 492 proposes, "await"
for "aiter()" is mandatory.

You have to write

ait = await aiter(xs)
async for x in ait:
print(c)

We can add some logic that will check that the iterator passed
to 'async for' is not an unresolved awaitable and resolve it
(instead of immediately checking if it has __anext__ method),
but that will complicate the implementation.  It will also
introduce more than one way of doing things.  I think that
users will recognize "async builtins" (when we add them) by
the first letter "a" and use them in "await" expressions
consistently.



```
The question here is whether the object returned by aiter(xs) has an
`__aiter__` method. Since it was intended to be the target of  `await`, it
has an `__await__` method. But that itself is mostly an alias for
`__iter__`, not `__aiter__`. I guess it can be made to work, the object
just has to implement a bunch of different protocols.

Correct.  And yes, we address this all by having iteration
protocols clearly separated.



*6. StopAsyncException*

I'm not sure about this. The motivation given in the PEP seems to focus on
the need for `__anext__` to be async. But is this really the right pattern?
What if we required `ait.__anext__()` to return a future, which can either
raise good old `StopIteration` or return the next value from the iteration
when awaited? I'm wondering if there are a few alternatives to be explored
around the async iterator protocol still.

__anext__ should return an awaitable (following the terminology
of the PEP), which can be a coroutine-object.  I'm not sure that
with semantics of PEP 479 it can actually raise StopIteration
(without some hacks in genobject).

I'm also trying to think forward about how we can add
generator-coroutines (the ones that combine 'await' and some
form of 'yield') to make writing asynchronous iterators
easier. I think that reusing StopIteration on that level
will be a very hard thing to understand and implement.

I'll experiment with reference implementation and update
the PEP.


Thank you,
Yury
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 492 vs. PEP 3152, new round

2015-04-24 Thread Ethan Furman

On 04/24, Yury Selivanov wrote:
> On 2015-04-24 1:03 PM, Guido van Rossum wrote:

>> Ditto for `__aiter__` and `__anext__`. I guess this means that the async
>> equivalent to obtaining an iterator through `it = iter(xs)` followed by
>> `for x over it` will have to look like `ait = await aiter(xs)` followed by
>> `for x over ait`, where an iterator is required to have an `__aiter__`
>> method that's an async function and returns self immediately. But what if
>> you left out the `await` from the first call? I.e. can this work?
>> ```
>> ait = aiter(xs)
>> async for x in ait:
>>  print(x)
> 
> With the current semantics that PEP 492 proposes, "await"
> for "aiter()" is mandatory.
> 
> You have to write
> 
> ait = await aiter(xs)
> async for x in ait:
> print(c)

As a new user to asyncio and this type of programming in general, 'await aiter'
feels terribly redundant.

--
~Ethan~
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 492 vs. PEP 3152, new round

2015-04-24 Thread Guido van Rossum

On Fri, Apr 24, 2015 at 11:03 AM, Ethan Furman  wrote:

> On 04/24, Yury Selivanov wrote:
> > On 2015-04-24 1:03 PM, Guido van Rossum wrote:
>
> >> Ditto for `__aiter__` and `__anext__`. I guess this means that the async
> >> equivalent to obtaining an iterator through `it = iter(xs)` followed by
> >> `for x over it` will have to look like `ait = await aiter(xs)` followed
> by
> >> `for x over ait`, where an iterator is required to have an `__aiter__`
> >> method that's an async function and returns self immediately. But what
> if
> >> you left out the `await` from the first call? I.e. can this work?
> >> ```
> >> ait = aiter(xs)
> >> async for x in ait:
> >>  print(x)
> >
> > With the current semantics that PEP 492 proposes, "await"
> > for "aiter()" is mandatory.
> >
> > You have to write
> >
> > ait = await aiter(xs)
> > async for x in ait:
> > print(c)
>
> As a new user to asyncio and this type of programming in general, 'await
> aiter'
> feels terribly redundant.
>

Yeah, but normally you would never do that. You'd just use `async for x in
xs`. I'm just bickering over the exact expansion of that.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] async/await in Python; v2

On Fri, Apr 24, 2015 at 09:32:51AM -0400, Barry Warsaw wrote:
> On Apr 24, 2015, at 11:17 PM, Steven D'Aprano wrote:
> 
> >It seems to me that tools that search for r"^\s*def\s+spam\s*\(" are
> 
> They would likely search for something like r"^\s*def\s+[a-zA-Z0-9_]+" which
> will hit "def async spam" but not "async def".

Unless somebody wants to do a survey of editors and IDEs and other 
tools, arguments about what regex they may or may not use to search for 
function definitions is an exercise in futility. They may use regexes 
anchored to the start of the line. They may not. They may deal with "def 
async" better than "async def", or the other way around. Either way, 
it's a pretty thin argument for breaking the invariant that the token 
following `def` is the name of the function.

Whatever new syntax is added, something is going to break.

-- 
Steve
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Type hints -- a mediocre programmer's reaction

2015-04-24 Thread Paul Sokolovsky

Hello,

On Fri, 24 Apr 2015 18:27:29 +0100
Ronan Lamy  wrote:

>  Also ask why no one used type specifier, they are possible since
>  Python 3.0 ?
>  Because it is the wrong way for Python.
> >>>
> >>> That's an example of how perceptions differ. In my list,
> >>> everyone(*) uses them - MyPy, MicroPython, etc. Even more should
> >>> use them (any JIT module, which are many), but sit in the bushes,
> >>> waiting for a kick, like PEP484 provides.
> >>
> >> It's OK that type hints are only to assist the programmer.
> >
> > Yes, it's OK to have a situation where type hints assist only a
> > programmer. It's not OK to think that type hints may be useful only
> > for programmer, instead of bunch more purposes, several of which
> > were already shown in the long previous discussion.
> >
> >> PyPy's FAQ
> >> has an explanation of why type hints are not for performance.
> >> http://pypy.readthedocs.org/en/latest/faq.html#would-type-annotations-help-pypy-s-performance
> >
> > You probably intended to write "why type hints are not for *PyPy's*
> > performance". There're many other language implementations and
> > modules for which it may be useful, please don't limit your
> > imagination by a single case.
> 
> Those points apply to basically any compliant implementation of
> Python relying on speculative optimisation. Python is simply too
> dynamic for PEP484-style hints to provide any useful performance
> improvement targets.

What's your point - saying that type annotations alone not enough to
achieve the best ("C-like") performance, which is true, or saying that
if they are alone not enough, then they are not needed at all, which
is ... strange ?

> > And speaking of PyPy, it really should think how to improve its
> > performance - not of generated programs, but of generation itself.
> > If compilation of a trivial program on a pumpy hardware takes 5
> > minutes and gigabytes of RAM and diskspace, few people will use it
> > for other purposes beyond curiosity. There's something very
> > un-Pythonic in waiting 5 mins just to run 10-line script. Type
> > hints can help here too ;-) (by not wasting resources propagating
> > types thru the same old standard library for example).
> 
> Sorry, but that's nonsense. PyPy would be a seriously useless 
> interpreter if running a 10-line script required such a lengthy 
> compilation, so, obviously, that's not what happens.
> 
> You seem to misunderstand what PyPy is: it's an interpreter with a 
> just-in-time compiler, not a static compiler. It doesn't generate 
> programs in any meaningful sense. Instead, it interprets the program, 
> and when it detects a hot code path, it compiles it to machine code 
> based on the precise types it sees. No resources are wasted on code
> that isn't actually executed.

Regardless of whether I understood that meta-meta stuff, I just
followed couple of tutorials, each of them warning of memory and disk
space issues, and both running long to get results. Everyone else
following tutorials will get the same message I did - PyPy is a
slow-to-work-with bloat.

As for uber-meta stuff PyPy offers - I'm glad that's all done in
my favorite language, leaving all other languages behind. I'm saddened
there's no mundane JIT or static compiler usable and accepted by all
community - many other languages have that.

This all goes pretty offtopic wrt to the original discussion, so again,
what's your point - you say that all these things can't be done in
Python, or there's no need for it to be done? That people should look
somewhere else? I submitted a bug to jinja2 project and posted message
on its mailing list - I didn't get reply for 3 months. Why? Because its
maintainer went hacking another language, how was it called, alGOl, or
something. You want me and other folks to go too? Sorry, I'm staying so
far, and keep dreaming of better Python's future (where for example if
I need to get more performance from existing app, I can gradually
optimize it based on need, not rewrite it in another language or be
hitting "not implemented" in uber-meta stuff).

-- 
Best regards,
 Paul  mailto:[email protected]
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] async/await in Python; v2

2015-04-24 Thread Łukasz Langa


> On Apr 24, 2015, at 6:32 AM, Barry Warsaw  wrote:
> 
> On Apr 24, 2015, at 11:17 PM, Steven D'Aprano wrote:
> 
>> It seems to me that tools that search for r"^\s*def\s+spam\s*\(" are
> 
> They would likely search for something like r"^\s*def\s+[a-zA-Z0-9_]+" which
> will hit "def async spam" but not "async def”.

Realistically that can’t be what they’re doing because of multiple string 
literals, internal-scope functions, etc.
But I agree with Steven that guessing here is pointless. More importantly, 
consider:

- if we optimize for some unproven backwards compatibility with tools, we’re 
sacrificing better readability of “async def foo()”
- if that tool wants to work with Python 3.5, it’ll still have to support 
“await” so we’re going to be incompatible anyway; let alone “async for” and 
“async with”

So all in all, I don’t buy this argument.

-- 
Best regards,
Łukasz Langa

WWW: http://lukasz.langa.pl/
Twitter: @llanga
IRC: ambv on #python-dev

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] async/await in Python; v2

2015-04-24 Thread Paul Sokolovsky

Hello,

On Fri, 24 Apr 2015 12:04:27 -0700
Łukasz Langa  wrote:

[]
> > 
> > They would likely search for something like
> > r"^\s*def\s+[a-zA-Z0-9_]+" which will hit "def async spam" but not
> > "async def”.
> 
> Realistically that can’t be what they’re doing because of multiple
> string literals, internal-scope functions, etc. But I agree with
> Steven that guessing here is pointless. More importantly, consider:
> 
> - if we optimize for some unproven backwards compatibility with
> tools, we’re sacrificing better readability of “async def foo()”

Yes, so hopefully another argument prevails: the sooner they break, the
sooner they're fixed (no irony here, I really consider it strange to
optimize language syntax based on background auxiliary utilities'
features or misfeatures).

[]


-- 
Best regards,
 Paul  mailto:[email protected]
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 492 vs. PEP 3152, new round

2015-04-24 Thread Łukasz Langa

> On Apr 24, 2015, at 10:03 AM, Guido van Rossum  wrote:
> 
> 1. precise syntax of `async def`
> 
> So I still prefer `async def`.

Me too. Also, we would use a similar vocabulary to existing users of the 
feature. This is exactly how Hack does it: 
http://docs.hhvm.com/manual/en/hack.async.php 
, how ECMAScript 7 proposes it: 
http://wiki.ecmascript.org/doku.php?id=strawman:async_functions 
 and similarly 
to how C# does it (async comes after the public/private modifier but before the 
return type): https://msdn.microsoft.com/en-us/library/hh156513.aspx 

> 2. do we need `async for` and `async with`
> 
> Yes we do.

+1

> (Though maybe we should consider `await for` and `await with`? That would 
> have the advantage of making it easy to scan for all suspension points by 
> searching for /await/. But being a verb it doesn't read very well.)

I’m on the fence here.

OT1H, I think “await for something in a_container” and “await with 
a_context_manager():” also read pretty well. It’s also more consistent with 
being the one way of finding “yield points”.

OTOH, “await with a_context_manager():” suggests the entire statement is 
awaited on, which is not true. “async with” is more opaque in this way, it 
simply states “there are going to be implementation-specific awaits inside”. So 
it’s more consistent with “async def foo()”.

All in all I think I’m leaning towards “async for” and “async with”. More 
importantly though, I’m wondering how obvious will the failure mode be when 
somebody uses a bare “for” instead of an “async for”. Ditto for “with” vs. 
“async with”. How much debugging will be necessary to find that it’s only a 
missing “async” before the loop?

Side note: to add to the confusion about syntax, Hack’s equivalent for “async 
for” - which doesn’t really translate well to Python - uses “await” and ties it 
to the iterable:

foreach ($list await as $input) { … }

The equivalent in Python would be:

for input in list await:

Notably, this is ugly and would be confused with `for input in await list:` 
which means something different.

Also, this particular construct represents less than 0.01% of all “await” 
occurences in Facebook code, suggesting it’s not performance critical.

> 3. syntactic priority of `await`
> 
> Yury, could you tweak the syntax for `await` so that we can write the most 
> common usages without parentheses?

+1

Yury points out there was likely a reason this wasn’t the case for `yield` in 
the first place. It would be good to revisit that. Maybe for yield itself, too?

> 4. `cocall` vs. `await`
> 
> I just can't get used to this aspect of PEP 3152, so I'm rejecting it.

+1

> (Yury: PEP 492 is not accepted yet, but you're getting closer.)

May I suggest using the bat-signal to summon Glyph to confirm this is going to 
be helpful/usable with Twisted as well?

> 5. do we really need `__aiter__` and friends
> 
> There's a lot of added complexity, but I think it's worth it. I don't think 
> we need to make the names longer, the 'a' prefix is fine for these methods.

+1

> 6. StopAsyncException
> 
> I'm not sure about this. The motivation given in the PEP seems to focus on 
> the need for `__anext__` to be async. But is this really the right pattern? 
> What if we required `ait.__anext__()` to return a future, which can either 
> raise good old `StopIteration` or return the next value from the iteration 
> when awaited? I'm wondering if there are a few alternatives to be explored 
> around the async iterator protocol still.

So are you suggesting to pass the returned value in a future? In this case the 
future would need to be passed to __anext__, so the Cursor example from the PEP 
would look like this:

class Cursor:
def __init__(self):
self.buffer = collections.deque()

def _prefetch(self):
...

async def __aiter__(self):
return self

async def __anext__(self, fut):
if not self.buffer:
self.buffer = await self._prefetch()
if self.buffer:
fut.set_result(self.buffer.popleft())
else:
fut.set_exception(StopIteration)

While this is elegant, my concern is that one-future-per-iteration-step might 
be bad for performance.

Maybe consider the following. The `async def` syntax decouples the concept of a 
coroutine from the implementation. While it’s still based on generators under 
the hood, the user no longer considers his “async function” to be a generator 
or conforming to the generator protocol. From the user’s perpective, it’s 
obvious that the return below means something different than the exception:

async def __anext__(self):
if not self.buffer:
self.buffer = await self._prefetch()
if not self.buffer:
raise StopIteration
return self.buffer.popleft()

Re: [Python-Dev] PEP 492 vs. PEP 3152, new round


Victor,

On 2015-04-24 5:32 PM, Victor Stinner wrote:

7. compatibility with asyncio and existing users of it

The current state of the PEP makes types.coroutine() mandatory. If a
generator-based coroutine is not modified with types.coroutine, await
cannot be used on it. To be more concrete: asyncio coroutines not
declared with @asyncio.coroutine cannot be used with await.

Would it be crazy to allow waiting on a generator-based coroutine
(current asyncio coroutines) without having to call types.coroutine()
on it?


I'd be big -1 on that.  The current PEP design is all about
strictly prohibiting users from calling regular generators
with 'await' expression.  And if a generator isn't decorated
with @coroutine - then it's a regular generator for us.



Maybe I just missed the purpose of disallow this.

It's also possible to modify asyncio to detect at runtime when an
asyncio coroutine is not decorated by @asyncio.coroutine (emit a
warning or even raise an exception).


I'd be +1 to add a warning to Task and other places where
we accept generator-based coroutines.

Thanks!
Yury

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 492 vs. PEP 3152, new round

2015-04-24 Thread Victor Stinner

Hi,

2015-04-24 19:03 GMT+02:00 Guido van Rossum :
> 1. precise syntax of `async def`
>
> Of all the places to put `async` I still like *before* the `def` the best.

So do I.


> 2. do we need `async for` and `async with`
>
> Yes we do.

I agree.


> 3. syntactic priority of `await`
>
> Yury, could you tweak the syntax for `await` so that we can write the most
> common usages without parentheses?

IMO another point must be discussed, corner cases in the grammar and
parser of the current PEP & implementation:
https://www.python.org/dev/peps/pep-0492/#transition-period-shortcomings

And the fact the Python 3.7 will make async & await keywords without
proving a way to prepare the code for this major change in the Python
syntax.

According to the PEP, patching the parser to detect async & await as
keywords is almost a hack, and there are corner cases where it doesn't
work "as expected".

That's why I suggest to reconsider the idea of supporting an
*optional* "from __future__ import async" to get async and await as
keywords in the current file. This import would allow all crazy
syntax. The parser might suggest to use the import when it fails to
parse an async or await keyword :-)


If you don't like the compromise of a parser with corner cases and an
optional __future__ to "workaround these cases", I'm also ok to
reproduce what we did with the introduction of the with keyword. I
mean not supporting async nor await by default, and maing __future__
mandatory to get the new feature.

=> 0 risk of backward compatibility issue
=> no more crazy hacks in the parser


> 4. `cocall` vs. `await`
>
> The asyncio library is getting plenty of adoption and it has the concept of
> separating the *getting* of a future[1] from *waiting* for it.

My rationale in my other email was similar (ability to get a function
without calling it, as you wrote, like bounded methods), so obviously
I agree with it :-)

I accept the compromise of creating a coroutine object without wait
for it (obvious and common bug when learning asyncio). Hopefully, we
keep the coroutine wrapper feature (ok, maybe I suggested this idea to
Yury because I suffered so much when I learnt how to use asyncio ;-)),
so it will still be easy to emit a warning in debug mode.

On Stackoverflow, when someone posts a code with bug, I'm now replying
"please rerun your code with asyncio debug mode enabled" ;-) I also
mentionned it at the *beginning* of the asyncio doc (it's documented
at the end of the asyncio doc!).



> 5. do we really need `__aiter__` and friends
>
> There's a lot of added complexity, but I think it's worth it.

I agree. I don't see how to keep the PEP consistent without having new
dedicated protocols.


> 6. StopAsyncException

(Sorry, I have no opinion on this point.)


> 7. compatibility with asyncio and existing users of it

The current state of the PEP makes types.coroutine() mandatory. If a
generator-based coroutine is not modified with types.coroutine, await
cannot be used on it. To be more concrete: asyncio coroutines not
declared with @asyncio.coroutine cannot be used with await.

Would it be crazy to allow waiting on a generator-based coroutine
(current asyncio coroutines) without having to call types.coroutine()
on it?

Maybe I just missed the purpose of disallow this.

It's also possible to modify asyncio to detect at runtime when an
asyncio coroutine is not decorated by @asyncio.coroutine (emit a
warning or even raise an exception).

Victor
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 492 vs. PEP 3152, new round


Lukasz,

On 2015-04-24 5:37 PM, Łukasz Langa wrote:



(Though maybe we should consider `await for` and `await with`? That would have 
the advantage of making it easy to scan for all suspension points by searching 
for /await/. But being a verb it doesn't read very well.)

I’m on the fence here.

OT1H, I think “await for something in a_container” and “await with 
a_context_manager():” also read pretty well. It’s also more consistent with 
being the one way of finding “yield points”.

OTOH, “await with a_context_manager():” suggests the entire statement is 
awaited on, which is not true. “async with” is more opaque in this way, it 
simply states “there are going to be implementation-specific awaits inside”. So 
it’s more consistent with “async def foo()”.


This.  I also think of 'await with' as I'm awaiting on the whole 
statement.  And I don't even know how to interpret that.



6. StopAsyncException

I'm not sure about this. The motivation given in the PEP seems to focus on the 
need for `__anext__` to be async. But is this really the right pattern? What if 
we required `ait.__anext__()` to return a future, which can either raise good 
old `StopIteration` or return the next value from the iteration when awaited? 
I'm wondering if there are a few alternatives to be explored around the async 
iterator protocol still.

So are you suggesting to pass the returned value in a future? In this case the 
future would need to be passed to __anext__, so the Cursor example from the PEP 
would look like this:

class Cursor:
 def __init__(self):
 self.buffer = collections.deque()

 def _prefetch(self):
 ...

 async def __aiter__(self):
 return self

 async def __anext__(self, fut):
 if not self.buffer:
 self.buffer = await self._prefetch()
 if self.buffer:
 fut.set_result(self.buffer.popleft())
 else:
 fut.set_exception(StopIteration)

While this is elegant, my concern is that one-future-per-iteration-step might 
be bad for performance.

Maybe consider the following. The `async def` syntax decouples the concept of a 
coroutine from the implementation. While it’s still based on generators under 
the hood, the user no longer considers his “async function” to be a generator 
or conforming to the generator protocol. From the user’s perpective, it’s 
obvious that the return below means something different than the exception:

 async def __anext__(self):
 if not self.buffer:
 self.buffer = await self._prefetch()
 if not self.buffer:
 raise StopIteration
 return self.buffer.popleft()

So, the same way we added wrapping in RuntimeErrors for generators in PEP 479, 
we might add transparent wrapping in a _StopAsyncIteration for CO_COROUTINE.


FWIW I have to experiment more with the reference implementation,
but at the moment I'm big -1 on touching StopIteration for
coroutines.  It's used for too many things.


Thanks!

Yury
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 492 vs. PEP 3152, new round

2015-04-24 Thread Łukasz Langa

> On Apr 24, 2015, at 10:03 AM, Guido van Rossum  wrote:
> 
> 6. StopAsyncException
> 
> What if we required `ait.__anext__()` to return a future?

On top of my previous response, one more thing to consider is that this idea 
brings a builtin Future back to the proposal, which has already been rejected 
in the "No implicit wrapping in Futures” section of the PEP.

PEP 492 manages to solve all issues without introducing a built-in Future.

-- 
Best regards,
Łukasz Langa

WWW: http://lukasz.langa.pl/
Twitter: @llanga
IRC: ambv on #python-dev

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 492 vs. PEP 3152, new round

2015-04-24 Thread Guido van Rossum

Sorry, when I wrote "future" (lower-case 'f') I really meant what Yury
calls *awaitable*. That's either a coroutine or something with an __await__
emthod.

On Fri, Apr 24, 2015 at 3:17 PM, Łukasz Langa  wrote:

>
> On Apr 24, 2015, at 10:03 AM, Guido van Rossum  wrote:
>
> *6. StopAsyncException*
>
> What if we required `ait.__anext__()` to return a future?
>
>
> On top of my previous response, one more thing to consider is that this
> idea brings a builtin Future back to the proposal, which has already been
> rejected in the "No implicit wrapping in Futures” section of the PEP.
>
> PEP 492 manages to solve all issues without introducing a built-in Future.
>
> --
> Best regards,
> Łukasz Langa
>
> WWW: http://lukasz.langa.pl/
> Twitter: @llanga
> IRC: ambv on #python-dev
>
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Type hints -- a mediocre programmer's reaction

2015-04-24 Thread Ronan Lamy

Le 24/04/15 19:45, Paul Sokolovsky a écrit :

Hello,

On Fri, 24 Apr 2015 18:27:29 +0100
Ronan Lamy wrote:

PyPy's FAQ
has an explanation of why type hints are not for performance.
http://pypy.readthedocs.org/en/latest/faq.html#would-type-annotations-help-pypy-s-performance

You probably intended to write "why type hints are not for *PyPy's*
performance". There're many other language implementations and
modules for which it may be useful, please don't limit your
imagination by a single case.

Those points apply to basically any compliant implementation of
Python relying on speculative optimisation. Python is simply too
dynamic for PEP484-style hints to provide any useful performance
improvement targets.

What's your point - saying that type annotations alone not enough to
achieve the best ("C-like") performance, which is true, or saying that
if they are alone not enough, then they are not needed at all, which
is ... strange ?

My point is that the arguments in the PyPy FAQ aren't actually specific
to PyPy, and therefore that the conclusion, that hints are almost
entirely useless if you’re looking at performance, holds in general.
So let me restate these arguments in terms of a generic,
performance-minded implementation of the full Python language spec:

* Hints have no run-time effect. The interpreter cannot assume that they
are obeyed.
* PEP484 hints are too high-level. Replacing an 'int' object with a
single machine word would be useful, but an 'int' annotation gives no
guarantee that it's correct (because Python 3 ints can have arbitrary
size and because subclasses of 'int' can override any operation to
invoke arbitrary code).
* A lot more information is needed to produce good code (e.g. “this f()
called here really means this function there, and will never be
monkey-patched” – same with len() or list(), btw).

* Most of this information cannot easily be expressed as a type
* If the interpreter gathers all that information, it'll probably have
gathered a superset of what PEP484 can provide anyway.

And speaking of PyPy, it really should think how to improve its
performance - not of generated programs, but of generation itself.
If compilation of a trivial program on a pumpy hardware takes 5
minutes and gigabytes of RAM and diskspace, few people will use it
for other purposes beyond curiosity. There's something very
un-Pythonic in waiting 5 mins just to run 10-line script. Type
hints can help here too ;-) (by not wasting resources propagating
types thru the same old standard library for example).

Sorry, but that's nonsense. PyPy would be a seriously useless
interpreter if running a 10-line script required such a lengthy
compilation, so, obviously, that's not what happens.

Regardless of whether I understood that meta-meta stuff, I just
followed couple of tutorials, each of them warning of memory and disk
space issues, and both running long to get results. Everyone else
following tutorials will get the same message I did - PyPy is a
slow-to-work-with bloat.

Ah, I suppose you're talking about the RPython tool chain, which is used
to build PyPy. Though it's an interesting topic in itself (and is pretty
much comparable to Cython wrt. type hints), it has about as much
relevance to PyPy users as the inner workings of GCC have to CPython users.

Well, the thing is that people don't seem to want to write PyPy
tutorials, because it's boring. However, I can give you the definitive
3-line version:

1. Download and install PyPy [http://pypy.org/download.html]
2. Launch the 'pypy' executable.
3. Go read https://docs.python.org/2/tutorial/

As for uber-meta stuff PyPy offers - I'm glad that's all done in
my favorite language, leaving all other languages behind. I'm saddened
there's no mundane JIT or static compiler usable and accepted by all
community - many other languages have that.

This all goes pretty offtopic wrt to the original discussion, so again,
what's your point - you say that all these things can't be done in
Python, or there's no need for it to be done? That people should look
somewhere else? I submitted a bug to jinja2 project and posted message
on its mailing list - I didn't get reply for 3 months. Why? Because its
maintainer went hacking another language, how was it called, alGOl, or
something. You want me and other folks to go too? Sorry, I'm staying so
far, and keep dreaming of better Python's future (where for example if
I need to get more performance from existing app, I can gradually
optimize it based on need, not rewrite it in another language or be
hitting "not implemented" in uber-meta stuff).

"If

Re: [Python-Dev] Type hints -- a mediocre programmer's reaction

2015-04-24 Thread Kevin Modzelewski

On Fri, Apr 24, 2015 at 6:05 PM, Ronan Lamy  wrote:

> Le 24/04/15 19:45, Paul Sokolovsky a écrit :
>
>> Hello,
>>
>> On Fri, 24 Apr 2015 18:27:29 +0100
>> Ronan Lamy  wrote:
>>
>>  PyPy's FAQ
> has an explanation of why type hints are not for performance.
>
> http://pypy.readthedocs.org/en/latest/faq.html#would-type-annotations-help-pypy-s-performance
>

 You probably intended to write "why type hints are not for *PyPy's*
 performance". There're many other language implementations and
 modules for which it may be useful, please don't limit your
 imagination by a single case.

>>>
>>> Those points apply to basically any compliant implementation of
>>> Python relying on speculative optimisation. Python is simply too
>>> dynamic for PEP484-style hints to provide any useful performance
>>> improvement targets.
>>>
>>
>> What's your point - saying that type annotations alone not enough to
>> achieve the best ("C-like") performance, which is true, or saying that
>> if they are alone not enough, then they are not needed at all, which
>> is ... strange ?
>>
>
> My point is that the arguments in the PyPy FAQ aren't actually specific to
> PyPy, and therefore that the conclusion, that hints are almost entirely
> useless if you’re looking at performance, holds in general.
> So let me restate these arguments in terms of a generic,
> performance-minded implementation of the full Python language spec:
>
> * Hints have no run-time effect. The interpreter cannot assume that they
> are obeyed.
> * PEP484 hints are too high-level. Replacing an 'int' object with a single
> machine word would be useful, but an 'int' annotation gives no guarantee
> that it's correct (because Python 3 ints can have arbitrary size and
> because subclasses of 'int' can override any operation to invoke arbitrary
> code).
> * A lot more information is needed to produce good code (e.g. “this f()
> called here really means this function there, and will never be
> monkey-patched” – same with len() or list(), btw).
> * Most of this information cannot easily be expressed as a type
> * If the interpreter gathers all that information, it'll probably have
> gathered a superset of what PEP484 can provide anyway.


I'm with the PyPy folks here -- I don't see any use for PEP 484 type hints
from a code generation perspective.  Even if the hints were guaranteed to
be correct, the PEP 484 type system doesn't follow substitutability.  I
don't mean that as a critique, I think it's a decision that makes it more
useful by keeping it in line with the majority of type usage in Python, but
it means that even if the hints are correct they don't really end up
providing any guarantees to the JIT.


>
>
>  And speaking of PyPy, it really should think how to improve its
 performance - not of generated programs, but of generation itself.
 If compilation of a trivial program on a pumpy hardware takes 5
 minutes and gigabytes of RAM and diskspace, few people will use it
 for other purposes beyond curiosity. There's something very
 un-Pythonic in waiting 5 mins just to run 10-line script. Type
 hints can help here too ;-) (by not wasting resources propagating
 types thru the same old standard library for example).

>>>
>>> Sorry, but that's nonsense. PyPy would be a seriously useless
>>> interpreter if running a 10-line script required such a lengthy
>>> compilation, so, obviously, that's not what happens.
>>>
>>> You seem to misunderstand what PyPy is: it's an interpreter with a
>>> just-in-time compiler, not a static compiler. It doesn't generate
>>> programs in any meaningful sense. Instead, it interprets the program,
>>> and when it detects a hot code path, it compiles it to machine code
>>> based on the precise types it sees. No resources are wasted on code
>>> that isn't actually executed.
>>>
>>
>> Regardless of whether I understood that meta-meta stuff, I just
>> followed couple of tutorials, each of them warning of memory and disk
>> space issues, and both running long to get results. Everyone else
>> following tutorials will get the same message I did - PyPy is a
>> slow-to-work-with bloat.
>>
>
> Ah, I suppose you're talking about the RPython tool chain, which is used
> to build PyPy. Though it's an interesting topic in itself (and is pretty
> much comparable to Cython wrt. type hints), it has about as much relevance
> to PyPy users as the inner workings of GCC have to CPython users.
>
> Well, the thing is that people don't seem to want to write PyPy tutorials,
> because it's boring. However, I can give you the definitive 3-line version:
> 1. Download and install PyPy [http://pypy.org/download.html]
> 2. Launch the 'pypy' executable.
> 3. Go read https://docs.python.org/2/tutorial/
>
>  As for uber-meta stuff PyPy offers - I'm glad that's all done in
>> my favorite language, leaving all other languages behind. I'm saddened
>> there's no mundane JIT or static compiler usable and accepted by all
>>

Re: [Python-Dev] Type hints -- a mediocre programmer's reaction

On Sat, Apr 25, 2015 at 02:05:15AM +0100, Ronan Lamy wrote:

> * Hints have no run-time effect. The interpreter cannot assume that they 
> are obeyed.

I know what you mean, but just for the record, annotations are runtime 
inspectable, so people can (and probably have already started) to write 
runtime argument checking decorators or frameworks which rely on the 
type hints.

> * PEP484 hints are too high-level. Replacing an 'int' object with a 
> single machine word would be useful, but an 'int' annotation gives no 
> guarantee that it's correct (because Python 3 ints can have arbitrary 
> size and because subclasses of 'int' can override any operation to 
> invoke arbitrary code).

Then create your own int16, uint64 etc types.

> * A lot more information is needed to produce good code (e.g. “this f() 
> called here really means this function there, and will never be 
> monkey-patched” – same with len() or list(), btw).
> * Most of this information cannot easily be expressed as a type
> * If the interpreter gathers all that information, it'll probably have 
> gathered a superset of what PEP484 can provide anyway.

All this is a red herring. If type hints are useful to PyPy, that's a 
bonus. Cython uses its own system of type hints, a future version may be 
able to use PEP 484 hints instead. But any performance benefit is a 
bonus. PEP 484 is for increasing correctness, not speed.

-- 
Steve
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How to behave regarding commiting

On 17 April 2015 at 11:41, Berker Peksağ  wrote:
> On Fri, Apr 17, 2015 at 4:32 AM, Facundo Batista
>  wrote:
>> Hola!
>>
>> I'm asking this because quite some time passed since I was active in
>> the development of our beloved language.
>>
>> I'm trying to not break any new rule not known by me.
>>
>> I opened a bug recently [0], somebody else proposed a patch that I
>> like. However, that patch has no test. I will do a test for that code,
>> but then what?
>>
>> Shall I just commit and push? Or the whole branch should be proposed
>> for further reviewing?
>
> Hi,
>
> Since writing a test for that patch is simple, I'd just commit it to
> the default branch.

(Catching up on several days of python-dev email)

The ideal case is getting pre-commit reviews, but as a matter of
practicality, we each get to decide whether or not we're OK with just
pushing a particular change and relying on the buildbots and
post-commit review.

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Changing PyModuleDef.m_reload to m_slots

On 18 April 2015 at 15:58, Stefan Behnel  wrote:
> Petr Viktorin schrieb am 17.04.2015 um 15:52:
>> As a background, the PyModuleDef structure [1] is currently:
>>
>> struct PyModuleDef{
>>   PyModuleDef_Base m_base;
>>   const char* m_name;
>>   const char* m_doc;
>>   Py_ssize_t m_size;
>>   PyMethodDef *m_methods;
>>   inquiry m_reload;
>>   traverseproc m_traverse;
>>   inquiry m_clear;
>>   freefunc m_free;
>> };
>>
>> ... where the m_reload pointer is unused, and must be NULL.
>> My proposal is to repurpose this pointer to hold an array of slots, in the
>> style of PEP 384's PyType_Spec [2], which would allow adding extensions --
>> both those needed for PEP 489 and future ones.
>
> FWIW, I'm +1 on this. It replaces a struct field that risks staying unused
> basically forever with an extensible interface that massively improves the
> current extension module protocol and allows future extensions (including
> getting back the pointer we replaced).
>
> The alternative of essentially duplicating all sorts of things to
> accommodate for a new metadata struct is way too cumbersome and ugly in
> comparison.

Sorry for the delayed reply (post-PyCon travel), but +1 from me as
well for the reasons Stefan gives.

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 492 vs. PEP 3152, new round


Guido van Rossum wrote:
Yury, could you tweak the syntax for `await` so that we can write the 
most common usages without parentheses? In particular I'd like to be 
able to write

```
return await foo()
with await foo() as bar: ...
foo(await bar(), await bletch())
```


Making 'await' a prefix operator with the same precedence
as unary minus would allow most reasonable usages, I think.

The only reason "yield from" has such a constrained syntax
is that it starts with "yield", which is similarly constrained.
Since 'await' is a brand new keyword isn't bound by those constraints.

--
Greg
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Surely "nullable" is a reasonable name?

On 22 April 2015 at 03:31, Larry Hastings  wrote:
>
> On 04/21/2015 04:50 AM, Tal Einat wrote:
>
> As for the default set of accepted types for various convertors, if we
> could choose any syntax we liked, something like "accept=+{NoneType}"
> would be much better IMO.
>
>
> In theory Argument Clinic could use any syntax it likes.  In practice, under
> the covers we tease out one or two bits of non-Python syntax, then run
> ast.parse over it.  Saved us a lot of work.
>
> "s: accept={str,NoneType}" is a legal Python parameter declaration; "s:
> accept+={NoneType}" is not.  If I could figure out a clean way to hack in
> support for += I'll support it.  Otherwise you'll be forced to spell it out.

Ellipsis seems potentially useful here to mean "whatever the default
accepted types are": "s: accept={...,NoneType}"

My other question would be whether we can use "None" in preference to
NoneType, as PEP 484 does:
https://www.python.org/dev/peps/pep-0484/#using-none

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Surely "nullable" is a reasonable name?

On 25 April 2015 at 14:44, Nick Coghlan  wrote:
> On 22 April 2015 at 03:31, Larry Hastings  wrote:
>>
>> On 04/21/2015 04:50 AM, Tal Einat wrote:
>>
>> As for the default set of accepted types for various convertors, if we
>> could choose any syntax we liked, something like "accept=+{NoneType}"
>> would be much better IMO.
>>
>>
>> In theory Argument Clinic could use any syntax it likes.  In practice, under
>> the covers we tease out one or two bits of non-Python syntax, then run
>> ast.parse over it.  Saved us a lot of work.
>>
>> "s: accept={str,NoneType}" is a legal Python parameter declaration; "s:
>> accept+={NoneType}" is not.  If I could figure out a clean way to hack in
>> support for += I'll support it.  Otherwise you'll be forced to spell it out.
>
> Ellipsis seems potentially useful here to mean "whatever the default
> accepted types are": "s: accept={...,NoneType}"

Ah, I misread Tal's suggestion. Using unary + is an even neater approach.

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] async/await in Python; v2


Wild idea:

Let "@" mean "async" when it's directly in front
of a keyword.

Then we would have:

  @def f():
 ...

  @for x in iter:
 ...

  @with context as thing:
 ...

--
Greg
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 492 vs. PEP 3152, new round


Victor Stinner wrote:


That's why I suggest to reconsider the idea of supporting an
*optional* "from __future__ import async" to get async and await as
keywords in the current file. This import would allow all crazy
syntax. The parser might suggest to use the import when it fails to
parse an async or await keyword :-)


To me, these new features *obviously* should require
a __future__ import. Anything else would be crazy.


I accept the compromise of creating a coroutine object without wait
for it (obvious and common bug when learning asyncio). Hopefully, we
keep the coroutine wrapper feature (ok, maybe I suggested this idea to
Yury because I suffered so much when I learnt how to use asyncio ;-)),
so it will still be easy to emit a warning in debug mode.


I'm disappointed that there will *still* be no direct and
reliable way to detect and clearly report this kind of
error, and that what there is will only be active in
a special debug mode.

--
Greg
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Type hints -- a mediocre programmer's reaction

On 22 April 2015 at 03:03, Carol Willing  wrote:
> 2. Clearly, great thought has been put into this PEP. If anyone has a good
> analysis of the potential impact on Python 3 adoption, please do pass along.
> I would be interested in reading the information.

I don't have hard data, but I do get to see quite a few of the
challenges that dynamically typed languages (including Python) have
scaling to large development teams. When you look at the way folks
actually *use* languages like Java and C# in practice, you find that
they're almost universally "tool mavens" (in Oliver Steele's sense:
http://blog.osteele.com/posts/2004/11/ides/) where automated tools are
taking care of most of the boilerplate for them. Find me a C#
developer, and I'll bet you they're a Visual Studio user, find me a
Java developer, and I'll bet you they're an Eclipse or IntelliJ user.
This approach actually offers a lot of benefits in putting a "skill
floor" under a development team - while you can't make people think,
you can at least automatically rule out broad categories of mundane
errors, and focus on the business logic of the problem you're aiming
to solve.

As a result, my main reaction to PEP 484 in a Python 3 adoption
context is that "Python 3 offers all the agility and flexibility of
Python 2, with all the structural assurances of Java or C#" is
actually a huge selling point for anyone in an institutional context
attempting to persuade their management chain to back a migration
effort from Python 2 to Python 3.

Another factor to consider here is that my understanding is that one
of the *reasons* folks want better structural analysis (by annotating
the Python 2 stdlib and key third part libraries in typeshed) is to
help automate Python 2 -> Python 3 conversions in the absence of
comprehensive test coverage. While to some degree this works against
the previous point, there's a difference between having this as an
addon vs having it as a standard feature (and unlike the network
security changes and bundling pip, this is one where I'm entirely
happy leaving it as the kind of carrot that can take pride of place in
a corporate business case).

The legitimate concerns that arise are around what happens to
*community* code bases, including the standard library itself, as well
as what folks are likely to see if they run "inspect.getsource()" on
standard library components. For that, I think there's a lot of value
in continuing to have explicit type hints be the exception rather than
the rule in the upstream community, so the idea of the typeshed
project is enormously appealing to me. If anyone doesn't want to deal
with type hints themselves, but has a contributor that really wants to
annotate their library, then "take it to typeshed" will hopefully
become a recurring refrain :)

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 492 vs. PEP 3152, new round

On 25 April 2015 at 07:37, Łukasz Langa  wrote:
> On Apr 24, 2015, at 10:03 AM, Guido van Rossum  wrote:
> 3. syntactic priority of `await`
>
> Yury, could you tweak the syntax for `await` so that we can write the most
> common usages without parentheses?
>
> +1
>
> Yury points out there was likely a reason this wasn’t the case for `yield`
> in the first place. It would be good to revisit that. Maybe for yield
> itself, too?

yield requires parentheses in most cases for readability purposes. For
example, does this pass one argument or two to "f"?:

f(yield a, b)

Answer: neither, it's a syntax error, as you have to supply the
parentheses to say whether you mean "f((yield a, b))" or "f((yield a),
b)".

Requiring parentheses except in a few pre-approved cases
(specifically, as a standalone statement and as the RHS of assignment
statements) eliminated all that potential ambiguity.

Allowing "return yield a, b" and "return yield from a, b" by analogy
with assignment statements would be fine, but getting more permissive
than that with yield expressions doesn't make sense due to the
ambiguity between yielding a tuple and other uses of commas as
separators in various parts of the grammar.

PEP 492's "await" is a different case, as asynchronously waiting for a
tuple doesn't make any sense. However, I'm not sure the grammar will
let you reasonably express "anything except a tuple literal" as the
subexpression after the await keyword, so it will likely still need
special case handling in the affected statements in order to avoid
"await a, b" being interpreted as waiting for a tuple.

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 492 vs. PEP 3152, new round


Guido van Rossum wrote:
Sorry, when I wrote "future" (lower-case 'f') I really meant what Yury 
calls *awaitable*. That's either a coroutine or something with an 
__await__ emthod.


But how is an awaitable supposed to raise StopIteration
if it's implemented by a generator or async def[*] function?
Those things use StopIteration to wrap return values.

I like the idea of allowing StopIteration to be raised
in an async def function and wrapping it somehow. I'd
add that it could also be unwrapped automatically when
it emerges from 'await', so that code manually invoking
__anext__ can catch StopIteration as usual.

I don't think this could conflict with any existing
uses of StopIteration, since raising it inside generators
is currently forbidden.

[*] I'm still struggling with what to call those things.
Calling them just "coroutines" seems far too ambiguous.
(There should be a Zen item something along the lines
of "If you can't think of a concise and unambiguous name
for it, it's probably a bad idea".)

--
Greg
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 492 vs. PEP 3152, new round