[Python-Dev] asyncio: how to interrupt an async def w/ finally: ( e.g. Condition.wait() )

[Matthias Urlichs]

The following code has a problem: the generator returned by .wait() has a 
finally: section. When self.stopped is set, it still needs to run. As it is 
asynchronous (it needs to re-acquire the lock), I need to come up with a 
reliable way to wait for it. If I don't, .release() will throw an exception 
because the lock is still unlocked.

The best method to do this that I've come up with is the five marked lines.
I keep thinking there must be a better way to do this (taking into account 
that I have no idea whether the 'await r' part of this is even necessary).


```
class StopMe(BaseException):
pass
class Foo:
async dev some_method(self):
self.uptodate = asyncio.Condition()
self.stopped = asyncio.Future()
…
await self.uptodate.acquire()
try:
while self.some_condition():
w = self.uptodate.wait()
await asyncio.wait([w,self.stopped], loop=self.conn._loop, 
return_when=asyncio.FIRST_COMPLETED)
with contextlib.suppress(StopMe):  # FIXME?
r = w.throw(StopMe())  # FIXME?
if r is not None:  # FIXME?
await r# FIXME?
await w# FIXME?
finally:
self.uptodate.release()
```
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Typo in PEP-0423

[Terry Reedy]

On 12/19/2015 5:55 AM, Nick Coghlan wrote:


Even once the new docs are in place, getting them to the top of search
of results ahead of archived material that may be years out of date is
likely to still be a challenge - for example, even considering just
the legacy distutils docs, the "3.1" and "2" docs appear in the
results at 
https://www.google.com/search?q=registering+with+the+package+index+site%3Apython.org=utf-8=utf-8,
but the latest "3" docs don't.


Can we retroactively modify old docs by adding a link to newer docs at 
the top?


"UPDATE 2016: This doc is obsolete.  You probably should be looking at
https: "

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncio: how to interrupt an async def w/ finally: ( e.g. Condition.wait() )

[Guido van Rossum]

Perhaps you can add a check for a simple boolean 'stop' flag to your
condition check, and when you want to stop the loop you set that flag and
then call notify() on the condition. Then you can follow the standard
condition variable protocol instead of all this nonsense. :-)

class Foo:
async def some_method(self):
self.uptodate = asyncio.Condition()
self.stopped = False
…
await self.uptodate.acquire()
try:
while (not self.stopped) and self.some_condition():
await self.uptodate.wait()
finally:
self.uptodate.release()
def stop_it(self):
self.stopped = True
self.uptodate.notify()

On Sat, Dec 19, 2015 at 7:43 AM, Matthias Urlichs 
wrote:

> The following code has a problem: the generator returned by .wait() has a
> finally: section. When self.stopped is set, it still needs to run. As it is
> asynchronous (it needs to re-acquire the lock), I need to come up with a
> reliable way to wait for it. If I don't, .release() will throw an exception
> because the lock is still unlocked.
>
> The best method to do this that I've come up with is the five marked lines.
> I keep thinking there must be a better way to do this (taking into account
> that I have no idea whether the 'await r' part of this is even necessary).
>
>
> ```
> class StopMe(BaseException):
> pass
> class Foo:
> async dev some_method(self):
> self.uptodate = asyncio.Condition()
> self.stopped = asyncio.Future()
> …
> await self.uptodate.acquire()
> try:
> while self.some_condition():
> w = self.uptodate.wait()
> await asyncio.wait([w,self.stopped], loop=self.conn._loop,
> return_when=asyncio.FIRST_COMPLETED)
> with contextlib.suppress(StopMe):  # FIXME?
> r = w.throw(StopMe())  # FIXME?
> if r is not None:  # FIXME?
> await r# FIXME?
> await w# FIXME?
> finally:
> self.uptodate.release()
> ```
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Typo in PEP-0423

[A.M. Kuchling]

On Sat, Dec 19, 2015 at 08:55:26PM +1000, Nick Coghlan wrote:
> Even once the new docs are in place, getting them to the top of search
> of results ahead of archived material that may be years out of date is
> likely to still be a challenge - for example, even considering just
> the legacy distutils docs, the "3.1" and "2" docs appear ...

We probably need to update https://docs.python.org/robots.txt, which
currently contains:

# Prevent development and old documentation from showing up in search results.
User-agent: *
# Disallow: /dev
Disallow: /release

The intent was to allow the latest version of the docs to be crawled.
Unfortunately, with the current hierarchy we'd have to disallow each
version, e.g.

Disallow: /2.6/*
Disallow: /3.0/*
Disallow: /3.1/*

And we'd need to update it for each new major release.

--amk
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncio: how to interrupt an async def w/ finally: ( e.g. Condition.wait() )

[Matthias Urlichs]

On 19.12.2015 20:25, Guido van Rossum wrote:
> Perhaps you can add a check for a simple boolean 'stop' flag to your
> condition check, and when you want to stop the loop you set that flag
> and then call notify() on the condition. Then you can follow the
> standard condition variable protocol instead of all this nonsense. :-)
Your example does not work.

>def stop_it(self):
>self.stopped = True
>self.uptodate.notify()

self.uptodate needs to be locked before I can call .notify() on it.
Creating a new task just for that seems like overkill, and I'd have to
add a generation counter to prevent a race condition. Doable, but ugly.

However, this doesn't fix the generic problem; Condition.wait() was just
what bit me today.
When a non-async generator goes out of scope, its finally: blocks will
execute. An async procedure call whose refcount reaches zero without
completing simply goes away; finally: blocks are *not* called and there
is *no* warning.
I consider that to be a bug.

-- 
-- Matthias Urlichs

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Webmaster] Python keeps installing as 32 bit

[Terry Reedy]

On 12/18/2015 4:34 PM, Mullins, Robb wrote:


Please remove these posts/liservs, etc. if possible, or strip my contact
info/name/phone/email off the posts please.  I’m getting calls from
people trying to help with my Python install issue.



http://code.activestate.com/lists/python-dev/138936/
http://blog.gmane.org/gmane.comp.python.devel


There is no connection between PSF and either Activestate or Gmane.org. 
 We have no control over their mirrors.  We occasionally remove 
defamatory material posted to python-list, but even that is somewhat 
futile as python list is mirrored on gmane, usenet, google-groups, and 
many other places.  pydev has fewer mirrors, but as you noticed, there 
are at least 2.



This was not supposed to be posted online.  I just wanted to know if
there was a trick to forcing x64 Python to install on x64 machines.


There is no easy way to edit and forward at the same time.


Thanks,

RM

Desktop Support Specialist

Center for Innovation in Teaching & Learning

citl-techsupp...@mx.uillinois.edu
 /(For computer issues, please
use the ticket system.)/

(217) 333-


This time, *you* publicly posted your phone number, which I obscured. 
Perhaps you should have two signatures or two accounts with different 
signatures, depending on your mail/news software.


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Typo in PEP-0423

[Guido van Rossum]

Maybe we need to find a S.E.O. expert. I betcha some are lurking on this
list.

On Sat, Dec 19, 2015 at 2:55 AM, Nick Coghlan  wrote:

> On 19 December 2015 at 03:44, Guido van Rossum  wrote:
> > On Fri, Dec 18, 2015 at 9:34 AM, Tim Legrand  >
> > wrote:
> >>
> >> Well, this looks like a rhetorical question :)
> >
> > It wasn't, I was hoping you'd be quicker at picking one than me (I don't
> > publish packages on PyPI much myself so the docs all look like Greek to
> me
> > :-).
>
> There's an effort currently underway to significantly improve the
> getting started tutorials on packaging.python.org, but it's
> unfortunately going to be a long time before we can retire the legacy
> docs completely - while parts of them have aged badly (and are
> entirely superseded by packaging.python.org), other parts
> unfortunately aren't covered anywhere else yet :(
>
> Even once the new docs are in place, getting them to the top of search
> of results ahead of archived material that may be years out of date is
> likely to still be a challenge - for example, even considering just
> the legacy distutils docs, the "3.1" and "2" docs appear in the
> results at
> https://www.google.com/search?q=registering+with+the+package+index+site%3Apython.org=utf-8=utf-8
> ,
> but the latest "3" docs don't.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncio: how to interrupt an async def w/ finally: ( e.g. Condition.wait() )

[Matthias Urlichs]

On 20.12.2015 01:26, Kevin Conway wrote:
> async def coroutine():
> try:
> await Awaitable()
> await Awaitable()
> finally:
> print('finally')
Try adding another "await Awaitable()" after the "finally:".

I have to take back my "doesn't print an error" comment, however;
there's another reference to the Condition.wait() generator (the task
asyncio.wait() creates to wrap the generator in), and the "Task was
destroyed but it is pending!" message got delayed sufficiently that I
missed it. (Dying test cases tend to spew many of these.)

Testcase:

import asyncio
import gc
cond = asyncio.Condition()
loop = asyncio.get_event_loop()

async def main():
async with cond:
# asyncio.wait() does this, if we don't
w = asyncio.ensure_future(cond.wait())
await asyncio.wait([w],timeout=1)
print(gc.get_referrers(w))
loop.run_until_complete(main())

Time to refactor my code to do the wait/timeout outside the "async with
cond".

-- 
-- Matthias Urlichs

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncio: how to interrupt an async def w/ finally: ( e.g. Condition.wait() )

[Guido van Rossum]

On Sat, Dec 19, 2015 at 1:40 PM, Matthias Urlichs 
wrote:

> On 19.12.2015 20:25, Guido van Rossum wrote:
> > Perhaps you can add a check for a simple boolean 'stop' flag to your
> > condition check, and when you want to stop the loop you set that flag
> > and then call notify() on the condition. Then you can follow the
> > standard condition variable protocol instead of all this nonsense. :-)
> Your example does not work.
>
> >def stop_it(self):
> >self.stopped = True
> >self.uptodate.notify()
>
> self.uptodate needs to be locked before I can call .notify() on it.
>

Fair enough.


> Creating a new task just for that seems like overkill, and I'd have to
> add a generation counter to prevent a race condition. Doable, but ugly.
>

I guess that's due to some application logic, but whatever. You don't
really seem to care about finding a solution for this problem anyways:


> However, this doesn't fix the generic problem; Condition.wait() was just
> what bit me today.
> When a non-async generator goes out of scope, its finally: blocks will
> execute. An async procedure call whose refcount reaches zero without
> completing simply goes away; finally: blocks are *not* called and there
> is *no* warning.
> I consider that to be a bug.
>

If that's so, can you demonstrate that without invoking all these other
things? Other traffic in this thread seems to indicate it may be not as
simple as that.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Typo in PEP-0423

[Nick Coghlan]

On 19 December 2015 at 03:44, Guido van Rossum  wrote:
> On Fri, Dec 18, 2015 at 9:34 AM, Tim Legrand 
> wrote:
>>
>> Well, this looks like a rhetorical question :)
>
> It wasn't, I was hoping you'd be quicker at picking one than me (I don't
> publish packages on PyPI much myself so the docs all look like Greek to me
> :-).

There's an effort currently underway to significantly improve the
getting started tutorials on packaging.python.org, but it's
unfortunately going to be a long time before we can retire the legacy
docs completely - while parts of them have aged badly (and are
entirely superseded by packaging.python.org), other parts
unfortunately aren't covered anywhere else yet :(

Even once the new docs are in place, getting them to the top of search
of results ahead of archived material that may be years out of date is
likely to still be a challenge - for example, even considering just
the legacy distutils docs, the "3.1" and "2" docs appear in the
results at 
https://www.google.com/search?q=registering+with+the+package+index+site%3Apython.org=utf-8=utf-8,
but the latest "3" docs don't.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncio: how to interrupt an async def w/ finally: ( e.g. Condition.wait() )

[Gustavo Carneiro]

I tried to reproduce the problem you describe, but failed.  Here's my test
program (forgive the awful tab indentation, long story):

--
import asyncio

async def foo():
print("resource acquire")
try:
await asyncio.sleep(100)
finally:
print("resource release")


async def main():
task = asyncio.ensure_future(foo())
print("task created")
await asyncio.sleep(0)
print("about to cancel task")
task.cancel()
print("task cancelled, about to wait for it")
try:
await task
except asyncio.CancelledError:
pass
print("waited for cancelled task")


if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
loop.close()
---

I get this output:


10:54:28 ~/Documents$ python3.5 foo.py
task created
resource acquire
about to cancel task
task cancelled, about to wait for it
resource release
waited for cancelled task


Which seems to indicate that the finally clause is correctly executed when
the task is waited for, after being cancelled.

But maybe I completely misunderstood your problem...


On 19 December 2015 at 21:40, Matthias Urlichs  wrote:

> On 19.12.2015 20:25, Guido van Rossum wrote:
> > Perhaps you can add a check for a simple boolean 'stop' flag to your
> > condition check, and when you want to stop the loop you set that flag
> > and then call notify() on the condition. Then you can follow the
> > standard condition variable protocol instead of all this nonsense. :-)
> Your example does not work.
>
> >def stop_it(self):
> >self.stopped = True
> >self.uptodate.notify()
>
> self.uptodate needs to be locked before I can call .notify() on it.
> Creating a new task just for that seems like overkill, and I'd have to
> add a generation counter to prevent a race condition. Doable, but ugly.
>
> However, this doesn't fix the generic problem; Condition.wait() was just
> what bit me today.
> When a non-async generator goes out of scope, its finally: blocks will
> execute. An async procedure call whose refcount reaches zero without
> completing simply goes away; finally: blocks are *not* called and there
> is *no* warning.
> I consider that to be a bug.
>
> --
> -- Matthias Urlichs
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/gjcarneiro%40gmail.com
>



-- 
Gustavo J. A. M. Carneiro
Gambit Research
"The universe is always one step beyond logic." -- Frank Herbert
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] asyncio: how to interrupt an async def w/ finally: ( e.g. Condition.wait() )

[Kevin Conway]

> An async procedure call whose refcount reaches zero without completing
simply goes away; finally: blocks are *not* called and there is *no*
warning.

I believe OP is looking at these two scenarios:

def generator():
try:
yield None
yield None
finally:
print('finally')


gen = generator()
gen.send(None)
del gen
# prints finally on GC


class Awaitable:
def __await__(self):
return self
def __next__(self):
return self


async def coroutine():
try:
await Awaitable()
await Awaitable()
finally:
print('finally')


coro = coroutine()
coro.send(None)
del coro
# prints finally on GC

I don't see any difference in the behaviour between the two. My guess is
that OP's code is not hitting a zero refcount.

On Sat, Dec 19, 2015 at 5:00 PM Gustavo Carneiro 
wrote:

> I tried to reproduce the problem you describe, but failed.  Here's my test
> program (forgive the awful tab indentation, long story):
>
> --
> import asyncio
>
> async def foo():
> print("resource acquire")
> try:
> await asyncio.sleep(100)
> finally:
> print("resource release")
>
>
> async def main():
> task = asyncio.ensure_future(foo())
> print("task created")
> await asyncio.sleep(0)
> print("about to cancel task")
> task.cancel()
> print("task cancelled, about to wait for it")
> try:
> await task
> except asyncio.CancelledError:
> pass
> print("waited for cancelled task")
>
>
> if __name__ == '__main__':
> loop = asyncio.get_event_loop()
> loop.run_until_complete(main())
> loop.close()
> ---
>
> I get this output:
>
> 
> 10:54:28 ~/Documents$ python3.5 foo.py
> task created
> resource acquire
> about to cancel task
> task cancelled, about to wait for it
> resource release
> waited for cancelled task
> 
>
> Which seems to indicate that the finally clause is correctly executed when
> the task is waited for, after being cancelled.
>
> But maybe I completely misunderstood your problem...
>
>
> On 19 December 2015 at 21:40, Matthias Urlichs 
> wrote:
>
>> On 19.12.2015 20:25, Guido van Rossum wrote:
>> > Perhaps you can add a check for a simple boolean 'stop' flag to your
>> > condition check, and when you want to stop the loop you set that flag
>> > and then call notify() on the condition. Then you can follow the
>> > standard condition variable protocol instead of all this nonsense. :-)
>> Your example does not work.
>>
>> >def stop_it(self):
>> >self.stopped = True
>> >self.uptodate.notify()
>>
>> self.uptodate needs to be locked before I can call .notify() on it.
>> Creating a new task just for that seems like overkill, and I'd have to
>> add a generation counter to prevent a race condition. Doable, but ugly.
>>
>> However, this doesn't fix the generic problem; Condition.wait() was just
>> what bit me today.
>> When a non-async generator goes out of scope, its finally: blocks will
>> execute. An async procedure call whose refcount reaches zero without
>> completing simply goes away; finally: blocks are *not* called and there
>> is *no* warning.
>> I consider that to be a bug.
>>
>> --
>> -- Matthias Urlichs
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>>
> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/gjcarneiro%40gmail.com
>>
>
>
>
> --
> Gustavo J. A. M. Carneiro
> Gambit Research
> "The universe is always one step beyond logic." -- Frank Herbert
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/kevinjacobconway%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Webmaster] Python keeps installing as 32 bit

[Mullins, Robb]

Hi,

Please remove these posts/liservs, etc. if possible, or strip my contact 
info/name/phone/email off the posts please.  I’m getting calls from people 
trying to help with my Python install issue.


http://code.activestate.com/lists/python-dev/138936/

http://blog.gmane.org/gmane.comp.python.devel


This was not supposed to be posted online.  I just wanted to know if there was 
a trick to forcing x64 Python to install on x64 machines.

Thanks,
RM



Desktop Support Specialist
Center for Innovation in Teaching & Learning
citl-techsupp...@mx.uillinois.edu 
(For computer issues, please use the ticket system.)
(217) 333-2146

From: Mullins, Robb
Sent: Wednesday, December 16, 2015 2:49 PM
To: 'Brett Cannon' ; Steve Holden 
Cc: webmas...@python.org; python-dev@python.org
Subject: RE: [Python-Dev] [Webmaster] Python keeps installing as 32 bit

Yeah, I was using  Windows x86-64 executable 
installer  from 
that page.  I tried unzipping it just in case, no luck.

I’m thinking I’ll probably just use 32-bit though.  I found a post saying 
64-bit might have issues compiling.  I don’t think users will know or care.  
And there x86 installers are there.
http://www.howtogeek.com/197947/how-to-install-python-on-windows/
[cid:image001.jpg@01D139A9.8D88BCE0]
The only other thing I was thinking was something with the chip maybe.  I ran 
into this about a year ago.  (Or more now…)  I Python down for 32 vs 64-bit.  
Then I noticed some 64-bit machines were still doing 32-bit, but I only have 
the x86-64.exe.  I can’t force x64 on it.
It’s not a huge issue at this point.  Once I figure it out, it will save time.  
I’m planning on manually uninstalling versions of Python and then installing 
the current one (leaning toward x86 now) so all the user machines are 
consistent.

Thanks,
Robb




Desktop Support Specialist
Center for Innovation in Teaching & Learning
citl-techsupp...@mx.uillinois.edu 
(For computer issues, please use the ticket system.)
(217) 333-2146

From: Brett Cannon [mailto:br...@python.org]
Sent: Wednesday, December 16, 2015 2:39 PM
To: Steve Holden >; Mullins, 
Robb >
Cc: webmas...@python.org; 
python-dev@python.org
Subject: Re: [Python-Dev] [Webmaster] Python keeps installing as 32 bit

I can say for certain that Python 3.5.1 will install as 64-bit as that's what 
I'm personally running on the Windows 10 laptop that I'm writing this email on.

If you look at 
https://www.python.org/downloads/release/python-351/
 you will notice there are explicit 64-bit installers that you can use. Did you 
get your copy of Python by going straight to 
python.org/download
 and clicking the yellow "Download Python 3.5.1" button?

On Wed, 16 Dec 2015 at 12:33 Steve Holden 
> wrote:
Hi Robb,

This address is really for web site issues, but we are mostly old hands, and 
reasonably well-connected, so we try to act as a helpful channel when we can.

In this case I can't personally help (though another webmaster may, if 
available, be able to offer advice). I stopped doing system administration for 
anything but my own machines a long time ago, having done far too much :-)

The many mailing list channels available are listed at 
https://mail.python.org/mailman/listinfo.
 I would recommend that you try the distutils list at