Kyle Stanley added the comment:
Thank you Arteem, that should help indicate where the memory leak is present.
--
___
Python tracker
<https://bugs.python.org/issue44
Kyle Stanley added the comment:
> But why does `asyncio.run` unconditionally create a new event loop instead of
> running on `asyncio.get_event_loop`?
AFAIK, it does so for purposes of compatibility in programs that need multiple
separate event loops and providing a degree o
Kyle Stanley added the comment:
Thanks for closing up the issue, Victor :)
--
___
Python tracker
<https://bugs.python.org/issue39995>
___
___
Python-bugs-list m
Kyle Stanley added the comment:
> Thanks, Ned <3
(For following up and closing the issue)
--
___
Python tracker
<https://bugs.python.org/issue37228>
___
__
Kyle Stanley added the comment:
> Since 3.5 has now reached end-of-life, this issue will not be fixed there so
> it looks like it can be closed.
Thanks, Ned <3
--
___
Python tracker
<https://bugs.python.or
Kyle Stanley added the comment:
Thanks for bringing attention to this, Julien.
While the regression is definitely unfortunate, I'm uncertain about whether the
behavior is *correct* to allow in the first place. Specifically, allowing the
registration of an atexit hook which u
Kyle Stanley added the comment:
> I'm dealing with a subtle deadlock involving
> concurrent.futures.ThreadPoolExecutor, and my solution that worked in Python
> 3.8 broke with 3.9. I'm running some long-running (possibly infinite) tasks
> in the thread pool, and I can
New submission from Kyle Stanley :
This issue was created for the purpose of tracking any changes related to PEP
594 (Removing dead batteries from stdlib), and any relevant discussions about
the modules being removed.
--
components: Library (Lib)
messages: 382815
nosy: aeros
Kyle Stanley added the comment:
New changeset b9127dd6eedd693cfd716a648864e2e00186 by Yurii Karabas in
branch 'master':
bpo-42392: Improve removal of *loop* parameter in asyncio primitives (GH-23499)
https://github.com/python/cpython/commit/b9127dd6eedd693cfd716a6488
Kyle Stanley added the comment:
> Is there anyone who is assigned to removing the deprecated `loop` parameter
> from `asyncio` codebase?
> If not I can take this task, I believe I have enough free time and curiosity
> to do that :)
You can certainly feel free to work on that
Kyle Stanley added the comment:
Regarding the example _get_loop():
```
def _get_loop(self):
loop = asyncio.get_running_loop()
if self._loop is None:
self._loop = loop
if loop is not self._loop: raise
if not loop.is_running
Kyle Stanley added the comment:
> Oh my. FWIW I think that we need to implement this differently. I don't
> think it matters where, say, an asyncio.Lock was instantiated. It can be
> created anywhere. So IMO its __init__ shouldn't try to capture the current
> loop
Kyle Stanley added the comment:
Sure, I would be interested in helping with this. Although if a newer
contributor takes it up before I'm able to, I wouldn't be opposed to that
either (my main priority at the moment is helping with PEP 594 since it's a
concrete goal of my in
Kyle Stanley added the comment:
> Regularly calling executor.shutdown() and then instantiating a new
> ThreadPoolExecutor in order to run an asyncio program does not seem like a
> good API to me.
Clarification: you're typically only supposed to instantiate a single
ThreadP
Kyle Stanley added the comment:
Also note that the difference in memory is much higher when an exception occurs
(presumably because the exception is stored on future._exception and not
cleaned up?):
```
[aeros:~/repos/cpython]$ ./python
~/programming/python/asyncio_run_in_exec_leak.py -n
Kyle Stanley added the comment:
In the snippet provided, at least part of the resources are not finalized
because executor.shutdown() was not called in the program (which should be done
when creating a local instance of the executors, either explicitly or using the
context manager). For the
Change by Kyle Stanley :
--
nosy: +aeros, bquinlan, pitrou
___
Python tracker
<https://bugs.python.org/issue41588>
___
___
Python-bugs-list mailing list
Unsub
Kyle Stanley added the comment:
Since this issue is now a significant blocker for PEP 594 (remove stdlib dead
batteries, which includes asyncore and asynchat), I'm going to prioritize
working on this and assign it to myself.
--
assignee: -&g
Kyle Stanley added the comment:
I've applied Justin's patch, and opened PRs for backporting to 3.9 and 3.8
(currently waiting on CI).
Based on my local testing and buildbot fleet results, it definitely reduces the
rate at which the failure occurs, but does not 100% resolve the E
Kyle Stanley added the comment:
New changeset de73d432bb29f6439f2db16cb991e15e09c70c26 by Justin Turner Arthur
in branch 'master':
bpo-38912: fix close before connect callback in test_asyncio SSL tests
(GH-22691)
https://github.com/python/cpyt
Kyle Stanley added the comment:
I can confirm that both on the latest version of the docs (for 3.8) and for the
version mentioned in the issue (3.6), the issue mentioned with
asyncio.subprocess.PIPE is no longer present. (It was likely fixed in the
asyncio documentation overhaul that
Kyle Stanley added the comment:
DanilZ, could you take a look at the superseding issue
(https://bugs.python.org/issue37297) and see if your exception raised within
the job is the same?
If it's not, I would suggest opening a separate issue (and linking to it in a
comment here), as I
Kyle Stanley added the comment:
I've received approval from Lukasz to backport to 3.9 and 3.8, and have now
merged the PRs.
It could use a post-commit review from a core dev that has some additional
experience with SQLite3, but it can be de-escalated from release bl
Change by Kyle Stanley :
--
nosy: +ghaering
___
Python tracker
<https://bugs.python.org/issue41815>
___
___
Python-bugs-list mailing list
Unsubscribe:
Kyle Stanley added the comment:
With this being a segfault that's simple to replicate under normal
circumstances, I've elevated the priority to release blocker. I'll wait on the
release manager (Lukasz) for explicit approval for backporting to 3.9 and 3.8,
with both branch
Kyle Stanley added the comment:
In addition to the above, if a major revision is made to pty, I'd suggest also
addressing the issue of "master/slave" terminology, and replace it with
something comparable like "parent/child". There's an open devguide issue
(http
Kyle Stanley added the comment:
Thanks for bringing attention to cancel_futures being missing in the base
Executor class and for the PR, Shantanu!
--
___
Python tracker
<https://bugs.python.org/issue39
Kyle Stanley added the comment:
New changeset a763ee3c583e6a2dfe1b1ac0600a48e8a978ed50 by Shantanu in branch
'3.9':
[3.9] bpo-39349: Add cancel_futures to Executor.shutdown base class (GH-22023)
(GH-22048)
https://github.com/python/cpython/commit/a763ee3c583e6a2dfe1b1ac0600a48
Kyle Stanley added the comment:
New changeset 17dc1b789ecc33b4a254eb3b799085f4b3624ca5 by Shantanu in branch
'master':
bpo-39349: Add cancel_futures to Executor.shutdown base class (GH-22023)
https://github.com/python/cpython/commit/17dc1b789ecc33b4a254eb3b799085
Change by Kyle Stanley :
--
nosy: -paul.moore, steve.dower, tim.golden, zach.ware
___
Python tracker
<https://bugs.python.org/issue41682>
___
___
Python-bug
Change by Kyle Stanley :
--
components: +Windows, asyncio
nosy: +paul.moore, steve.dower, tim.golden, zach.ware
___
Python tracker
<https://bugs.python.org/issue41
Kyle Stanley added the comment:
In case the link to the logs are lost from closing and reopening the PR, it was
a Windows CI test (GitHub actions) ran with the following configuration:
D:\a\cpython\cpython>"D:\a\cpython\cpython\PCbuild\amd64\python.exe" -u -Wd -E
-bb -m test
Change by Kyle Stanley :
--
components: +Tests
nosy: +asvetlov, yselivanov
type: -> behavior
versions: +Python 3.10
___
Python tracker
<https://bugs.python.org/issu
New submission from Kyle Stanley :
In an unrelated PR to add a new argument to the base Executor class for
concurrent.futures, there was a test_asyncio failure in test_sendfile for the
ProactorEventLoop tests:
==
FAIL
Change by Kyle Stanley :
--
nosy: aeros
priority: normal
severity: normal
status: open
title: test_asyncio: Proactor
___
Python tracker
<https://bugs.python.org/issue41
Kyle Stanley added the comment:
Good catch, Shantanu. It was not intentional on my part, and it would make
sense to include *cancel_futures* in the base Executor class as documented. If
you'd like to submit a PR to add it (attaching it to this issue), I should be
able to review it wit
Kyle Stanley added the comment:
Due to the way pickle works, it's not presently possible to serialize wrapped
functions directly, at least not in a way that allows you to pass it to
executor.submit() within the inner function (AFAICT). I'm also not certain what
it would involve
Kyle Stanley added the comment:
> Should I set status for this issue for closed with resolution rejected ?
I'll proceed with closing the issue.
> Should I delete branch on my forked git repo ?
> Can I delete my forked git repo ?
Might as well delete the branch, but the forked
Kyle Stanley added the comment:
Yep, having to set a custom exception handler definitely constitutes as needing
"finer control over the event loop behavior". There's absolute nothing wrong
with using the low-level API when you need further customization, but we try to
mi
Kyle Stanley added the comment:
Thanks Rémi for working on this.
--
priority: release blocker -> normal
stage: patch review -> commit review
___
Python tracker
<https://bugs.python.org/i
Kyle Stanley added the comment:
> With beta 4 coming in 2 days it would perhaps good to choose now whether this
> must be done for 3.9 or 3.10
Agreed. I'm definitely leaning towards 3.10 with it being decently well into
the beta, but I'll try to bring this to Yury'
Kyle Stanley added the comment:
Thanks again for working on this @Fantix, and for the continued vigilance on
the issue after the test failures occurred in the buildbots. I think this
recent PR-20868 will do the trick. At the very least, it will make the failure
much more rare than before
Change by Kyle Stanley :
--
nosy: +aeros
___
Python tracker
<https://bugs.python.org/issue40973>
___
___
Python-bugs-list mailing list
Unsubscribe:
Change by Kyle Stanley :
--
nosy: +jaraco, lemburg
___
Python tracker
<https://bugs.python.org/issue40973>
___
___
Python-bugs-list mailing list
Unsubscribe:
Kyle Stanley added the comment:
> Last question before PR pls: given the fact that this test is to cover a
> fixed case when loop.sock_*() was hanging forever, should I keep the
> wait_for(..., timeout=10)?
For now, I think we can keep the asyncio.wait_for() w/ 10 the sec time
Kyle Stanley added the comment:
No problem at all, and thanks for clarifying.
> If enlarging the timeout works in some of your cases, that really inspires me
> on a potential cause of the issue in the test. To simulate the race condition
> specifically for loop.sock_sendall(), I
Kyle Stanley added the comment:
I believe that I might have identified a simple fix for the issue, but first
I'd like to clarify on something:
What exactly is the reason that we need to use asyncio.wait_for() with a
timeout of 10 seconds in these tests? We typically avoid using
Kyle Stanley added the comment:
I was able to replicate the failure locally by running the large number of jobs
in parallel. We typically use this to test for tricky race conditions, to
simulate maximum load:
[aeros:~/repos/cpython]$ ./python -m test test_asyncio.test_sock_lowlevel
--match
Kyle Stanley added the comment:
Thanks for bringing attention to this Rémi.
I'm a bit concerned that it might be a bit late into the 3.9 beta to
incorporate a removal and that we might need to consider delaying to 3.10, but
I'll leave that up to Lukasz. Feel free to open a
Kyle Stanley added the comment:
Thanks for the additional feedback, Caleb.
> I think `gather()` should work the same. It would be confusing if
> `future_gather.cancelled()` is false if a child is cancelled, while a plain
> old outer future returns `future.cancelled() == true` i
Kyle Stanley added the comment:
Upon further investigation, I've realized that the issue is just that the
cancel() override for `_GatheringFuture` never sets its state to CANCELLED at
any point (unlike its parent), and is instead going to always be set to
FINISHED because o
Kyle Stanley added the comment:
> So, we can't rely on checking ``self.done() and self._cancel_requested`` for
> future.cancelled() as this would mean that future.cancelled() would return
> true for a future that fully completed if `future.cancel()` was called after
> it f
Kyle Stanley added the comment:
> Specifically a future can't be cancelled once it reaches the PENDING state,
> this is indicated when future.cancel() returns false; see
> https://github.com/python/cpython/blob/0e96c419d7287c3c7f155c9f2de3c61020386256/Lib/asyncio/futures.py#L
Kyle Stanley added the comment:
Thank you for the bug report Timm.
While I can certainly understand the source of the confusion here, I think you
may be misunderstanding an important part of cancellation for futures.
Specifically a future can't be cancelled once it reaches the PENDING
Kyle Stanley added the comment:
The main issue is that shutdown_default_executor() (or something that does the
same thing) is necessary to ensure the resources of the event loop's
ThreadPoolExecutor are finalized in a safe manner. Otherwise, it frequently
results in dangling threads
Kyle Stanley added the comment:
Thanks for the PR, Zackery.
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Kyle Stanley added the comment:
> is there a workaround for earlier Python versions that does not involve
> patching the standard library?
You could potentially try using the new default watcher,
`ThreadedChildWatcher`, by implementing it locally and setting it as the child
watcher
Kyle Stanley added the comment:
> The new test_sock_client_racing() test seems to have... a race condition...
I'm also a bit skeptical about relying on `time.sleep(0.01)` at the end of the
loop in `_basetest_sock_recv_racing()`, but I don't have an idea for a better
approach
Kyle Stanley added the comment:
>From looking at the commit history of AbstactEventLoop.run_in_executor(), it
>seems that it was previously be a non-coroutine method prior to the conversion
>from the `@asyncio.coroutine` decorator to `async def` (PR-4753). See
>https://githu
Kyle Stanley added the comment:
> I made the suggested change to just print the os.remove() statements (instead
> of executing them) and also removed the 'skip news'.
I think you may have misunderstood the suggestion.
Specifically, the key part was "I would suggest add
Kyle Stanley added the comment:
Thanks for bring this to our attention and working on this Bar Harel, and
thanks to Yury for the suggestions.
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Pytho
Kyle Stanley added the comment:
New changeset 13206b52d16c2489f4c7dd2dce2a7f48a554b5ed by Bar Harel in branch
'master':
bpo-40405: Fix asyncio.as_completed docs (GH-19753)
https://github.com/python/cpython/commit/13206b52d16c2489f4c7dd2dce2a7f
Kyle Stanley added the comment:
Now that the `versionadded` label has been added and the contextvar issue was
addressed, this issue can be closed.
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Pytho
Change by Kyle Stanley :
--
pull_requests: +19555
pull_request: https://github.com/python/cpython/pull/20278
___
Python tracker
<https://bugs.python.org/issue32
Change by Kyle Stanley :
--
pull_requests: +19448
pull_request: https://github.com/python/cpython/pull/20143
___
Python tracker
<https://bugs.python.org/issue32
Kyle Stanley added the comment:
> Would it make sense to deprecate it and stick with ThreadedChildWatcher?
I had proposed deprecating it in another bpo issue, but it was brought up that
MutliLoopWatcher had only been added recently in 3.8. So, it might be a bit too
soon to deprecate
Change by Kyle Stanley :
--
pull_requests: +19324
pull_request: https://github.com/python/cpython/pull/20013
___
Python tracker
<https://bugs.python.org/issue38
Kyle Stanley added the comment:
> What other MultiLoopWatcher tests are currently having random hangs?
Oh, never mind. I see `test_stdin_stdout` is also hanging in
https://bugs.python.org/issue38182. I would like to take a closer look at that
one before skipping it as w
Kyle Stanley added the comment:
> There are more MultiLoopWatcher tests which hang randomly, it's not only
> test_close_kill_running().
What other MultiLoopWatcher tests are currently having random hangs? From
searching "MultiLoopWatcher" on bpo, I'm only finding
Kyle Stanley added the comment:
I presume this is referring to the following example on the first page of the
docs:
```
import asyncio
async def main():
print('Hello ...')
await asyncio.sleep(1)
print('... World!')
# Python 3.7+
asyncio.run(main())
```
If
Kyle Stanley added the comment:
> Is the new asyncio.loop.shutdown_default_executor() suitable for event loops
> that are run in single-step mode?
I honestly can't say for certain; the primary intended use case for
shutdown_default_executor() was to provide a means of properly
Kyle Stanley added the comment:
I have also explored a few different solutions and was unable to find a fix for
this issue. If it's still causing intermittent hangs after Andrew's patch, we
may want to consider skipping `test_close_kill_running` for `MultiLoopWatcher`
until we ca
Change by Kyle Stanley :
--
nosy: +aeros
___
Python tracker
<https://bugs.python.org/issue40512>
___
___
Python-bugs-list mailing list
Unsubscribe:
Kyle Stanley added the comment:
John Smith wrote:
> The code works normally again if I remove the return statement from the
> `run_coro` function.
I am a bit surprised that the code works as intended simply by removing the
return statement. Perhaps Yury or Andrew can clarify on that
Kyle Stanley added the comment:
@Bar as requested by Guido, let's continue the discussion in the PR:
https://github.com/python/cpython/pull/19753#pullrequestreview-403185142.
--
___
Python tracker
<https://bugs.python.org/is
Kyle Stanley added the comment:
Alternatively, I think "can wait for" would also read better than "allows to
wait for", if that is preferred over my above suggestion.
--
___
Python tracker
<https://bug
Kyle Stanley added the comment:
> based upon the order of completion
I forgot to explain this in the above message. I think something that mentions
"order of completion" should be included. Although it's implied in the name
`as_completed`, to me it seems worthwhile to be f
Kyle Stanley added the comment:
Yury Selivanov wrote:
> I'd suggest to change to: "Return an iterator of coroutines. Each coroutine
> allows to wait for the earliest next result from the set of the remaining
> awaitables."
The first sentence looks good to me, but I
Change by Kyle Stanley :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Kyle Stanley added the comment:
New changeset cc011b5190b63f0be561ddec38fc4cd9e60cbf6a by Kyle Stanley in
branch '3.8':
[3.8] bpo-40431: Fix syntax typo in turtledemo (GH-19777) (#19784)
https://github.com/python/cpython/commit/cc011b5190b63f0be561ddec38fc4c
Kyle Stanley added the comment:
Also, thanks to Victor for coming up with an interim solution, and for the
feedback from Antoine and Thomas. GH-19760 is a significant improvement from my
initial proposal in GH-19751.
--
___
Python tracker
<ht
Kyle Stanley added the comment:
I am a bit concerned about the performance implication of accessing the thread
wakeup through a lock in the call queue, but for now I think it's reasonable to
address the buildbot failure with a locking solution while we try to find a
better one.
I&
Change by Kyle Stanley :
--
pull_requests: +19107
pull_request: https://github.com/python/cpython/pull/19784
___
Python tracker
<https://bugs.python.org/issue40
Change by Kyle Stanley :
--
versions: -Python 3.7
___
Python tracker
<https://bugs.python.org/issue40431>
___
___
Python-bugs-list mailing list
Unsubscribe:
Kyle Stanley added the comment:
Victor Stinner wrote:
> If you backport the change, please remove the NEWS entry, since it's not a
> SyntaxError in 3.7 and 3.8.
Good point. In that case, I'll do a manual backport of the PR just to 3.8 and
skip the news entry. As mentione
Kyle Stanley added the comment:
For addressing the backwards compatibility concern, I think we should just
convert it to a SyntaxWarning for cases like the above to indicate that it's
not really correct syntax, but not harmful enough to justify code breakage. I
think it fits the docum
Kyle Stanley added the comment:
> I don't think that it's worth it to backport the change to 3.7 and 3.8: in
> Python 3.7 and 3.8, the code is accepted.
> Python 3.7 and 3.8 are not broken, so there is no need to fix them :-)
I think we could very reasonably change `e
Kyle Stanley added the comment:
I decided to close PR-19751. Both because it does not correctly address the
race condition (due to an oversight on my part) and it would add substantial
overhead to _ThreadWakeup. Instead, I agree that we should explore a
non-locking solution
Kyle Stanley added the comment:
Oops, it seems that I opened PR-19751 a bit preemptively. When I get the
chance, I'll see if Antoine's implementation can address the failures and do
some comparisons.
--
___
Python track
Kyle Stanley added the comment:
Antoine Pitrou wrote:
> You're right that a Py_DECREF is missing there. However, it seems unlikely
> that this is triggering the test failure, because `PyList_New(1)` will
> practically never fail in normal conditions (and even making it delibe
Change by Kyle Stanley :
--
pull_requests: +19072
pull_request: https://github.com/python/cpython/pull/19751
___
Python tracker
<https://bugs.python.org/issue39
Kyle Stanley added the comment:
paul rubin wrote:
> Yeah I think the basic answer to this ticket is "Python doesn't really have
> multisets and a proposal to add them should go somewhere else". Fair
> enough-- consider the request withdrawn from here.
Not necessari
Kyle Stanley added the comment:
paul rubin wrote:
> In that case, it works for what it is made for, so it really then is a
> separate question of whether implementing multisets properly is worthwhile.
Indeed, that seems to be the primary question to address. Personally, I don'
Kyle Stanley added the comment:
After writing the above out and a bit of further consideration, I think it
might make more sense to wait for the event after setting `self._closed = True`
so that it prevents future wakeup() and clear() calls from reading/writing to
the pipe, while still
Kyle Stanley added the comment:
I can't be certain for the other failures, but I'm currently exploring a
potential solution for addressing the `test_killed_child` failure. To me, it
seems to be a race condition with attempting to call _ThreadWakeup.close()
while there are still b
Change by Kyle Stanley :
--
nosy: +pitrou
___
Python tracker
<https://bugs.python.org/issue39995>
___
___
Python-bugs-list mailing list
Unsubscribe:
Change by Kyle Stanley :
--
nosy: +aeros
___
Python tracker
<https://bugs.python.org/issue39995>
___
___
Python-bugs-list mailing list
Unsubscribe:
Kyle Stanley added the comment:
Also, adding Raymond to the nosy list in case he has any specific comments
about a frozen collections.Counter.
--
nosy: +rhettinger
versions: +Python 3.9
___
Python tracker
<https://bugs.python.org/issue40
Kyle Stanley added the comment:
Rather than starting this out as a bpo issue, I would highly recommend bringing
forth a proposal for this in python-id...@python.org and explaining in as much
detail as possible:
1) How a frozen variation of collection.Counter would benefit your specific use
Change by Kyle Stanley :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
1 - 100 of 416 matches
Mail list logo