[issue9248] multiprocessing.pool: Proposal: waitforslot
R. David Murray added the comment: Closed per OP's request. -- nosy: +r.david.murray resolution: - out of date stage: patch review - resolved status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9248 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9248] multiprocessing.pool: Proposal: waitforslot
Ask Solem added the comment: This patch is quite dated now and I have fixed many bugs since. The feature is available in billiard and is working well but The code has diverged quite a lot from python trunk. I will be updating billiard to reflect the changes for Python 3.4 soon (billiard is currently 3.3). I think we can forget about taking individual patches from billiard for now, and instead maybe merge the codebases at some point if there's interest. we have a version of multiprocessing.Pool using async IO and one pipe per process that drastically improves performance and also avoids the threads+forking issues (well, not the initial fork), but I have not yet adapted it to use the new asyncio module in 3.4 So suggestion is to close this and rather get a discussion going for combining our efforts. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9248 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9248] multiprocessing.pool: Proposal: waitforslot
Richard Oudkerk added the comment: Since there are no new features added to Python 2, this would be a Python 3 only feature. I think for Python 3 it is better to concentrate on developing concurrent.futures rather than multiprocessing.Pool. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9248 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9248] multiprocessing.pool: Proposal: waitforslot
Mark Lawrence added the comment: Could somebody please review the attached patch. -- nosy: +BreamoreBoy versions: +Python 3.5 -Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9248 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9248] multiprocessing.pool: Proposal: waitforslot
Changes by Ned Deily n...@acm.org: -- nosy: +sbt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9248 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9248] multiprocessing.pool: Proposal: waitforslot
Changes by R. David Murray rdmur...@bitdance.com: -- stage: - patch review type: - feature request versions: +Python 3.3 -Python 2.6, Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9248 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9248] multiprocessing.pool: Proposal: waitforslot
New submission from Ask Solem a...@opera.com: This patch adds the `waitforslot` argument to apply_async. If set to `True`, apply_async will not return until there is a worker available to process the job. This is implemented by a semaphore that is released by the result handler whenever a new result is ready. The semaphore is also released when the supervisor (worker_handler) finds a worker process that has been unexpectedly terminated. This is already in use by Celery 2.0, which ships with its own modified version of multiprocessing.pool. I'm not sure about the name ``waitforslot``, I think I may even hate it, but haven't been able to come up with a better name for it yet. -- components: Library (Lib) messages: 110193 nosy: asksol, jnoller priority: normal severity: normal status: open title: multiprocessing.pool: Proposal: waitforslot versions: Python 2.6, Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9248 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9248] multiprocessing.pool: Proposal: waitforslot
Changes by Ask Solem a...@opera.com: -- keywords: +patch Added file: http://bugs.python.org/file17985/multiprocessing-tr...@82502-apply-semaphore.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9248 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
