[issue7946] Convoy effect with I/O bound threads and New GIL

Sophist added the comment: > https://docs.google.com/document/d/18CXhDb1ygxg-YXNBJNzfzZsDFosB5e6BfnXLlejd9l0/edit 1. The steering committee hasn't given the go ahead for this yet, and we have no idea when such a decision will be made nor whether the decision with be yes or no. 2. Even

[issue7946] Convoy effect with I/O bound threads and New GIL

Guido van Rossum added the comment: Start here: https://docs.google.com/document/d/18CXhDb1ygxg-YXNBJNzfzZsDFosB5e6BfnXLlejd9l0/edit AFAICT the SC hasn't made up their minds about this. -- ___ Python tracker

[issue7946] Convoy effect with I/O bound threads and New GIL

Sophist added the comment: > I think that we should focus our efforts on removing the GIL, now that we > have a feasible solution for doing so without breaking anything Is this really a thing? Something that is definitely happening in a reasonable timescale? Or are there some big

[issue7946] Convoy effect with I/O bound threads and New GIL

Omer Katz added the comment: I think that we should focus our efforts on removing the GIL, now that we have a feasible solution for doing so without breaking anything (hopefully) however the removal of the GIL is still far from being complete and will need to be rebased upon the latest Python

[issue7946] Convoy effect with I/O bound threads and New GIL

Change by Guido van Rossum : -- nosy: +gvanrossum ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue7946] Convoy effect with I/O bound threads and New GIL

Sophist added the comment: Please see also https://github.com/faster-cpython/ideas/discussions/328 for a proposal for a simple (much simpler than BFS) GIL scheduler only allocating the GIL between runable O/S threads waiting to have ownership of the GIL, and using the O/S scheduler for

[issue7946] Convoy effect with I/O bound threads and New GIL

Change by Mark Dickinson : -- nosy: +mark.dickinson ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue7946] Convoy effect with I/O bound threads and New GIL

Change by STINNER Victor : -- nosy: -vstinner ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue7946] Convoy effect with I/O bound threads and New GIL

Change by Charles-François Natali : -- nosy: -neologix ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue7946] Convoy effect with I/O bound threads and New GIL

Maarten Breddels added the comment: In case someone finds it useful, I've written a blog post on how to visualize the GIL: https://www.maartenbreddels.com/perf/jupyter/python/tracing/gil/2021/01/14/Tracing-the-Python-GIL.html In the comments (or at

[issue7946] Convoy effect with I/O bound threads and New GIL

Stuart Axon added the comment: Catching up on the comments on this, it seems like nobody has enough certainty to say it will work well enough. In Linux, the scheduler is pluggable, which lets other non-default schedulers be shipped and tried in the real world. - See schedutil, introduced

[issue7946] Convoy effect with I/O bound threads and New GIL

Change by Gregory P. Smith : -- resolution: -> wont fix ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue7946] Convoy effect with I/O bound threads and New GIL

Change by David Beazley : -- stage: -> resolved status: open -> closed ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue7946] Convoy effect with I/O bound threads and New GIL

STINNER Victor added the comment: If someone wants to close this issue, I suggest to write a short section in the Python documentation to give some highlights on the available options and stategies to maximize performances and list drawbacks of each method. Examples: * Multiple threads

[issue7946] Convoy effect with I/O bound threads and New GIL

Dima Tisnek added the comment: My 2c as Python user: Back in 2010, I've used multithreading extensively, both for concurrency and performance. Others used multiprocessing or just shelled out. People talked about using **the other** core, or sometimes the other socket on a server. Now in

[issue7946] Convoy effect with I/O bound threads and New GIL

Gregory P. Smith added the comment: It's a known issue and has been outlined very well and still comes up from time to time in real world applications, which tend to see this issue and Dave's presentation and just work around it in any way possible for their system and move on with life.

[issue7946] Convoy effect with I/O bound threads and New GIL

David Beazley added the comment: About nine years ago, I stood in front of a room of Python developers, including many core developers, and gave a talk about the problem described in this issue. It included some live demos and discussion of a possible fix.

[issue7946] Convoy effect with I/O bound threads and New GIL

Larry Hastings added the comment: FWIW: I think David's cited behavior proves that the GIL is de facto a scheduler. And, in case you missed it, scheduling is a hard problem, and not a solved problem. There are increasingly complicated schedulers with new approaches and heuristics.

[issue7946] Convoy effect with I/O bound threads and New GIL

Change by Dong-hee Na : -- nosy: +corona10 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue7946] Convoy effect with I/O bound threads and New GIL

Change by Joshua Bronson : -- nosy: +jab ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue7946] Convoy effect with I/O bound threads and New GIL

Change by Dirkjan Ochtman : -- nosy: -djc ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue7946] Convoy effect with I/O bound threads and New GIL

Gregory P. Smith added the comment: I suggest: (1) turning one of the patches (probably the last BFS one?) into a PR against the github master branch (3.9) and, (2) if none of the existing pyperformance workloads already demonstrates the problems with the existing GIL implementation, adopt

[issue7946] Convoy effect with I/O bound threads and New GIL

Gregory P. Smith added the comment: (unassigning as it doesn't make sense to assign to anyone unless they're actually working on it) -- assignee: pitrou -> ___ Python tracker

[issue7946] Convoy effect with I/O bound threads and New GIL

Change by Gregory P. Smith : -- priority: low -> normal versions: +Python 3.9 -Python 3.3 ___ Python tracker ___ ___

[issue7946] Convoy effect with I/O bound threads and New GIL

Omer Katz added the comment: FYI I can verify that the original benchmark is still valid on Python 3.7.3. I'm running the client on an 8 core CPU. The result is 30.702 seconds (341534.322 bytes/sec). I'll need somebody to decide how we're going to fix this problem. I can do the legwork.

[issue7946] Convoy effect with I/O bound threads and New GIL

Armin Rigo added the comment: Note that PyPy has implemented a GIL which does not suffer from this problem, possibly using a simpler approach than the patches here do. The idea is described and implemented here:

[issue7946] Convoy effect with I/O bound threads and New GIL

Omer Katz added the comment: Celery 5 is going async and in order to isolate the main event loop from task execution, the tasks are going to be executed in a different thread with it's own event loop. This thread may or may not be CPU bound. The main thread is I/O bound. This patch should

[issue7946] Convoy effect with I/O bound threads and New GIL

Changes by Stefan Behnel sco...@users.sourceforge.net: -- nosy: +scoder ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___ ___

[issue7946] Convoy effect with I/O bound threads and New GIL

Dima Tisnek added the comment: What happened to this bug and patch? -- nosy: +Dima.Tisnek ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___

[issue7946] Convoy effect with I/O bound threads and New GIL

Antoine Pitrou added the comment: Not much :) The patch is complex and the issue hasn't proved to be significant in production code. Do you have a (real-world) workload where this shows up? Le 15/07/2014 09:52, Dima Tisnek a écrit : Dima Tisnek added the comment: What happened to this bug

[issue7946] Convoy effect with I/O bound threads and New GIL

STINNER Victor victor.stin...@gmail.com added the comment: gettimeofday returns you wall clock time: if a process that modifies time is running, e.g. ntpd, you'll likely to run into trouble. the value returned is _not_ monotonic, ... The issue #12822 asks to use monotonic clocks when

[issue7946] Convoy effect with I/O bound threads and New GIL

Changes by Julian Mehnle jul...@mehnle.net: -- nosy: +jmehnle ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___ ___ Python-bugs-list mailing

[issue7946] Convoy effect with I/O bound threads and New GIL

Changes by Antoine Pitrou pit...@free.fr: -- priority: high - low versions: -Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___

[issue7946] Convoy effect with I/O bound threads and New GIL

Changes by P. Henrique Silva ph.si...@gmail.com: -- nosy: +phsilva ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___ ___ Python-bugs-list

[issue7946] Convoy effect with I/O bound threads and New GIL

Changes by Hans Lellelid h...@velum.net: -- nosy: +hozn ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___ ___ Python-bugs-list mailing list

[issue7946] Convoy effect with I/O bound threads and New GIL

Changes by Nir Aides n...@winpdb.org: Removed file: http://bugs.python.org/file17356/bfs.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___ ___

[issue7946] Convoy effect with I/O bound threads and New GIL

Nir Aides n...@winpdb.org added the comment: Updated bfs.patch with BSD license and copyright notice. ! Current version patches cleanly and builds with Python revision svn r81201. Issue 7946 and proposed patches were put on hold indefinitely following this python-dev discussion:

[issue7946] Convoy effect with I/O bound threads and New GIL

Gregory P. Smith g...@krypto.org added the comment: Thanks for all your work Nir! I personally think the BFS approach is the best we've seen yet for this problem! Having read the thread you linked to in full (ignoring the tagents bikeshedding and mudslinging that went on there), it sounds

[issue7946] Convoy effect with I/O bound threads and New GIL

Changes by Peter Portante peter.a.porta...@gmail.com: -- nosy: +portante ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___ ___

[issue7946] Convoy effect with I/O bound threads and New GIL

Michele vo.s...@gmail.com added the comment: Attached ccbench-osx.log made today on OSX on latest svn checkout. Hope it helps -- nosy: +Michele Added file: http://bugs.python.org/file17393/ccbench-osx.log ___ Python tracker rep...@bugs.python.org

[issue7946] Convoy effect with I/O bound threads and New GIL

Changes by Victor Godoy Poluceno victorpoluc...@gmail.com: -- nosy: +victorpoluceno ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___ ___

[issue7946] Convoy effect with I/O bound threads and New GIL

Changes by Nir Aides n...@winpdb.org: Added file: http://bugs.python.org/file17370/nir-ccbench-linux.log ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___

[issue7946] Convoy effect with I/O bound threads and New GIL

Nir Aides n...@winpdb.org added the comment: A link to ccbench results comparing old GIL, old GIL with long check interval, new GIL and BFS: http://bugs.python.org/file17370/nir-ccbench-linux.log Summary: Results for ccbench latency and bandwidth test run on Ubuntu Karmic 64bit, q9400

[issue7946] Convoy effect with I/O bound threads and New GIL

Changes by Nick Coghlan ncogh...@gmail.com: -- nosy: +ncoghlan ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___ ___ Python-bugs-list mailing

[issue7946] Convoy effect with I/O bound threads and New GIL

Changes by Nir Aides n...@winpdb.org: Removed file: http://bugs.python.org/file17330/bfs.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___ ___

[issue7946] Convoy effect with I/O bound threads and New GIL

Nir Aides n...@winpdb.org added the comment: Updated bfs.patch to patch cleanly updated py3k branch. Use: $ patch -p1 bfs.patch -- Added file: http://bugs.python.org/file17356/bfs.patch ___ Python tracker rep...@bugs.python.org

[issue7946] Convoy effect with I/O bound threads and New GIL

Changes by Nir Aides n...@winpdb.org: Removed file: http://bugs.python.org/file17195/bfs.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___ ___

[issue7946] Convoy effect with I/O bound threads and New GIL

Nir Aides n...@winpdb.org added the comment: Duck, here comes another update to bfs.patch. This one with some cleanups which simplify the code and improve behavior (on Windows XP), shutdown code, comments, and experimental use of TSC for timestamps, which eliminates timestamp reading

[issue7946] Convoy effect with I/O bound threads and New GIL

Changes by Nir Aides n...@winpdb.org: Added file: http://bugs.python.org/file17194/nir-ccbench-xp32.log ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___

[issue7946] Convoy effect with I/O bound threads and New GIL

Changes by Nir Aides n...@winpdb.org: Removed file: http://bugs.python.org/file16967/bfs.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___ ___

[issue7946] Convoy effect with I/O bound threads and New GIL

Nir Aides n...@winpdb.org added the comment: I updated bfs.patch with improvements on Windows XP. The update disables priority boosts associated with the scheduler condition on Windows for CPU bound threads. Here is a link to ccbench results:

[issue7946] Convoy effect with I/O bound threads and New GIL

Nir Aides n...@winpdb.org added the comment: Dave, The behavior of your patch on Windows XP/2003 (and earlier) might be related to the way Windows boosts thread priority when it is signaled. Try to increase priority of monitor thread and slice size. Another thing to look at is how to

[issue7946] Convoy effect with I/O bound threads and New GIL

Nir Aides n...@winpdb.org added the comment: On Thu, Apr 29, 2010 at 2:03 AM, David Beazley wrote: Wow, that is a *really* intriguing performance result with radically different behavior than Unix. Do you have any ideas of what might be causing it? Instrument the code and I'll send you a

[issue7946] Convoy effect with I/O bound threads and New GIL

Nir Aides n...@winpdb.org added the comment: On Wed, Apr 28, 2010 at 12:41 AM, Larry Hastings wrote: The simple solution: give up QPC and use timeGetTime() with timeBeginPeriod(1), which is totally reliable but only has millisecond accuracy at best. It is preferable to use a high

[issue7946] Convoy effect with I/O bound threads and New GIL

Nir Aides n...@winpdb.org added the comment: Dave, there seems to be some problem with your patch on Windows: F:\devz:\dabeaz-wcg\PCbuild\python.exe y:\ccbench.py -b == CPython 3.2a0.0 (py3k) == == x86 Windows on 'x86 Family 6 Model 23 Stepping 10, GenuineIntel' == --- I/O bandwidth ---

[issue7946] Convoy effect with I/O bound threads and New GIL

Charles-Francois Natali neolo...@free.fr added the comment: @dabeaz I'm getting random segfaults with your patch (even with the last one), pretty much everywhere malloc or free is called. Ater skimming through the code, I think the problem is due to gil_last_holder: In drop_gil and take_gil,

[issue7946] Convoy effect with I/O bound threads and New GIL

Changes by David Beazley d...@dabeaz.com: Removed file: http://bugs.python.org/file17102/dabeaz_gil.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___

[issue7946] Convoy effect with I/O bound threads and New GIL

David Beazley d...@dabeaz.com added the comment: Added extra pointer check to avoid possible segfault. -- Added file: http://bugs.python.org/file17104/dabeaz_gil.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946

[issue7946] Convoy effect with I/O bound threads and New GIL

Charles-Francois Natali neolo...@free.fr added the comment: I don't see segfaults anymore, but there's still an unsafe dereference of gil_last_holder inside take_gil: /* Wait on the appropriate GIL depending on thread's classification */ if (!tstate-cpu_bound) { /* We are I/O

[issue7946] Convoy effect with I/O bound threads and New GIL

donal djeo donaldje...@gmail.com added the comment: I'm getting random segfaults with your patch (even with the last one), pretty much everywhere malloc or free is called. Ater skimming through the code, I think the problem is due to gil_last_holder: In drop_gil and take_gil, you dereference

[issue7946] Convoy effect with I/O bound threads and New GIL

Changes by Antoine Pitrou pit...@free.fr: -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue7946] Convoy effect with I/O bound threads and New GIL

Changes by Antoine Pitrou pit...@free.fr: -- components: +Interpreter Core -None ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___ ___

[issue7946] Convoy effect with I/O bound threads and New GIL

Changes by Guilherme Salgado gsalg...@gmail.com: -- nosy: -salgado ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___ ___ Python-bugs-list

[issue7946] Convoy effect with I/O bound threads and New GIL

David Beazley d...@dabeaz.com added the comment: That second access of gil_last_holder-cpu_bound is safe because that block of code is never entered unless some other thread currently holds the GIL. If a thread holds the GIL, then gil_last_holder is guaranteed to have a valid value.

[issue7946] Convoy effect with I/O bound threads and New GIL

Charles-Francois Natali neolo...@free.fr added the comment: Didn't have much sleep last night, so please forgive me if I say something stupid, but: Python/pystate.c: void PyThreadState_DeleteCurrent() { PyThreadState *tstate = _PyThreadState_Current; if (tstate == NULL)

[issue7946] Convoy effect with I/O bound threads and New GIL

David Beazley d...@dabeaz.com added the comment: I stand corrected. However, I'm going to have to think of a completely different approach for carrying out that functionality as I don't know how the take_gil() function is able to determine whether gil_last_holder has been deleted or not.

[issue7946] Convoy effect with I/O bound threads and New GIL

Changes by David Beazley d...@dabeaz.com: Removed file: http://bugs.python.org/file17104/dabeaz_gil.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___

[issue7946] Convoy effect with I/O bound threads and New GIL

Charles-Francois Natali neolo...@free.fr added the comment: Do you have any examples or insight you can provide about how these segfaults have shown up in Python code? I'm not able to observe any such behavior on OS-X or Linux. Is this happening while running the ccbench program? Some

[issue7946] Convoy effect with I/O bound threads and New GIL

David Beazley d...@dabeaz.com added the comment: One more attempt at fixing tricky segfaults. Glad someone had some eagle eyes on this :-). -- Added file: http://bugs.python.org/file17106/dabeaz_gil.patch ___ Python tracker rep...@bugs.python.org

[issue7946] Convoy effect with I/O bound threads and New GIL

Nir Aides n...@winpdb.org added the comment: On Tue, Apr 27, 2010 at 12:23 PM, Charles-Francois Natali wrote: @nirai I have some more remarks on your patch: - /* Diff timestamp capping results to protect against clock differences * between cores. */ _LOCAL(long double) _bfs_diff_ts(long

[issue7946] Convoy effect with I/O bound threads and New GIL

Antoine Pitrou pit...@free.fr added the comment: I stand corrected. However, I'm going to have to think of a completely different approach for carrying out that functionality as I don't know how the take_gil() function is able to determine whether gil_last_holder has been deleted or not.

[issue7946] Convoy effect with I/O bound threads and New GIL

Larry Hastings la...@hastings.org added the comment: In Windows the high-precision counter might return different results on different cores in some hardware configurations (older multi-core processors). More specifically: some older multi-core processors where the HAL implements

[issue7946] Convoy effect with I/O bound threads and New GIL

Antoine Pitrou pit...@free.fr added the comment: Dave, In the current implementation, threads perform a timed-wait on a condition variable. If time expires and no thread switches have occurred, the currently running thread is forced to drop the GIL. A problem, as far as I can see, is that

[issue7946] Convoy effect with I/O bound threads and New GIL

David Beazley d...@dabeaz.com added the comment: Greg, I like the idea of the monitor suspending if no thread owns the GIL. Let me work on that. Good point on embedded systems. Antoine, Yes, the gil monitor is completely independent and simply ticks along every 5 ms. A worst case

[issue7946] Convoy effect with I/O bound threads and New GIL

Changes by David Beazley d...@dabeaz.com: Removed file: http://bugs.python.org/file17084/dabeaz_gil.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___

[issue7946] Convoy effect with I/O bound threads and New GIL

David Beazley d...@dabeaz.com added the comment: I've updated the GIL patch to reflect concerns about the monitor thread running forever. This version has a suspension mechanism where the monitor goes to sleep if nothing is going on for awhile. It gets resumed if threads try to acquire the

[issue7946] Convoy effect with I/O bound threads and New GIL

David Beazley d...@dabeaz.com added the comment: I've also attached a new file schedtest.py that illustrates a subtle difference between having the GIL monitor thread and not having the monitor. Without the monitor, every thread is responsible for its own scheduling. If you have a lot of

[issue7946] Convoy effect with I/O bound threads and New GIL

Nir Aides n...@winpdb.org added the comment: Dave, there seems to be a bug in your patch on Windows XP. It crashes in ccbench.py with the following output: python_d.exe y:\ccbench.py == CPython 3.2a0.0 (py3k) == == x86 Windows on 'x86 Family 6 Model 23 Stepping 10, GenuineIntel' == ---

[issue7946] Convoy effect with I/O bound threads and New GIL

Changes by David Beazley d...@dabeaz.com: Removed file: http://bugs.python.org/file17094/dabeaz_gil.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___

[issue7946] Convoy effect with I/O bound threads and New GIL

David Beazley d...@dabeaz.com added the comment: New version of patch that will probably fix Windows-XP problems. Was doing something stupid in the monitor (not sure how it worked on Unix). -- Added file: http://bugs.python.org/file17102/dabeaz_gil.patch

[issue7946] Convoy effect with I/O bound threads and New GIL

Changes by Ray.Allen ysj@gmail.com: -- nosy: +ysj.ray ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___ ___ Python-bugs-list mailing list

[issue7946] Convoy effect with I/O bound threads and New GIL

David Beazley d...@dabeaz.com added the comment: The attached patch makes two simple refinements to the new GIL implemented in Python 3.2. Each is briefly described below. 1. Changed mechanism for thread time expiration In the current implementation, threads perform a timed-wait on a

[issue7946] Convoy effect with I/O bound threads and New GIL

David Beazley d...@dabeaz.com added the comment: One comment on that patch I just submitted. Basically, it's an attempt to make an extremely simple tweak to the GIL that fixes most of the problems discussed here in an extremely simple manner. I don't have any special religious attachment to

[issue7946] Convoy effect with I/O bound threads and New GIL

David Beazley d...@dabeaz.com added the comment: Here is the result of running the writes.py test with the patch I submitted. This is on OS-X. bash-3.2$ ./python.exe writes.py t1 2.83990693092 0 t2 3.27937912941 0 t1 5.54346394539 1 t2 6.68237304688 1 t1 8.9648039341 2 t2 9.60041999817 2 t1

[issue7946] Convoy effect with I/O bound threads and New GIL

Gregory P. Smith g...@krypto.org added the comment: Nice dabeaz. One potential concern with dabeaz_gil.patch 2010-04-25 21:13 is that it appears to always leave the gil_monitor thread running. This is bad on mobile/embedded platforms where waking up at regular intervals prevents advanced

[issue7946] Convoy effect with I/O bound threads and New GIL

Changes by Andres Moreira elkpich...@gmail.com: -- nosy: +andrix ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___ ___ Python-bugs-list

[issue7946] Convoy effect with I/O bound threads and New GIL

Nir Aides n...@winpdb.org added the comment: the scheduling function bfs_find_task returns the first task that has an expired deadline. since an expired deadline probably means that the scheduler hasn't run for a while, it might be worth it to look for the thread with the oldest deadline

[issue7946] Convoy effect with I/O bound threads and New GIL

Changes by Nir Aides n...@winpdb.org: Removed file: http://bugs.python.org/file16947/bfs.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___ ___

[issue7946] Convoy effect with I/O bound threads and New GIL

Nir Aides n...@winpdb.org added the comment: Yet another update to bfs.patch. I upload a variation on Florent's write test which prints progress of background CPU bound threads as: thread-name timestamp progress Here are some numbers from Windows XP 32bit with Intel q9400 (4 cores). Builds

[issue7946] Convoy effect with I/O bound threads and New GIL

Changes by Nir Aides n...@winpdb.org: Removed file: http://bugs.python.org/file16830/bfs.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___ ___

[issue7946] Convoy effect with I/O bound threads and New GIL

Nir Aides n...@winpdb.org added the comment: I uploaded an update to bfs.patch which improves behavior in particular on non-Linux multi-core (4+) machines. Hi Charles-Francois, Thanks for taking the time to review this patch! - nothing guarantees that you'll get a msec resolution Right,

[issue7946] Convoy effect with I/O bound threads and New GIL

Charles-Francois Natali neolo...@free.fr added the comment: Please disregard my remark on COND_TIMED_WAIT not updating timeout_result, it's wrong (it's really a macro, not a function...) -- ___ Python tracker rep...@bugs.python.org

[issue7946] Convoy effect with I/O bound threads and New GIL

Charles-Francois Natali neolo...@free.fr added the comment: Some more remarks: - COND_TIMED_WAIT macro modifies timeout_result when pthread_cond_timewait expires. But timeout_result is not an int pointer, just an int. So it is never updated, and as a result, bfs_check_depleted is never set

[issue7946] Convoy effect with I/O bound threads and New GIL

Charles-Francois Natali neolo...@free.fr added the comment: A couple remarks on BFS-based patch: - nothing guarantees that you'll get a msec resolution - gettimeofday returns you wall clock time: if a process that modifies time is running, e.g. ntpd, you'll likely to run into trouble. the value

[issue7946] Convoy effect with I/O bound threads and New GIL

Changes by Nir Aides n...@winpdb.org: Removed file: http://bugs.python.org/file16710/bfs.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___ ___

[issue7946] Convoy effect with I/O bound threads and New GIL

Nir Aides n...@winpdb.org added the comment: Uploaded an update. -- Added file: http://bugs.python.org/file16830/bfs.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___

[issue7946] Convoy effect with I/O bound threads and New GIL

Changes by Thouis (Ray) Jones tho...@gmail.com: -- nosy: +thouis ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___ ___ Python-bugs-list

[issue7946] Convoy effect with I/O bound threads and New GIL

Changes by Nir Aides n...@winpdb.org: Removed file: http://bugs.python.org/file16680/bfs.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___ ___

[issue7946] Convoy effect with I/O bound threads and New GIL

Nir Aides n...@winpdb.org added the comment: I upload a new update to bfs.patch which improves scheduling and reduces overhead. -- Added file: http://bugs.python.org/file16710/bfs.patch ___ Python tracker rep...@bugs.python.org

[issue7946] Convoy effect with I/O bound threads and New GIL

Changes by Ram Rachum cool...@cool-rr.com: -- nosy: +cool-RR ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7946 ___ ___ Python-bugs-list mailing

  1   2   >