Nir Aides n...@winpdb.org added the comment:
gilinter.patch has good IO latency in UDP test on my system when built with
--with-computed-gotos:
In [34]: %timeit -n3 client.work()
0.320 seconds (32782026.509 bytes/sec)
0.343 seconds (30561727.443 bytes/sec)
0.496 seconds (21154075.417
Antoine Pitrou pit...@free.fr added the comment:
Hmm, the gilinter patch shouldn't be sensitive to whether computed gotos are
enabled or not. Here is an updated patch, though, the previous one didn't apply
cleanly anymore. I've also made the priority condition a bit stricter.
--
Added
Changes by Antoine Pitrou pit...@free.fr:
Removed file: http://bugs.python.org/file16238/issuegil.txt
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7946
___
Changes by Antoine Pitrou pit...@free.fr:
Removed file: http://bugs.python.org/file16317/gilprio.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7946
___
Changes by Antoine Pitrou pit...@free.fr:
Removed file: http://bugs.python.org/file16329/gilinter.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7946
___
Changes by Nir Aides n...@winpdb.org:
Removed file: http://bugs.python.org/file16663/bfs.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7946
___
___
Nir Aides n...@winpdb.org added the comment:
I update bfs.patch. It now builds on Windows (and Posix).
--
Added file: http://bugs.python.org/file16679/bfs.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7946
Changes by Nir Aides n...@winpdb.org:
Removed file: http://bugs.python.org/file16679/bfs.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7946
___
___
Changes by Nir Aides n...@winpdb.org:
Added file: http://bugs.python.org/file16680/bfs.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7946
___
___
Changes by Dirkjan Ochtman dirk...@ochtman.nl:
--
nosy: +djc
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7946
___
___
Python-bugs-list mailing
Changes by Konrad Delong kon...@gmail.com:
--
nosy: +konryd
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7946
___
___
Python-bugs-list mailing
Changes by Alexandre Conrad alexandre.con...@gmail.com:
--
nosy: +aconrad
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7946
___
___
Changes by Guilherme Salgado gsalg...@gmail.com:
--
nosy: +salgado
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7946
___
___
Python-bugs-list
Antoine Pitrou pit...@free.fr added the comment:
I also switched to gettimeofday() so it should work now on all Posix
with high resolution timer
But on a busy system, won't measuring wall clock time rather than CPU time give
bogus results?
--
Nir Aides n...@winpdb.org added the comment:
But on a busy system, won't measuring wall clock time rather than CPU time
give bogus results?
This was the motivation for using clock_gettime(). I tried the wall clock
version under load (including on single core system) and it seems to behave.
Changes by Larry Hastings la...@hastings.org:
--
nosy: +larry
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7946
___
___
Python-bugs-list mailing
Changes by Augie Fackler li...@durin42.com:
--
nosy: +durin42
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7946
___
___
Python-bugs-list mailing
Nir Aides n...@winpdb.org added the comment:
I upload an updated bfs.patch. Apply to updated py32 and ignore the error with:
$ patch -fp1 bfs.patch
$ ./configure
Please give understandable benchmark numbers, including an explicit
comparison with baseline 3.2, and patched 3.2 (e.g.
Changes by Nir Aides n...@winpdb.org:
Removed file: http://bugs.python.org/file16634/bfs.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7946
___
___
Changes by Nir Aides n...@winpdb.org:
Removed file: http://bugs.python.org/file16567/linux-7946.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7946
___
Antoine Pitrou pit...@free.fr added the comment:
I use clock_gettime() to get the thread running time to calculate slice
depletion.
Ouch. CLOCK_THREAD_CPUTIME_ID is not a required part of the standard. Only
CLOCK_REALTIME is guaranteed to exist.
By the way, it's not obvious cpued tests
Nir Aides n...@winpdb.org added the comment:
Ouch. CLOCK_THREAD_CPUTIME_ID is not a required part of the standard. Only
CLOCK_REALTIME is guaranteed to exist.
Right, however the man page at kernel.org says the following on
CLOCK_THREAD_CPUTIME_ID:
Sufficiently recent versions of glibc and
Antoine Pitrou pit...@free.fr added the comment:
Please specify system and test details so I can try to look into it.
It's a dual-core Linux x86-64 system. But, looking at the patch again,
the reason is obvious:
#define CHECK_SLICE_DEPLETION(tstate) (bfs_check_depleted ||
Nir Aides n...@winpdb.org added the comment:
It's a dual-core Linux x86-64 system. But, looking at the patch again, the
reason is obvious:
#define CHECK_SLICE_DEPLETION(tstate) (bfs_check_depleted || (tstate
tick_counter % 1000 == 0))
`tstate-tick_counter % 1000` is replicating the
Antoine Pitrou pit...@free.fr added the comment:
What distribution and version of GNU/Linux are you using?
$ cat /etc/mandriva-release
Mandriva Linux release 2010.0 (Official) for x86_64
$ rpm -qv glibc
glibc-2.10.1-6.2mnb2
$ uname -a
Linux localhost 2.6.31.5-desktop-1mnb #1 SMP Fri Oct 23
Nir Aides n...@winpdb.org added the comment:
Well, on initial check the scheduler seems to work well with regular
gettimeofday() wall clock instead of clock_gettime().
:)
/* Return thread running time in seconds (with nsec precision). */
static inline long double get_thread_timestamp(void) {
Changes by Nir Aides n...@winpdb.org:
Removed file: http://bugs.python.org/file16644/bfs.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7946
___
___
Nir Aides n...@winpdb.org added the comment:
Uploaded an updated bfs.patch
The latency problem was related to the --with-computed-gotos flag. I fixed it
and it seems to work fine now.
I also switched to gettimeofday() so it should work now on all Posix with high
resolution timer.
Changes by John Levon movem...@users.sourceforge.net:
--
nosy: +movement
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7946
___
___
Changes by Jesús Cea Avión j...@jcea.es:
--
nosy: +jcea
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7946
___
___
Python-bugs-list mailing list
Nir Aides n...@winpdb.org added the comment:
I upload bfs.patch
To apply the patch use the following commands on updated python 3.2:
$ patch -fp1 bfs.patch
$ ./configure
The patch replaces the GIL with a scheduler. The scheduler is a simplified
implementation of the recent kernel Brain F**k
Antoine Pitrou pit...@free.fr added the comment:
I upload bfs.patch
Interesting patch, but:
- Please give understandable benchmark numbers, including an explicit
comparison with baseline 3.2, and patched 3.2 (e.g. gilinter.patch)
- Please also measure single-thread performance, because it
Nir Aides n...@winpdb.org added the comment:
I tried Florent's modification to the write test and did not see the effect on
my machine with an updated revision of Python32.
I am running Ubuntu Karmic 64 bit.
7s - no background threads.
20s - one background thread.
According to the following
Nir Aides n...@winpdb.org added the comment:
I updated the patch with a small fix and increased the ticks
countdown-to-release considerably. This seems to help the OS classify CPU bound
threads as such and actually improves IO performance.
--
Added file:
David Beazley d...@dabeaz.com added the comment:
Here's a short benchmark for everyone who thinks that my original benchmark was
somehow related to TCP behavior. This one doesn't even involve sockets:
from threading import Thread
import time
def writenums(f,n):
start = time.time()
Florent Xicluna florent.xicl...@gmail.com added the comment:
On some platforms the difference is not so important.
I ran it in Debian Lenny AMD64 Core2 Duo P9600 @2.66GHz.
# Python 3.2a0 (py3k:78982M, Mar 15 2010, 15:40:42)
# [GCC 4.3.4] on linux2
0.67s without thread
0.84s with spinning
Florent Xicluna florent.xicl...@gmail.com added the comment:
With line buffering, I see the issue.
* 6 s without thread
* 115 s with the spinning thread (varying: 60 s, 98 s)
* 16 s with the spinning thread and the last gilinter.patch
# Modified version of the test case, with bufsize=1
David Beazley d...@dabeaz.com added the comment:
Whoa, that's pretty diabolically evil with bufsize=1. On my machine, doing
that just absolutely kills the performance (13 seconds without the spinning
thread versus 557 seconds with the thread!). Or, put another way, the writing
performance
David Beazley d...@dabeaz.com added the comment:
Almost forgot--if I turn off one of the CPU cores, the time drops from 557
seconds to 32 seconds. Gotta love it!
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7946
Changes by Nir Aides n...@winpdb.org:
--
nosy: +nirai
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7946
___
___
Python-bugs-list mailing list
Antoine Pitrou pit...@free.fr added the comment:
One thing to think about with
this example is the proper priority of I/O handling generally. What
if, instead of a file, this example code was writing on a pipe to
another process? For that, you would probably want that I/O thread
to be
Antoine Pitrou pit...@free.fr added the comment:
Oh the situation definitely matters. Although, in the big picture,
most programmers would probably prefer to have fast I/O performance
over slow I/O performance :-).
Yes, of course. But that's not the point. We could try to improve GIL
David Beazley d...@dabeaz.com added the comment:
Oh the situation definitely matters. Although, in the big picture, most
programmers would probably prefer to have fast I/O performance over slow I/O
performance :-).
--
___
Python tracker
David Beazley d...@dabeaz.com added the comment:
I absolutely agree 100% that it is not worth trying to fix the GIL for every
conceivable situation (although if you could, I wouldn't complain).
To me, there are really only two scenarios worth worrying about:
1. Get rid of all of that
Antoine Pitrou pit...@free.fr added the comment:
And here is an experimental patch which enables the priority requests
mechanism which was in the original new GIL patch. Experimental because it
only enables them on a couple of socket methods (enough to affect the
benchmarks).
Here are the
David Beazley d...@dabeaz.com added the comment:
I posted some details about the priority GIL modifications I showed during my
PyCON open-space session here:
http://www.dabeaz.com/blog/2010/02/revisiting-thread-priorities-and-new.html
I am attaching the .tar.gz file with modifications if
Antoine Pitrou pit...@free.fr added the comment:
Here is another patch based on a slightly different approach. Instead of being
explicitly triggered in I/O methods, priority requests are decided based on the
computed interactiveness of a thread. Interactiveness itself is a simple
saturated
Antoine Pitrou pit...@free.fr added the comment:
We could try not to release the GIL when socket methods are called on a
non-blocking socket.
Regardless, I've re-run the tests under the Linux machine, with two spinning
threads:
* python 2.7:
25.580 seconds (409914.612 bytes/sec)
* python
New submission from David Beazley d...@dabeaz.com:
Background
---
In order to multitask with threads, a critical part of the Python
interpreter implementation concerns the behavior of I/O operations
such as read, write, send, and receive. Specifically, whenever an I/O
operation is
Antoine Pitrou pit...@free.fr added the comment:
Just a quick test under Linux (on a dual quad core machine):
- with iotest.py and echo_client.py both running Python 2.7: 25.562 seconds
(410212.450 bytes/sec)
- with iotest.py and echo_client.py both running Python 3.2: 28.160 seconds
David Beazley d...@dabeaz.com added the comment:
The comment on the CPU-bound workload is valid--it is definitely true that
Python 2.6 results will degrade as the workload of each tick is increased.
Maybe a better way to interpreter those results is as a baseline of what kind
of I/O
101 - 151 of 151 matches
Mail list logo
