Paul Rubin http://phr...@nospam.invalid wrote:
Hendrik van Rooyen m...@microcorp.co.za writes:
I think that this is because (like your link has shown) the problem
is really not trivial, and also because the model that can bring
sanity to the party (independent threads/processes that
Look, guys, here's the thing:
In the company I work at we decided to rewrite our MRP system in
Python. I was one of the main proponents of it since it's nicely cross
platform and allows for quite rapid application development. The
language and it's built in functions are simply great. The
On 21 juin, 03:27, Jure Erznožnik jure.erznoz...@gmail.com wrote:
Add:
Carl, Olivier co. - You guys know exactly what I wanted.
Others: Going back to C++ isn't what I had in mind when I started
initial testing for my project.
Do you think multiprocessing can help you seriously ?
Can you
On Jun 21, 9:32 am, OdarR olivier.da...@gmail.com wrote:
Do you think multiprocessing can help you seriously ?
Can you benefit from multiple cpu ?
did you try to enhance your code with numpy ?
Olivier
(installed a backported multiprocessing on his 2.5.1 Python, but need
installation of
Kay Schluehr k...@fiber-space.de wrote:
This implies that people stay defensive concerning concurrency ( like
me right now ) and do not embrace it like e.g. Erlang does. Sometimes
there is a radical change in the way we design applications and a
language is the appropriate medium to express
Christian Heimes wrote:
Hard computations gain more speed from carefully crafted C or Fortran
code that utilizes features like the L1 and L2 CPU cache, SIMD etc. or
parallelized algorithms. If you start sharing values between multiple
cores you have a serious problem.
Oh, and use NumPy for
Jure Erznožnik wrote:
On Jun 20, 1:36 am, a...@pythoncraft.com (Aahz) wrote:
You should put up or shut up -- I've certainly seen multi-core speedup
with threaded software, so show us your benchmarks!
--
Sorry, no intent to offend anyone here. Flame wars are not my thing.
I have shown my
Jesse Noller wrote:
Sorry, you're incorrect. I/O Bound threads do in fact, take
advantage
of multiple cores.
I don't know whether anyone else brought this up, but it looks
like Python has problems with even this form of threading
http://www.dabeaz.com/python/GIL.pdf
It's certainly a very
In article 90303b55-8686-4d56-b89c-01e31d0a6...@l8g2000vbp.googlegroups.com,
=?windows-1252?Q?Jure_Erzno=9Enik?= jure.erznoz...@gmail.com wrote:
So, recently I started writing a part of this new system in Python. A
report generator to be exact. Let's not go into existing offerings,
they are
In article h1l6m3$3f...@gemini.csx.cam.ac.uk,
Jeremy Sanders jeremy+complangpyt...@jeremysanders.net wrote:
Jesse Noller wrote:
Sorry, you're incorrect. I/O Bound threads do in fact, take advantage
of multiple cores.
I don't know whether anyone else brought this up, but it looks
like Python
Hendrik van Rooyen m...@microcorp.co.za writes:
I think that this is because (like your link has shown) the problem
is really not trivial, and also because the model that can bring
sanity to the party (independent threads/processes that communicate
with queued messages) is seen as inefficient
Jure Erznožnik jure.erznoz...@gmail.com (JE) wrote:
JE Sorry, just a few more thoughts:
JE Does anybody know why GIL can't be made more atomic? I mean, use
JE different locks for different parts of code?
JE This way there would be way less blocking and the plugin interface
JE could remain the
Jure Erznožnik jure.erznoz...@gmail.com (JE) wrote:
JE I have shown my benchmarks. See first post and click on the link.
JE That's the reason I started this discussion.
JE All I'm saying is that you can get threading benefit, but only if the
JE threading in question is implemented in C plugin.
Ross Ridge rri...@csclub.uwaterloo.ca (RR) wrote:
RR By definition an I/O bound thread isn't CPU bound so won't benefit from
RR improved CPU resources.
But doing I/O is not the same as being I/O bound. And Python allows
multithreading when a thread does I/O even if that thread is not I/O
bound
On Jun 19, 4:42 pm, Christian Heimes li...@cheimes.de wrote:
OdarR schrieb:
On 19 juin, 21:41, Carl Banks pavlovevide...@gmail.com wrote:
He's saying that if your code involves extensions written in C that
release the GIL, the C thread can run on a different core than the
Python-thread
On Jun 19, 4:35 pm, a...@pythoncraft.com (Aahz) wrote:
In article 157e0345-74e0-4144-a2e6-2b4cc854c...@z7g2000vbh.googlegroups.com,
Carl Banks pavlovevide...@gmail.com wrote:
I wish Pythonistas would be more willing to acknowledge the (few)
drawbacks of the language (or implementation, in
You might want to read about The Problem with Threads:
http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.pdf
and then decide to switch to an appropriate concurrency model for your use
case.
and to a programming language that supports it.
--
On 20 juin, 11:02, Carl Banks pavlovevide...@gmail.com wrote:
Here's the thing: not everyone complaining about the GIL is trying to
get the raw power of their machines. They just want to take
advantage of multiple cores so that their Python program runs
faster.
It would be rude and
Carl Here's the thing: not everyone complaining about the GIL is trying
Carl to get the raw power of their machines. They just want to take
Carl advantage of multiple cores so that their Python program runs
Carl faster.
If their code is CPU-bound it's likely that rewriting
Kay Schluehr wrote:
You might want to read about The Problem with Threads:
http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.pdf
and then decide to switch to an appropriate concurrency model for your use
case.
and to a programming language that supports it.
Maybe, yes. But many
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Jean-Paul Calderone wrote:
On Sat, 20 Jun 2009 00:07:27 GMT, Lie Ryan lie.1...@gmail.com wrote:
[snip]
Perhaps we should have more built-in/stdlib operations that can release
GIL safely to release GIL by default? And perhaps some builtin/stdlib
On Jun 20, 6:36 am, s...@pobox.com wrote:
Carl Here's the thing: not everyone complaining about the GIL is trying
Carl to get the raw power of their machines. They just want to take
Carl advantage of multiple cores so that their Python program runs
Carl faster.
If their code
Add:
Carl, Olivier co. - You guys know exactly what I wanted.
Others: Going back to C++ isn't what I had in mind when I started
initial testing for my project.
--
http://mail.python.org/mailman/listinfo/python-list
Carl I'm sure you think you're trying to be helpful, but you're coming
Carl off as really presumptuous with this casual dismissal of their
Carl concerns.
My apologies, but in most cases there is more than one way to skin a cat.
Trust me, if removing the global interpreter lock was
On 20 Jun., 17:28, Stefan Behnel stefan...@behnel.de wrote:
Kay Schluehr wrote:
You might want to read about The Problem with Threads:
http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.pdf
and then decide to switch to an appropriate concurrency model for your use
case.
and
On Jun 20, 8:18 pm, s...@pobox.com wrote:
Carl Maybe you don't intend to sound like you're saying shut up and
Carl use C, but to me, that's how you come off. If you're going to
Carl advise someone to use C, at least try to show some understanding
Carl for their concerns--it
This is a very long-running issue, that has been discussed many times. Here
are the two sides to keeping the gil or removing it:
Remove the GIL:
- True multi-threaded programming
- Scalable performance across a multi-core machine
- Unfortunately, this causes a slow-down in single
Jure Erznožnik wrote:
See here for introduction:
http://groups.google.si/group/comp.lang.python/browse_thread/thread/370f8a1747f0fb91
Digging through my problem, I discovered Python isn't exactly thread
safe and to solve the issue, there's this Global Interpreter Lock
(GIL) in place.
Digging through my problem, I discovered Python isn't exactly thread
safe and to solve the issue, there's this Global Interpreter Lock
(GIL) in place.
It's the opposite: Python is exactly thread safe precisely because it
has the GIL in place.
Is there any other way to work around the issue
Digging through my problem, I discovered Python isn't exactly thread
safe and to solve the issue, there's this Global Interpreter Lock
(GIL) in place.
It's the opposite: Python is exactly thread safe precisely because it
has the GIL in place.
Is there any other way to work around the issue
On 19 juin, 11:52, Jure Erznožnik jure.erznoz...@gmail.com wrote:
See here for
introduction:http://groups.google.si/group/comp.lang.python/browse_thread/thread/3...
Digging through my problem, I discovered Python isn't exactly thread
safe and to solve the issue, there's this Global
Jure Erznožnik wrote:
See here for introduction:
http://groups.google.si/group/comp.lang.python/browse_thread/thread/370f8a1747f0fb91
Digging through my problem, I discovered Python isn't exactly thread
safe and to solve the issue, there's this Global Interpreter Lock
(GIL) in place.
Jure Erznožnik wrote:
See here for introduction:
http://groups.google.si/group/comp.lang.python/browse_thread/thread/370f8a1747f0fb91
Digging through my problem, I discovered Python isn't exactly thread
safe and to solve the issue, there's this Global Interpreter Lock
(GIL) in place.
On 19 juin, 16:16, Martin von Loewis martin.vonloe...@hpi.uni-:
If you know that your (C) code is thread safe on its own, you can
release the GIL around long-running algorithms, thus using as many
CPUs as you have available, in a single process.
what do you mean ?
Cpython can't benefit from
CPython itself can't... but the c extension can. Mine did.
On Fri, Jun 19, 2009 at 9:50 AM, OdarR olivier.da...@gmail.com wrote:
On 19 juin, 16:16, Martin von Loewis martin.vonloe...@hpi.uni-:
If you know that your (C) code is thread safe on its own, you can
release the GIL around
If you know that your (C) code is thread safe on its own, you can
release the GIL around long-running algorithms, thus using as many
CPUs as you have available, in a single process.
Olivier what do you mean ?
Olivier Cpython can't benefit from multi-core without multiple
On 19 juin, 19:13, s...@pobox.com wrote:
Olivier what do you mean ?
Olivier Cpython can't benefit from multi-core without multiple
Olivier processes.
It can, precisely as Martin indicated. Only one thread at a time can hold
the GIL. That doesn't mean that multiple threads
OdarR wrote:
I don't see such improvement in the Python library, or maybe you can
indicate us some meaningfull example...?
I currently only use CPython, with PIL, Reportlab...etc.
I don't see improvement on a Core2duo CPU and Python. How to proceed
(following what you wrote) ?
I've seen a
On Jun 19, 6:53 am, Ben Charrow bchar...@csail.mit.edu wrote:
Jure Erznožnik wrote:
See here for introduction:
http://groups.google.si/group/comp.lang.python/browse_thread/thread/3...
Digging through my problem, I discovered Python isn't exactly thread
safe and to solve the issue, there's
On Jun 19, 11:08 am, OdarR olivier.da...@gmail.com wrote:
On 19 juin, 19:13, s...@pobox.com wrote:
Olivier what do you mean ?
Olivier Cpython can't benefit from multi-core without multiple
Olivier processes.
It can, precisely as Martin indicated. Only one thread at a time
Thanks guys, for all the replies.
They were some very interesting reading / watching.
Seems to me, the Unladen-Swallow might in time produce code which will
have this problem lessened a bit. Their roadmap suggests at least
modifying the GIL principles if not fully removing it. On top of this,
On 19 juin, 21:05, Christian Heimes li...@cheimes.de wrote:
I've seen a single Python process using the full capacity of up to 8
CPUs. The application is making heavy use of lxml for large XSL
transformations, a database adapter and my own image processing library
based upon FreeImage.
Sorry, just a few more thoughts:
Does anybody know why GIL can't be made more atomic? I mean, use
different locks for different parts of code?
This way there would be way less blocking and the plugin interface
could remain the same (the interpreter would know what lock it used
for the plugin, so
On 19 juin, 21:41, Carl Banks pavlovevide...@gmail.com wrote:
He's saying that if your code involves extensions written in C that
release the GIL, the C thread can run on a different core than the
Python-thread at the same time. The GIL is only required for Python
code, and C code that uses
On Fri, Jun 19, 2009 at 12:50 PM, OdarRolivier.da...@gmail.com wrote:
On 19 juin, 16:16, Martin von Loewis martin.vonloe...@hpi.uni-:
If you know that your (C) code is thread safe on its own, you can
release the GIL around long-running algorithms, thus using as many
CPUs as you have available,
On Jun 19, 11:45 pm, OdarR olivier.da...@gmail.com wrote:
On 19 juin, 21:05, Christian Heimes li...@cheimes.de wrote:
I've seen a single Python process using the full capacity of up to 8
CPUs. The application is making heavy use of lxml for large XSL
transformations, a database adapter and
On Jun 19, 11:59 pm, Jesse Noller jnol...@gmail.com wrote:
On Fri, Jun 19, 2009 at 12:50 PM, OdarRolivier.da...@gmail.com wrote:
On 19 juin, 16:16, Martin von Loewis martin.vonloe...@hpi.uni-:
If you know that your (C) code is thread safe on its own, you can
release the GIL around
On 19 Jun, 21:41, Carl Banks pavlovevide...@gmail.com wrote:
(Note: I'm not talking about releasing the GIL for I/O operations,
it's not the same thing. I'm talking about the ability to run
computations on multiple cores at the same time, not to block in 50
threads at the same time.
On Fri, Jun 19, 2009 at 6:10 PM, Jure Erznožnikjure.erznoz...@gmail.com wrote:
On Jun 19, 11:59 pm, Jesse Noller jnol...@gmail.com wrote:
On Fri, Jun 19, 2009 at 12:50 PM, OdarRolivier.da...@gmail.com wrote:
On 19 juin, 16:16, Martin von Loewis martin.vonloe...@hpi.uni-:
If you know that
In article d6d695d8-2af6-4713-bcbf-587e19e16...@n4g2000vba.googlegroups.com,
=?windows-1252?Q?Jure_Erzno=9Enik?= jure.erznoz...@gmail.com wrote:
I do aggree though that threading is important. Regardless of any
studies showing that threads suck, they are here and they offer
relatively simple
In article 157e0345-74e0-4144-a2e6-2b4cc854c...@z7g2000vbh.googlegroups.com,
Carl Banks pavlovevide...@gmail.com wrote:
I wish Pythonistas would be more willing to acknowledge the (few)
drawbacks of the language (or implementation, in this case) instead of
all this rationalization.
Please
In article f9ff1a89-813b-4367-9e95-570ab2bac...@h23g2000vbc.googlegroups.com,
=?windows-1252?Q?Jure_Erzno=9Enik?= jure.erznoz...@gmail.com wrote:
On Jun 19, 11:59=A0pm, Jesse Noller jnol...@gmail.com wrote:
Sorry, you're incorrect. I/O Bound threads do in fact, take advantage
of multiple
OdarR schrieb:
On 19 juin, 21:41, Carl Banks pavlovevide...@gmail.com wrote:
He's saying that if your code involves extensions written in C that
release the GIL, the C thread can run on a different core than the
Python-thread at the same time. The GIL is only required for Python
code, and C
On Jun 20, 1:36 am, a...@pythoncraft.com (Aahz) wrote:
You should put up or shut up -- I've certainly seen multi-core speedup
with threaded software, so show us your benchmarks!
--
Sorry, no intent to offend anyone here. Flame wars are not my thing.
I have shown my benchmarks. See first post
Aahz wrote:
In article 157e0345-74e0-4144-a2e6-2b4cc854c...@z7g2000vbh.googlegroups.com,
Carl Banks pavlovevide...@gmail.com wrote:
I wish Pythonistas would be more willing to acknowledge the (few)
drawbacks of the language (or implementation, in this case) instead of
all this
Jesse Noller jnol...@gmail.com wrote:
Sorry, you're incorrect. I/O Bound threads do in fact, take advantage
of multiple cores.
jure.erznoz...@gmail.com wrote:
Incorrect. They take advantage of OS threading support where another
thread can run while one is blocked for I/O. That is not equal to
Jure Erznožnik wrote:
On Jun 20, 1:36 am, a...@pythoncraft.com (Aahz) wrote:
You should put up or shut up -- I've certainly seen multi-core speedup
with threaded software, so show us your benchmarks!
--
Sorry, no intent to offend anyone here. Flame wars are not my thing.
I have shown my
On Sat, 20 Jun 2009 00:07:27 GMT, Lie Ryan lie.1...@gmail.com wrote:
[snip]
Perhaps we should have more built-in/stdlib operations that can release
GIL safely to release GIL by default? And perhaps some builtin/stdlib
should receive an optional argument that instruct them to release GIL
and by
58 matches
Mail list logo
