Kent Johnson wrote:
On Mon, Apr 27, 2009 at 2:46 AM, A.T.Hofkamp <[email protected]> wrote:
you are making
a IO-bound app, so threading doesn't buy you anything in performance and
gives a big headache in data protection
Please explain. Presumably the single-threaded app is not IO-bound.
Adding threads can push it to the point of being IO-bound. The
Euh, what?
Suppose I have a CPU-intensive task. Now by adding threads (ie more
CPU-intensive tasks running at the same time) I can make it an IO-bound
application?
Wow.
Let's call s...@home and climateprediction.net .
As far as I know, doing DB queries is an IO-bound process, since you must make
a connection to the remote DB server, and send data back and forth to the
server over the network (which is ssssllllooooowwww compared to a calculation).
With threading, you will have more network connections that you are waiting
for. W.r.t. performance you don't gain anything, since the CPU is already
mostly idle.
> requirements for communicating between threads in this app are modest
> and not likely to cause a big headache.
With respect to the data problem, I don't know what the OP wants, but maybe he
wants to run different tests at different times (ie first 1 minute test A,
then 1 minute test B etc) or different mixes of queries. He may also want to
collect performance results (avg x queries/second).
Such data is shared between the threads, so it must be protected to prevent it
from getting corrupted.
Maybe it is modest in requirements, but you must do it none the less.
Also note that the protection itself may also become a bottle neck in the
program (you must constantly claim and release locks which adds overhead even
when there is no other thread waiting).
With Twisted, you can have at least the same number of IO connections as with
threading, while you don't get the data protection problems and overhead for
the simple reason that it is still a single-threaded program.
Maybe threading is the correct solution here. However the OP also asked for
"...or is there another way to go about it?".
Twisted is definitely another way to go about it, and nobody mentioned it in
previous posts, so I thought it correct to mention its existence here.
Sincerely,
Albert
_______________________________________________
Tutor maillist - [email protected]
http://mail.python.org/mailman/listinfo/tutor
