>>> I have found the stop-and-go between two processes on the same machine >>> leads to very poor throughput. By stop-and-go, I mean the producer and >>> consumer are constantly getting on and off of the CPU since the pipe >>> gets full (or empty for consumer). Note that a producer can't run at >>> its top speed as the scheduler will pull it out since it's output pipe >>> got filled up. > > On a single core CPU when only one process can be running, the > producer must get off the CPU so that the consumer may start the > draining process.
It's still not clear why you say that the producer can run "at its top speed". You seem to be suggesting that in such a setup, the CPU would be idle, i.e. not 100% loaded. Assuming that the consumer won't block for something else, then both processes will run at their "top speed". Of course, for two processes running at a single CPU, the top speed won't be the MIPs of a single processor, as they have to share the CPU. So when you say it leads to very poor throughput, I ask: compared to what alternative? Regards, Martin -- http://mail.python.org/mailman/listinfo/python-list
