On Mon, 25 Feb 2013, Jay Dobies wrote:
Touch base with Randy. He tweaked a bunch of those numbers before v2 released and should be able to point you to the best places to start playing with.
I had done some lab testing with the --num-threads parameter in December. I had learned that our Grinder code was using threads for CPU intensive work during downloads. Due to the Python GIL, this was actually causing the threads to thrash each other, which significantly lowered performance for synchronization. For my test, I used traffic control to limit my bandwidth to 20 Mbps, 10 Mpbs, and 1 Mbps between myself and a LAN reachable CentOS repository, and in all three cases I found that having one thread resulted in the best performance. Due to this finding, I set the default number of threads to 1. The --num-threads flag can be used to override the default. One thing I did not simulate in my testing was network latency. If there were high network latency, I would guess that adding more threads might eventually lead to better performance, as more of them would be in a waiting state instead of thrashing each other. I didn't much time to simulate this scenario, so if you find that adding threads improves performance, it might be due to latency. I'd like to know if that does help, as it would warrant another test. Thanks! -- Randy Barlow _______________________________________________ Pulp-list mailing list [email protected] https://www.redhat.com/mailman/listinfo/pulp-list
