Julian wrote: > > One of the many things I would like to do for version 2 is to find > a way > to parallelize more, so that quad core and such can be used. I'm still > not exactly sure how is the best way to do that. > > > I might be wrong and it is probably not that simple (and I can't > really check right now), but I think I remember seeing something in > the Win32 API to set the processor affinity of a thread, after > checking for >2 processors. Not sure what the linux equivalent is. > Processor affinity is a tweak. A micro-optimization, if you like. It is a technique that is meant for reducing the overhead just a little more, to squeeze a bit more performance out of a program. That has nothing to do with the problem at hand.
Our problem is this. We have X mips at our disposal, but instead of having all X at our disposal, we have X/n mips at our disposal which we can parallelize up to a parallel level of n. The question is, how do we create an algorithm that can take advantage of this. The current algorithm for rsyncrypto is this. One thread of execution encrypts. Another thread compresses. That's it. If n=2, you get full utilization on one CPU, and some utilization on the other one (because one of the operations runs faster than the other one it is almost guaranteed that one of the CPUs will not reach maximal capacity). If n is greater than 2, we have one CPU encrypting full steam, another one that uses some of its time for compressing, and the rest of the CPUs are not utilized by rsyncrypto. This is wasted processing power. You can anchor which CPU does what all you want (that's what affinity does), but it won't change the fact that rsyncrypto does not know how to use more than two CPUs at the moment. Shachar ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ Rsyncrypto-devel mailing list Rsyncrypto-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/rsyncrypto-devel
