Julian wrote:

>
>     One of the many things I would like to do for version 2 is to find
>     a way
>     to parallelize more, so that quad core and such can be used. I'm still
>     not exactly sure how is the best way to do that.
>
>
> I might be wrong and it is probably not that simple (and I can't 
> really check right now), but I think I remember seeing something in 
> the Win32 API to set the processor affinity of a thread, after 
> checking for >2 processors. Not sure what the linux equivalent is.
>
Processor affinity is a tweak. A micro-optimization, if you like. It is 
a technique that is meant for reducing the overhead just a little more, 
to squeeze a bit more performance out of a program. That has nothing to 
do with the problem at hand.

Our problem is this. We have X mips at our disposal, but instead of 
having all X at our disposal, we have X/n mips at our disposal which we 
can parallelize up to a parallel level of n. The question is, how do we 
create an algorithm that can take advantage of this.

The current algorithm for rsyncrypto is this. One thread of execution 
encrypts. Another thread compresses. That's it. If n=2, you get full 
utilization on one CPU, and some utilization on the other one (because 
one of the operations runs faster than the other one it is almost 
guaranteed that one of the CPUs will not reach maximal capacity). If n 
is greater than 2, we have one CPU encrypting full steam, another one 
that uses some of its time for compressing, and the rest of the CPUs are 
not utilized by rsyncrypto. This is wasted processing power. You can 
anchor which CPU does what all you want (that's what affinity does), but 
it won't change the fact that rsyncrypto does not know how to use more 
than two CPUs at the moment.

Shachar

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Rsyncrypto-devel mailing list
Rsyncrypto-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsyncrypto-devel

Reply via email to