> However, ssh encryption is pretty expensive. Rough measurements show > that transferring a file across ssh takes CPU time comparable to > compiling it, which would make it fairly impractical for distcc.
You must be using 3des... [aaronl@vitelus:~/openssl-0.9.6g]$ cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 8 model name : Celeron (Coppermine) stepping : 10 cpu MHz : 1097.099 cache size : 128 KB fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 2 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 mmx fxsr sse bogomips : 2188.90 [aaronl@vitelus:~/openssl-0.9.6g]$ ./apps/openssl speed des-ede3 Doing des ede3 for 3s on 8 size blocks: 3257959 des ede3's in 3.00s Doing des ede3 for 3s on 64 size blocks: 424293 des ede3's in 3.00s Doing des ede3 for 3s on 256 size blocks: 106682 des ede3's in 3.00s Doing des ede3 for 3s on 1024 size blocks: 26712 des ede3's in 3.00s Doing des ede3 for 3s on 8192 size blocks: 3338 des ede3's in 3.00s OpenSSL 0.9.6g 9 Aug 2002 built on: Fri Sep 6 15:24:07 PDT 2002 options:bn(64,32) md2(int) rc4(idx,int) des(ptr,risc1,16,long) idea(int) blowfish(idx) compiler: gcc -DTHREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DL_ENDIAN -DTERMIO -O3 -fomit-frame-pointer -m486 -Wall -DSHA1_ASM -DMD5_ASM -DRMD160_ASM The 'numbers' are in 1000s of bytes per second processed. type 8 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes des ede3 8687.89k 9051.58k 9103.53k 9117.70k 9114.97k [aaronl@vitelus:~/openssl-0.9.6g]$ ./apps/openssl speed bf-cbc Doing blowfish cbc for 3s on 8 size blocks: 12446868 blowfish cbc's in 3.00s Doing blowfish cbc for 3s on 64 size blocks: 1787532 blowfish cbc's in 3.00s Doing blowfish cbc for 3s on 256 size blocks: 459200 blowfish cbc's in 3.00s Doing blowfish cbc for 3s on 1024 size blocks: 115553 blowfish cbc's in 3.00s Doing blowfish cbc for 3s on 8192 size blocks: 14450 blowfish cbc's in 3.00s OpenSSL 0.9.6g 9 Aug 2002 built on: Fri Sep 6 15:24:07 PDT 2002 options:bn(64,32) md2(int) rc4(idx,int) des(ptr,risc1,16,long) idea(int) blowfish(idx) compiler: gcc -DTHREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DL_ENDIAN -DTERMIO -O3 -fomit-frame-pointer -m486 -Wall -DSHA1_ASM -DMD5_ASM -DRMD160_ASM The 'numbers' are in 1000s of bytes per second processed. type 8 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes blowfish cbc 33191.65k 38134.02k 39185.07k 39442.09k 39458.13k These results are directly applicable to SSH since OpenSSH uses OpenSSL's cryptographic primitives. 40MB/s is way faster than I can send to my network, not to mention quite a bit faster than GCC. I don't deny that there may be further overhead in SSH protocol, including hashing (although OpenSSL claims a rate of 85996.82k/s for SHA-1 on my modest machine). I don't think that very much inefficiency beyond encryption and hashing is mandated by the protocols, so ssh's speed problems are most likely bugs in OpenSSH that could be addressed with patches much more easily and charitably then writing an alternative secure authentication system just for distcc. I was pondering recommending a compromise where distcc would use ssh to transmit commands, but actual transfer of data would occur over unsecured sockets. The benefit is that this system removes the "any command can be run through a simple network connection" issue. I've come to think that this is a bad idea, because if you're on an untrusted network you can't trust unsecured data transfer anyway. Source code and the resulting binary aren't usually very private, but SSH also protects the integrity of data that travels over it. Without some cryptographic integrity verification, a malicious user could modify either the code on its way to compilation or the object code on the way back to the client. This would be a great way to break into any computer which runs the binary that distcc orchestrated the compilation of.
msg00161/pgp00000.pgp
Description: PGP signature
