On Thu, Jan 15, 2009 at 02:06:58AM +0000, RW wrote: > On Thu, 15 Jan 2009 00:20:54 +0100 > Roland Smith <rsm...@xs4all.nl> wrote: > > > On Wed, Jan 14, 2009 at 10:55:38PM +0000, RW wrote: > > > > Not just in reduced transfer rates, but also in terms of CPU cycles > > > used - a sustained geli to geli file copy makes things really slow > > > for me. > > > > That's probably because two geli kernel threads are competing for time > > on a single core. I've had problems with that as well (geli-encrypted > > USB drive stalling). > > > > Since I've switched to a multi-core machine (where the number of cores > > should be at least equal to the number of geli-encrypted devices), CPU > > load for gele has dropped to barely noticable. > > > > I find that puzzling; have you measured that on sustained geli to > geli transfers (with GB size files).
Not really. My previous single core machine kept stalling the drive. So instead of storing dumps for my biggest (/home) partition, I've turned to rsync-ing that, and using dumps for /, /usr and /var. I just did a simple test: copying a 5249634304 byte file from a non-encrypted disk to an encrypted one. This took 3:22 with the CPU load hovering at 10-13% at full speed. That's around 24.8 MiB/s. Copying the same file to another location on the unencrypted disk takes 2:57 with the CPU at around 5% at about 3/4 speed. That comes down to 28.3 MiB/s. Not a big difference IMHO. In normal usage the encrypted partition doesn't feel slower, because there are still at least two cores fully available for other jobs so interactivity doesn't suffer. A single-core machine would obviously feel different. The only thing that takes a while is compiling ports and big LaTeX jobs (compiling 300 pages five times with makeindex and bibtex runs in between). > The reason I'm a bit sceptical is that dd'ing /dev/random to /dev/null > runs at about 20MBytes/s on my single core (verses 700MBytes/s > for /dev/zero). File copies into geli run at about 15Mbytes/s, openssl > enc -aes-256-cbs runs at about the same ballpark figure. I get: dd if=/dev/random of=/dev/null bs=64k count=10000 10000+0 records in 10000+0 records out 655360000 bytes transferred in 10.693598 secs (61285266 bytes/sec) say 58 MiB/s dd if=/dev/zero of=/dev/null bs=64k count=10000 10000+0 records in 10000+0 records out 655360000 bytes transferred in 0.281247 secs (2330194568 bytes/sec) or around 2200 MiB/s And that is obviously running on _one_ core (CPU at approx. 25%). > Even if I had > multi-cores I would still be cpu-limited to 20MB/s, and that would fully > occupy two cores on geli to geli transfers. Your cores are probably > faster, but I'd expect a factor of two or so would be swallowed-up by > faster transfers. I don't see how cpu usage would be negligible unless > your individual cores are an order of magnitude faster than that. It turns out that on a multi-core machine a geli thread is started on each core for each disk (4 cores, two disks): ps -xa | grep g_eli 92 ?? DL 5:31.75 [g_eli[0] ad4s1g] 93 ?? DL 6:33.76 [g_eli[1] ad4s1g] 94 ?? DL 6:26.02 [g_eli[2] ad4s1g] 95 ?? DL 7:41.18 [g_eli[3] ad4s1g] 98 ?? DL 1:49.60 [g_eli[0] ad6s1g] 99 ?? DL 1:30.30 [g_eli[1] ad6s1g] 100 ?? DL 1:53.87 [g_eli[2] ad6s1g] 101 ?? DL 1:50.64 [g_eli[3] ad6s1g] This can be tuned with the sysctl 'kern.geom.eli.threads'. > Just out of curiosity what rate do you get on > dd if=/dev/random of=/dev/null bs=64k count=10000 See above. Roland -- R.F.Smith http://www.xs4all.nl/~rsmith/ [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated] pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725)
pgpA9YGVk7E0A.pgp
Description: PGP signature