Re: [Haifux] Whole disk encryption, because it costs nothing?
I think you guys always keep forgetting about the gremlins.
On Tue, Jan 12, 2010 at 1:25 PM, Eli Billauer e...@billauer.co.il wrote:
Etzion Bar-Noy wrote:
Sequential IO is very simple, relatively, so that you will hardly feel
the performance impact testing it.
Test random IO loads with small packets (0.5K-4K) and you will
probably feel the performance impact there.
It crossed my mind that a sequential test was too simplistic. My ideal
test would be a database server running like crazy on the data, but this
is not my application anyhow. I agree that I should test the system
under load but I can't really think about a realistic under load
scenario for 4 hyperthreaded CPUs. At least not something that I could
test without turning it into a project. So I went for the session below.
/usr is unencypted. /secret is encrypted. My home directory sits on a
740 GB ext4 partition, which has seen some activity. /secret and /fresh
are 20 GB, and are pretty untouched.
The whole system will be reinstalled pretty soon, so I don't care about
being root and fooling around. ;)
My finding is that encryption still speeds up writes. This is really
weird. I swear that /fresh was created a minute before I used it on this
test, so I suppose it can't be in a better shape. Still, writing to
/secret is faster. The only thing that was slower was removing a lot of
files in one go.
It's also interesting to note that writing to my home directory was
considerably slower. So even if encryption turns out to have some
impact, other factors (partition size? Fragmentation?) seem to be more
dominant.
So while I still agree with the concept that encryption must surely slow
down disk writes, my tests show me the contrary.
Insights, anyone?
Eli
-- Session follows ---
[r...@short eli]# tar -c /usr | { cd /secret/delete-this/ time tar -x ;
}
tar: Removing leading `/' from member names
tar: Removing leading `/' from hard link targets
real5m12.709s
user0m2.351s
sys0m31.486s
[r...@short eli]# time rm -rf /secret/delete-this
real0m13.319s
user0m0.180s
sys0m5.154s
[r...@short eli]# tar -c /usr | { cd /home/eli/delete-this/ time tar
-x ; }
tar: Removing leading `/' from member names
tar: Removing leading `/' from hard link targets
real7m0.826s
user0m2.344s
sys0m32.340s
[r...@short eli]# time rm -rf /home/eli/delete-this
real0m8.195s
user0m0.196s
sys0m5.620s
[r...@short eli]# tar -c /usr | { cd /fresh/delete-this/ time tar -x ;
}
tar: Removing leading `/' from member names
tar: Removing leading `/' from hard link targets
real5m42.639s
user0m2.332s
sys0m36.879s
[r...@short eli]# time rm -rf /fresh/delete-this
real0m9.612s
user0m0.203s
sys0m6.275s
--
Web: http://www.billauer.co.il
___
Haifux mailing list
Haifux@haifux.org
http://hamakor.org.il/cgi-bin/mailman/listinfo/haifux
--
Maxim Kovgan
___
Haifux mailing list
Haifux@haifux.org
http://hamakor.org.il/cgi-bin/mailman/listinfo/haifux
Re: [Haifux] Whole disk encryption, because it costs nothing?
How about doing incremental backups every night(or week) to amazon S3? Duplicity tool can achieve this easily. Also TimeDrive is a nice GUI for it: http://www.oak-tree.us/blog/index.php/science-and-technology/time-drive. Thanks, Zaar, I just might do that! -- Dotan Cohen http://what-is-what.com http://gibberish.co.il ___ Haifux mailing list Haifux@haifux.org http://hamakor.org.il/cgi-bin/mailman/listinfo/haifux
Re: [Haifux] Whole disk encryption, because it costs nothing?
It crossed my mind that a sequential test was too simplistic. My ideal test would be a database server running like crazy on the data, but this is not my application anyhow. I agree that I should test the system under load but I can't really think about a realistic under load scenario for 4 hyperthreaded CPUs. At least not something that I could test without turning it into a project. So I went for the session below. Why not use the Phoronix test suite: http://www.phoronix-test-suite.com/ -- Dotan Cohen http://what-is-what.com http://gibberish.co.il ___ Haifux mailing list Haifux@haifux.org http://hamakor.org.il/cgi-bin/mailman/listinfo/haifux
[Haifux] Whole disk encryption, because it costs nothing?
Hello, I suppose that by now some of you have realized that I'm working on setting up my new computer. ;) The new question is whether I should encrypt the whole hard disk, including the swap partition (minus, possibly, a read-only mounted /boot). Just so I don't need to worry in case my computer gets stolen for worth in metal one day. Mind you, I have a quadcore (the kernel counts 8 CPUs), and I plan on hardware RAID-5 (Intel P55) with three 1 TB hard disks, not yet implemented (when I upgrade, I upgrade). RAM sums up to 4 GB. So I ran a small test. /secret is an encrypted partition. My home directory is not. While the encrypted write ran, I had more or less one CPU at 100% and seven others doing nothing. The results below are consistent and repeatable on my computer. Reads to /dev/null take grossly the same time as write. [...@short ~]$ time dd if=/dev/zero of=/secret/zeros.delme bs=1M count=16k 16384+0 records in 16384+0 records out 17179869184 bytes (17 GB) copied, 158.784 s, 108 MB/s real2m38.822s user0m0.015s sys0m13.655s [...@short ~]$ time dd if=/dev/zero of=zeros.delme bs=1M count=16k 16384+0 records in 16384+0 records out 17179869184 bytes (17 GB) copied, 228.711 s, 75.1 MB/s real3m49.069s user0m0.010s sys0m25.029s Aha! Encryption actually speeds up the write! Well, not really, I suppose. Maybe it has to do with /secret being untouched until now, and the cleartext disk being somewhat fragmented by now. But this little test makes me wonder if I pay anything at all for this (expect for a piece of unused CPU power). If there is any reason in the world not to encrypt the whole chunk. Inputs are welcome. Eli -- Web: http://www.billauer.co.il ___ Haifux mailing list Haifux@haifux.org http://hamakor.org.il/cgi-bin/mailman/listinfo/haifux
Re: [Haifux] Whole disk encryption, because it costs nothing?
Eli Billauer wrote: Inputs are welcome. Eli In my laptop (dual core, used for development) I found that compiling from an encrypted partition was noticeably slower. I wound up setting up an encrypted partition, but only placing there things I consider sensitive (which does include the swap, mail, client's sources, my family images etc.), as well as some other automatically generated stuff (such as /var/lib/mlocate and ~/.bash_history, both via symlinks). I guess it really depends on your use scenario. I'm using the laptop to compile Android (1.5GB) and Linux kernels. Also, I do not run any virtual machine from the encrypted partition - too much trouble. Shachar -- Shachar Shemesh Lingnu Open Source Consulting Ltd. http://www.lingnu.com ___ Haifux mailing list Haifux@haifux.org http://hamakor.org.il/cgi-bin/mailman/listinfo/haifux
Re: [Haifux] Whole disk encryption, because it costs nothing?
I wound up setting up an encrypted partition, but only placing there things I consider sensitive (which does include the swap, mail, client's sources, my family images etc.), as well as some other automatically generated stuff (such as /var/lib/mlocate and ~/.bash_history, both via symlinks). I guess it really depends on your use scenario. I also considered putting personal things such as photos in an encrypted partition, but I wonder about data recovery. I backup every month, but for the most recent things, can data recovery be performed? I usually use JFS partitions, but I am open to others if need be. -- Dotan Cohen http://what-is-what.com http://gibberish.co.il ___ Haifux mailing list Haifux@haifux.org http://hamakor.org.il/cgi-bin/mailman/listinfo/haifux
Re: [Haifux] Whole disk encryption, because it costs nothing?
Shachar Shemesh wrote: Also, I do not run any virtual machine from the encrypted partition - too much trouble. Since that's exactly what I want to do, could you be more specific about what kind of trouble? Just slow? Or other issues? And when you say dual core, how many CPUs does the kernel count? (my quad core counts as eight, as I've already mentioned) Eli -- Web: http://www.billauer.co.il ___ Haifux mailing list Haifux@haifux.org http://hamakor.org.il/cgi-bin/mailman/listinfo/haifux
Re: [Haifux] Whole disk encryption, because it costs nothing?
Please note that the next generation of Intel CPUs (starting at February!) are going to have an AES instruction built in them, making harddisk encryption (using AES) extremely efficient (we're talking at about 4 cycles/byte in stupid modes and about 0.7 cycles per byte in more useful modes). AMD to follow in 2011. Also, I do not run any virtual machine from the encrypted partition - too much trouble. Since that's exactly what I want to do, could you be more specific about what kind of trouble? Just slow? Or other issues? And when you say dual core, how many CPUs does the kernel count? (my quad core counts as eight, as I've already mentioned) Eli -- Web: http://www.billauer.co.il ___ Haifux mailing list Haifux@haifux.org http://hamakor.org.il/cgi-bin/mailman/listinfo/haifux -- Orr Dunkelman, orr.dunkel...@gmail.com GPG fingerprint: C2D5 C6D6 9A24 9A95 C5B3 2023 6CAB 4A7C B73F D0AA (This key will never sign Emails, only other PGP keys. The key corresponds to o...@vipe.technion.ac.il) ___ Haifux mailing list Haifux@haifux.org http://hamakor.org.il/cgi-bin/mailman/listinfo/haifux
Re: [Haifux] Whole disk encryption, because it costs nothing?
On Mon, Jan 11, 2010 at 9:43 PM, Dotan Cohen dotanco...@gmail.com wrote: I wound up setting up an encrypted partition, but only placing there things I consider sensitive (which does include the swap, mail, client's sources, my family images etc.), as well as some other automatically generated stuff (such as /var/lib/mlocate and ~/.bash_history, both via symlinks). I guess it really depends on your use scenario. I also considered putting personal things such as photos in an encrypted partition, but I wonder about data recovery. I backup every month, but for the most recent things, can data recovery be performed? I usually use JFS partitions, but I am open to others if need be. How about doing incremental backups every night(or week) to amazon S3? Duplicity tool can achieve this easily. Also TimeDrive is a nice GUI for it: http://www.oak-tree.us/blog/index.php/science-and-technology/time-drive. -- Dotan Cohen http://what-is-what.com http://gibberish.co.il ___ Haifux mailing list Haifux@haifux.org http://hamakor.org.il/cgi-bin/mailman/listinfo/haifux -- Zaar ___ Haifux mailing list Haifux@haifux.org http://hamakor.org.il/cgi-bin/mailman/listinfo/haifux
Re: [Haifux] Whole disk encryption, because it costs nothing?
Sequential IO is very simple, relatively, so that you will hardly feel the performance impact testing it. Test random IO loads with small packets (0.5K-4K) and you will probably feel the performance impact there. Ez On Mon, Jan 11, 2010 at 7:00 PM, Eli Billauer e...@billauer.co.il wrote: Hello, I suppose that by now some of you have realized that I'm working on setting up my new computer. ;) The new question is whether I should encrypt the whole hard disk, including the swap partition (minus, possibly, a read-only mounted /boot). Just so I don't need to worry in case my computer gets stolen for worth in metal one day. Mind you, I have a quadcore (the kernel counts 8 CPUs), and I plan on hardware RAID-5 (Intel P55) with three 1 TB hard disks, not yet implemented (when I upgrade, I upgrade). RAM sums up to 4 GB. So I ran a small test. /secret is an encrypted partition. My home directory is not. While the encrypted write ran, I had more or less one CPU at 100% and seven others doing nothing. The results below are consistent and repeatable on my computer. Reads to /dev/null take grossly the same time as write. [...@short ~]$ time dd if=/dev/zero of=/secret/zeros.delme bs=1M count=16k 16384+0 records in 16384+0 records out 17179869184 bytes (17 GB) copied, 158.784 s, 108 MB/s real2m38.822s user0m0.015s sys0m13.655s [...@short ~]$ time dd if=/dev/zero of=zeros.delme bs=1M count=16k 16384+0 records in 16384+0 records out 17179869184 bytes (17 GB) copied, 228.711 s, 75.1 MB/s real3m49.069s user0m0.010s sys0m25.029s Aha! Encryption actually speeds up the write! Well, not really, I suppose. Maybe it has to do with /secret being untouched until now, and the cleartext disk being somewhat fragmented by now. But this little test makes me wonder if I pay anything at all for this (expect for a piece of unused CPU power). If there is any reason in the world not to encrypt the whole chunk. Inputs are welcome. Eli -- Web: http://www.billauer.co.il ___ Haifux mailing list Haifux@haifux.org http://hamakor.org.il/cgi-bin/mailman/listinfo/haifux ___ Haifux mailing list Haifux@haifux.org http://hamakor.org.il/cgi-bin/mailman/listinfo/haifux
Re: [Haifux] Whole disk encryption, because it costs nothing?
Eli Billauer wrote: Shachar Shemesh wrote: Also, I do not run any virtual machine from the encrypted partition - too much trouble. Since that's exactly what I want to do, could you be more specific about what kind of trouble? Just slow? Or other issues? No other issues. As far as I remember, I didn't even bother to try. I suspect it'll be too slow. And when you say dual core, how many CPUs does the kernel count? Two. (my quad core counts as eight, as I've already mentioned) Which means, if I understand correctly, that you have four hyper threaded cores, which is not the same as eight CPUs. This means your test was meaningless. You should test the performance hit under load, not when the other CPUs are idle, as hyper threading does not partition the task completely between the cores. Eli -- Shachar Shemesh Lingnu Open Source Consulting Ltd. http://www.lingnu.com ___ Haifux mailing list Haifux@haifux.org http://hamakor.org.il/cgi-bin/mailman/listinfo/haifux
Re: [Haifux] Whole disk encryption, because it costs nothing?
Dotan Cohen wrote: I also considered putting personal things such as photos in an encrypted partition, but I wonder about data recovery. I backup every month, but for the most recent things, can data recovery be performed? Theoretically, yes. In practice, keep a good backup. Then again, you should keep a good backup anyways. Never rely on data recovery on a working system. Shachar -- Shachar Shemesh Lingnu Open Source Consulting Ltd. http://www.lingnu.com ___ Haifux mailing list Haifux@haifux.org http://hamakor.org.il/cgi-bin/mailman/listinfo/haifux
