Re: Softraid crypto metadata backup
>On Sat, Jan 07, 2023 at 02:33:31PM +, Nathan Carruth wrote: >>The way I see it, this depends on one's use case. >>There certainly are cases where it is important to be able >>to irrevocably destroy all data in an instant. But there are >>also use cases where one is only interested in making sure >>that the average person couldn’t access one’s data if one lost >>one’s laptop/external drive. >> >>I still think that anyone with the second use case could benefit >>from more documentation as I suggested, but I get the feeling >>this opinion is in the distinct minority here. > >If you're part of that supposed minority: count me in. If it's true >that the headers of encrypted disks on Openbsd are set up in a similar >way as on e.g. Linux, then it's actually a good idea to be able to >have precise knowledge about how to backup that header on OBSD. > From what I learned the preparation for that failure includes first a >copy of the data on that encrypted disk to a second or - even better - >a third encrypted one. > >But copying back a whole disk because on the original broken one it's >just the header that went south might be an effort that is a least >avoidable. And this when part two of disaster preparation might be >helpful: the so-called header backup. > >I don't know how often, and if, such header corruptions on encrypted >disks happen on OBSD, but on LUKS/cryptsetup encrypted disks on >Linux this does not seem to be that unusual - from the LUKS FAQ: > >"By far the most questions on the cryptsetup mailing list are from > people that managed to damage the start of their LUKS partitions, > i.e. the LUKS header. In most cases, there is nothing that can be done > to help these poor souls recover their data. Make sure you understand > the problem and limitations imposed by the LUKS security model BEFORE > you face such a disaster! In particular, make sure you have a current > header backup before doing any potentially dangerous operations." > >https://gitlab.com/cryptsetup/cryptsetup/-/blob/main/FAQ.md > >And so, that no one is getting me wrong: I don't expect anyone to >create the documentation for me. Definitively not. But if disk header >corruption can be a problem on Openbsd, too, then this very >possibility helps at least to understand the point the OP was trying >to make when starting this thread. > >Regards, >Wolfgang I wasn't going to say anything more, but after reading this I figured I ought to at least suggest an update to the documentation. Since I'm definitely not qualified to document the details, all I can really do is something like what is posted below (these should be diffs to the current versions of src/share/man/man4/softraid.4 and www/faq/faq14.html on CVS). Thanks, Nathan For softraid.4: @@ -270,0 +271,4 @@ +.Pp +The CRYPTO discipline emphasizes confidentiality over integrity. +In particular, corruption of the on-disk metadata will render all encrypted +data permanently inaccessible. For faq14.html: @@ -835,0 +836,9 @@ +Note + +Decryption requires the use of on-disk metadata documented in the +https://cvsweb.openbsd.org/src/sys/dev/softraidvar.h;>source. +Corruption of this metadata can easily render all encrypted data +permanently inaccessible. +There is at present no automated way to backup this metadata. +
Re: Softraid crypto metadata backup
On Sat, Jan 07, 2023 at 02:33:31PM +, Nathan Carruth wrote: The way I see it, this depends on one's use case. There certainly are cases where it is important to be able to irrevocably destroy all data in an instant. But there are also use cases where one is only interested in making sure that the average person couldn’t access one’s data if one lost one’s laptop/external drive. I still think that anyone with the second use case could benefit from more documentation as I suggested, but I get the feeling this opinion is in the distinct minority here. If you're part of that supposed minority: count me in. If it's true that the headers of encrypted disks on Openbsd are set up in a similar way as on e.g. Linux, then it's actually a good idea to be able to have precise knowledge about how to backup that header on OBSD. I followed this thread. But kept silent so far - simply because I basically don't know much about how disk headers are organized on Openbsd. So — thanks to everyone for the answers, I’m signing off this question now. Take care and stay secure, Nathan Nathan Carruth writes: permanently and irrevocably destroy all data on your entire disk”. This is a feature. Can be a feature in some situations, yes. But that's just one part of the story, if I understand correctly. More so, it's the very point in an encrypted filesystem. If you haven't planned for this failure scenario The OP was trying to prepare exactly for this very scenario. That was obviously the whole purpose of starting the thread. From what I learned the preparation for that failure includes first a copy of the data on that encrypted disk to a second or - even better - a third encrypted one. But copying back a whole disk because on the original broken one it's just the header that went south might be an effort that is a least avoidable. And this when part two of disaster preparation might be helpful: the so-called header backup. I don't know how often, and if, such header corruptions on encrypted disks happen on OBSD, but on LUKS/cryptsetup encrypted disks on Linux this does not seem to be that unusual - from the LUKS FAQ: "By far the most questions on the cryptsetup mailing list are from people that managed to damage the start of their LUKS partitions, i.e. the LUKS header. In most cases, there is nothing that can be done to help these poor souls recover their data. Make sure you understand the problem and limitations imposed by the LUKS security model BEFORE you face such a disaster! In particular, make sure you have a current header backup before doing any potentially dangerous operations." https://gitlab.com/cryptsetup/cryptsetup/-/blob/main/FAQ.md And so, that no one is getting me wrong: I don't expect anyone to create the documentation for me. Definitively not. But if disk header corruption can be a problem on Openbsd, too, then this very possibility helps at least to understand the point the OP was trying to make when starting this thread. Regards, Wolfgang then what are you doing using a device which *by design* can irrevocably trash its contents in an instant?
Re: ??????: Softraid crypto metadata backup
Hi, Please fix your email client to correctly attribute quotes in list mail that you reply to. On Thu, Jan 05, 2023 at 02:13:53PM +, Nathan Carruth wrote: > Thank you for your response (apologies that I just saw this). > > I will have a look at the file you mentioned. > > I am curious what you mean by this: > > ??? Backing up, restoring or > otherwise messing with the softraid metadata without using the standard tools > is an advanced subject??? > > as far as I know there aren???t any standard tools for doing any > of this? If there is, it is probably all I need. The standard tools, (basically bioctl), allow you to create softraid volumes, change the passphrase and do a few other tasks. I published a separate program to resize crypto volumes. This is all that most users need. You are asking about and trying to do something that is completely outside the scope of being 'supported'. It's not recommended, nor considered to be useful.
Re: Softraid crypto metadata backup
On Thu, Jan 05, 2023 at 05:13:05AM +, Nathan Carruth wrote: > Perhaps I should have clarified my use case. I have data which > is potentially legally privileged and which I also cannot afford > to lose. Thus an unencrypted backup is out of the question, and > my first thought was to use full-disk encryption for the backup. For this particular use-case, you can just pipe tar through the openssl command line utility and write the backup to removable media either directly or as a file on a non-encrypted filesystem. I wrote in detail about this method in an article about doing encrypted backups to blu-ray disc: https://research.exoticsilicon.com/articles/crystal_does_optical
Re: Softraid crypto metadata backup
On Thu, Jan 05, 2023 at 05:13:05AM +, Nathan Carruth wrote: > I presume that OpenBSD also writes on-disk metadata of the > same sort somewhere. Where? Look at /usr/src/sys/dev/softraidvar.h. The structures that contain the softraid metadata are defined there. There is general softraid metadata, and crypto specific metadata. These are stored near the beginning of the RAID partition as defined in the disklabel. In fact, they are SR_META_OFFSET blocks from the start, which is currently 8192 bytes. You can also look at this on your own disk with dd and hexdump to familiarise yourself with what the layout looks like, (useful for future reference). Or read my article about resizing softraid volumes for some examples. > I know I could dig this out of > the source code The source code is the definitive reference. And it can change. > As it stands, the documentation gives no hint that softraid > crypto gives any additional risk of data loss. Just about any additional layer on top of a storage volume increases the complexity of the system, which some people might regard as 'additional risk'. This is in no way specific to softraid crypto. > If there are in > fact e.g. salt values written in an unknown location on the > disk It's not unknown, it's documented quite clearly in the source code. > whose loss renders THE ENTIRE DISK cryptographically > inaccessible, surely this ought to be documented somewhere? By definition, losing the salt value used with any effective crypto system _should_ make it inaccessible! This is even considered a feature, because you can effectively erase the disk just by destroying the metadata. > While I agree with you that there are > definite security risks in backing up such metadata, surely > the decision as to what to do ought to be left to the end user, > rather than being enforced by lack of documentation? The source code is the definitive documentation. Backing up, restoring or otherwise messing with the softraid metadata without using the standard tools is an advanced subject, so it's quite reasonable to expect anybody wanting to do this to read and understand the source rather than having it spelt out in a manual page or other documentation. If it was documented elsewhere, that documentation would have to be kept up to date with the current source, otherwise it could end up causing more problems than it solves. In any case, what you are proposing to do, (back up the softraid crypto metadata), is almost certainly a waste of time, as it is extremely unlikely that you will ever be in a situation where such a backup would be useful. Additionally, if you _do_ decide to go ahead with this, then in the very unlikely event that you corrupt the metadata on the main disk and want to restore it from a backup, please do your research _before_ trying to restore it. It would be very easy to corrupt the disk further by dd'ing the wrong data to the wrong place. There have been a lot of posts to the mailing lists in the past by people who have tried to fix disk partitioning problems by themselves and made the situation worse. What you are proposing sounds to me like a foot gun.
Re: Softraid crypto metadata backup
On 1/2/23 22:22, Nathan Carruth wrote: Does a softraid(4) crypto volume require metadata backup? (I am running amd64 OpenBSD 6.9 if it is relevant, will probably upgrade in the next few months.) I understand FreeBSD GELI (e.g.) requires such a backup to protect against crypto-related metadata corruption rendering the encrypted volume inaccessible. Neither the OpenBSD disk FAQ nor the man pages for softraid(4) or bioctl(8) have anything to say about the matter. Web searches also turn up no relevant information. Storage requires backup. Encrypted storage is (by design) more fragile than unencrypted storage. Sounds like you are trying to protect against ONE form of storage failure and avoid the solution you really need to have: a good backup system, to deal with *all* forms of storage failure. I'd suggest a good backup system...to deal with ALL forms of data loss. Yes, encrypted storage implies a certain care has to be taken with the backups as well, you need to pick a solution that is appropriate for your needs -- or accept that yeah, stuff will go bye-bye someday. I don't see a benefit to trying to protect against some single failure mode when all the other failure modes still exist. If you have good backups, you are good. If you don't, dealing with a 1% problem isn't going to change much. Nick.
