RE: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
> From: Brandon Nielsen [mailto:niels...@jetfuse.net] > Sent: Wednesday, January 26, 2022 5:14 PM > On 1/26/22 3:25 AM, Roberto Sassu via devel wrote: > > [Snip] > > > > > - web servers or other kind of servers where you, as client, would > >like the guarantee that your data is processed only if the software > >running in the server is not compromised > > > > For what it's worth, I, and several people I work with, have been hoping > for something to enable this kind of functionality for quite some time > now. Hope to see more about it soon! Thanks Brandon! That is very encouraging. All the pieces have been already developed and demonstrated at the final review meeting of the FutureTPM European project. Here, at the bottom of the email, you can find some links: https://www.spinics.net/lists/fedora-devel/msg295543.html I planned to create a dedicated Fedora Change for the remote attestation part, which would depend on: https://fedoraproject.org/wiki/Changes/DIGLIM I'm still looking for some support from people interested like you, to comment in the kernel mailing list and help for the acceptance of the kernel patches in the upstream kernel. Thanks Roberto HUAWEI TECHNOLOGIES Duesseldorf GmbH, HRB 56063 Managing Director: Li Peng, Zhong Ronghua ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On 1/26/22 3:25 AM, Roberto Sassu via devel wrote: [Snip] - web servers or other kind of servers where you, as client, would like the guarantee that your data is processed only if the software running in the server is not compromised For what it's worth, I, and several people I work with, have been hoping for something to enable this kind of functionality for quite some time now. Hope to see more about it soon! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
Dne 21. 01. 22 v 17:08 Roberto Sassu via devel napsal(a): (note for the infrastructure mailing list: please check if the changes I'm proposing could be tested in the Fedora infrastructure, like Copr) Copr does not use `rpmsign` at all. Copr uses `obs-sign` [1]. The benefits of obs-sign is that you do not need to send whole package to the signing machine and then back. You send just hash there and signature back. So to test this in Copr the --signverity has to be first implemented in obs-sign. [1] https://github.com/openSUSE/obs-sign Miroslav ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Wed, 2022-01-26 at 09:25 +, Roberto Sassu via devel wrote: > > ...snip > > If the users often make changes on their system, with high > privileges, > I agree that DIGLIM would simply cause too much overhead for > the configuration (every time the users make a change, they have > to whitelist their software). > Yeah that will definitely be a problem in my use cases. You know, if I wanted to have something interfere with my work, like Windows, I would just install Windows. Stephen > Roberto > > HUAWEI TECHNOLOGIES Duesseldorf GmbH, HRB 56063 > Managing Director: Li Peng, Zhong Ronghua > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: > https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > Do not reply to spam on the list, report it: > https://pagure.io/fedora-infrastructure ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
RE: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
> From: Kevin Fenzi [mailto:ke...@scrye.com] > Sent: Tuesday, January 25, 2022 7:30 PM > On Fri, Jan 21, 2022 at 04:08:04PM +, Roberto Sassu via devel wrote: > > Hi everyone > > > > (note for the infrastructure mailing list: please check if the changes > > I'm proposing could be tested in the Fedora infrastructure, like Copr) > > copr uses a different signing setup... so probibly won't work there. > > We could perhaps work with you to try and get it in our staging env to > test, but I am not sure how involved that might be. I guess the first > step would be filing a infratructure ticket and explain what you would > need to change/test. Thanks, Kevin. Will do. > So, question for the change owners: > > This is not enabled by default, I assume you wish to enable it for > something, can you expand on what you will use it for and what that > helps you with? > > Also, is there any plan to enable it by default? Is there some way this > could be useful to all our users? or more of our users? > > This change and the DIGLIM change both feel to me like we are spending > time and effort enabling something that only helps a tiny fraction of > our users. Is there some way this could be more generally useful to our > users? I guess not many would use the remote attestation capability offered by DIGLIM (sealing a TPM key to the software running in the machine). The two scenarios in which that could be used are: - web servers or other kind of servers where you, as client, would like the guarantee that your data is processed only if the software running in the server is not compromised - closed networks where you, as network administrator, would like the guarantee that your clients can access those networks only if their software is not compromised Probably, the secure boot functionality at application level is much more appealing, and less invasive. It should have almost zero impact if the users use their system for web browsing or writing. Although it is unlikely to happen, this feature prevents arbitrary execution of software, even by root. The trust anchor will be the kernel, and one would not have to rely on the correct execution of the package manager (and on its verification of the package signature). If the users often make changes on their system, but confined to their environment, DIGLIM could still be helpful, as the damage of running arbitrary software will be limited to the users environment (no arbitrary software execution as root). If the users often make changes on their system, with high privileges, I agree that DIGLIM would simply cause too much overhead for the configuration (every time the users make a change, they have to whitelist their software). Roberto HUAWEI TECHNOLOGIES Duesseldorf GmbH, HRB 56063 Managing Director: Li Peng, Zhong Ronghua ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Fri, Jan 21, 2022 at 04:08:04PM +, Roberto Sassu via devel wrote: > Hi everyone > > (note for the infrastructure mailing list: please check if the changes > I'm proposing could be tested in the Fedora infrastructure, like Copr) copr uses a different signing setup... so probibly won't work there. We could perhaps work with you to try and get it in our staging env to test, but I am not sure how involved that might be. I guess the first step would be filing a infratructure ticket and explain what you would need to change/test. So, question for the change owners: This is not enabled by default, I assume you wish to enable it for something, can you expand on what you will use it for and what that helps you with? Also, is there any plan to enable it by default? Is there some way this could be useful to all our users? or more of our users? This change and the DIGLIM change both feel to me like we are spending time and effort enabling something that only helps a tiny fraction of our users. Is there some way this could be more generally useful to our users? kevin signature.asc Description: PGP signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
RE: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
Hi everyone (note for the infrastructure mailing list: please check if the changes I'm proposing could be tested in the Fedora infrastructure, like Copr) I made the first version of the rpm extension to sign fsverity digests with a GPG key. The patch set (with some bug fixes) is available here: https://github.com/robertosassu/rpm/commits/fsverity-gpg-v1 I tested it locally with my own GPG key. I took an existing Fedora 34 package and signed it with rpmsign: $ usr/bin/rpmsign --define "%_gpg_name testhost " \ --define "%_file_signing_key _GPG_" \ --define "%_file_signing_cert _GPG_" \ --addsign --signverity tmux-3.1c-2.fc34.x86_64.rpm I then checked that the package has now fsverity signatures: $ usr/bin/rpm -qp tmux-3.1c-2.fc34.x86_64.rpm \ --queryformat '[%{RPMTAG_FILENAMES} %{RPMTAG_VERITYSIGNATURES}\n]' [...] /usr/bin/tmux iQHHBAABCgAxFiEEEiFa0dGZVYzTrIN+rxtXRMfK0McFAmHq0+4THHRlc3Rob3N0 QHRlc3QudGVzdAAKCRCvG1dEx8rQx81nC/42NW9xJx3rcTiR6/5oL55GPkan+OIq t2dW1clJUOrxOGVy/5JQTQf0MQXA7gzH1yPgcrskkahjSfWlp4pt7oOw3rukUyaO zVZxue4XE6XESYtolczK4VEhpc8lbm4hj0e4NCg/dKri/+L5wIdJvmqWNeCfl7uZ [...] In a VM I tried to install the modified package. The root filesystem is ext4 and has the fsverity feature enabled. The fsverity rpm plugin is also enabled and hasn't been modified to work with the new PGP signatures. The kernel includes the patch set I recently sent to the kernel mailing lists to add support for PGP keys and signatures: https://lore.kernel.org/linux-integrity/2022080318.591029-1-roberto.sa...@huawei.com/ and another patch that calls verify_pgp_signature() in fs/verity/signature.c. The first installation attempt fails, due to the missing key in the .fs-verity keyring: # usr/bin/rpm -Uhvi ../tmux-3.1c-2.fc34.x86_64.rpm --debug [...] D: Plugin: calling hook fsm_file_prepare in fsverity plugin D: applying signature: [...] D: failed to enable verity (errno 126) for /usr/bin/tmux;61ead62d Then, I added the required GPG key to the .fs-verity keyring: # cat /mnt/repos/linux/certs/pubring.gpg | keyctl padd asymmetric test %keyring:.fs-verity 76292211 The key is now loaded: # keyctl show %keyring:.fs-verity Keyring 66741466 --a-swrv 0 0 keyring: .fs-verity 76292211 --als--v 0 0 \_ asymmetric: test I retried the tmux installation: # usr/bin/rpm -Uhvi ../tmux-3.1c-2.fc34.x86_64.rpm --debug [...] D: Plugin: calling hook fsm_file_prepare in fsverity plugin D: applying signature: [...] D: fsverity enabled signature for: path /usr/bin/tmux;61ead713 dest /usr/bin/tmux This time the installation is successful, which means that the PGP signature has been successfully verified. Roberto HUAWEI TECHNOLOGIES Duesseldorf GmbH, HRB 56063 Managing Director: Li Peng, Zhong Ronghua ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Fri, Dec 17, 2021 at 4:59 PM Colin Walters wrote: > > > > On Mon, Dec 13, 2021, at 5:21 PM, Tom Stellard wrote: > > > > Did you test the impact this has on package build times? Particularly > > packages like llvm, clang, webkit2gtk3, etc. that have very large > > debuginfo files? > > I think far too often the culture here is "make $change for all RPMs". But > this "everything is an RPM" mindset can lead to outcomes and methodology that > is at best weird. > > For e.g. "let's try building with newer gcc", it would seem far better to me > to e.g. start with the things that are in Fedora CoreOS or Workstation or > whatever. (And, optionally their build dependencies) > Subset validation is very useful, yes, but fundamentally the Rawhide corpus is used to shake out GCC in the first place. It's a big part of how GCC new stable releases become so good. If we don't do it, basically nobody will. > For *this* particular change, the value of pre-signing the -debug RPMs > seems...weak. Or even the `-devel` RPMs. Now, obviously choosing *which* > binaries to sign would require some thought. > But I think that's worth doing instead of blindly doing everything. I'm not sure I agree. Outputs become inputs for other processes, and I think people generally prefer that the integrity of those inputs could be assured. If we'd want the integrity of the filesystem to be assured for runtime components, I see no reason that we wouldn't want it for development components and build-time components too. Moreover, as infrastructure for granular, network-based access for those things becomes a reality, having signed blobs for those things becomes more valuable so that privileged processes can be reasonably assured that they aren't tampered with. -- 真実はいつも一つ!/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
RE: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
> From: Neal Gompa [mailto:ngomp...@gmail.com] > Sent: Friday, December 17, 2021 11:17 AM > On Fri, Dec 17, 2021 at 5:14 AM Roberto Sassu via devel > wrote: > > > > > In Fedora, we use a new package signing key for each Fedora release. > > > What key would be used for the fs-verity signatures: the same key, > > > a separate key? Edit: I see that the Change page says a dedicated key is > > > used. > > > > Hi all > > > > I'm doing related work in this area. I'll provide some additional > > thoughts. > > > > Probably, it could be possible to use the package signing key > > for the fsverity signatures. However, this would require the > > kernel to be able to load the PGP keys in the builtin keyring > > and support PGP signatures. > > > > David Howells did that some time ago, and I adapted his > > patches for the latest kernel. Without going into too much > > detail, I've modified the kernel build to take the Fedora > > keys. They are available to verify PGP signatures (I use > > this feature to verify RPM headers). > > > > If there is interest, I could propose the patch set to the kernel > > mailing list. > > > > That would be phenomenal. That would allow us to tie things to our > existing process of key rotation for each Fedora release and leverage > the existing infrastructure for package signing for fsverity > signatures too. Great. I will propose the patch set again. For another Fedora feature I'm proposing (related to this one): https://fedoraproject.org/wiki/Changes/DIGLIM I went as far as to be able to appraise with IMA existing RPM headers (I added support for PGP appended signatures). I'm using this information in a different way: loading the file digests from the RPM header to the kernel and querying them with IMA during appraisal. That would work with fsverity digests too. If the concern is the increased package size, adding fsverity digests to the RPM header instead of signatures might be more acceptable. I made a small demo: https://lore.kernel.org/linux-integrity/a16a628b9e21433198c490500a987...@huawei.com/ where loading of kernel modules is granted by IPE if their fsverity digest is found in the DIGLIM hash table. Roberto HUAWEI TECHNOLOGIES Duesseldorf GmbH, HRB 56063 Managing Director: Li Peng, Zhong Ronghua > > > IIUC, I need to do some steps at each boot: > > > 1. add all the keys to the keyring > > > 2. set sysctl fs.verity.require_signatures=1 > > > > > > So… in 1., do I always load all the keys that Fedora has used for this > > > purpose, in case there are still some files on disk? Or is there some > > > mechanism to say that e.g. keys older than F(N-2) are now not necessary? > > > > > > Who does 2.? > > > > > > I think it'd be hard to enable this during a system upgrade: one would > > > need > > > to reinstall all rpms (with new rpms carrying the fsverity metadata) > > > or some files become unreadable once 2. is done. This brings a question: > > > is there some rpm virtual Provides/Requires to specify that the fs-verity > > > stuff is present? I assume the user would want to triple-check that they > > > don't have any rpms without the metadata before enabling verification. > > > > The kernel does not enforce signature verification unless fsverity > > is explicitly enabled on a file. I guess the rpm plugin can be configured > > to enable fsverity only if it finds a signature in the RPM header. That > > would allow a mixed configuration where some files are protected, > > other not. > > > > This does not seem ideal for mandatory protection, where you want > > to be sure that integrity is checked, even if only on a subset of files > > (e.g. executables and shared libraries), regardless of whether fsverity > > was enabled or not on a file. > > > > It could be task of the security subsystem to do this type of enforcement. > > At the moment, IMA and IPE (Integrity Protection Enforcement) are > > planning to support fsverity and do the enforcement based on a policy. > > > > So that means there *will* be a policy control mechanism for this > enforced by the kernel, cool! > > > > > -- > 真実はいつも一つ!/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Fri, Dec 17, 2021 at 5:14 AM Roberto Sassu via devel wrote: > > > In Fedora, we use a new package signing key for each Fedora release. > > What key would be used for the fs-verity signatures: the same key, > > a separate key? Edit: I see that the Change page says a dedicated key is > > used. > > Hi all > > I'm doing related work in this area. I'll provide some additional > thoughts. > > Probably, it could be possible to use the package signing key > for the fsverity signatures. However, this would require the > kernel to be able to load the PGP keys in the builtin keyring > and support PGP signatures. > > David Howells did that some time ago, and I adapted his > patches for the latest kernel. Without going into too much > detail, I've modified the kernel build to take the Fedora > keys. They are available to verify PGP signatures (I use > this feature to verify RPM headers). > > If there is interest, I could propose the patch set to the kernel > mailing list. > That would be phenomenal. That would allow us to tie things to our existing process of key rotation for each Fedora release and leverage the existing infrastructure for package signing for fsverity signatures too. > > IIUC, I need to do some steps at each boot: > > 1. add all the keys to the keyring > > 2. set sysctl fs.verity.require_signatures=1 > > > > So… in 1., do I always load all the keys that Fedora has used for this > > purpose, in case there are still some files on disk? Or is there some > > mechanism to say that e.g. keys older than F(N-2) are now not necessary? > > > > Who does 2.? > > > > I think it'd be hard to enable this during a system upgrade: one would need > > to reinstall all rpms (with new rpms carrying the fsverity metadata) > > or some files become unreadable once 2. is done. This brings a question: > > is there some rpm virtual Provides/Requires to specify that the fs-verity > > stuff is present? I assume the user would want to triple-check that they > > don't have any rpms without the metadata before enabling verification. > > The kernel does not enforce signature verification unless fsverity > is explicitly enabled on a file. I guess the rpm plugin can be configured > to enable fsverity only if it finds a signature in the RPM header. That > would allow a mixed configuration where some files are protected, > other not. > > This does not seem ideal for mandatory protection, where you want > to be sure that integrity is checked, even if only on a subset of files > (e.g. executables and shared libraries), regardless of whether fsverity > was enabled or not on a file. > > It could be task of the security subsystem to do this type of enforcement. > At the moment, IMA and IPE (Integrity Protection Enforcement) are > planning to support fsverity and do the enforcement based on a policy. > So that means there *will* be a policy control mechanism for this enforced by the kernel, cool! -- 真実はいつも一つ!/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On 12/12/21 8:26 AM, Florian Weimer wrote: > * Zbigniew Jędrzejewski-Szmek: > >> Some more questions about how the verification happens… IIUC, I need to >> load some keys to the kernel keyring, and then set >> fs.verity.require_signatures. >> >> Where do the keys come from? How are the keys themselves verified? >> At what time are they loaded and by whom? >> >> (Let's say that I'm an attacker with access to the file system. If >> the keys are loaded from the file system, can I just drop in a rogue key, >> similarly to what happens when new keys are distributed as part of >> distro upgrades?) >> >> If fs-verity verification prevents me from successfully modifying or >> replacing /usr/bin/foo or /usr/lib/systemd/system/foo.service, is >> there anything which hinders just adding /etc/systemd/system/foo.service >> that does whatever I want? There is not. dm-verity is the only solution to that that I am aware of in Linux. dm-verity also solves the problem that someone could tamper with the block device and exploit a vulnerability in the kernel's filesystem drivers. > Even if we do that, we can't rule out two scenarios: > > * The attacker builds a (perhaps unrelated) Fedora package and lets > Fedora sign it with the official key. The file signature is as good > as any other produced by Fedora. > > (Alternatively, the attacker reuses file contents signed elsewhere in > Fedora because it has an unintended side effect when used in a > different context. This attack requires finding a suitable gadget in > some Fedora package, but a dodgy Fedora package build is not needed.) NetBSD’s Veriexec has a neat solution to this problem, but it is significantly more complicated. It uses additional metadata to validate a file. > * The attacker (who could be the legitimate user) changes the system > configuration so that the feature is disabled during boot. > > Neither we or hardware suppliers installing Fedora on their devices are > permitted to plug the second hole because it's required for software > freedom (or Fedora would have to share its private signing keys). The obvious solution to the second problem is some form of measured boot: if the feature is disabled, it shows up in the system measurements, and access to various secrets may be denied by the TPM. Android uses a similar solution: at least on Google Pixel devices, one can unlock the bootloader, but doing so erases all data on the device. > The combination of these two unsolvable issues suggests to me that > anyone who wants to deploy this is better off with their own trust root, > and that approach will include their particular version of key > management as well. But this also means that pre-computed file > signatures are not particular useful to them. They would have to > discard them anyway before deployment. Their own signing process might > as well check the RPM header signature instead. +1 on this. There have also been bugs in RPM's handling of IMA signatures, and fs-verity signature handling could have similar issues. Since IMA and fs-verity signatures are currently stored in the signature header, they can be modified without changing the package signature, so they increase the attack surface of RPM. -- Sincerely, Demi Marie Obenour (she/her/hers) OpenPGP_0xB288B55FFF9C22C1.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
> On Wed, Dec 15, 2021, at 1:45 PM, Luca Boccassi wrote: > > Hmm. Some interesting stuff going on there but I would have started with a > new SELinux > access vector. That'd allow fine-grained constraints, e.g. disallowing > `init_t` but > allowing `unconfined_service_t`. Possibly also landlock should be able to > hook into this. > IOW it's not clear to me that a new LSM is the best thing for the ecosystem > here. > > But bigger picture I'd agree that fs-verity is significantly stronger when > coupled > with such a policy - strong enough to block exploits like the runc one: > https://unit42.paloaltonetworks.com/breaking-docker-via-runc-explaining-c... We use this in production, and those kind of constraints are just not enough, because there's too many obvious ways around them. What we needed is for the kernel to enforce that only signed code shall run, and thus IPE was born, so that when coupled with dm-verity/fs-verity it allows to enforce a policy like that. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Mon, Dec 13, 2021, at 5:21 PM, Tom Stellard wrote: > > Did you test the impact this has on package build times? Particularly > packages like llvm, clang, webkit2gtk3, etc. that have very large > debuginfo files? I think far too often the culture here is "make $change for all RPMs". But this "everything is an RPM" mindset can lead to outcomes and methodology that is at best weird. For e.g. "let's try building with newer gcc", it would seem far better to me to e.g. start with the things that are in Fedora CoreOS or Workstation or whatever. (And, optionally their build dependencies) For *this* particular change, the value of pre-signing the -debug RPMs seems...weak. Or even the `-devel` RPMs. Now, obviously choosing *which* binaries to sign would require some thought. But I think that's worth doing instead of blindly doing everything. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Wed, Dec 15, 2021, at 1:45 PM, Luca Boccassi wrote: >> On Fri, Dec 10, 2021 at 10:47:52AM +0100, Vít Ondruch wrote: >> >> Any file covered by fs-verity is immutable after installation. So you >> cannot modify the contents, the kernel refuses. But you can just >> replace the file (like during an upgrade), and of course copy and edit >> in a different location. If replaced, no fs-verity checking is done >> any more by the kernel. There was some talk about high-level solution >> to prevent such files from being executed, e.g. an LSM module, but no >> details... (Thinking about this, it would be pretty hard, because the >> LSM would need to be smart enough to know which files are installed >> through rpm, and which files are not. I would love to hear more details >> about what is planned here.) >> >> Zbyszek > > There is such an LSM that supports fs-verity (and dm-verity), being > reviewed right now: IPE > > https://lore.kernel.org/lkml/81d5e825-1ee2-8f6b-cd9d-07b0f8bd3...@linux.microsoft.com/T/ > https://microsoft.github.io/ipe/ Hmm. Some interesting stuff going on there but I would have started with a new SELinux access vector. That'd allow fine-grained constraints, e.g. disallowing `init_t` but allowing `unconfined_service_t`. Possibly also landlock should be able to hook into this. IOW it's not clear to me that a new LSM is the best thing for the ecosystem here. But bigger picture I'd agree that fs-verity is significantly stronger when coupled with such a policy - strong enough to block exploits like the runc one: https://unit42.paloaltonetworks.com/breaking-docker-via-runc-explaining-cve-2019-5736/ (Of course that one was *also* stopped by ostree's default read-only bind mount, which I keep saying is not really primarily about security, but hey it worked there) I said this about the IMA-for-Fedora proposal too - to me the approach of "sign all the RPMs" is not a good idea. Instead the focus should be on ensuring the capability works, and is usable from tools to build custom systems. And this of course runs into a bigger picture question of whether we should emphasize the entry point into Fedora being Anaconda/FCOS like (provision and configure a "pre-built" system) or more like Image Builder and tools like that. I think the answer has to be "we do both" really - it's just hard. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
> We don't have a proof of concept for the LSM module. I agree with you that in > practice > it would probably need to implement some kind of "list of files we care > about", > but I do not have an intelligent opinion about that. > > Based on Roberto's comment in a different sub-thread, there could be some > ongoing work > integrating with IMA that might be past the hypothetical stage? IPE supports fs-verity and dm-verity, and allows to write a policy such as "only allow execution of binaries/libraries from fs-verity verified files or dm-verity verified volumes): https://lore.kernel.org/lkml/81d5e825-1ee2-8f6b-cd9d-07b0f8bd3...@linux.microsoft.com/T/ https://microsoft.github.io/ipe/ ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
> On Fri, Dec 10, 2021 at 10:47:52AM +0100, Vít Ondruch wrote: > > Any file covered by fs-verity is immutable after installation. So you > cannot modify the contents, the kernel refuses. But you can just > replace the file (like during an upgrade), and of course copy and edit > in a different location. If replaced, no fs-verity checking is done > any more by the kernel. There was some talk about high-level solution > to prevent such files from being executed, e.g. an LSM module, but no > details... (Thinking about this, it would be pretty hard, because the > LSM would need to be smart enough to know which files are installed > through rpm, and which files are not. I would love to hear more details > about what is planned here.) > > Zbyszek There is such an LSM that supports fs-verity (and dm-verity), being reviewed right now: IPE https://lore.kernel.org/lkml/81d5e825-1ee2-8f6b-cd9d-07b0f8bd3...@linux.microsoft.com/T/ https://microsoft.github.io/ipe/ ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Tue, Dec 14, 2021 at 4:20 PM Zbigniew Jędrzejewski-Szmek wrote: > > On Tue, Dec 14, 2021 at 08:08:19PM +0100, Fabio Valentini wrote: > > I thought fsverity was about determining at runtime that the system > > has not been tampered with? But if somebody who has (physical) access > > to the device can just ... move verified files out of the way and put > > their own (unverified) files there (which then apparently does not > > trigger red warning signs?) - doesn't that defeat the whole purpose of > > enabling fsverity? > > That's a good question. fs-verity and dm-verity share the same > underlying concept (merkle trees and signature verification by the > kernel). So this raises the question that you asked… which can also be > phrased as "why would you even use fs-verity, if you can do dm-verity"? dm-verity presents a read-only block device. It can't be modified once created. fs-verity works at the VFS layer. The block device and file system remain read-write capable, only the fs-verity enabled file contents are read-only. Also, fs-verity only really provides integrity checking in an efficient manner, with an optional mechanism for user space to enforce authentication - the kernel doesn't do this. A further option for user space is auditing. There is this dance in computer security where you're constantly plugging holes, looking for holes, but also assuming certain holes are definitely filled. It might seem like a contradiction. But of course we say physical access means the system is compromised, so you have to assume that hole is filled or all bets are off. Here you have to assume enough of user space is not compromised enough that you can trust that the fs-verity mechanisms can be used (create and verify), and the reason why you're using this regime is to make certain modifications impossible without detection so that the system doesn't become compromised, or at least compromised in silent fashion. fs-verity does not completely provide a secure and closed system, it's just one aspect of creating a more secure system. I'd go so far as to say it takes some esoteric knowledge to understand that this is only a "probably important small thing", and not a "critical big thing", to do. This is a few years old, 2018, but it discusses some of the confusing points when fs-verity was merged. https://lwn.net/Articles/763729/ Chris Murphy > My understanding it the following: fs-verity originated in the Android > world where you can have an unprivileged process downloading a file, > e.g. a jar. This unprivileged process manages the download, but the > file is only trusted and executed when it has a matching signature > from some central authority. The file contains the whole app, > including all resources, so there is no question of other unverified > files being used by the app. And the file can be large enough that > it's practical to do chunked verification, since checksumming the whole > file on first use would be slow. > > We don't really have the same considerations: the download process > has full privileges, and the download is exploded into individual files, > and more importantly, unpackaged files are also used. -- Chris Murphy ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Tue, Dec 14, 2021 at 09:18:20PM +, Zbigniew Jędrzejewski-Szmek wrote: > My understanding it the following: fs-verity originated in the Android > world where you can have an unprivileged process downloading a file, > e.g. a jar. This unprivileged process manages the download, but the > file is only trusted and executed when it has a matching signature > from some central authority. The file contains the whole app, > including all resources, so there is no question of other unverified > files being used by the app. And the file can be large enough that > it's practical to do chunked verification, since checksumming the whole > file on first use would be slow. This does seem rather reminiscent of our LiveCD squashfs situation -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Tue, Dec 14, 2021 at 08:08:19PM +0100, Fabio Valentini wrote: > On Tue, Dec 14, 2021 at 1:45 AM Davide Cavalca via devel > wrote: > > > > On Mon, 2021-12-13 at 16:00 +0100, Vít Ondruch wrote: > > > Would it be possible to document the editing of protected file in the > > > change proposal, probably including example of the best way to do it > > > (is > > > it possible to replace the file by symlink?) Or is there a way to > > > temporary enable the editing with some overlay? Is there any other way > > > to restore the original file except "dnf reinstall"? > > > > I've added this to the wiki: > > https://fedoraproject.org/wiki/Changes/FsVerityRPM#Can_the_user_modify_a_file_shipped_by_a_package_.28e.g._to_edit_a_script_while_debugging.29_.3F > > > > You could restore the original file via "dnf reinstall", or by moving > > it back into place (rename() and unlink() are allowed on fs-verity > > enabled files). > > I thought fsverity was about determining at runtime that the system > has not been tampered with? But if somebody who has (physical) access > to the device can just ... move verified files out of the way and put > their own (unverified) files there (which then apparently does not > trigger red warning signs?) - doesn't that defeat the whole purpose of > enabling fsverity? That's a good question. fs-verity and dm-verity share the same underlying concept (merkle trees and signature verification by the kernel). So this raises the question that you asked… which can also be phrased as "why would you even use fs-verity, if you can do dm-verity"? My understanding it the following: fs-verity originated in the Android world where you can have an unprivileged process downloading a file, e.g. a jar. This unprivileged process manages the download, but the file is only trusted and executed when it has a matching signature from some central authority. The file contains the whole app, including all resources, so there is no question of other unverified files being used by the app. And the file can be large enough that it's practical to do chunked verification, since checksumming the whole file on first use would be slow. We don't really have the same considerations: the download process has full privileges, and the download is exploded into individual files, and more importantly, unpackaged files are also used. Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Tue, Dec 14, 2021 at 1:45 AM Davide Cavalca via devel wrote: > > On Mon, 2021-12-13 at 16:00 +0100, Vít Ondruch wrote: > > Would it be possible to document the editing of protected file in the > > change proposal, probably including example of the best way to do it > > (is > > it possible to replace the file by symlink?) Or is there a way to > > temporary enable the editing with some overlay? Is there any other way > > to restore the original file except "dnf reinstall"? > > I've added this to the wiki: > https://fedoraproject.org/wiki/Changes/FsVerityRPM#Can_the_user_modify_a_file_shipped_by_a_package_.28e.g._to_edit_a_script_while_debugging.29_.3F > > You could restore the original file via "dnf reinstall", or by moving > it back into place (rename() and unlink() are allowed on fs-verity > enabled files). I thought fsverity was about determining at runtime that the system has not been tampered with? But if somebody who has (physical) access to the device can just ... move verified files out of the way and put their own (unverified) files there (which then apparently does not trigger red warning signs?) - doesn't that defeat the whole purpose of enabling fsverity? Fabio ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
I don't believe we systematically tested this. We will collect that along with the detailed size increase data. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Mon, 2021-12-13 at 16:00 +0100, Vít Ondruch wrote: > Would it be possible to document the editing of protected file in the > change proposal, probably including example of the best way to do it > (is > it possible to replace the file by symlink?) Or is there a way to > temporary enable the editing with some overlay? Is there any other way > to restore the original file except "dnf reinstall"? I've added this to the wiki: https://fedoraproject.org/wiki/Changes/FsVerityRPM#Can_the_user_modify_a_file_shipped_by_a_package_.28e.g._to_edit_a_script_while_debugging.29_.3F You could restore the original file via "dnf reinstall", or by moving it back into place (rename() and unlink() are allowed on fs-verity enabled files). Cheers Davide ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On 12/2/21 11:36, Ben Cotton wrote: https://fedoraproject.org/wiki/Changes/FsVerityRPM == Summary == Enable the use of fsverity for installed RPM files validation. == Owners == * Name: [[User:Dcavalca|Davide Cavalca]], [[User:Borisb|Boris Burkov]], [[User:Filbranden|Filipe Brandenburger]], [[User:Salimma|Michel Alexandre Salim]], [[User:Malmond|Matthew Almond]] * Email: dcava...@fb.com, bor...@fb.com, filbran...@fb.com, mic...@fb.com, malm...@fb.com == Detailed description == fs-verity is a [https://www.kernel.org/doc/html/latest/filesystems/fsverity.html Linux kernel feature] that does transparent on-demand integrity/authenticity verification of the contents of read-only files, using a hidden Merkle tree (hash tree) associated with the file. The mechanism is similar to dm-verity, but implemented at the file level rather than at the block device level. When fsverity is enabled for a file, the kernel reads every block and generates a hash tree for it, which is stored within the filesystem. On subsequent reads, the kernel computes the block hash and compares it with the one stored in the tree, protecting against alterations and corruption. Because this happens at the filesystem data block read layer, it encompasses all file operations (open, mmap,exec, etc.). In the context of rpm, there are two parts to this: * at build time, we compute the Merkle tree for the files within a package, then sign it and ship it as part of the rpm metadata; * at run time, if the fsverity rpm plugin is enabled, rpm will install the fsverity signature key and enable fsverity on files that are installed. This proposal is primarily concerned with the first part, which will make it possible for users to leverage fs-verity for RPM if they so desire. Specifically, installing and enabling the fs-verity rpm plugin by default is explicitly considered out of scope here. === Caveats === Merkle tree cost The Merkle tree used by fsverity needs to be generated (once at build time, once when the package is installed) and stored on disk. The generation process involves reading all blocks and computing the hash, which has a non-trivial cost; however, it does not appear to meaningfully slow down package installs during empirical testing. Once Did you test the impact this has on package build times? Particularly packages like llvm, clang, webkit2gtk3, etc. that have very large debuginfo files? -Tom ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Mon, Dec 13, 2021 at 07:32:34PM -, Boris Burkov via devel wrote: > Sorry this wasn't clear. > > The rpm carries just 'c' (as well as some small, fixed-size metadata for > interpreting it, like hash algorithm) > > Just to explain that comment which suggested 'a': we have to compute the > Merkle tree at build time in order to get a root hash to sign. The Merkle > tree is then fully re-computed by the fs-verity code in the kernel for > storage by the local filesystem when the rpm is ultimately installed. OK, so please adjust the change page not to say "it", but "the signature" or something like that. Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
We don't have a proof of concept for the LSM module. I agree with you that in practice it would probably need to implement some kind of "list of files we care about", but I do not have an intelligent opinion about that. Based on Roberto's comment in a different sub-thread, there could be some ongoing work integrating with IMA that might be past the hypothetical stage? ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
Sorry this wasn't clear. The rpm carries just 'c' (as well as some small, fixed-size metadata for interpreting it, like hash algorithm) Just to explain that comment which suggested 'a': we have to compute the Merkle tree at build time in order to get a root hash to sign. The Merkle tree is then fully re-computed by the fs-verity code in the kernel for storage by the local filesystem when the rpm is ultimately installed. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
Dne 12. 12. 21 v 12:33 Zbigniew Jędrzejewski-Szmek napsal(a): On Fri, Dec 10, 2021 at 10:47:52AM +0100, Vít Ondruch wrote: Dne 10. 12. 21 v 0:08 Davide Cavalca via devel napsal(a): On Fri, 2021-12-03 at 22:08 +, Richard W.M. Jones wrote: I'm unclear about the threat model - this is an attacker who is someone able to overwrite single files (eg. /bin/ls) but cannot turn off the fs-verity system as a whole? Also if RPM can update /bin/ls then surely an attacker who can widely compromise system files must also be able to update /bin/ls in the same way? Once fs-verity is enabled for a given file (which, in the RPM case, happens at package installation time), it cannot be disabled, and the file becomes immutable. One can still rename() or unlink() it (and this is indeed how rpm is able to replace files when upgrading packages), but the actual contents cannot be altered. Trying to debug some issue in shell/ruby/python script, will it be possible to modify such file? Any file covered by fs-verity is immutable after installation. So you cannot modify the contents, the kernel refuses. But you can just replace the file (like during an upgrade), and of course copy and edit in a different location. If replaced, no fs-verity checking is done any more by the kernel. There was some talk about high-level solution to prevent such files from being executed, e.g. an LSM module, but no details... (Thinking about this, it would be pretty hard, because the LSM would need to be smart enough to know which files are installed through rpm, and which files are not. I would love to hear more details about what is planned here.) Thx for explanation. Would it be possible to document the editing of protected file in the change proposal, probably including example of the best way to do it (is it possible to replace the file by symlink?) Or is there a way to temporary enable the editing with some overlay? Is there any other way to restore the original file except "dnf reinstall"? I am asking because I am torn between the security and OTOH convenient debugging if something goes wrong. Don't take me wrong I am far from editing system file on daily basis, but I have certainly did that. Vít OpenPGP_signature Description: OpenPGP digital signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
> In Fedora, we use a new package signing key for each Fedora release. > What key would be used for the fs-verity signatures: the same key, > a separate key? Edit: I see that the Change page says a dedicated key is used. Hi all I'm doing related work in this area. I'll provide some additional thoughts. Probably, it could be possible to use the package signing key for the fsverity signatures. However, this would require the kernel to be able to load the PGP keys in the builtin keyring and support PGP signatures. David Howells did that some time ago, and I adapted his patches for the latest kernel. Without going into too much detail, I've modified the kernel build to take the Fedora keys. They are available to verify PGP signatures (I use this feature to verify RPM headers). If there is interest, I could propose the patch set to the kernel mailing list. > IIUC, I need to do some steps at each boot: > 1. add all the keys to the keyring > 2. set sysctl fs.verity.require_signatures=1 > > So… in 1., do I always load all the keys that Fedora has used for this > purpose, in case there are still some files on disk? Or is there some > mechanism to say that e.g. keys older than F(N-2) are now not necessary? > > Who does 2.? > > I think it'd be hard to enable this during a system upgrade: one would need > to reinstall all rpms (with new rpms carrying the fsverity metadata) > or some files become unreadable once 2. is done. This brings a question: > is there some rpm virtual Provides/Requires to specify that the fs-verity > stuff is present? I assume the user would want to triple-check that they > don't have any rpms without the metadata before enabling verification. The kernel does not enforce signature verification unless fsverity is explicitly enabled on a file. I guess the rpm plugin can be configured to enable fsverity only if it finds a signature in the RPM header. That would allow a mixed configuration where some files are protected, other not. This does not seem ideal for mandatory protection, where you want to be sure that integrity is checked, even if only on a subset of files (e.g. executables and shared libraries), regardless of whether fsverity was enabled or not on a file. It could be task of the security subsystem to do this type of enforcement. At the moment, IMA and IPE (Integrity Protection Enforcement) are planning to support fsverity and do the enforcement based on a policy. Roberto HUAWEI TECHNOLOGIES Duesseldorf GmbH, HRB 56063 Managing Director: Li Peng, Zhong Ronghua ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Do, 09.12.21 23:55, Fedora Development ML (devel@lists.fedoraproject.org) wrote: > > On Do, 02.12.21 14:36, Ben Cotton (bcotton(a)redhat.com) wrote: > > > > Hmm, so what I am really missing on the feature page: what's the > > attack scenario here? Usually security features come with an attack > > scenario they are supposed to address. But there's no discussion about > > that. > > > Good point, we were skimpy on that. In case you missed it, Davide > just laid it out in his reply to the same question from Richard > Jones higher in the thread. The short answer is that just this > change as-is has no practical threat model, we viewed it as an > enablement step that we believe has a realistic path to broadly > authenticated rpm contents. Alternatively, it does give a > sophisticated user an opportunity to build something like that for > themselves as well. This sounds strange to me. I mean, it's fine if new features initially only implement a subset of what they eventually are supposed to accomplish, but it appears to me you add only infrastructure without *any* use? Infrastructure that has no consuming feature typically doesn't work. Experience tells us that. Hence: I think it's fine if you don't go from 0 to 100 right away but there should be at least *some* benefit if you add such complex infra to the distro. > > This protects file contents, not the metada, right? So what about the > > metadata? if I see a fs-verity enabled inode with libssl.so data in it > > today, and it's a vulnerably version, and I make a hardlink to it, and > > then it gets replaced by a fixed version (with a slightly updated > > name) — how you intend to make sure, that i can't fool you into loading > > my copy of the old file but under the new name? > > If the file is still properly signed with a trusted matching > certificate in the kernel keyring we can't/won't do anything about > it. But that's an extremely weak property then? If I can have "ping" and "poweroff" binaries both signed by Fedora, but then swap them without the signing being able to detect that, why do this at all? > > What about signing anyway? What is this precisely signed with? The > > keys for that when are they rolled over? Who manages those keys? The > > tea for that, who's doing that in Fedora? Is there any protection > > against downgrades between RPM package versions? Does this in any way > > protect combinations of binaries/libraries? I mean, pretty much all > > programs we ship consist of a large number of ELF objects, and you > > probably need to sign the combination of them, but this model doesn't > > look like it offers that at all? If a security bug is found in library > > X, how is it taken out of equation? What would the workflow for that > > look like on the Fedora side of things? > > > I don't have a particularly specific answer to all of these > questions, unfortunately. Perhaps someone with more knowledge about > Fedora's existing key management can weigh in. But imagining we have > the LSM hooks setup so that we never load anything executable > without it being signed by a trusted key, then in my mind, banishing > a bad shared library would require revoking the certificate. Uff. That might work in focussed deployments, but across a distribution like Fedora? Everytime there's an exploitable bug in *any* of the software Fedora ships you want to invalidate basically the *whole* distro? That sounds absolutely unmanagable and unrealistic to me. > I think one could draw a reasonable parallel between package signing > and file signing here, though. Users assemble meaningful system > functionality out of a collection of packages, which are each > individually signed. One of those packages could then be found to be > compromised, and need to be removed, otherwise a user could download > and install a perfectly valid signed package and have a bad time, > which undermines the value of the signing. Is there something about > composing executables out of files that is meaningfully different in > this sense from composing systems out of packages? RPMs sign metadata. You can't trick RPM that easily into accepting "ping" as "poweroff" binary or vice versa. I mean, RPM's security model is from the 1990's, and it only does validation on request, not on access. So it's not great, but it *did* get right to authenticate a whole package in combination with all its metadata. And since versioned deps are part of that metadata you can express relationships. Still, this feature appears totally not throught to the end. I don't get the usecase? Lennart -- Lennart Poettering, Berlin ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/l
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
* Zbigniew Jędrzejewski-Szmek: > Some more questions about how the verification happens… IIUC, I need to > load some keys to the kernel keyring, and then set > fs.verity.require_signatures. > > Where do the keys come from? How are the keys themselves verified? > At what time are they loaded and by whom? > > (Let's say that I'm an attacker with access to the file system. If > the keys are loaded from the file system, can I just drop in a rogue key, > similarly to what happens when new keys are distributed as part of > distro upgrades?) > > If fs-verity verification prevents me from successfully modifying or > replacing /usr/bin/foo or /usr/lib/systemd/system/foo.service, is > there anything which hinders just adding /etc/systemd/system/foo.service > that does whatever I want? Even if we do that, we can't rule out two scenarios: * The attacker builds a (perhaps unrelated) Fedora package and lets Fedora sign it with the official key. The file signature is as good as any other produced by Fedora. (Alternatively, the attacker reuses file contents signed elsewhere in Fedora because it has an unintended side effect when used in a different context. This attack requires finding a suitable gadget in some Fedora package, but a dodgy Fedora package build is not needed.) * The attacker (who could be the legitimate user) changes the system configuration so that the feature is disabled during boot. Neither we or hardware suppliers installing Fedora on their devices are permitted to plug the second hole because it's required for software freedom (or Fedora would have to share its private signing keys). The combination of these two unsolvable issues suggests to me that anyone who wants to deploy this is better off with their own trust root, and that approach will include their particular version of key management as well. But this also means that pre-computed file signatures are not particular useful to them. They would have to discard them anyway before deployment. Their own signing process might as well check the RPM header signature instead. Thanks, Florian ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Fri, Dec 10, 2021 at 10:47:52AM +0100, Vít Ondruch wrote: > > Dne 10. 12. 21 v 0:08 Davide Cavalca via devel napsal(a): > >On Fri, 2021-12-03 at 22:08 +, Richard W.M. Jones wrote: > >>I'm unclear about the threat model - this is an attacker who is > >>someone able to overwrite single files (eg. /bin/ls) but cannot turn > >>off the fs-verity system as a whole? > >> > >>Also if RPM can update /bin/ls then surely an attacker who can widely > >>compromise system files must also be able to update /bin/ls in the > >>same way? > >Once fs-verity is enabled for a given file (which, in the RPM case, > >happens at package installation time), it cannot be disabled, and the > >file becomes immutable. One can still rename() or unlink() it (and this > >is indeed how rpm is able to replace files when upgrading packages), > >but the actual contents cannot be altered. > > > Trying to debug some issue in shell/ruby/python script, will it be possible > to modify such file? Any file covered by fs-verity is immutable after installation. So you cannot modify the contents, the kernel refuses. But you can just replace the file (like during an upgrade), and of course copy and edit in a different location. If replaced, no fs-verity checking is done any more by the kernel. There was some talk about high-level solution to prevent such files from being executed, e.g. an LSM module, but no details... (Thinking about this, it would be pretty hard, because the LSM would need to be smart enough to know which files are installed through rpm, and which files are not. I would love to hear more details about what is planned here.) Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
Some more questions about how the verification happens… IIUC, I need to load some keys to the kernel keyring, and then set fs.verity.require_signatures. Where do the keys come from? How are the keys themselves verified? At what time are they loaded and by whom? (Let's say that I'm an attacker with access to the file system. If the keys are loaded from the file system, can I just drop in a rogue key, similarly to what happens when new keys are distributed as part of distro upgrades?) If fs-verity verification prevents me from successfully modifying or replacing /usr/bin/foo or /usr/lib/systemd/system/foo.service, is there anything which hinders just adding /etc/systemd/system/foo.service that does whatever I want? On Thu, Dec 09, 2021 at 11:08:53PM +, Davide Cavalca via devel wrote: > - you could use a LSM to enforce that exec() can only happen on files > with valid fs-verity signatures; this would protect any binary > - you could use a launcher booted from secure storage (say, a dm-verity Is there some LSM module like this ready for use? Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
> * at run time, if the fsverity rpm plugin is enabled, rpm will install > the fsverity signature key and enable fsverity on files that are > installed. This requires CONFIG_FS_VERITY_BUILTIN_SIGNATURES=y. Currently our kernels are built without that. It seems like a simple addition (the amount of code guarded by this is very small), but this should be added to the Scope. > In the context of rpm, there are two parts to this: > * at build time, we compute the Merkle tree for the files within a > package, then sign it and ship it as part of the rpm metadata; There was already some chatter about signatures and verification in the thread, but I didn't see a definite answer, and I think this is the part that is currently the most underspecified. In Fedora, we use a new package signing key for each Fedora release. What key would be used for the fs-verity signatures: the same key, a separate key? Edit: I see that the Change page says a dedicated key is used. OK, so is this key rotated one per release or some different schedule? How would the public part of the key be delivered to users? How does one actually enable signature verification? Let's say I have a kernel with CONFIG_FS_VERITY_BUILTIN_SIGNATURES=y and rpm-plugin-fsverity installed, and I installed some rpms, and the files from those rpms now have now had the FS_IOC_ENABLE_VERITY ioctl done. IIUC, I need to do some steps at each boot: 1. add all the keys to the keyring 2. set sysctl fs.verity.require_signatures=1 So… in 1., do I always load all the keys that Fedora has used for this purpose, in case there are still some files on disk? Or is there some mechanism to say that e.g. keys older than F(N-2) are now not necessary? Who does 2.? I think it'd be hard to enable this during a system upgrade: one would need to reinstall all rpms (with new rpms carrying the fsverity metadata) or some files become unreadable once 2. is done. This brings a question: is there some rpm virtual Provides/Requires to specify that the fs-verity stuff is present? I assume the user would want to triple-check that they don't have any rpms without the metadata before enabling verification. Zbyszek P.S. Those questions are about the mechanism, not policy or the threat model. I have some questions about that, but I'll ask separately, since this mail is lengthy already. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Fri, Dec 03, 2021 at 05:31:21PM -, Boris Burkov via devel wrote: > The top-level hash is calculated for each file, then that hash is signed with > the inputted rsa key pair and the signed hash is appended to the array of > signed hashes in the rpm metadata. I am guessing the way we worded the > proposal is a little unclear because we call it "the" signature when it's one > rpm metadata item that's an array of the signatures. > > fs-verity the kernel feature operates on a per-file basis, and since the > ultimate goal is to deliver fs-verity enabled files on the installer's > system, we need each file's signature in the rpm. At install, we call the > fs-verity enable ioctl for each file, passing in its signature to make use of > the kernel authentication functionality. What exactly is appended to the rpm: a) the merkle tree b) the top-level hash from the merkle tree c) the signature for b d) some combination of a, b, c? Above you seem to say "b+c" ("signed hash"). In the Change page: > at build time, we compute the Merkle tree for the files within a > package, then sign it and ship it as part of the rpm metadata; …which is "a". "c" would make sense to me. Please clarify. Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Thu, 2 Dec 2021 at 14:37, Ben Cotton wrote: > > https://fedoraproject.org/wiki/Changes/FsVerityRPM > > == Summary == > > Enable the use of fsverity for installed RPM files validation. > > == Owners == > > * Name: [[User:Dcavalca|Davide Cavalca]], [[User:Borisb|Boris > Burkov]], [[User:Filbranden|Filipe Brandenburger]], > [[User:Salimma|Michel Alexandre Salim]], [[User:Malmond|Matthew > Almond]] > * Email: dcava...@fb.com, bor...@fb.com, filbran...@fb.com, > mic...@fb.com, malm...@fb.com > ** koji integration: koji will need to add the fs-verity metadata to > packages when signing them I am going to say that this is a bigger sticking point than people are realising and this is not a F36 but a F37 change. The changes to koji, signing infrastructure and testing have to be put in place before January 19 2022 (2022-01-19) for the mass rebuild[1]. They need to have been in staging and testing before then so that bugs could be worked out and various groups could work things out. Due to various end of year holidays and 'use your PTO before its lost' schedules, many of the developers who could sign off on the changes to koji/bodhi/whatever-testing/sigul/etc depending on the changes needed are not going to be available. Even if the changes are trivial ones, this is a very short window to land things. At best for F36, I could see the changes getting put into Fedora before the beta and some packages signed. If you want this for F37, then all that work needs to be ready and done by (2022-07-20)[2]. [1] https://fedorapeople.org/groups/schedule/f-36/f-36-key-tasks.html [2] https://fedorapeople.org/groups/schedule/f-37/f-37-key-tasks.html -- Stephen J Smoogen. Let us be kind to one another, for most of us are fighting a hard battle. -- Ian MacClaren ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
Dne 10. 12. 21 v 0:08 Davide Cavalca via devel napsal(a): On Fri, 2021-12-03 at 22:08 +, Richard W.M. Jones wrote: I'm unclear about the threat model - this is an attacker who is someone able to overwrite single files (eg. /bin/ls) but cannot turn off the fs-verity system as a whole? Also if RPM can update /bin/ls then surely an attacker who can widely compromise system files must also be able to update /bin/ls in the same way? Once fs-verity is enabled for a given file (which, in the RPM case, happens at package installation time), it cannot be disabled, and the file becomes immutable. One can still rename() or unlink() it (and this is indeed how rpm is able to replace files when upgrading packages), but the actual contents cannot be altered. Trying to debug some issue in shell/ruby/python script, will it be possible to modify such file? Vít OpenPGP_signature Description: OpenPGP digital signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
> On Do, 02.12.21 14:36, Ben Cotton (bcotton(a)redhat.com) wrote: > > Hmm, so what I am really missing on the feature page: what's the > attack scenario here? Usually security features come with an attack > scenario they are supposed to address. But there's no discussion about > that. > Good point, we were skimpy on that. In case you missed it, Davide just laid it out in his reply to the same question from Richard Jones higher in the thread. The short answer is that just this change as-is has no practical threat model, we viewed it as an enablement step that we believe has a realistic path to broadly authenticated rpm contents. Alternatively, it does give a sophisticated user an opportunity to build something like that for themselves as well. > This protects file contents, not the metada, right? So what about the > metadata? if I see a fs-verity enabled inode with libssl.so data in it > today, and it's a vulnerably version, and I make a hardlink to it, and > then it gets replaced by a fixed version (with a slightly updated > name) — how you intend to make sure, that i can't fool you into loading > my copy of the old file but under the new name? > If the file is still properly signed with a trusted matching certificate in the kernel keyring we can't/won't do anything about it. > What about signing anyway? What is this precisely signed with? The > keys for that when are they rolled over? Who manages those keys? The > tea for that, who's doing that in Fedora? Is there any protection > against downgrades between RPM package versions? Does this in any way > protect combinations of binaries/libraries? I mean, pretty much all > programs we ship consist of a large number of ELF objects, and you > probably need to sign the combination of them, but this model doesn't > look like it offers that at all? If a security bug is found in library > X, how is it taken out of equation? What would the workflow for that > look like on the Fedora side of things? > I don't have a particularly specific answer to all of these questions, unfortunately. Perhaps someone with more knowledge about Fedora's existing key management can weigh in. But imagining we have the LSM hooks setup so that we never load anything executable without it being signed by a trusted key, then in my mind, banishing a bad shared library would require revoking the certificate. I think one could draw a reasonable parallel between package signing and file signing here, though. Users assemble meaningful system functionality out of a collection of packages, which are each individually signed. One of those packages could then be found to be compromised, and need to be removed, otherwise a user could download and install a perfectly valid signed package and have a bad time, which undermines the value of the signing. Is there something about composing executables out of files that is meaningfully different in this sense from composing systems out of packages? > The security model seems really strange to me I must say: so much > infrastructure, and it doesn't protect at all how our programs are > usually combined? > > Puzzled, > > Lennart ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Sat, 2021-12-04 at 23:46 +0100, Kevin Kofler via devel wrote: > Davide Cavalca via devel wrote: > > To clarify: RPM does support files validation, but fs-verity is > > more > > than just that. With RPM, the validation only happens on install > > time, > > and when one runs rpm -V manually. With fs-verity, the validation > > happens on-demand whenever a block of a file that originated from > > an > > RPM is accessed. This means, for example, that if an attacker > > replaces > > /bin/ls on disk with a compromised one, the next time it's read > > from > > disk (e.g. because you ran it) you will see a validation failure > > and > > the syscall will be blocked, preventing the compromised code from > > being > > executed. > > This means that there is a performance cost in addition to the disk > space > cost (because something has to compute those checksums each time the > file is > acessed). There's only a performance cost if fs-verity is actually enabled (which is not in scope for this proposal). The checksums are computed on- demand on a per-block basis, so you only end up checksumming the pages you're actually accessing. > It also means that it is harder for users to exercise their right > to modify the Free Software (because replacing or patching RPM- > installed > binaries will lead to them failing to execute). There's nothing stopping the user from loading their own key in the kernel keyring and then installing their own locally-build RPM that has been verity-signed with their own key. And again, this only becomes a concern if one is actually enabling fs-verity in the first place. > > Since the change also adds to the metadata in the RPM, that means > that it > also increases the size of the RPMs. With keepcache=1, this also > translates > to increased disk space use. But even if the user does not keep > cached RPMs, > the download sizes will increase, which can cost time and for some > users > even money. That's correct, there will be (modest) increase in the size of the RPMs. We're going to collect some more data to quantify this more concretely. Cheers Davide ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
> If I enable FS-verity and later find that I need to patch a file to fix > some problem, how do I as the sysadmin tell Linux that this change is > authorized? Do I disable FS-verity for that specific file? Disable > FS-verity globally? Add my own key to the kernel's keyring? Build and > sign my own RPM package? > > What prevents an attacker from doing the same? I think this is a good, fair point and is a serious tradeoff in authenticating distributed files. However, I believe it should be possible for the user to securely configure a keypair and load the certificate in the fs-verity keyring s.t. they can sign the files they craft themselves, without allowing an attacker to, just like they would to normally sign things. So you could copy the file, modify it to your liking, (or just rebuild the rpm locally) then enable verity with your own signature. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Sat, 2021-12-04 at 09:37 -0500, Stephen John Smoogen wrote: > > Or just pad /usr/bin/rpm with some null characters at the end to break > its signature and also stop updates from happening. [Or the fs-verity > daemon which will report that these problems are occuring. ] If the attacker has filesystem access, this wouldn't work, as fs-verity makes the file immutable. If they have block level access to the underlying storage, they can alter the data blocks of the rpm binary, and that would indeed result in failure on the next exec as the signature wouldn't match. Cheers Davide ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Fri, 2021-12-03 at 22:08 +, Richard W.M. Jones wrote: > I'm unclear about the threat model - this is an attacker who is > someone able to overwrite single files (eg. /bin/ls) but cannot turn > off the fs-verity system as a whole? > > Also if RPM can update /bin/ls then surely an attacker who can widely > compromise system files must also be able to update /bin/ls in the > same way? Once fs-verity is enabled for a given file (which, in the RPM case, happens at package installation time), it cannot be disabled, and the file becomes immutable. One can still rename() or unlink() it (and this is indeed how rpm is able to replace files when upgrading packages), but the actual contents cannot be altered. Where is this useful? For example, fs-verity can help in the scenario where an attacker has out-of-band access to the storage device (say, they pull a hard drive from a colo'd server or a sdcard from an embedded device, or they boot into a liveusb, or they access a VM image directly from a host). Let's say that happens, and the attacker changes a few blocks of /bin/ls on the device to make it run nefarious code. When you boot your system again, it would fail at exec() time because the Merkle tree wouldn't match. Let's say that instead the attacker mounts (or gains access to) your filesystem, unlinks /bin/ls and replaces it wholesale with a new copy (hence creating a new inode). The attacker doesn't have your signing key, so they can't resign the file and enable fs-verity on it (they could resign it with their own key, but unless they can then find a way to load its cert into the kernel keyring it won't do much good). To protect against this, you now have a few options: - you could use a LSM to enforce that exec() can only happen on files with valid fs-verity signatures; this would protect any binary - you could use a launcher booted from secure storage (say, a dm-verity volume, which could even be the initrd), and have this launcher perform the verification; this of course only protects against binaries executed from the launcher, but depending on your threat model it might be enough Like most security solutions, this isn’t a silver bullet and it’s not something that in and of itself would necessarily prevent all possible attacks. However, fs-verity can be a useful building block in a defense-in-depth approach against specific attacks, depending on your threat model. Cheers Davide ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
You could reinstall the rpm, the fs will reclaim the verity metadata along with the rest of the old file. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Do, 02.12.21 14:36, Ben Cotton (bcot...@redhat.com) wrote: Hmm, so what I am really missing on the feature page: what's the attack scenario here? Usually security features come with an attack scenario they are supposed to address. But there's no discussion about that. This protects file contents, not the metada, right? So what about the metadata? if I see a fs-verity enabled inode with libssl.so data in it today, and it's a vulnerably version, and I make a hardlink to it, and then it gets replaced by a fixed version (with a slightly updated name) — how you intend to make sure, that i can't fool you into loading my copy of the old file but under the new name? What about signing anyway? What is this precisely signed with? The keys for that when are they rolled over? Who manages those keys? The tea for that, who's doing that in Fedora? Is there any protection against downgrades between RPM package versions? Does this in any way protect combinations of binaries/libraries? I mean, pretty much all programs we ship consist of a large number of ELF objects, and you probably need to sign the combination of them, but this model doesn't look like it offers that at all? If a security bug is found in library X, how is it taken out of equation? What would the workflow for that look like on the Fedora side of things? The security model seems really strange to me I must say: so much infrastructure, and it doesn't protect at all how our programs are usually combined? Puzzled, Lennart ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Thu, Dec 02, 2021 at 02:36:51PM -0500, Ben Cotton wrote: > Enable the use of fsverity for installed RPM files validation. Can we use this to validate the install media at runtime rather than as a separate boot step? -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
Davide Cavalca via devel wrote: > To clarify: RPM does support files validation, but fs-verity is more > than just that. With RPM, the validation only happens on install time, > and when one runs rpm -V manually. With fs-verity, the validation > happens on-demand whenever a block of a file that originated from an > RPM is accessed. This means, for example, that if an attacker replaces > /bin/ls on disk with a compromised one, the next time it's read from > disk (e.g. because you ran it) you will see a validation failure and > the syscall will be blocked, preventing the compromised code from being > executed. This means that there is a performance cost in addition to the disk space cost (because something has to compute those checksums each time the file is acessed). It also means that it is harder for users to exercise their right to modify the Free Software (because replacing or patching RPM-installed binaries will lead to them failing to execute). > About filesystem usage: unless you install rpm-plugin-fsverity (which > is not and will not be installed by default), there is no disk space > increase for verity-signed RPM packages. If you do install rpm-plugin- > fsverity, some disk space will be used for the Merkle tree as described > in the Change. Since the change also adds to the metadata in the RPM, that means that it also increases the size of the RPMs. With keepcache=1, this also translates to increased disk space use. But even if the user does not keep cached RPMs, the download sizes will increase, which can cost time and for some users even money. Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
Vitaly Zaitsev via devel wrote: > -1. RPM already supports files validation and this feature will waste > file system space. I agree with you. This is yet another "feature" that increases both the size of RPMs and, if enabled by default, the size of default installations. We need to stop tolerating creeping bloat! So -1 to the Change proposal. Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Fri, 3 Dec 2021 at 17:09, Richard W.M. Jones wrote: > > On Fri, Dec 03, 2021 at 06:08:49PM +, Davide Cavalca via devel wrote: > > Broadly speaking, fs-verity makes it possible to ensure that files that > > were installed via an RPM have not been modified. It is useful in > > environments where an attacker might be able to modify system files > > (say, replace /bin/ls with a compromised version) and you want to > > protect against that. For example, consider an appliance-like system > > placed in an untrusted location where you may not be able to control > > who has physical access (this could be a server, but it could also be a > > kiosk in an internet point or a school). In this scenario, fs-verity > > can be one of the building blocks to ensure and maintain system trust. > > I'm unclear about the threat model - this is an attacker who is > someone able to overwrite single files (eg. /bin/ls) but cannot turn > off the fs-verity system as a whole? > > Also if RPM can update /bin/ls then surely an attacker who can widely > compromise system files must also be able to update /bin/ls in the > same way? > Or just pad /usr/bin/rpm with some null characters at the end to break its signature and also stop updates from happening. [Or the fs-verity daemon which will report that these problems are occuring. ] -- Stephen J Smoogen. Let us be kind to one another, for most of us are fighting a hard battle. -- Ian MacClaren ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Friday, 03 December 2021 at 18:49, Davide Cavalca via devel wrote: [...] > About filesystem usage: unless you install rpm-plugin-fsverity (which > is not and will not be installed by default), there is no disk space > increase for verity-signed RPM packages. If you do install rpm-plugin- > fsverity, some disk space will be used for the Merkle tree as described > in the Change. How do I reclaim that space after removing rpm-plugin-fsverity? Regards, Dominik -- Fedora https://getfedora.org | RPM Fusion http://rpmfusion.org There should be a science of discontent. People need hard times and oppression to develop psychic muscles. -- from "Collected Sayings of Muad'Dib" by the Princess Irulan ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
> * at build time, we compute the Merkle tree for the files within a > package, then sign it and ship it as part of the rpm metadata; [...] > Note that the Merkle tree > is ''not'' shipped with the RPM itself (only its signature is) In that case, "ship it" above should be changed to "ship the signature", unless this is some distinction between "the RPM metadata" and "the RPM itself". If I enable FS-verity and later find that I need to patch a file to fix some problem, how do I as the sysadmin tell Linux that this change is authorized? Do I disable FS-verity for that specific file? Disable FS-verity globally? Add my own key to the kernel's keyring? Build and sign my own RPM package? What prevents an attacker from doing the same? Will files under /etc be covered, or will local configuration still be possible? Björn Persson pgpbMIhTbl4rJ.pgp Description: OpenPGP digital signatur ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Fri, Dec 03, 2021 at 06:08:49PM +, Davide Cavalca via devel wrote: > Broadly speaking, fs-verity makes it possible to ensure that files that > were installed via an RPM have not been modified. It is useful in > environments where an attacker might be able to modify system files > (say, replace /bin/ls with a compromised version) and you want to > protect against that. For example, consider an appliance-like system > placed in an untrusted location where you may not be able to control > who has physical access (this could be a server, but it could also be a > kiosk in an internet point or a school). In this scenario, fs-verity > can be one of the building blocks to ensure and maintain system trust. I'm unclear about the threat model - this is an attacker who is someone able to overwrite single files (eg. /bin/ls) but cannot turn off the fs-verity system as a whole? Also if RPM can update /bin/ls then surely an attacker who can widely compromise system files must also be able to update /bin/ls in the same way? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Fri, Dec 3, 2021 at 1:15 PM Davide Cavalca wrote: > > On Thu, 2021-12-02 at 19:10 -0500, Josh Boyer wrote: > > On Thu, Dec 2, 2021, 5:33 PM Davide Cavalca via devel > > wrote: > > > > > Correct, XFS doesn't support fs-verity at the moment (though it > > > could > > > be implemented if one wanted to). > > > > That means it would exclude Fedora Server and ELN as they are XFS > > based. > > There isn't any impact on XFS-based editions by default. Building and > signing RPMs with fs-verity works fine, as there's no filesystem > dependency for that. Installing RPMs also works fine, with one caveat: > if you're running a system with an unsupported filesystem and have rpm- > plugin-fsverity (which is not installed by default), fs-verity will not > be enabled for the installed files so there will be no verification > (but the RPM installation itself should succeed). Yes, thank you. I meant it would render those Editions/projects unable to use this Change. It's not the end of the world, just an observation. josh ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
Errors at installation time should be fully diagnosable, and even if the output today doesn't make it totally obvious what happened, it would be easy to fix in rpm. The errors post-install are a bit trickier. Imagine you install your rpm, and kick off some long running daemon from it. A month later, a block gets corrupted in a way fs checksums don't catch (e.g. ext4, btrfs nodatasum, evil maid), and suddenly that daemon receives a SIGBUS and crashes. You would be able to see clearly that it was a verity issue in dmesg, but I don't think the binary could reasonably know what happened or write a meaningful log. In that sense, I think it's actually pretty similar to the experience if you have corruption in your disk and start getting btrfs checksum errors on a file--you'd have to look in dmesg to know why your file is broken. The middle ground is when opening/exec-ing the file fails. In that case, you might get a sufficiently specific error code you could figure out it's verity, and the full error would be in dmesg as well. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
> I think there are two cases of interest: > > 1) a file or signature in the rpm is corrupted, the signature doesn't have a > matching > cert installed, etc... > in this case, if the plugin is present, when you attempt to install the rpm > the verity > enable ioctl will explicitly fail, and presumably so will the rpm install. > You can see > most of the details for this sort of case here in the rpm plugin code: > https://github.com/rpm-software-management/rpm/blob/master/plugins/fsveri > > 2) after installation, a file from an fs-verity enabled rpm gets one or more > blocks > corrupted > The first read of a corrupted block from disk (the good uncorrupted page > might survive in > page cache for a while) will result in EIO for read-like system calls and > SIGBUS if the > file is mapped (executables, mmap). Thanks for the information. So in the event of an error, it's expected that the user would be informed that the error was due to a rpm-fsverity check and they could potentially reinstall the rpm from a trusted source to resolve the problem? Of course there's the underlying issue of _why_ it happened, but from the standpoint of wanting an immediate path forward. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
I omitted one more interesting case. If the verity metadata (signature, root hash) is corrupted after installation but before the file is opened, then opening/exec-ing the file can fail. Also, if pages from a binary read in during the exec itself are corrupted, the system call itself could fail (rather than the process getting sigbus like for a random page during execution) ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Thu, 2021-12-02 at 19:10 -0500, Josh Boyer wrote: > On Thu, Dec 2, 2021, 5:33 PM Davide Cavalca via devel > wrote: > > > Correct, XFS doesn't support fs-verity at the moment (though it > > could > > be implemented if one wanted to). > > That means it would exclude Fedora Server and ELN as they are XFS > based. There isn't any impact on XFS-based editions by default. Building and signing RPMs with fs-verity works fine, as there's no filesystem dependency for that. Installing RPMs also works fine, with one caveat: if you're running a system with an unsupported filesystem and have rpm- plugin-fsverity (which is not installed by default), fs-verity will not be enabled for the installed files so there will be no verification (but the RPM installation itself should succeed). Cheers Davide ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Thu, 2021-12-02 at 20:05 -0500, Josh Boyer wrote: > Yes, I saw that and I appreciate it. That's a comparison between the > two implementations. I am asking about what benefits and use cases > fs-verity solves in Fedora. Right now, the change simply says: > > "The main benefit is the ability to do block-level verification of > RPM-installed files. In turn, this can be used to implement > usecase-specific validation and verification policies depending on > the > environment requirements." > > which is also largely true of IMA. The IMA change went into more > detailed use cases, which perhaps may have been it's downfall. So > can > you describe what most Fedora users would use this for or benefit > from > it? Or if "most users" is not an applicable qualifier, can you at > least give some more detailed use cases that you would expect people > to use it for? Broadly speaking, fs-verity makes it possible to ensure that files that were installed via an RPM have not been modified. It is useful in environments where an attacker might be able to modify system files (say, replace /bin/ls with a compromised version) and you want to protect against that. For example, consider an appliance-like system placed in an untrusted location where you may not be able to control who has physical access (this could be a server, but it could also be a kiosk in an internet point or a school). In this scenario, fs-verity can be one of the building blocks to ensure and maintain system trust. This Change is mostly about putting in place the necessary plumbing for this to be at all possible. We'll try to expand the Change proposal and flesh out potential usecases a bit more. > OK. I guess I was looking for some side-by-side data comparisons in > the overhead between IMA metadata and fs-verity. "1/127th of the > original Merkel tree size" doesn't tell me much. > > Are there some test runs with numbers to show before/after data for > both the RPM size and installed FS usage? Perhaps with an example > install. The IMA change attempted to document this and seeing a 1.1% > average increase in RPM size was easier to understand. We've done some empirical testing on this (showing neglibible increases), but still need to gather more formal data. We'll try to prioritize that and add it to the Change once it's available. Cheers Davide ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
I think there are two cases of interest: 1) a file or signature in the rpm is corrupted, the signature doesn't have a matching cert installed, etc... in this case, if the plugin is present, when you attempt to install the rpm the verity enable ioctl will explicitly fail, and presumably so will the rpm install. You can see most of the details for this sort of case here in the rpm plugin code: https://github.com/rpm-software-management/rpm/blob/master/plugins/fsverity.c#L114. 2) after installation, a file from an fs-verity enabled rpm gets one or more blocks corrupted The first read of a corrupted block from disk (the good uncorrupted page might survive in page cache for a while) will result in EIO for read-like system calls and SIGBUS if the file is mapped (executables, mmap). ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Fri, 2021-12-03 at 12:21 +0100, Vitaly Zaitsev via devel wrote: > On 02/12/2021 20:36, Ben Cotton wrote: > > Enable the use of fsverity for installed RPM files validation. > > -1. RPM already supports files validation and this feature will waste > file system space. To clarify: RPM does support files validation, but fs-verity is more than just that. With RPM, the validation only happens on install time, and when one runs rpm -V manually. With fs-verity, the validation happens on-demand whenever a block of a file that originated from an RPM is accessed. This means, for example, that if an attacker replaces /bin/ls on disk with a compromised one, the next time it's read from disk (e.g. because you ran it) you will see a validation failure and the syscall will be blocked, preventing the compromised code from being executed. About filesystem usage: unless you install rpm-plugin-fsverity (which is not and will not be installed by default), there is no disk space increase for verity-signed RPM packages. If you do install rpm-plugin- fsverity, some disk space will be used for the Merkle tree as described in the Change. Cheers Davide ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
The top-level hash is calculated for each file, then that hash is signed with the inputted rsa key pair and the signed hash is appended to the array of signed hashes in the rpm metadata. I am guessing the way we worded the proposal is a little unclear because we call it "the" signature when it's one rpm metadata item that's an array of the signatures. fs-verity the kernel feature operates on a per-file basis, and since the ultimate goal is to deliver fs-verity enabled files on the installer's system, we need each file's signature in the rpm. At install, we call the fs-verity enable ioctl for each file, passing in its signature to make use of the kernel authentication functionality. I'm happy to explain the exact flow at signing and at install in more detail if that would be helpful. Boris ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
Perhaps I glossed over it in the description, but what is the expected user experience in the event of a RPM fs-verity mismatch/error? ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On 02/12/2021 20:36, Ben Cotton wrote: Enable the use of fsverity for installed RPM files validation. -1. RPM already supports files validation and this feature will waste file system space. -- Sincerely, Vitaly Zaitsev (vit...@easycoding.org) ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
> The signature size is roughly proportional to the number of files in > the package. Can you explain how the signature is performed? I assume that the verity top-level hash is calculated for each file and then … ? And if you have all the per-file hashes, why not do one more level of Merkle and calculate a single hash for all the files in the rpm and sign that (so that the signature overhead is of fixed size)? Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Thu, Dec 2, 2021 at 7:27 PM Michel Alexandre Salim wrote: > > Hello, > > On Thu, Dec 02, 2021 at 07:10:32PM -0500, Josh Boyer wrote: > > On Thu, Dec 2, 2021, 5:33 PM Davide Cavalca via devel < > > devel@lists.fedoraproject.org> wrote: > > > > > On Thu, 2021-12-02 at 13:09 -0800, Kevin Fenzi wrote: > > > > On Thu, Dec 02, 2021 at 02:36:51PM -0500, Ben Cotton wrote: > > > > ...snip... > > > > > > > > > > In the context of rpm, there are two parts to this: > > > > > * at build time, we compute the Merkle tree for the files within a > > > > > package, then sign it and ship it as part of the rpm metadata; > > > > > > > > This is some kind of seperate signing that happens at build time? > > > > > > > > Or it's added to the rpm metadata and covered by the normal package > > > > signing if/when the package is signed? > > > > > > As part of the signing flow (e.g. via rpmsign), the Merkle tree is > > > generated and a signature is computed from it, which is then added to > > > the rpm metadata. > > > > > > > IMA received significant pushback on the impact to RPMs and signing > > implications in Fedora. How does fs-verity compare here? > We wrote up a comparison here: > https://fedoraproject.org/wiki/Changes/FsVerityRPM#Relationship_with_IMA Yes, I saw that and I appreciate it. That's a comparison between the two implementations. I am asking about what benefits and use cases fs-verity solves in Fedora. Right now, the change simply says: "The main benefit is the ability to do block-level verification of RPM-installed files. In turn, this can be used to implement usecase-specific validation and verification policies depending on the environment requirements." which is also largely true of IMA. The IMA change went into more detailed use cases, which perhaps may have been it's downfall. So can you describe what most Fedora users would use this for or benefit from it? Or if "most users" is not an applicable qualifier, can you at least give some more detailed use cases that you would expect people to use it for? > > Alternatively/additionally, why is fs-verity worth the hit for Fedora where > > IMA was not. > > The hit is much smaller; per > https://fedoraproject.org/wiki/Changes/FsVerityRPM#Merkle_tree_cost > - if the plugin is installed, the Merkle tree is stored but it's 1/127th > the original file size > - the RPM only ships the signature, not the tree itself; per > https://fedoraproject.org/wiki/Changes/FsVerityRPM#Signature_overhead_cost > in practice we see a minimal to no increase in the size of the RPM > > So as proposed in this Change, users can opt in by installing the > plugin, otherwise they will be mostly unimpacted. OK. I guess I was looking for some side-by-side data comparisons in the overhead between IMA metadata and fs-verity. "1/127th of the original Merkel tree size" doesn't tell me much. Are there some test runs with numbers to show before/after data for both the RPM size and installed FS usage? Perhaps with an example install. The IMA change attempted to document this and seeing a 1.1% average increase in RPM size was easier to understand. josh > As discussed in the write-up - IMA does have a richer policy system, and > could potentially be integrated (so we use IMA but with a fsverity > backend) to get the best of both worlds. > > Best regards, > > -- > Michel Alexandre Salim > profile: https://keyoxide.org/mic...@michel-slm.name ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On 12/2/21 6:46 PM, Michael Cronenworth wrote: Could this be directly added to rpm instead of an external tool set? I see you did. It helps to read the Change link... Sorry. :) ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On 12/2/21 4:32 PM, Davide Cavalca via devel wrote: There's support in robosignatory to ask to sign files (used for the short lived IMA stuff), but I suspect it would need a new ability for this. Finally who is going to write this? Change owners? Or do you expect robosignatory maintainers to do so? Thanks for clarifying! Yes, I think robosignatory is likely what we want here. We (the Change owners) expect to do the work, though we'll likely need some advice/help around code review, testing and deployment. Could this be directly added to rpm instead of an external tool set? rpm --addsign ./$RPM to rpm --addsign --addfsverity ./$RPM I'm thinking about those who might also want to use it... end-users and third-party repos such as RPMFusion. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
Hello, On Thu, Dec 02, 2021 at 07:10:32PM -0500, Josh Boyer wrote: > On Thu, Dec 2, 2021, 5:33 PM Davide Cavalca via devel < > devel@lists.fedoraproject.org> wrote: > > > On Thu, 2021-12-02 at 13:09 -0800, Kevin Fenzi wrote: > > > On Thu, Dec 02, 2021 at 02:36:51PM -0500, Ben Cotton wrote: > > > ...snip... > > > > > > > > In the context of rpm, there are two parts to this: > > > > * at build time, we compute the Merkle tree for the files within a > > > > package, then sign it and ship it as part of the rpm metadata; > > > > > > This is some kind of seperate signing that happens at build time? > > > > > > Or it's added to the rpm metadata and covered by the normal package > > > signing if/when the package is signed? > > > > As part of the signing flow (e.g. via rpmsign), the Merkle tree is > > generated and a signature is computed from it, which is then added to > > the rpm metadata. > > > > IMA received significant pushback on the impact to RPMs and signing > implications in Fedora. How does fs-verity compare here? We wrote up a comparison here: https://fedoraproject.org/wiki/Changes/FsVerityRPM#Relationship_with_IMA > Alternatively/additionally, why is fs-verity worth the hit for Fedora where > IMA was not. The hit is much smaller; per https://fedoraproject.org/wiki/Changes/FsVerityRPM#Merkle_tree_cost - if the plugin is installed, the Merkle tree is stored but it's 1/127th the original file size - the RPM only ships the signature, not the tree itself; per https://fedoraproject.org/wiki/Changes/FsVerityRPM#Signature_overhead_cost in practice we see a minimal to no increase in the size of the RPM So as proposed in this Change, users can opt in by installing the plugin, otherwise they will be mostly unimpacted. As discussed in the write-up - IMA does have a richer policy system, and could potentially be integrated (so we use IMA but with a fsverity backend) to get the best of both worlds. Best regards, -- Michel Alexandre Salim profile: https://keyoxide.org/mic...@michel-slm.name signature.asc Description: PGP signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Thu, Dec 2, 2021, 5:33 PM Davide Cavalca via devel < devel@lists.fedoraproject.org> wrote: > On Thu, 2021-12-02 at 13:09 -0800, Kevin Fenzi wrote: > > On Thu, Dec 02, 2021 at 02:36:51PM -0500, Ben Cotton wrote: > > ...snip... > > > > > > In the context of rpm, there are two parts to this: > > > * at build time, we compute the Merkle tree for the files within a > > > package, then sign it and ship it as part of the rpm metadata; > > > > This is some kind of seperate signing that happens at build time? > > > > Or it's added to the rpm metadata and covered by the normal package > > signing if/when the package is signed? > > As part of the signing flow (e.g. via rpmsign), the Merkle tree is > generated and a signature is computed from it, which is then added to > the rpm metadata. > IMA received significant pushback on the impact to RPMs and signing implications in Fedora. How does fs-verity compare here? Alternatively/additionally, why is fs-verity worth the hit for Fedora where IMA was not. I'm not looking to create a false equivalence or compare apples to oranges, but I want to better understand why this is a more acceptable approach for Fedora. As an aside, a few people have asked why RHEL didn't get IMA into Fedora first. Frankly, we tried and got feedback that it didn't apply to Fedora's needs and use cases. As of right now, we have no plans to force something into Fedora that the community doesn't want. If fs-verity is more acceptable or is described in a way that is better understood with enough similarities to IMA, we may try again. > > * at run time, if the fsverity rpm plugin is enabled, rpm will > > > install > > > the fsverity signature key and enable fsverity on files that are > > > installed. > > > > Is this signature key the fedora rpm package signing key? > > Or some seperate fsverity key? > > fs-verity needs a RSA key/cert pair for file signing at package > signature time. At package install time, the cert needs to be loaded in > the appropriate kernel keyring. We've always used a dedicated keypair > during testing -- I'm not actually sure if the package signing key > could be reused for this, as it's a GPG key, but this is something we > should follow up on. > > > > fs-verity works by using a Merkle tree to generate a checksum for > > > every data block in the system, and reads will fail if a single > > > data > > > > every data block? or every packaged in rpms data block? > > fs-verity only operates on files where it has been enabled via its > ioctl (which, if you install the RPM plugin, is taken care of by RPM on > your behalf). For those, fs-verity will checksum every data block > whenever it's accessed and validate it still matches. > > > > block read fails it’s checksum. The signature of the the file is > > > validated against a public key loaded into the kernel keyring. > > > Because > > > fsverity operates on block reads, its runtime cost is small (as it > > > only needs to verify the block that is being accessed), and it can > > > protect from alterations at any point in time. > > > > Is this via dm_verity kernel module? Or thats a completely seperate > > thing? > > It's somewhat inspired by dm-verity, but it's a separate > implementation, the only shared logic is the hash computation code in > the kernel. > > > > == Scope == > > > > > > * Proposal owners > > > ** btrfs kernel enablement work > > > ([ > > > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=146054090b0859b28fc39015c7704ccc3c3a347f > > > landed in 5.15]); see this > > > [ > > > > https://developers.facebook.com/blog/post/2021/10/19/fs-verity-support-in-btrfs/ > > > blog post] for more details > > > > What does this mean for all the other filesystems? They would just be > > slower since btrfs is computing the trees as part of it's normal > > checksumming? > > fs-verity requires support in the underlying filesystem. If you're > using a filesystem that doesn't support it and attempt to enable fs- > verity on a file, the ioctl will fail. Note that this is only a concern > at runtime, not at build time. > > > > ** koji integration: koji will need to add the fs-verity metadata > > > to > > > packages when signing them > > > > Well, koji doesn't sign packages. robosignatory listens for messages > > on > > the message bus for koji tagging and based on it's config, it asks > > sigul > > to sign rpms and import the signatures into koji. > > > > There's support in robosignatory to ask to sign files (used for the > > short lived IMA stuff), but I suspect it would need a new ability for > > this. > > > > Finally who is going to write this? Change owners? > > Or do you expect robosignatory maintainers to do so? > > Thanks for clarifying! Yes, I think robosignatory is likely what we > want here. We (the Change owners) expect to do the work, though we'll > likely need some advice/help around code review, testing and > deployment. > > > > * fs-verity requires filesystem support; currently suppor
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Thu, 2021-12-02 at 13:09 -0800, Kevin Fenzi wrote: > On Thu, Dec 02, 2021 at 02:36:51PM -0500, Ben Cotton wrote: > ...snip... > > > > In the context of rpm, there are two parts to this: > > * at build time, we compute the Merkle tree for the files within a > > package, then sign it and ship it as part of the rpm metadata; > > This is some kind of seperate signing that happens at build time? > > Or it's added to the rpm metadata and covered by the normal package > signing if/when the package is signed? As part of the signing flow (e.g. via rpmsign), the Merkle tree is generated and a signature is computed from it, which is then added to the rpm metadata. > > * at run time, if the fsverity rpm plugin is enabled, rpm will > > install > > the fsverity signature key and enable fsverity on files that are > > installed. > > Is this signature key the fedora rpm package signing key? > Or some seperate fsverity key? fs-verity needs a RSA key/cert pair for file signing at package signature time. At package install time, the cert needs to be loaded in the appropriate kernel keyring. We've always used a dedicated keypair during testing -- I'm not actually sure if the package signing key could be reused for this, as it's a GPG key, but this is something we should follow up on. > > fs-verity works by using a Merkle tree to generate a checksum for > > every data block in the system, and reads will fail if a single > > data > > every data block? or every packaged in rpms data block? fs-verity only operates on files where it has been enabled via its ioctl (which, if you install the RPM plugin, is taken care of by RPM on your behalf). For those, fs-verity will checksum every data block whenever it's accessed and validate it still matches. > > block read fails it’s checksum. The signature of the the file is > > validated against a public key loaded into the kernel keyring. > > Because > > fsverity operates on block reads, its runtime cost is small (as it > > only needs to verify the block that is being accessed), and it can > > protect from alterations at any point in time. > > Is this via dm_verity kernel module? Or thats a completely seperate > thing? It's somewhat inspired by dm-verity, but it's a separate implementation, the only shared logic is the hash computation code in the kernel. > > == Scope == > > > > * Proposal owners > > ** btrfs kernel enablement work > > ([ > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=146054090b0859b28fc39015c7704ccc3c3a347f > > landed in 5.15]); see this > > [ > > https://developers.facebook.com/blog/post/2021/10/19/fs-verity-support-in-btrfs/ > > blog post] for more details > > What does this mean for all the other filesystems? They would just be > slower since btrfs is computing the trees as part of it's normal > checksumming? fs-verity requires support in the underlying filesystem. If you're using a filesystem that doesn't support it and attempt to enable fs- verity on a file, the ioctl will fail. Note that this is only a concern at runtime, not at build time. > > ** koji integration: koji will need to add the fs-verity metadata > > to > > packages when signing them > > Well, koji doesn't sign packages. robosignatory listens for messages > on > the message bus for koji tagging and based on it's config, it asks > sigul > to sign rpms and import the signatures into koji. > > There's support in robosignatory to ask to sign files (used for the > short lived IMA stuff), but I suspect it would need a new ability for > this. > > Finally who is going to write this? Change owners? > Or do you expect robosignatory maintainers to do so? Thanks for clarifying! Yes, I think robosignatory is likely what we want here. We (the Change owners) expect to do the work, though we'll likely need some advice/help around code review, testing and deployment. > > * fs-verity requires filesystem support; currently support for ext4 > > and f2fs is already available; support for btrfs landed in 5.15 > > No xfs support? Correct, XFS doesn't support fs-verity at the moment (though it could be implemented if one wanted to). Cheers Davide ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Thu, Dec 02, 2021 at 02:36:51PM -0500, Ben Cotton wrote: ...snip... > > In the context of rpm, there are two parts to this: > * at build time, we compute the Merkle tree for the files within a > package, then sign it and ship it as part of the rpm metadata; This is some kind of seperate signing that happens at build time? Or it's added to the rpm metadata and covered by the normal package signing if/when the package is signed? > * at run time, if the fsverity rpm plugin is enabled, rpm will install > the fsverity signature key and enable fsverity on files that are > installed. Is this signature key the fedora rpm package signing key? Or some seperate fsverity key? ...snip... > fs-verity works by using a Merkle tree to generate a checksum for > every data block in the system, and reads will fail if a single data every data block? or every packaged in rpms data block? > block read fails it’s checksum. The signature of the the file is > validated against a public key loaded into the kernel keyring. Because > fsverity operates on block reads, its runtime cost is small (as it > only needs to verify the block that is being accessed), and it can > protect from alterations at any point in time. Is this via dm_verity kernel module? Or thats a completely seperate thing? ...snip... > == Scope == > > * Proposal owners > ** btrfs kernel enablement work > ([https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=146054090b0859b28fc39015c7704ccc3c3a347f > landed in 5.15]); see this > [https://developers.facebook.com/blog/post/2021/10/19/fs-verity-support-in-btrfs/ > blog post] for more details What does this mean for all the other filesystems? They would just be slower since btrfs is computing the trees as part of it's normal checksumming? > ** koji integration: koji will need to add the fs-verity metadata to > packages when signing them Well, koji doesn't sign packages. robosignatory listens for messages on the message bus for koji tagging and based on it's config, it asks sigul to sign rpms and import the signatures into koji. There's support in robosignatory to ask to sign files (used for the short lived IMA stuff), but I suspect it would need a new ability for this. Finally who is going to write this? Change owners? Or do you expect robosignatory maintainers to do so? > * Other developers: > ** deploy the koji integration changes to production > * Release engineering: tbd > * Policies and guidelines: N/A > * Trademark approval: N/A > > == Upgrade/compatibility impact == > > None > > == How to test == > > Install the fs-verity RPM plugin to validate package contents: > > $ sudo dnf install rpm-plugin-fsverity > Note that this will only be useful if the packages being installed > contain the appropriate fs-verity metadata (which, for Fedora upstream > packages, requires Koji integration that is part of this Change). > However, you should still be able to test this if you locally sign a > package with rpmsign --addverity. > > == User experience == > > This Change is fully transparent and there is no user impact by > default. If the user chooses to enable the fs-verity RPM plugin, they > can then leverage the additional verification features provided by > fs-verity. > > == Dependencies == > > * fs-verity support is available in RPM as of 4.17, which is available > as of Fedora 35 and is already enabled in rpm-4.17.0-0.rc1.0.fc36 > * CONFIG_FS_VERITY in the kernel config; this is already enabled > * fs-verity requires filesystem support; currently support for ext4 > and f2fs is already available; support for btrfs landed in 5.15 No xfs support? Thanks in advance for answers. kevin signature.asc Description: PGP signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
On Thu, 2021-12-02 at 15:08 -0500, Frank Ch. Eigler wrote: > > > === Relationship with IMA === > > > > [https://sourceforge.net/p/linux-ima/wiki/Home/ IMA] is another > > technology meant to provide detection of file alterations. IMA and > > fsverity operate very differently, and are somewhat complementary. > > [...] > > Do these two systems use the same per-file signature metadata in the > RPMs? Both fs-verity and IMA use file signatures, but they each have their own dedicated flags and signing flows in RPM (e.g. see https://github.com/rpm-software-management/rpm/blob/4afe2d14d33db82ccb41c0a8d5eb1a4db90762fc/rpmsign.c for the signing implementation). The signatures themselves are not interchangeable -- fs-verity's signature is based off the Merkle tree (which itself is block-based), while IMA measures the file as a whole. Cheers Davide ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
> === Relationship with IMA === > > [https://sourceforge.net/p/linux-ima/wiki/Home/ IMA] is another > technology meant to provide detection of file alterations. IMA and > fsverity operate very differently, and are somewhat complementary. > [...] Do these two systems use the same per-file signature metadata in the RPMs? - FChE ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure