Re: [qubes-devel] Re: Is there a way to save dispvm snapshots for fast startup?
On Thursday, 9 November 2017 23:25:34 UTC+8, Marek Marczykowski-Górecki wrote: > Whatever it did, it certainly made the experience of using DVM's much > > nicer, and the question is if something similar is available in 4.0? > > Currently not. We'll work on improving DispVM performance, but after > final 4.0 release. Some ideas include: > - having PVHv2 really working (it will cut stubdomain and bootloader >out of boot process - about 1-2s improvement) > - pre-starting a DispVM so it sits there ready for use[2] - this have >noticeable memory impact, but similar to 3.2 - copy of the savefile >is stored in dom0 RAM > Is that how it worked in 3.2? Because I'm sure it was very fast there (less than 10 seconds, if I remember correctly). > > As for qrexec timeouts, this is unrelated issue[1], which we have some > (hopefully sufficient) fix - it will land in testing repo today. Did it land yet? I did a dom0-update from testing and I'm still suffering from this issue (although I can't say if the frequency has been reduced). Regards, Elias -- You received this message because you are subscribed to the Google Groups "qubes-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to qubes-devel+unsubscr...@googlegroups.com. To post to this group, send email to qubes-devel@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-devel/5fa1d8fe-3783-46c3-a47d-28dde037401c%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [qubes-devel] Re: Is there a way to save dispvm snapshots for fast startup?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On Mon, Nov 13, 2017 at 01:45:32AM -0800, Elias Mårtenson wrote: > On Thursday, 9 November 2017 23:25:34 UTC+8, Marek Marczykowski-Górecki > wrote: > > > Whatever it did, it certainly made the experience of using DVM's much > > > nicer, and the question is if something similar is available in 4.0? > > > > Currently not. We'll work on improving DispVM performance, but after > > final 4.0 release. Some ideas include: > > - having PVHv2 really working (it will cut stubdomain and bootloader > >out of boot process - about 1-2s improvement) > > - pre-starting a DispVM so it sits there ready for use[2] - this have > >noticeable memory impact, but similar to 3.2 - copy of the savefile > >is stored in dom0 RAM > > > > Is that how it worked in 3.2? Because I'm sure it was very fast there (less > than 10 seconds, if I remember correctly). There is a chance it will be even faster (the second option). > > As for qrexec timeouts, this is unrelated issue[1], which we have some > > (hopefully sufficient) fix - it will land in testing repo today. > > > Did it land yet? I did a dom0-update from testing and I'm still suffering > from > this issue (although I can't say if the frequency has been reduced). No, there were some infrastructure-related problems. It should be there tomorrow... - -- Best Regards, Marek Marczykowski-Górecki Invisible Things Lab A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBCAAGBQJaBkTUAAoJENuP0xzK19csiF8H/iGUy8dHdpkR9hQhCPqqj5cp zXAwveTmuVX4ghbA+gYaLTAtIWwiSFh+73LRnB6QRGemGAa5RLflo7iTqeH+oLfe XkDyJ4TaQbO2d8meZOa0uOj2zNjprhKaEahsrvx7ql4R9GbJMr3rZkD/agYDQvIE VQsszPOIiUEDvlHutOnMk/9d6DkzRm8H09evu1D4OAw9eD1gTb19bE/S0dw/fX1P QpEpkAgjvjmN1aqA3cBRhSjVSEQbLOn/oGTOMtagmAiwjupFiFMy7UstTr/FzFVU SRw9ojNaTOsVUQuMrgCsvxRhAZb1U7f6xbcZlRddUzZHviYQz5oHnD+qNRi6I6k= =QmFM -END PGP SIGNATURE- -- You received this message because you are subscribed to the Google Groups "qubes-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to qubes-devel+unsubscr...@googlegroups.com. To post to this group, send email to qubes-devel@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-devel/20171113095033.GA1061%40mail-itl. For more options, visit https://groups.google.com/d/optout.
Re: [qubes-devel] qvm-open-in-vm (Q4.0rc2)
On Tuesday, November 7, 2017 at 7:27:48 PM UTC, Marek Marczykowski-Górecki wrote: > > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > On Tue, Nov 07, 2017 at 04:33:27AM -0800, Frédéric Pierret (fepitre) > wrote: > > Hi, > > I'm experiencing something strange in Qubes 4.0rc2: when I try to > > qvm-open-in-vm several images in the same vm it seems to does not work. > For > > e.g. from work appvm: > > > > qvm-open-in-vm '$default' appel.jpg (then I choose e.g. personal), It > opens > > great and I keep the image open in personal, and then I do the same with > > appel2.jpg but the image does not appear and does not exist in > /tmp/work-* > > > > Doing this with two differents targets VM or with e.g. PDF or TXT works > > great. Can someone test it please (just before trying to eventually > deeply > > debug...)? > > I can confirm this behavior. (I guess) it is because the opening the > second file in fact open it in the same viewer/editor instance. And that > command exit immediately after sending a message to main (first) > instance. So qvm-open-in-vm (in fact, /usr/lib/qubes/vm-file-editor) > thinks you already closed the file and remove it from the target VM. > This is very similar to more general issue we have with gnome-terminal > in DispVM - the command you use to open app/file do not wait until you > close app/file. > > - -- > Best Regards, > Marek Marczykowski-Górecki > Invisible Things Lab > A: Because it messes up the order in which people normally read text. > Q: Why is top-posting such a bad thing? > -BEGIN PGP SIGNATURE- > Version: GnuPG v2 > > iQEcBAEBCAAGBQJaAgksAAoJENuP0xzK19csi/AH+gOZ1aUZVuLv6cKAjNBSckdK > o7hwZBhdwwyeeBJ3NYdl0R1VDXWOSK+RMdh5jPXnXVdJHjJb2CXJ2nwrofWwP8C0 > Kv4O3scER6Iq7iFOns/0C9pvXAVs9FI0T9rZ98kGy9isIugDDIJJR2ie7hvKBhg1 > BeSXLFIBNPv3TERRxOqiZpOkWh7+YmDZWsIpTX1nldsEZDzWxvkbKbOK+fg7RhY1 > AkBJFjNTbNPIhwIqXVBIBBB7Ygu4J0TNP6k7f9eWYqCJrGgc4ykAglK/UhUf4Cwx > 4lyhRWveNt4td5bGBOCslnIQNQYdztViIN/ubxAcQMcIk4kaXHaqR19zc1MxNEE= > =M00S > -END PGP SIGNATURE- > Update: The new released dom0 updates seems to have done the trick, debian-8 now works flawlessly. Much appreciated! No discovered delay or issues either for qvm-run or qvm-open-in-vm in debian.8. Debian-9 works too, but it needs to sit idle for a minute or two when started up, before qvm-run works. Also pressing the qvm-run impatiently or too early, seems like it can bug out and then nothing starts at all. Which means the VM has to be shutdown and re-started before trying again. So in regard of debian-9, start it up, be patient for a minute or two, before starting anything. The exact same applies to any App-VM running on the debian.9 template. Debian-8 works normal and is quick. If others are in the same situation, II'd stick with debian-8 for now. Also for anyone trying to get debian-9 running with this delay, then don't trust the circular status thingy in the Qubes widget in regard to whether debian-9 is ready or not. The extra minute or two, is after it's done starting according to the widget, and not from the point you first started it. The amount of time may also vary from machine to machine? What was done on my side: - I rebooted all of Qubes after the new dom0 update, (just to be sure everything was in proper place). - Unlink any debian-8 templates. - sudo dnf remove qubes-template-debian-8 - sudo qubes-dom0-update qubes-template-debian-8 - Profit. Further details: I'm not 100% sure it was the updates, but given it did not work before the updates, and worked afterwards, with no other chances, I strongly assume its the updates that fixed it. Also, here are some more complicated details, that may or may not matter. At the time of the update, I did not have debian-8 installed, which I had removed the day before during my own attempts to fix it. I only had debian-9 installed when the updates came out. After running the dom0 updates, and rebooting, debian-9 still did not work. So I went ahead and installed the debian-8, which to my pleasant surprise, unlike the many previous attempts, now worked right out of the box after the install, and seemingly for now flawlessly at that. However, debian-9 did still not work, so I re-installed debian-9. When I started debian-9, it did still not work, but I stayed with it for a bit, trying to see if I could get it working. I found out that debian-9 has a time-lag, or delay, before it works after starting up. And if spamming the qvm-run, it'll stop working entirely. debian-9 must run idle after it has booted up, for a minute or two. But once there, it works flawlessly too (it seems, did not try debian-9 much). Now I do not know if these templates really required a re-install to fix them, after all, I had debian-9 at the time of the updates. As you must see by now, I did not test whether debian-9 had a time delay before trying to re-install it, and debian-8 was not installed. I tr
Re: [qubes-devel] (trying to avoid) unpacking before checking signatures
On 08/11/17 10:51 PM, Jean-Philippe Ouellet wrote: I could of course verify the signature of the auto-generated sha256sums.asc file which covers all the files (including compressed ones), but that means trusting kernel.org infrastructure - which was compromised in 2011 and may well be compromised again in the future... Not to argue anything in particular, but sometimes I do have moments where I wonder if moving our ultimate trust to developer workstations (where the signature is generated) was such a great idea. I can assure you that kernel.org infrastructure is, in the majority of cases, vastly better protected than developer workstations -- together with the signing keys that live on them and into which we put the ultimate trust. I don't have any answers to this, just wanted to share my inner conflict with others. :) Best, -- Konstantin Ryabitsev Director, IT Infrastructure Security The Linux Foundation -- You received this message because you are subscribed to the Google Groups "qubes-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to qubes-devel+unsubscr...@googlegroups.com. To post to this group, send email to qubes-devel@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-devel/996e1530-0ac9-513e-aa4e-38eb64c8006f%40linuxfoundation.org. For more options, visit https://groups.google.com/d/optout.
Re: [qubes-devel] (trying to avoid) unpacking before checking signatures
On Mon, Nov 13, 2017 at 10:28:15AM -0500, Konstantin Ryabitsev wrote: > On 08/11/17 10:51 PM, Jean-Philippe Ouellet wrote: > >I could of course verify the signature of the auto-generated > >sha256sums.asc file which covers all the files (including compressed > >ones), but that means trusting kernel.org infrastructure - which was > >compromised in 2011 and may well be compromised again in the future... > > Not to argue anything in particular, but sometimes I do have moments where I > wonder if moving our ultimate trust to developer workstations (where the > signature is generated) was such a great idea. I can assure you that > kernel.org infrastructure is, in the majority of cases, vastly better > protected than developer workstations -- together with the signing keys that > live on them and into which we put the ultimate trust. The thing is developer workstations are *already* trusted, because if those workstations are compromised they can compromise the code itself, which in practice is quite difficult to effectively audit. So why add an additional trusted entity when you already have one? That said, with deterministic builds you can get the best of both worlds: build on both kernel.org infrastructure and dev workstations and only release if the builds match 100% -- https://petertodd.org 'peter'[:-1]@petertodd.org -- You received this message because you are subscribed to the Google Groups "qubes-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to qubes-devel+unsubscr...@googlegroups.com. To post to this group, send email to qubes-devel@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-devel/20171113172131.GA20733%40savin.petertodd.org. For more options, visit https://groups.google.com/d/optout. signature.asc Description: Digital signature
Re: [qubes-devel] (trying to avoid) unpacking before checking signatures
On Mon, Nov 13, 2017 at 12:21:31PM -0500, Peter Todd wrote: Not to argue anything in particular, but sometimes I do have moments where I wonder if moving our ultimate trust to developer workstations (where the signature is generated) was such a great idea. I can assure you that kernel.org infrastructure is, in the majority of cases, vastly better protected than developer workstations -- together with the signing keys that live on them and into which we put the ultimate trust. The thing is developer workstations are *already* trusted, because if those workstations are compromised they can compromise the code itself, which in practice is quite difficult to effectively audit. So why add an additional trusted entity when you already have one? I would argue the point you're making here. Compromising code that is headed for git repositories is significantly more difficult than code that is going into a tarball. A malicious commit going into a git repository at least has the potential to be reviewed and analyzed (and will almost certainly be noticed if it lands in the Linux kernel repository). However, a 2-line change somewhere deep in the released tarball will likely never be noticed without someone actually checking for such sneaky changes. If I were an attacker in control of a developer workstation, I would target specifically tarballs -- I would make use of the race condition between when "git archive" completes and when the gpg sign command is invoked to substitute the generated tarball for a malicious one. That said, with deterministic builds you can get the best of both worlds: build on both kernel.org infrastructure and dev workstations and only release if the builds match 100% It's not quite what we do, but the process of releasing a kernel does not rely on us accepting tarballs from Linus or Greg. Instead, we take the git tree reference, the tarball prefix, and the gpg signature -- and then generate the tarball on our end. If the signature we received successfully verifies the tarball we generated on our end, that means that our git tree is identical to Linus's (or Greg's) tree, and therefore we can trust our generated tarball. This was written as a convenience, not as a security check (Linus oftentimes goes diving to remote parts of the world and does not want to spend most of his day uploading 600-megabyte tarballs over bad connections). However, it also acts as a useful double-check to verify both that git repositories on our end do not differ from the ones on Linus's workstation, and to make sure "use a race condition to sneak in a trojaned tarball" attack cannot succeed. It is my intent to deprecate the ability to upload tarballs entirely and force everyone to use the same process Linus and Greg use to generate tarballs on our end and then use the provided signature to verify it. (If you're interested in how the kernel release process works, you can see my talk from last year: https://www.youtube.com/watch?v=9ZIu3a3gocM) -K -- You received this message because you are subscribed to the Google Groups "qubes-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to qubes-devel+unsubscr...@googlegroups.com. To post to this group, send email to qubes-devel@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-devel/20171113182915.iaqd7kpuh2hbrkfs%40gmail.com. For more options, visit https://groups.google.com/d/optout. signature.asc Description: PGP signature
Re: [qubes-devel] (trying to avoid) unpacking before checking signatures
On 13 November 2017 at 13:29, Konstantin Ryabitsev wrote: > (If you're interested in how the kernel release process works, you can see my > talk from last year: https://www.youtube.com/watch?v=9ZIu3a3gocM) Looks like I gave the wrong link. Here is the direct one: https://www.youtube.com/watch?v=vohrz14S6JE -- Konstantin Ryabitsev Director, IT Infrastructure Security The Linux Foundation Montréal, Québec +1-971-258-2363 -- You received this message because you are subscribed to the Google Groups "qubes-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to qubes-devel+unsubscr...@googlegroups.com. To post to this group, send email to qubes-devel@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-devel/CAMwyc-RUu%3DPMm9%3DPq8Gvxup2JRWLhd40THuBFxh%2B%3DtUshuPvnw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: [qubes-devel] (trying to avoid) unpacking before checking signatures
On Mon, Nov 13, 2017 at 01:29:15PM -0500, Konstantin Ryabitsev wrote: > >The thing is developer workstations are *already* trusted, because if those > >workstations are compromised they can compromise the code itself, which in > >practice is quite difficult to effectively audit. So why add an additional > >trusted entity when you already have one? > > I would argue the point you're making here. Compromising code that is headed > for git repositories is significantly more difficult than code that is going > into a tarball. A malicious commit going into a git repository at least has > the potential to be reviewed and analyzed (and will almost certainly be > noticed if it lands in the Linux kernel repository). However, a 2-line > change somewhere deep in the released tarball will likely never be noticed > without someone actually checking for such sneaky changes. I do agree that theres more potential for a change in git to be noticed, but I have to wonder how much that's actually true in practice? A backdoor can easily be a one character code change, and that's rather difficult to spot. > If I were an attacker in control of a developer workstation, I would target > specifically tarballs -- I would make use of the race condition between when > "git archive" completes and when the gpg sign command is invoked to > substitute the generated tarball for a malicious one. > > >That said, with deterministic builds you can get the best of both worlds: > >build > >on both kernel.org infrastructure and dev workstations and only release if > >the > >builds match 100% > > It's not quite what we do, but the process of releasing a kernel does not > rely on us accepting tarballs from Linus or Greg. Instead, we take the git > tree reference, the tarball prefix, and the gpg signature -- and then > generate the tarball on our end. If the signature we received successfully > verifies the tarball we generated on our end, that means that our git tree > is identical to Linus's (or Greg's) tree, and therefore we can trust our > generated tarball. When you say "GPG signature", do you mean the GPG signature on the git commit? Or is this a GPG signature on a tarball? > This was written as a convenience, not as a security check (Linus oftentimes > goes diving to remote parts of the world and does not want to spend most of > his day uploading 600-megabyte tarballs over bad connections). However, it > also acts as a useful double-check to verify both that git repositories on > our end do not differ from the ones on Linus's workstation, and to make sure > "use a race condition to sneak in a trojaned tarball" attack cannot succeed. > > It is my intent to deprecate the ability to upload tarballs entirely and > force everyone to use the same process Linus and Greg use to generate > tarballs on our end and then use the provided signature to verify it. That process to generate that tarball is deterministic right? I might be misunderstanding how all this works, but maybe what we actually need here is a new type of tarball signature verifier, that takes the GPG signature from the signed git commit, and verifies that the tarball matches the tree that the git commit signed. It should be possible to do this with just the tarball and part of the git commit if the tarball is in fact deterministically generated from the Git commit. -- https://petertodd.org 'peter'[:-1]@petertodd.org -- You received this message because you are subscribed to the Google Groups "qubes-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to qubes-devel+unsubscr...@googlegroups.com. To post to this group, send email to qubes-devel@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-devel/20171113184627.GA22583%40savin.petertodd.org. For more options, visit https://groups.google.com/d/optout. signature.asc Description: Digital signature
Re: [qubes-devel] (trying to avoid) unpacking before checking signatures
On Mon, Nov 13, 2017 at 1:46 PM, Peter Todd wrote: > On Mon, Nov 13, 2017 at 01:29:15PM -0500, Konstantin Ryabitsev wrote: >> >The thing is developer workstations are *already* trusted, because if those >> >workstations are compromised they can compromise the code itself, which in >> >practice is quite difficult to effectively audit. So why add an additional >> >trusted entity when you already have one? >> >> I would argue the point you're making here. Compromising code that is headed >> for git repositories is significantly more difficult than code that is going >> into a tarball. A malicious commit going into a git repository at least has >> the potential to be reviewed and analyzed (and will almost certainly be >> noticed if it lands in the Linux kernel repository). However, a 2-line >> change somewhere deep in the released tarball will likely never be noticed >> without someone actually checking for such sneaky changes. > > I do agree that theres more potential for a change in git to be noticed, but I > have to wonder how much that's actually true in practice? A backdoor can > easily > be a one character code change, and that's rather difficult to spot. > >> If I were an attacker in control of a developer workstation, I would target >> specifically tarballs -- I would make use of the race condition between when >> "git archive" completes and when the gpg sign command is invoked to >> substitute the generated tarball for a malicious one. >> >> >That said, with deterministic builds you can get the best of both worlds: >> >build >> >on both kernel.org infrastructure and dev workstations and only release if >> >the >> >builds match 100% >> >> It's not quite what we do, but the process of releasing a kernel does not >> rely on us accepting tarballs from Linus or Greg. Instead, we take the git >> tree reference, the tarball prefix, and the gpg signature -- and then >> generate the tarball on our end. If the signature we received successfully >> verifies the tarball we generated on our end, that means that our git tree >> is identical to Linus's (or Greg's) tree, and therefore we can trust our >> generated tarball. > > When you say "GPG signature", do you mean the GPG signature on the git commit? > Or is this a GPG signature on a tarball? > >> This was written as a convenience, not as a security check (Linus oftentimes >> goes diving to remote parts of the world and does not want to spend most of >> his day uploading 600-megabyte tarballs over bad connections). However, it >> also acts as a useful double-check to verify both that git repositories on >> our end do not differ from the ones on Linus's workstation, and to make sure >> "use a race condition to sneak in a trojaned tarball" attack cannot succeed. >> >> It is my intent to deprecate the ability to upload tarballs entirely and >> force everyone to use the same process Linus and Greg use to generate >> tarballs on our end and then use the provided signature to verify it. > > That process to generate that tarball is deterministic right? > > > I might be misunderstanding how all this works, but maybe what we actually > need > here is a new type of tarball signature verifier, that takes the GPG signature > from the signed git commit, and verifies that the tarball matches the tree > that > the git commit signed. It should be possible to do this with just the tarball > and part of the git commit if the tarball is in fact deterministically > generated from the Git commit. What is there to be gained by doing that instead of just verifying signatures on git tags/commits? -- You received this message because you are subscribed to the Google Groups "qubes-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to qubes-devel+unsubscr...@googlegroups.com. To post to this group, send email to qubes-devel@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-devel/CABQWM_CFmPa8xNr%3D9m86C4z0Ratcc50oDrW2ruaD1NQUn7agSg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: [qubes-devel] (trying to avoid) unpacking before checking signatures
On Mon, Nov 13, 2017 at 01:46:27PM -0500, Peter Todd wrote: I would argue the point you're making here. Compromising code that is headed for git repositories is significantly more difficult than code that is going into a tarball. A malicious commit going into a git repository at least has the potential to be reviewed and analyzed (and will almost certainly be noticed if it lands in the Linux kernel repository). However, a 2-line change somewhere deep in the released tarball will likely never be noticed without someone actually checking for such sneaky changes. I do agree that theres more potential for a change in git to be noticed, but I have to wonder how much that's actually true in practice? A backdoor can easily be a one character code change, and that's rather difficult to spot. It isn't nearly as difficult to spot when it's a part of a git commit, because it will stand so much more in a diff than it would inside a 600MB tarball. :) It's not quite what we do, but the process of releasing a kernel does not rely on us accepting tarballs from Linus or Greg. Instead, we take the git tree reference, the tarball prefix, and the gpg signature -- and then generate the tarball on our end. If the signature we received successfully verifies the tarball we generated on our end, that means that our git tree is identical to Linus's (or Greg's) tree, and therefore we can trust our generated tarball. When you say "GPG signature", do you mean the GPG signature on the git commit? Or is this a GPG signature on a tarball? On the tarball. We do not do anything with gpg signatures on git trees because we need to provide a sig file someone can run against the tarball, not git. It is my intent to deprecate the ability to upload tarballs entirely and force everyone to use the same process Linus and Greg use to generate tarballs on our end and then use the provided signature to verify it. That process to generate that tarball is deterministic right? Yes, "git archive" will generate deterministic tarballs -- at least if git versions are fairly close together. This has broken a couple of times in the past, but in both cases the changes in git were rolled back because it was judged that having a mechanism to have deterministic tarball generation was more valuable than having "more correct" tarballs. I might be misunderstanding how all this works, but maybe what we actually need here is a new type of tarball signature verifier, that takes the GPG signature from the signed git commit, and verifies that the tarball matches the tree that the git commit signed. It should be possible to do this with just the tarball and part of the git commit if the tarball is in fact deterministically generated from the Git commit. You can do this right now -- it would be a 5-line script, honestly, but would still require unpacking the tarball and ultimately you will not really gain anything that you don't already have right now by comparing to the .sig file provided by Linus or Greg. It already assures you that the tarball you've downloaded was generated directly from a git tree that is identical to the git tree on the developer's workstation. -K -- You received this message because you are subscribed to the Google Groups "qubes-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to qubes-devel+unsubscr...@googlegroups.com. To post to this group, send email to qubes-devel@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-devel/20171113190124.r7nkn4jxca6swctc%40gmail.com. For more options, visit https://groups.google.com/d/optout. signature.asc Description: PGP signature
Re: [qubes-devel] (trying to avoid) unpacking before checking signatures
On Mon, Nov 13, 2017 at 01:59:41PM -0500, Jean-Philippe Ouellet wrote: > I might be misunderstanding how all this works, but maybe what we actually > need > > here is a new type of tarball signature verifier, that takes the GPG > > signature > > from the signed git commit, and verifies that the tarball matches the tree > > that > > the git commit signed. It should be possible to do this with just the > > tarball > > and part of the git commit if the tarball is in fact deterministically > > generated from the Git commit. > > What is there to be gained by doing that instead of just verifying > signatures on git tags/commits? For some workflows you might not want to have to get the whole git history... but come to think of it, Git supports shallow checkouts with the git clone --depth option, and you can verify the PGP signatures on those shallow checkouts. So I think everything I said above can already be done with git with some minor changes to workflows. -- https://petertodd.org 'peter'[:-1]@petertodd.org -- You received this message because you are subscribed to the Google Groups "qubes-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to qubes-devel+unsubscr...@googlegroups.com. To post to this group, send email to qubes-devel@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-devel/20171113190723.GA22972%40savin.petertodd.org. For more options, visit https://groups.google.com/d/optout. signature.asc Description: Digital signature
Re: [qubes-devel] (trying to avoid) unpacking before checking signatures
On Mon, Nov 13, 2017 at 02:01:24PM -0500, Konstantin Ryabitsev wrote: > >I do agree that theres more potential for a change in git to be noticed, but > >I > >have to wonder how much that's actually true in practice? A backdoor can > >easily > >be a one character code change, and that's rather difficult to spot. > > It isn't nearly as difficult to spot when it's a part of a git commit, > because it will stand so much more in a diff than it would inside a 600MB > tarball. :) Yes, we agree one has a 0% chance of being noticed; the other has a 0% + epsilon chance. :) > Yes, "git archive" will generate deterministic tarballs -- at least if git > versions are fairly close together. This has broken a couple of times in the > past, but in both cases the changes in git were rolled back because it was > judged that having a mechanism to have deterministic tarball generation was > more valuable than having "more correct" tarballs. Good to hear! > >I might be misunderstanding how all this works, but maybe what we actually > >need > >here is a new type of tarball signature verifier, that takes the GPG > >signature > >from the signed git commit, and verifies that the tarball matches the tree > >that > >the git commit signed. It should be possible to do this with just the tarball > >and part of the git commit if the tarball is in fact deterministically > >generated from the Git commit. > > You can do this right now -- it would be a 5-line script, honestly, but > would still require unpacking the tarball and ultimately you will not really > gain anything that you don't already have right now by comparing to the .sig > file provided by Linus or Greg. It already assures you that the tarball > you've downloaded was generated directly from a git tree that is identical > to the git tree on the developer's workstation. Alright, sounds like I had a misunderstanding of how this all worked. If I understand it correctly, you're able to provide a signature on a tarball that's signed directly from the developer themselves, from a tarball they personally generated. We also have a mechanism to deterministically regenerate those tarballs from the git commits they were supposed to have been generated. In this scenario, once we get past the problem of uncompressing the tarball - which I think is adequately addressed by doing that in a DispVM as the OP suggested - we do in fact have a signature directly from the developer on that tarball with no additional trust dependencies, and we can deterministically check it against the git commit it was supposed to have come from as well to make sure it matches the reviewed code in git. Sounds good to me! :) -- https://petertodd.org 'peter'[:-1]@petertodd.org -- You received this message because you are subscribed to the Google Groups "qubes-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to qubes-devel+unsubscr...@googlegroups.com. To post to this group, send email to qubes-devel@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-devel/20171113191514.GB22972%40savin.petertodd.org. For more options, visit https://groups.google.com/d/optout. signature.asc Description: Digital signature
Re: [qubes-devel] (trying to avoid) unpacking before checking signatures
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Peter Todd: > On Mon, Nov 13, 2017 at 02:01:24PM -0500, Konstantin Ryabitsev wrote: >>> I do agree that theres more potential for a change in git to be noticed, >>> but I >>> have to wonder how much that's actually true in practice? A backdoor can >>> easily >>> be a one character code change, and that's rather difficult to spot. >> >> It isn't nearly as difficult to spot when it's a part of a git commit, >> because it will stand so much more in a diff than it would inside a 600MB >> tarball. :) > > Yes, we agree one has a 0% chance of being noticed; the other has a 0% + > epsilon > chance. :) Could you explain why you think so? As discussed below the tarball is generated (almost) deterministically so spotting a tarball which does not match the git commit is trivial (and even possible to automate). So why is spotting a manipulated tarball harder? >> Yes, "git archive" will generate deterministic tarballs -- at least if git >> versions are fairly close together. This has broken a couple of times in the >> past, but in both cases the changes in git were rolled back because it was >> judged that having a mechanism to have deterministic tarball generation was >> more valuable than having "more correct" tarballs. > > Good to hear! > >>> I might be misunderstanding how all this works, but maybe what we actually >>> need >>> here is a new type of tarball signature verifier, that takes the GPG >>> signature >> >from the signed git commit, and verifies that the tarball matches the tree >> >that >>> the git commit signed. It should be possible to do this with just the >>> tarball >>> and part of the git commit if the tarball is in fact deterministically >>> generated from the Git commit. >> >> You can do this right now -- it would be a 5-line script, honestly, but >> would still require unpacking the tarball and ultimately you will not really >> gain anything that you don't already have right now by comparing to the .sig >> file provided by Linus or Greg. It already assures you that the tarball >> you've downloaded was generated directly from a git tree that is identical >> to the git tree on the developer's workstation. > > Alright, sounds like I had a misunderstanding of how this all worked. > > If I understand it correctly, you're able to provide a signature on a tarball > that's signed directly from the developer themselves, from a tarball they > personally generated. We also have a mechanism to deterministically regenerate > those tarballs from the git commits they were supposed to have been generated. -BEGIN PGP SIGNATURE- iQJDBAEBCgAtFiEEqieyzvOmi9FGaQcT5KzJJ4pkaBYFAloJ9JkPHGh3NDJAaXBz dW1qLmRlAAoJEOSsySeKZGgW7moP/RDMN8oSwENDyxkfrxVVfskIRAK5hW7vk5qo niYJUw/kmRtbxD+a8xWZ0NoM5pZiNxS7McsaUFa/rMZtw7su7c0sGBMBzFANwpu9 lBNTC/8B7Xm9z4EsbXHYH7mM+NPN789mOQq6is2wzqQ7Aec/04QUIXPAwxjhLIy4 Dz3JCnGPzKx+EwDIPCPdRRQxGstnlo4YzMEl0E6pfbK56ePyWMSgQJ3Lw16l2HCZ GuwTExDABH4KpLxNMRSertLVZW+i5BncrlCuVMhmtx3AiajCpvYYSGxQhRC0WTxu p+saDCd6TrYdMQYFSKOm/kjLE9DE0RWmIcZaas2PXZhseRuVo+/qQntNQKwTCC4J 2l9YK6p7mmNwmjPZEUEta2Cba3jEhEw4u5gnC7OCnsFWX/S1V8kaFYZ9NIo4PfUA pVCcuIm1aWKg82Lyh6Crmu6k5SS6drcPyMb6+6Ge9A4mirsVMDHAzGQsbHa5F32O YH6w8hyVj91nQET4UBph/a6SP5NSSH3shlwmQeNbW5PCUdiGY5ODKcu5Ahk4+43j qMTjUgHuVd4ghOqyBVP0HzAOlIK0thJt+TCRjVBitwg8HVUDKSASOROj86uuK0Z0 NKaj/rAeRMEqnkF56Ydq1WjNLgH9b9epaPQFAqOohP5NUkGjaQSFm1PsRa/s4OGl G8zV2eSM =SmWM -END PGP SIGNATURE- -- You received this message because you are subscribed to the Google Groups "qubes-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to qubes-devel+unsubscr...@googlegroups.com. To post to this group, send email to qubes-devel@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-devel/ac7b273e-30e9-2602-2c5f-4b5086fbea7b%40ipsumj.de. For more options, visit https://groups.google.com/d/optout.
Re: [qubes-devel] (trying to avoid) unpacking before checking signatures
On Mon, Nov 13, 2017 at 2:38 PM, HW42 wrote: > Peter Todd: >> On Mon, Nov 13, 2017 at 02:01:24PM -0500, Konstantin Ryabitsev wrote: I do agree that theres more potential for a change in git to be noticed, but I have to wonder how much that's actually true in practice? A backdoor can easily be a one character code change, and that's rather difficult to spot. >>> >>> It isn't nearly as difficult to spot when it's a part of a git commit, >>> because it will stand so much more in a diff than it would inside a 600MB >>> tarball. :) >> >> Yes, we agree one has a 0% chance of being noticed; the other has a 0% + >> epsilon >> chance. :) > > Could you explain why you think so? As discussed below the tarball is > generated (almost) deterministically so spotting a tarball which does > not match the git commit is trivial (and even possible to automate). So > why is spotting a manipulated tarball harder? Because in practice people do not do that. Even I did not consider doing that in the context of this post. My assumption was that "if the tarball signed by the developer is somehow evil, then the developer is either compromised or themselves malicious, and in both cases we're screwed anyway so there's no point checking against git". Perhaps this is flawed reasoning, perhaps not, but it does allow for the possibility of unnoticed changes in the tarball not present in git. -- You received this message because you are subscribed to the Google Groups "qubes-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to qubes-devel+unsubscr...@googlegroups.com. To post to this group, send email to qubes-devel@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-devel/CABQWM_BYwjH%2BTLxV8Ubr%3D4uoyT1Fd5sWuLimT30t%2BSNg8gLWUg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: [qubes-devel] (trying to avoid) unpacking before checking signatures
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512
Konstantin Ryabitsev:
> On Mon, Nov 13, 2017 at 01:46:27PM -0500, Peter Todd wrote:
>>> I would argue the point you're making here. Compromising code that is headed
>>> for git repositories is significantly more difficult than code that is going
>>> into a tarball. A malicious commit going into a git repository at least has
>>> the potential to be reviewed and analyzed (and will almost certainly be
>>> noticed if it lands in the Linux kernel repository). However, a 2-line
>>> change somewhere deep in the released tarball will likely never be noticed
>>> without someone actually checking for such sneaky changes.
>>
>> I do agree that theres more potential for a change in git to be noticed, but
>> I
>> have to wonder how much that's actually true in practice? A backdoor can
>> easily
>> be a one character code change, and that's rather difficult to spot.
>
> It isn't nearly as difficult to spot when it's a part of a git commit,
> because it will stand so much more in a diff than it would inside a
> 600MB tarball. :)
>
>>> It's not quite what we do, but the process of releasing a kernel does not
>>> rely on us accepting tarballs from Linus or Greg. Instead, we take the git
>>> tree reference, the tarball prefix, and the gpg signature -- and then
>>> generate the tarball on our end. If the signature we received successfully
>>> verifies the tarball we generated on our end, that means that our git tree
>>> is identical to Linus's (or Greg's) tree, and therefore we can trust our
>>> generated tarball.
>>
>> When you say "GPG signature", do you mean the GPG signature on the git
>> commit?
>> Or is this a GPG signature on a tarball?
>
> On the tarball. We do not do anything with gpg signatures on git trees
> because we need to provide a sig file someone can run against the
> tarball, not git.
>
>>> It is my intent to deprecate the ability to upload tarballs entirely and
>>> force everyone to use the same process Linus and Greg use to generate
>>> tarballs on our end and then use the provided signature to verify it.
What's the reason for not signing the compressed files in the first
place? Compressing the files two times and then uploading three
signatures (for .tar, .tar.xz and .tar.gz) doesn't sound so bad.
>> That process to generate that tarball is deterministic right?
>
> Yes, "git archive" will generate deterministic tarballs -- at least if
> git versions are fairly close together. This has broken a couple of
> times in the past, but in both cases the changes in git were rolled
> back because it was judged that having a mechanism to have
> deterministic tarball generation was more valuable than having "more
> correct" tarballs.
The compression process is also reproducible. After a bit testing I
found the parameters which seem to be currently used (see attached
script). For the last few mainline/stable/longterm releases this works
fine. But it seems that not long ago the parameters changed (for example
the 4.11 tar.xz used other parameters).
If someone wants to play around with generating the archives from git
you can run the attached script in an checked out linux repository. It
will verify the git tag so you need the matching key in your gpg
keyring.
- --
df8a8178824e5e8b96c0ec6e7f4ab9d020d66286965ae053f6709c9daad01fe2 *gen-linux-tar
(Google Groups breaks PGP/MIME, so I need to use inline signatures,
which don't cover attachments)
-BEGIN PGP SIGNATURE-
iQJDBAEBCgAtFiEEqieyzvOmi9FGaQcT5KzJJ4pkaBYFAloKQT4PHGh3NDJAaXBz
dW1qLmRlAAoJEOSsySeKZGgWV9oQALQxQnsCngR+R53eSCUdWRBYyDeWibNMZDlW
upA5q2phi+rzCTFDUpuo13e1JjeQoTTFcNMLtdwQ80PxAQBGy1e7u6fpXEC1AtXo
PT/WMQHLHCjC8oLcJxo6/51cFL/bDIfmHN3CIRxfv5P2rGE6bo8HCcoj29Bp06aR
7GLACOPWoUrRSq7f8JiwhofC/CnhILSS20PFXyYMAC0LkWo0hX/iFj26GCm/WkHq
gIyzuZ9VLnB9yIWyn0HVQsJMo+z8cxkDfyX3AhtII90qiUDbfVzlfepTBRaWxWfv
0O661gkVeiYcPrsEquetVxc8EkZVORZU8etzh31su7iL4v5fEZNS9PqYtKeiC07f
Uh7mKzDnBhOfTcmzvdLBR/M5kkQx0nU5kfszgGGH2GxCnhd9jBLfVeG9vtbmEWz7
OM3tcqdydS+c+nvFNzXFS/H93dH0tyF73qA5fB/+HHNLyz2G+UOnUpTrk0vrBkK1
udjXfF2eQZscxpAXveGGbRMTKXZOSx30rWfGduxfu/G6QTaq9dPPOfzcdiGqzHM7
EFzEhzy6QILljEUfuiA2n9xEZnwAxLGlGQ/UTn++4fnDXsNdyk9Khfzfsa+BIVTP
xqskbQX7R9iOkpY4Jq60c8mRdF/1PbsIKuk7yvDE3gw8aKiY8bnpmgPnt2jEd2jV
AgDfemQ1
=mMvD
-END PGP SIGNATURE-
--
You received this message because you are subscribed to the Google Groups
"qubes-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to qubes-devel+unsubscr...@googlegroups.com.
To post to this group, send email to qubes-devel@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/qubes-devel/b4a8a034-3fed-fa65-0e31-af4c7cb82183%40ipsumj.de.
For more options, visit https://groups.google.com/d/optout.
#!/bin/bash
set -eu -o pipefail
if [ $# -ne 1 ]; then
printf 'Usage: gen-linux-tar 4.13.10\n' >&2
exit 1
fi
v="$1"
tag="v${v}"
git tag -v "$tag"
git archive --format=tar --prefix="linux
Re: [qubes-devel] Re: (trying to avoid) unpacking before checking signatures
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Jean-Philippe Ouellet: > On Wed, Nov 8, 2017 at 10:51 PM, Jean-Philippe Ouellet wrote: >> Hello, >> >> The way some things are distributed on kernel.org (e.g. util-linux >> [1], cryptsetup [2], etc.) is such that the authors upload .tar and >> .tar.sign files, and then the kernel.org infrastructure compresses >> those (creating .tar.gz & .tar.xz) and signs all resulting files >> (creating sha256sums.asc) using its own key. More info here [3] >> >> Kernel.org does not make the original .tar files available, which >> means there is no file available for which a signature directly from >> the developers is also available. In order to check the developer's >> provided signature, you must first unpack the file. I consider >> unpackers to be of sufficient complexity that I would rather not run >> them on arbitrary attacker-provided input. > > Specifically: I would rather not run unpackers on unverified > attacker-controlled input, unsandboxed, in a trusted part of the > builder. I think it's good to improve the situation here. But keep in mind that the IMO bigger problem is git. I.e. we already do a lot more processing of unverified input. But I don't see a solution for this one currently. (And of course ideally we would use a simpler signature verification tool than GnuPG) > Normally I'm more pragmatic and just don't care. Hooray for DispVMs :) > >> I could of course verify the signature of the auto-generated >> sha256sums.asc file which covers all the files (including compressed >> ones), but that means trusting kernel.org infrastructure - which was >> compromised in 2011 and may well be compromised again in the future... >> >> If I want to follow qubes packaging best practices [4] and ensure that >> no untrusted code gets processed (including unpacked) by the builder, >> it seems my best option is to manually download the .tar.gz, verify >> the kernel.org sig, unpack it (possibly in a DispVM), verify the >> developer's sig, and then pin the sha512 of the original file for >> qubes-builder's verify-sources. Pinning the hash is probably the less intrusive way. Another option would be to use a shallow git-clone. I think it's important to keep qubes-builder working on a non-Qubes system. But with a fallback the DispVM solution would also be an option. I personally would prefer pinning the hash. >> To be extra sure I can also re-compress and reproduce (almost) the >> original .tar.gz file from the verified .tar file with `gzip --no-name >> --best`, and then verify that only the 4 bytes for the timestamp [5] >> are different. The .xz is reproducible, see the other mail. >> [1]: https://www.kernel.org/pub/linux/utils/util-linux/v2.31/ >> [2]: https://www.kernel.org/pub/linux/utils/cryptsetup/v1.7/ >> [3]: >> https://www.kernel.org/signature.html#kernel-org-checksum-autosigner-and-sha256sums-asc >> [4]: https://www.qubes-os.org/news/2016/05/30/build-security/ >> [5]: http://www.gzip.org/zlib/rfc-gzip.html#file-format -BEGIN PGP SIGNATURE- iQJDBAEBCgAtFiEEqieyzvOmi9FGaQcT5KzJJ4pkaBYFAloKRFsPHGh3NDJAaXBz dW1qLmRlAAoJEOSsySeKZGgWsMUP/1tsiUzzE7L4RPqsJ0c5KMKJ3nyI5O11ifor am+7DMAm8Effl5LbvISalrxcDltGJjYmvzx/a5zPegTr1Jrt5s7WVAjHPHuJwhSQ 8M8sqfqJ1fYJ1jyc0VH1fSraVlLGqVMypzEMboOfJO6nuaKVV6F1kzlg56pNAglk YnxAyoXSnp++SoHLjhmp+GivDTZFj0a+3epAX5HemdJpTyLZkgMAEg8IvFGS4ln6 gdEoqXT6rEaN6qvG0XHZbHb6GZ8gAI+hZboqycN/puWxUXvK5u8KojEclrH3x5Qf lY+XBzPkho7wqRdF9iJgbn5+TZXYSjFVBLFHUMvIxYlycrYBaCZuv7XWzA70BwoX lb5JwwehVy/sLSmA20IduTl+mtoksaBm9/YVN7SOIFGg7zetPCz/+g4FGQVrFqBR +ZD7DVpd2PLF+Bc8R1xHjQyuJZQz/mcDz/oRWW72M2xIJCG+0+ht8nqZDwMFolRi JL0jOTJ2uaxSxNCdsYtuE4agi1swN5iKLsR5xsLcABQag4ByHmbnf9+uWgwW2v+N YAoBq18ermEqcixfn9f0t2RdE3qngXFj1Bk07xmmI6RiGIG8RJvPgS5OmjRB5chS k5FjNQSGk0yqXrPdH3bUf+TTENdpzht6RtKgtGtHvgIyM3ZwjkxenTbQl+trQ9zb z8fiDcAm =1VTB -END PGP SIGNATURE- -- You received this message because you are subscribed to the Google Groups "qubes-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to qubes-devel+unsubscr...@googlegroups.com. To post to this group, send email to qubes-devel@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-devel/bcde686f-d239-51b5-b380-6b2ac9d22c15%40ipsumj.de. For more options, visit https://groups.google.com/d/optout.
