Re: [openstack-dev] [nova] Native QEMU LUKS decryption review overview ahead of FF
On 24 January 2018 at 22:57, Matt Riedemannwrote: > On 1/22/2018 8:22 AM, Lee Yarwood wrote: > >> Hello, >> >> With M3 and FF rapidly approaching this week I wanted to post a brief >> overview of the QEMU native LUKS series. >> >> The full series is available on the following topic, I'll go into more >> detail on each of the changes below: >> >> https://review.openstack.org/#/q/topic:bp/libvirt-qemu-nativ >> e-luks+status:open >> >> libvirt: Collocate encryptor and volume driver calls >> https://review.openstack.org/#/c/460243/ (Missing final +2 and +W) >> >> This refactor of the Libvirt driver connect and disconnect volume code >> has the added benefit of also correcting a number of bugs around the >> attaching and detaching of os-brick encryptors. IMHO this would be >> useful in Queens even if the rest of the series doesn't land. >> >> libvirt: Introduce disk encryption config classes >> https://review.openstack.org/#/c/464008/ (Missing final +2 and +W) >> >> This is the most straight forward change of the series and simply >> introduces the required config classes to wire up native LUKS decryption >> within the domain XML of an instance. Hopefully nothing controversial. >> >> libvirt: QEMU native LUKS decryption for encrypted volumes >> https://review.openstack.org/#/c/523958/ (Missing both +2s and +W) >> >> This change carries the bulk of the implementation, wiring up encrypted >> volumes during their initial attachment. The commit message has a >> detailed run down of the various upgrade and LM corner cases we attempt >> to handle here, such as LM from a P to Q compute, detaching a P attached >> encrypted volume after upgrading to Q etc. >> >> Upgrade and LM testing is enabled by the following changes: >> >> fixed_key: Use a single hardcoded key across devstack deployments >> https://review.openstack.org/#/c/536343/ >> >> compute: Introduce an encrypted volume LM test >> https://review.openstack.org/#/c/536177/ >> >> This is being tested by tempest-dsvm-multinode-live-migration and >> grenade-dsvm-neutron-multinode-live-migration in the following DNM Nova >> change, enabling volume backed LM tests: >> >> DNM: Test LM with encrypted volumes >> https://review.openstack.org/#/c/536350/ >> >> Hopefully that covers everything but please feel free to ping if you >> would like more detail, background etc. Thanks in advance, >> >> Lee >> >> >> >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib >> e >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > The patch is already approved, and I asked melwitt to write a release > note, at which point it was noted that swap volume will not work with > native luks encrypted volumes. That's a regression. > It's only a regression since swap_volume with encrypted volumes was fixed in https://review.openstack.org/#/c/460243/, which landed on Monday as part of this series. Prior to Monday, swap_volume with encrypted volumes would result in the raw encrypted volume being presented to the guest after the swap. We need to at least report a nova bug for this so we can work on some kind > of fallback to the non-native decryption until there is a libvirt/qemu fix > upstream and we can put version conditionals in place for when we can > support swap volume with a native luks-encrypted volume. > In the context of the above, I don't think this is a priority as clearly nobody is currently doing it. There's already a bug to track the problem in libvirt, which is linked in a code comment. Admittedly that BZ is unnecessarily private, which I noted in review, but we've reached out to the author to ask them to open it up as there's nothing sensitive going on in there. In general, anything qemu can do natively makes Nova both simpler and more robust because we don't have to modify the host configuration. This eliminates a whole class of error states and race conditions, because when we kill qemu there's nothing left to clean up. Matt -- Matthew Booth Red Hat OpenStack Engineer, Compute DFG Phone: +442070094448 (UK) __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Native QEMU LUKS decryption review overview ahead of FF
On 1/22/2018 8:22 AM, Lee Yarwood wrote: Hello, With M3 and FF rapidly approaching this week I wanted to post a brief overview of the QEMU native LUKS series. The full series is available on the following topic, I'll go into more detail on each of the changes below: https://review.openstack.org/#/q/topic:bp/libvirt-qemu-native-luks+status:open libvirt: Collocate encryptor and volume driver calls https://review.openstack.org/#/c/460243/ (Missing final +2 and +W) This refactor of the Libvirt driver connect and disconnect volume code has the added benefit of also correcting a number of bugs around the attaching and detaching of os-brick encryptors. IMHO this would be useful in Queens even if the rest of the series doesn't land. libvirt: Introduce disk encryption config classes https://review.openstack.org/#/c/464008/ (Missing final +2 and +W) This is the most straight forward change of the series and simply introduces the required config classes to wire up native LUKS decryption within the domain XML of an instance. Hopefully nothing controversial. libvirt: QEMU native LUKS decryption for encrypted volumes https://review.openstack.org/#/c/523958/ (Missing both +2s and +W) This change carries the bulk of the implementation, wiring up encrypted volumes during their initial attachment. The commit message has a detailed run down of the various upgrade and LM corner cases we attempt to handle here, such as LM from a P to Q compute, detaching a P attached encrypted volume after upgrading to Q etc. Upgrade and LM testing is enabled by the following changes: fixed_key: Use a single hardcoded key across devstack deployments https://review.openstack.org/#/c/536343/ compute: Introduce an encrypted volume LM test https://review.openstack.org/#/c/536177/ This is being tested by tempest-dsvm-multinode-live-migration and grenade-dsvm-neutron-multinode-live-migration in the following DNM Nova change, enabling volume backed LM tests: DNM: Test LM with encrypted volumes https://review.openstack.org/#/c/536350/ Hopefully that covers everything but please feel free to ping if you would like more detail, background etc. Thanks in advance, Lee __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev The patch is already approved, and I asked melwitt to write a release note, at which point it was noted that swap volume will not work with native luks encrypted volumes. That's a regression. We need to at least report a nova bug for this so we can work on some kind of fallback to the non-native decryption until there is a libvirt/qemu fix upstream and we can put version conditionals in place for when we can support swap volume with a native luks-encrypted volume. -- Thanks, Matt __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Native QEMU LUKS decryption review overview ahead of FF
On 23-01-18 13:44:49, Lee Yarwood wrote: > A breif progress update in-line below. > > On 22-01-18 14:22:12, Lee Yarwood wrote: > > Hello, > > > > With M3 and FF rapidly approaching this week I wanted to post a brief > > overview of the QEMU native LUKS series. > > > > The full series is available on the following topic, I'll go into more > > detail on each of the changes below: > > > > https://review.openstack.org/#/q/topic:bp/libvirt-qemu-native-luks+status:open > > > > libvirt: Collocate encryptor and volume driver calls > > https://review.openstack.org/#/c/460243/ (Missing final +2 and +W) > > > > This refactor of the Libvirt driver connect and disconnect volume code > > has the added benefit of also correcting a number of bugs around the > > attaching and detaching of os-brick encryptors. IMHO this would be > > useful in Queens even if the rest of the series doesn't land. > > > > libvirt: Introduce disk encryption config classes > > https://review.openstack.org/#/c/464008/ (Missing final +2 and +W) > > > > This is the most straight forward change of the series and simply > > introduces the required config classes to wire up native LUKS decryption > > within the domain XML of an instance. Hopefully nothing controversial. > > Both of these have landed, my thanks to jaypipes for his reviews! > > > libvirt: QEMU native LUKS decryption for encrypted volumes > > https://review.openstack.org/#/c/523958/ (Missing both +2s and +W) > > > > This change carries the bulk of the implementation, wiring up encrypted > > volumes during their initial attachment. The commit message has a > > detailed run down of the various upgrade and LM corner cases we attempt > > to handle here, such as LM from a P to Q compute, detaching a P attached > > encrypted volume after upgrading to Q etc. > > Thanks to melwitt and mdbooth for your reviews! I've respun to address > the various nits and typos pointed out in this change. Ready and waiting > to respin again if any others crop up. My thanks again to melwitt for another review on this final patch. I'm going to be offline for most of Thursday ahead of the FF deadline so if any non-RH core reviewers are able to look at this today I'll do my best to address any nits, concerns, facepalms etc ASAP. Cheers, Lee -- Lee Yarwood A5D1 9385 88CB 7E5F BE64 6618 BCA6 6E33 F672 2D76 signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Native QEMU LUKS decryption review overview ahead of FF
On 23-01-18 16:52:30, Corey Bryant wrote: > On Tue, Jan 23, 2018 at 8:44 AM, Lee Yarwoodwrote: >> grenade-dsvm-neutron-multinode-live-migration is currently failing due >> to our use of the Ocata UCA on stable/pike leading to the following >> issue with the libvirt 2.5.0 build it provides: >> >> libvirt 2.5.0-3ubuntu5.6~cloud0 appears to be compiled without gnutls >> https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1744758 >> > Hey Lee, > > We have a new version of libvirt in ocata-proposed now that should fix your > issue and is ready for testing. Thanks for your work on this and for > opening the bug. Thanks Corey, as reported in the bug this WORKSFORME. Thanks for the quick turn around with this, it's really appreciated! >> I've cherry-picked the following devstack change back to stable/pike and >> pulled it into the test change above for Nova, hopefully working around >> these failures: >> >> Update to using pike cloud-archive >> https://review.openstack.org/#/c/536798/ FWIW I still think we should enable the Pike UCA for our stable/pike jobs. As noted in the stable review, testing the Ocata UCA with stable/pike strikes me as pointless as no one will ever use that combination of UCA and stable/pike bits in a real world deployment. Cheers, Lee -- Lee Yarwood A5D1 9385 88CB 7E5F BE64 6618 BCA6 6E33 F672 2D76 signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Native QEMU LUKS decryption review overview ahead of FF
On Tue, Jan 23, 2018 at 8:44 AM, Lee Yarwoodwrote: > A breif progress update in-line below. > > On 22-01-18 14:22:12, Lee Yarwood wrote: > > Hello, > > > > With M3 and FF rapidly approaching this week I wanted to post a brief > > overview of the QEMU native LUKS series. > > > > The full series is available on the following topic, I'll go into more > > detail on each of the changes below: > > > > https://review.openstack.org/#/q/topic:bp/libvirt-qemu- > native-luks+status:open > > > > libvirt: Collocate encryptor and volume driver calls > > https://review.openstack.org/#/c/460243/ (Missing final +2 and +W) > > > > This refactor of the Libvirt driver connect and disconnect volume code > > has the added benefit of also correcting a number of bugs around the > > attaching and detaching of os-brick encryptors. IMHO this would be > > useful in Queens even if the rest of the series doesn't land. > > > > libvirt: Introduce disk encryption config classes > > https://review.openstack.org/#/c/464008/ (Missing final +2 and +W) > > > > This is the most straight forward change of the series and simply > > introduces the required config classes to wire up native LUKS decryption > > within the domain XML of an instance. Hopefully nothing controversial. > > Both of these have landed, my thanks to jaypipes for his reviews! > > > libvirt: QEMU native LUKS decryption for encrypted volumes > > https://review.openstack.org/#/c/523958/ (Missing both +2s and +W) > > > > This change carries the bulk of the implementation, wiring up encrypted > > volumes during their initial attachment. The commit message has a > > detailed run down of the various upgrade and LM corner cases we attempt > > to handle here, such as LM from a P to Q compute, detaching a P attached > > encrypted volume after upgrading to Q etc. > > Thanks to melwitt and mdbooth for your reviews! I've respun to address > the various nits and typos pointed out in this change. Ready and waiting > to respin again if any others crop up. > > > Upgrade and LM testing is enabled by the following changes: > > > > fixed_key: Use a single hardcoded key across devstack deployments > > https://review.openstack.org/#/c/536343/ > > > > compute: Introduce an encrypted volume LM test > > https://review.openstack.org/#/c/536177/ > > > > This is being tested by tempest-dsvm-multinode-live-migration and > > grenade-dsvm-neutron-multinode-live-migration in the following DNM Nova > > change, enabling volume backed LM tests: > > > > DNM: Test LM with encrypted volumes > > https://review.openstack.org/#/c/536350/ > > > > Hopefully that covers everything but please feel free to ping if you > > would like more detail, background etc. Thanks in advance, > > grenade-dsvm-neutron-multinode-live-migration is currently failing due > to our use of the Ocata UCA on stable/pike leading to the following > issue with the libvirt 2.5.0 build it provides: > > libvirt 2.5.0-3ubuntu5.6~cloud0 appears to be compiled without gnutls > https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1744758 > > Hey Lee, We have a new version of libvirt in ocata-proposed now that should fix your issue and is ready for testing. Thanks for your work on this and for opening the bug. Corey I've cherry-picked the following devstack change back to stable/pike and > pulled it into the test change above for Nova, hopefully working around > these failures: > > Update to using pike cloud-archive > https://review.openstack.org/#/c/536798/ > > tempest-dsvm-multinode-live-migration is also failing but AFAICT they > are unrelated to this overall series and appear to be more generic > volume backed live migration failures. > > Thanks again! > > Lee > -- > Lee Yarwood A5D1 9385 88CB 7E5F BE64 6618 BCA6 6E33 F672 > 2D76 > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Native QEMU LUKS decryption review overview ahead of FF
A breif progress update in-line below. On 22-01-18 14:22:12, Lee Yarwood wrote: > Hello, > > With M3 and FF rapidly approaching this week I wanted to post a brief > overview of the QEMU native LUKS series. > > The full series is available on the following topic, I'll go into more > detail on each of the changes below: > > https://review.openstack.org/#/q/topic:bp/libvirt-qemu-native-luks+status:open > > libvirt: Collocate encryptor and volume driver calls > https://review.openstack.org/#/c/460243/ (Missing final +2 and +W) > > This refactor of the Libvirt driver connect and disconnect volume code > has the added benefit of also correcting a number of bugs around the > attaching and detaching of os-brick encryptors. IMHO this would be > useful in Queens even if the rest of the series doesn't land. > > libvirt: Introduce disk encryption config classes > https://review.openstack.org/#/c/464008/ (Missing final +2 and +W) > > This is the most straight forward change of the series and simply > introduces the required config classes to wire up native LUKS decryption > within the domain XML of an instance. Hopefully nothing controversial. Both of these have landed, my thanks to jaypipes for his reviews! > libvirt: QEMU native LUKS decryption for encrypted volumes > https://review.openstack.org/#/c/523958/ (Missing both +2s and +W) > > This change carries the bulk of the implementation, wiring up encrypted > volumes during their initial attachment. The commit message has a > detailed run down of the various upgrade and LM corner cases we attempt > to handle here, such as LM from a P to Q compute, detaching a P attached > encrypted volume after upgrading to Q etc. Thanks to melwitt and mdbooth for your reviews! I've respun to address the various nits and typos pointed out in this change. Ready and waiting to respin again if any others crop up. > Upgrade and LM testing is enabled by the following changes: > > fixed_key: Use a single hardcoded key across devstack deployments > https://review.openstack.org/#/c/536343/ > > compute: Introduce an encrypted volume LM test > https://review.openstack.org/#/c/536177/ > > This is being tested by tempest-dsvm-multinode-live-migration and > grenade-dsvm-neutron-multinode-live-migration in the following DNM Nova > change, enabling volume backed LM tests: > > DNM: Test LM with encrypted volumes > https://review.openstack.org/#/c/536350/ > > Hopefully that covers everything but please feel free to ping if you > would like more detail, background etc. Thanks in advance, grenade-dsvm-neutron-multinode-live-migration is currently failing due to our use of the Ocata UCA on stable/pike leading to the following issue with the libvirt 2.5.0 build it provides: libvirt 2.5.0-3ubuntu5.6~cloud0 appears to be compiled without gnutls https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1744758 I've cherry-picked the following devstack change back to stable/pike and pulled it into the test change above for Nova, hopefully working around these failures: Update to using pike cloud-archive https://review.openstack.org/#/c/536798/ tempest-dsvm-multinode-live-migration is also failing but AFAICT they are unrelated to this overall series and appear to be more generic volume backed live migration failures. Thanks again! Lee -- Lee Yarwood A5D1 9385 88CB 7E5F BE64 6618 BCA6 6E33 F672 2D76 signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] Native QEMU LUKS decryption review overview ahead of FF
Hello, With M3 and FF rapidly approaching this week I wanted to post a brief overview of the QEMU native LUKS series. The full series is available on the following topic, I'll go into more detail on each of the changes below: https://review.openstack.org/#/q/topic:bp/libvirt-qemu-native-luks+status:open libvirt: Collocate encryptor and volume driver calls https://review.openstack.org/#/c/460243/ (Missing final +2 and +W) This refactor of the Libvirt driver connect and disconnect volume code has the added benefit of also correcting a number of bugs around the attaching and detaching of os-brick encryptors. IMHO this would be useful in Queens even if the rest of the series doesn't land. libvirt: Introduce disk encryption config classes https://review.openstack.org/#/c/464008/ (Missing final +2 and +W) This is the most straight forward change of the series and simply introduces the required config classes to wire up native LUKS decryption within the domain XML of an instance. Hopefully nothing controversial. libvirt: QEMU native LUKS decryption for encrypted volumes https://review.openstack.org/#/c/523958/ (Missing both +2s and +W) This change carries the bulk of the implementation, wiring up encrypted volumes during their initial attachment. The commit message has a detailed run down of the various upgrade and LM corner cases we attempt to handle here, such as LM from a P to Q compute, detaching a P attached encrypted volume after upgrading to Q etc. Upgrade and LM testing is enabled by the following changes: fixed_key: Use a single hardcoded key across devstack deployments https://review.openstack.org/#/c/536343/ compute: Introduce an encrypted volume LM test https://review.openstack.org/#/c/536177/ This is being tested by tempest-dsvm-multinode-live-migration and grenade-dsvm-neutron-multinode-live-migration in the following DNM Nova change, enabling volume backed LM tests: DNM: Test LM with encrypted volumes https://review.openstack.org/#/c/536350/ Hopefully that covers everything but please feel free to ping if you would like more detail, background etc. Thanks in advance, Lee -- Lee Yarwood A5D1 9385 88CB 7E5F BE64 6618 BCA6 6E33 F672 2D76 signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev