Hi Wido,
Try this: https://github.com/apache/cloudstack/pull/2505/commits/9e6bd8aedb7e3a555bcb64e9220b9facb5aef879 CloudStack attempts to make a libvirt connection to the host's private ip using the qemu+tls:// uri to detect if libvirtd if running with tls enabled or not (default tls port 16514). This check does not involve reading libvirtd's config file. For an already added/connected KVM host, the certificates are renewed via cmd-answer pattern and not ssh-based provisioning. SSH-based certificate provisioning is only used when adding a new host. The CA framework uses a ca strictness global setting to ascertain whether the clients connecting to a management need to present certificates or not. This global setting is true only for fresh deployments, otherwise false for upgraded/existing ones. If this setting is true in your env, but you don't have certificates setup on KVM hosts, management server won't allow that agent to connect; if it is false, there will be no issues. For out-of-band addition of host such as yours, you can either use a sudoer user [1] to add a KVM host, or implement your own workflow, for example: Call the provisionCertificate API (let's say from the KVM host) and passing suitable API arguments such as the host's hostname, ips etc. From the response, save the files (cloud.crt, cloud.key and cloud.ca.crt) and create the keystore (cloud.jks, save its passphrase in agent.properties file) at agent's /etc/cloudstack/agent. For reference, you can see how the keystore scripts do this [2]. [1] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Allow+non-root+user+to+add+KVM+hypervisor [2] https://github.com/apache/cloudstack/tree/master/scripts/util - Rohit <https://cloudstack.apache.org> ________________________________ From: Wido den Hollander <w...@widodh.nl> Sent: Saturday, March 24, 2018 4:42:45 PM To: dev@cloudstack.apache.org Subject: Re: [DISCUSS] Enhancement: Use CA framework to enable secured live KVM VM migration On 03/23/2018 11:21 AM, Rohit Yadav wrote: > Wido - fair point, considering this I'll add some notes in our release notes. > With the suggested change, during pkg upgrade libvirtd will not be > reconfigured: > > https://github.com/shapeblue/cloudstack-apple/pull/65/commits/9dd93ee3c7b0a31cbd262a51aaed2b325decfbd8 > 404 ;) > > Instead, KVM hosts that don't meet following conditions will show up as > 'unsecure' when they connect to management server: > > - Host does not have certificates setup at /etc/cloudstack/agent > > - Host's libvirtd is not tls enabled with any provisioned certificate > How are you going to check the last part? Would you have the cloudstack agent read the libvirt config files? That might not work as you might have SELinux or AppArmor preventing that. I don't know you can ask libvirt that information when connected to it. > > The admin can then simply choose to provision certificate to secure > 'unsecure' hosts post upgrade via api or the newly added button. This process > will provision (or renew) certificates, configure only tls settings in > libvirtd and restart libvirtd and then restart the agent. See the PR for > details. Thanks. > Yes, but that will SSH to the hosts, correct? We for example don't have SSH from the mgmt server enabled and only populate agent.properties on the KVM hosts when we add them to CloudStack. Wido > > > - Rohit > > <https://cloudstack.apache.org> > > > > ________________________________ > From: Wido den Hollander <w...@widodh.nl> > Sent: Thursday, March 22, 2018 10:14:31 PM > To: dev@cloudstack.apache.org > Subject: Re: [DISCUSS] Enhancement: Use CA framework to enable secured live > KVM VM migration > > > > On 03/21/2018 10:06 AM, Rohit Yadav wrote: >> Thanks Wido for your comments. >> >> >> Yes, for any changes to libvirtd the proposal is to re-use >> cloudstack-setup-agent which in fact reconfigures libvirtd config at the >> time of the addition of host and also configure iptables rule. As part of >> upgrading a KVM agent, the post-install script (part of deb/rpm pkg) can >> also run the same to secure libvirt tls configuration only on KVM hosts that >> have any existing certificates/keystore. >> > > Hmm, we might want to be careful with a postinst. I'm not against it > being handled by a postinst, but we should watch out with overwriting > config files without the user knowing. > > Wido > >> >> - Rohit >> >> <https://cloudstack.apache.org> >> >> >> >> ________________________________ >> From: Wido den Hollander <w...@widodh.nl> >> Sent: Wednesday, March 21, 2018 1:38:19 PM >> To: dev@cloudstack.apache.org >> Subject: Re: [DISCUSS] Enhancement: Use CA framework to enable secured live >> KVM VM migration >> >> >> >> On 03/21/2018 08:05 AM, Rohit Yadav wrote: >>> All, >>> >>> >>> With the introduction of a native CA framework in CloudStack, with 4.11+ it >>> will be used to secure addition of KVM hosts and agents (cpvm, ssvm). >>> However, the KVM host agent may be secured while it communicates to the >>> management server, the live VM migration still happens on insecure tcp >>> connection. >>> >>> >>> It is proposed to re-use the existing mechanism introduced in 4.11 and >>> re-use host certificates that are used to secure a KVM host to secure >>> libvirt for allowing secured TLS-enabled VM migration. Further, the UI may >>> be enhanced to discover unsecured KVM hosts and allow securing (or >>> renewal/provisioning of certificates) through a button. Please find the FS >>> for the proposed enhancement: >>> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Secure+Live+VM+Migration+for+KVM >>> >> >> Seems good! As long as we make sure that only cloudstack-setup-agent >> touches the libvirt config files I'm good with it. >> >> Many people (like us) have the libvirt config files managed through a >> tool like Salt/Puppet/Chef and don't like it when daemons suddenly start >> changing configuration files. >> >> But this looks good to me! >> >> Wido >> >>> >>> - Rohit >>> >>> <https://cloudstack.apache.org> >>> >>> >>> >>> rohit.ya...@shapeblue.com >>> www.shapeblue.com<http://www.shapeblue.com> >>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK >>> @shapeblue >>> >>> >>> >>> >> >> rohit.ya...@shapeblue.com >> www.shapeblue.com<http://www.shapeblue.com> >> 53 Chandos Place, Covent Garden, London WC2N 4HSUK >> @shapeblue >> >> >> > > rohit.ya...@shapeblue.com > www.shapeblue.com<http://www.shapeblue.com> > 53 Chandos Place, Covent Garden, London WC2N 4HSUK > @shapeblue > > > rohit.ya...@shapeblue.com www.shapeblue.com 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
