[
https://issues.apache.org/jira/browse/CLOUDSTACK-9965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053167#comment-16053167
]
John McEleney commented on CLOUDSTACK-9965:
-------------------------------------------
It is also worth noting that the Console Proxy VM certificate deployment worked
flawlessly, so there is some difference between the SSVM and CPVM code.
> Custom SSL Certificate deployment on SSVM is broken
> ---------------------------------------------------
>
> Key: CLOUDSTACK-9965
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9965
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: Secondary Storage
> Affects Versions: 4.10.0.0
> Environment: Hypervisor: KVM.
> CloudStack version: 4.10.0-SNAPSHOT from 7-June-2016.
> SSVM template source:
> http://cloudstack.apt-get.eu/systemvm/4.10/RC4/systemvm64template-master-4.10.0-kvm.qcow2.bz2
> Reporter: John McEleney
>
> I've seen this bug with both the RC4 SSVM template and the prior release.
> When a custom SSL certificate is added to CloudStack, the Secondary Storage
> VM (SSVM) deployment fails. In spite of the custom certificate being copied
> to the VM, Apache remains configured to use the default Snakeoil certificate.
> I've looked through some of the scripting on the SSVM to try to work out why
> it's failing, and I've been able to tweak the running system to get the
> certificate working.
> Relevant logs entries in /varlog/cloud.log:
> {noformat}
> 2017-06-18 08:58:42,170 DEBUG [utils.script.Script]
> (agentRequest-Handler-1:null) Executing:
> /usr/local/cloud/systemvm/config_ssl.sh -i 94.229.139.152 -h s-22-VM -k
> /tmp/prvkey2407180375973626296.tmp -p /tmp/pubcert1028956520690640221.tmp -t
> /tmp/certchain5546051081886085440.tmp -u /tmp/rootcert3072981443992074804.tmp
> sed: can't read /etc/apache2/ports.conf: No such file or directory
> sed: can't read /etc/apache2/ports.conf: No such file or directory
> sed: can't read /etc/apache2/ports.conf: No such file or directory
> adding rewrite rules to file: /etc/apache2/sites-available/default-ssl
> adding cors rules to file: /etc/apache2/sites-available/default-ssl
> Stopping web server: apache2apache2: Could not reliably determine the
> server's fully qualified domain name, using 94.229.139.152 for ServerName
> ... waiting .
> Starting web server: apache2apache2: Could not reliably determine the
> server's fully qualified domain name, using 94.229.139.152 for ServerName
> .
> {noformat}
> At this point Apache is running and continuing to serve the snakeoil
> certificate.
> The config_ssl.sh script is carrying out actions on the following files:
> /etc/httpd/conf/httpd.conf
> /etc/apache2/sites-available/default
> /etc/apache2/sites-available/default-ssl
> /etc/apache2/ports.conf
> Of these files /etc/apache2/sites-available/default-ssl is clearly the most
> relevant. However, I see no evidence here of that file being linked to
> /etc/apache2/sites-enabled, so without that step, this work is wasted.
> site-enabled contains one file:
> /etc/apache2/sites-enabled/vhost-[IPADDRESS].conf
> That is the file that makes reference to the default snakeoil certiifcate:
> {noformat}
> SSLCertificateFile /etc/ssl/certs/ssl-cert-snakeoil.pem
> SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
> {noformat}
> The custom certificate has been successfully copied to /etc/httpd/ssl/certs
> folder, but is not referenced.
> To try to fix this, I do the following:
> - Delete vhost-[IPADDRESS].conf
> - Symlink default-ssl into sites-enabled
> - Restart apache
> {noformat}
> # rm -f /etc/apache2/sites-enabled/vhost-*.conf
>
>
> # ln -s /etc/apache2/sites-available/default-ssl /etc/apache2/sites-enabled/
> # service apache2 restart
> Restarting web server: apache2apache2: Could not reliably determine the
> server's fully qualified domain name, using [IPADDRESS] for ServerName
> ... waiting apache2: Could not reliably determine the server's fully
> qualified domain name, using [IPADDRESS] for ServerName
> no listening sockets available, shutting down
> Unable to open logs
> Action 'start' failed.
> The Apache error log may have more information.
> failed!
> {noformat}
> That's two remaining problems to solve. Apache needs a ServerName, and
> listening sockets need to be defined.
> The listening socket is the interesting one and relates to the errors in the
> log. Looking in config_ssl.sh, we can see where it tried and fail to
> manipulate the Listen IP/socket:
> {noformat}
> sed -i -e "s/Listen .*:80/Listen $ip:80/g" /etc/apache2/ports.conf
> sed -i -e "s/Listen .*:443/Listen $ip:443/g" /etc/apache2/ports.conf
> {noformat}
> As the log entries say, ports.conf does not exist. How did that happen? I
> found the answer to that in /etc/init.d/cloud-early:
> {noformat}
> clean_ipalias_config() {
> # Old
> rm -f /etc/apache2/conf.d/ports.*.meta-data.conf
> rm -f /etc/apache2/sites-available/ipAlias*
> rm -f /etc/apache2/sites-enabled/ipAlias*
> rm -f /etc/apache2/conf.d/vhost*.conf
> rm -f /etc/apache2/ports.conf
> rm -f /etc/apache2/vhostexample.conf
> rm -f /etc/apache2/sites-available/default
> rm -f /etc/apache2/sites-available/default-ssl
> rm -f /etc/apache2/sites-enabled/default
> rm -f /etc/apache2/sites-enabled/default-ssl
> # New
> rm -f /etc/apache2/sites-enabled/vhost-*.conf
> rm -f /etc/apache2/sites-enabled/000-default
> rm -rf /etc/failure_config
> }
> {noformat}
> The actions taken in /etc/init.d/cloud-early-config directly cause the Listen
> socket actions in config_ssl.sh to fail.
> My next step is to bring back the ServerName and Listen directives:
> {noformat}
> # cat > /etc/apache2/conf.d/ssvm <<'EOF'
> > ServerName server.mydomain.com
> > Listen 0.0.0.0:80
> >
> > <IfModule ssl_module>
> > Listen 0.0.0.0:443
> > </IfModule>
> >
> > <IfModule mod_gnutls.c>
> > Listen 0.0.0.0:443
> > </IfModule>
> >
> > # vim: syntax=apache ts=4 sw=4 sts=4 sr noet
> > EOF
> #
> {noformat}
> Next I tweak config_ssl.sh to have it operate on this file rather than
> ports.conf. I also have it create the sites/enabled symlinks and remove the
> vhost file:
> {noformat}
> --- config_ssl.sh 2017-06-06 21:51:12.000000000 +0000
> +++ config_ssl.sh.new 2017-06-18 11:05:38.197646907 +0000
> @@ -47,9 +47,9 @@
> cp -f /etc/apache2/sites-available/default-ssl.orig
> /etc/apache2/sites-available/default-ssl
> sed -i -e "s/<VirtualHost.*>/<VirtualHost $ip:80>/"
> /etc/apache2/sites-available/default
> sed -i -e "s/<VirtualHost.*>/<VirtualHost $ip:443>/"
> /etc/apache2/sites-available/default-ssl
> - sed -i -e "s/Listen .*:80/Listen $ip:80/g" /etc/apache2/ports.conf
> - sed -i -e "s/Listen .*:443/Listen $ip:443/g" /etc/apache2/ports.conf
> - sed -i -e "s/NameVirtualHost .*:80/NameVirtualHost $ip:80/g"
> /etc/apache2/ports.conf
> + sed -i -e "s/Listen .*:80/Listen $ip:80/g" /etc/apache2/conf.d/ssvm
> + sed -i -e "s/Listen .*:443/Listen $ip:443/g" /etc/apache2/conf.d/ssvm
> + sed -i -e "s/NameVirtualHost .*:80/NameVirtualHost $ip:80/g"
> /etc/apache2/conf.d/ssvm
> sed -i 's/ssl-cert-snakeoil.key/cert_apache.key/'
> /etc/apache2/sites-available/default-ssl
> sed -i 's/ssl-cert-snakeoil.pem/cert_apache.crt/'
> /etc/apache2/sites-available/default-ssl
> sed -i 's/SSLProtocol.*$/SSLProtocol all -SSLv2 -SSLv3/'
> /etc/apache2/sites-available/default-ssl
> @@ -80,7 +80,8 @@
> sed -i -e "s/<\/VirtualHost>/Header always set
> Access-Control-Allow-Headers \"x-requested-with, Content-Type, origin,
> authorization, accept, client-security-token, x-signature, x-metadata,
> x-expires\" \n&/" $SSL_FILE
> fi
> fi
> -
> + ln -s /etc/apache2/sites-available/default
> /etc/apache2/sites-available/default-ssl /etc/apache2/sites-enabled/
> + rm -f /etc/apache2/sites-enabled/vhost-*
> }
>
> copy_certs() {
> {noformat}
> I also found that my changes to config_ssl.sh would be overwritten at reboot,
> so I made it immutable with "chattt -i".
> What I end up with is a system working as expected but relying on a really
> dirty hack. I'm afraid I don't know enough about the conflicting needs of
> those that wrote these conflicting scripts to see a clean fix.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
