Hello, sorry for a late response, I have had a busy week.
I have made a few modification to your patch, namely if M2Crypto library is not available, it uses a custom HTTPS connection module which verifies the server certificate. This means that SSL certificate verification can also be used if the M2Crypto library is not available. Note: It looks like if you try to install the M2Crypto library on Unix using pip it fails, because it downloads the Windows package by default (it works fine if you install it manually using setup.py or install a .deb package if you are on Ubuntu / Debain). Now someone from the commit team needs to look at the patch and merge it into trunk. Tomaž On Sat, Nov 13, 2010 at 12:55 AM, Pietro Battiston <[email protected]>wrote: > Il giorno ven, 12/11/2010 alle 17.54 -0500, Tom Davis ha scritto: > > > > > > As already said, I have never used libcloud and maybe should hence not > > > care that much, but I still want you to notice that it will probably be > > > removed from Debian if the bug is not fixed: > > > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=598463#30 > > > > > > If we're going to get kicked from repos for this, I think it's time to > drop > > the "dependencies are generically bad" thing and fix this issue. It > doesn't > > seem like we would even need to explicitly *require* M2Crypto > > The patch I proposed doesn't. > > Pietro > > > > (though, IMO, > > we might as well since it's on pypi). > > > > On Fri, Nov 12, 2010 at 5:05 PM, Pietro Battiston <[email protected] > >wrote: > > > > > Il giorno lun, 08/11/2010 alle 19.40 +0100, Pietro Battiston ha > scritto: > > > > Il giorno lun, 08/11/2010 alle 11.18 +0100, Tomaž Muraus ha scritto: > > > > > Hello, > > > > > > > > > > A user has already created an issue on Jira about this[1] some time > ago > > > and > > > > > because the root issue is in the Python module only a warning has > been > > > added > > > > > to the README. > > > > > > > > > > I still personally think that the better solution would to fix the > > > problem > > > > > and subclass the HTTPSConnection class and manually check the > hostname > > > or > > > > > switch to the M2Crypto library like you have suggested. > > > > > > > > > > Only problem with switching to the M2Crypto library is that it adds > an > > > extra > > > > > dependency. > > > > > > > > > > > > Sure. Hence, using M2Crypto if available and printing a warning > > > > otherwise is to my eyes the optimum. > > > > > > > > That's what I'm doing in the attached patch (from "svn diff" on svn > > > > trunk). > > > > > > > > Would you mind reviewing/testing it? Would you/some other developer > > > > suggest if there are other places (a quick grep found none) in > libcloud > > > > where https connections are made? > > > > > > > > thanks a lot > > > > > > > > Pietro > > > > > > > > > As already said, I have never used libcloud and maybe should hence not > > > care that much, but I still want you to notice that it will probably be > > > removed from Debian if the bug is not fixed: > > > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=598463#30 > > > > > > bye > > > > > > Pietro > > > > > > > > > > > > > > > > > > > > [1]: https://issues.apache.org/jira/browse/LIBCLOUD-55 > > > > > > > > > > On Mon, Nov 8, 2010 at 11:00 AM, Pietro Battiston < > > > [email protected]>wrote: > > > > > > > > > > > Hello, > > > > > > > > > > > > I' coping with bug > > > > > > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=598463 > > > > > > and it seems the only upstream reference to it is > > > > > > > > > > > > > > > > http://mail-archives.apache.org/mod_mbox/incubator-libcloud/201009.mbox/%3c5860913.463891285776633273.javamail.j...@thor%3e > > > > > > > > > > > > Now, there is no doubt that it's indeed an annoying thing, and > that > > > many > > > > > > other > > > > > > projects just fixed it (waiting for python devs doing it). But > I'm > > > not > > > > > > in search of a flame: I just would like to fix it (as a Debian > patch, > > > if > > > > > > you are not interested). > > > > > > > > > > > > In a project of mine, the analogous fix took very few lines of > code: > > > > > > > > > > > > > > > > > > > > > > http://code.google.com/p/galleryremote/source/diff?spec=svn6&r=6&format=side&path=/trunk/galleryremote/gallery.py > > > > > > > > > > > > and I would be happy to try to do the same on libcloud, though I > > > > > > perfectly know it will be slightly harder. > > > > > > > > > > > > But the main point is: I never used this library, neither have an > > > > > > account on any cloud provider, so I would totally appreciate if > some > > > dev > > > > > > or at least user could cooperate with me. Feel free to answer in > > > mailing > > > > > > list of contact me privately. > > > > > > > > > > > > Thanks > > > > > > > > > > > > Pietro Battiston > > > > > > > > > > > > > > > > > > > > > > > > > > > >
From aa6f213f8d2232edc05870e8f590a9d363afcd39 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Toma=C5=BE=20Muraus?= <[email protected]> Date: Sat, 13 Nov 2010 04:47:15 +0100 Subject: [PATCH] If VERIFY_SSL_CERT is True, use the M2Crypto library httpslib module which verifies the server SSL certificate and if M2Crypto library is not available, use the custom HTTPS connection module which verifies the server certificate. --- libcloud/base.py | 64 +++++++++++++++++++++++++++++++------ libcloud/httplib_ssl.py | 80 +++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 134 insertions(+), 10 deletions(-) create mode 100644 libcloud/httplib_ssl.py diff --git a/libcloud/base.py b/libcloud/base.py index f3235c4..60ee798 100644 --- a/libcloud/base.py +++ b/libcloud/base.py @@ -20,14 +20,45 @@ import httplib, urllib import libcloud from libcloud.types import NodeState, DeploymentError from libcloud.ssh import SSHClient +from libcloud.httplib_ssl import VerifiedHTTPSConnection import time import hashlib import StringIO +import ssl import os import socket import struct from pipes import quote as pquote +# For backward compatibility this option is disabled by default +VERIFY_SSL_CERT = False + +# File containing one or more PEM-encoded CA certificates concatenated together +CA_CERTS_FILE_PATH = '/etc/ssl/certs/ca-certificates.crt' + +if VERIFY_SSL_CERT: + try: + from M2Crypto import httpslib + from M2Crypto import SSL + from M2Crypto.SSL import SSLError + + M2CRYPTO = True + HTTPSConnection = httpslib.HTTPSConnection + except ImportError: + # If M2Crypto library is not available custom HTTPS connection module + # which verifies the server certificate is used. + M2CRYPTO = False + SSLError = None + HTTPSConnection = VerifiedHTTPSConnection +else: + HTTPSConnection = httplib.HTTPSConnection + +if not VERIFY_SSL_CERT: + import warnings + warnings.warn('SSL certificate verification is disabled, this can pose a ' + 'security risk. For more information how to enable the SSL ' + 'certificate verification, please visit the libcloud ' + 'documentation.') class Node(object): """ @@ -257,13 +288,13 @@ class LoggingConnection(): cmd.extend([pquote("https://%s:%d%s"; % (self.host, self.port, url))]) return " ".join(cmd) -class LoggingHTTPSConnection(LoggingConnection, httplib.HTTPSConnection): +class LoggingHTTPSConnection(LoggingConnection, HTTPSConnection): """ Utility Class for logging HTTPS connections """ def getresponse(self): - r = httplib.HTTPSConnection.getresponse(self) + r = HTTPSConnection.getresponse(self) if self.log is not None: r, rv = self._log_response(r) self.log.write(rv + "\n") @@ -277,8 +308,7 @@ class LoggingHTTPSConnection(LoggingConnection, httplib.HTTPSConnection): self.log.write(pre + self._log_curl(method, url, body, headers) + "\n") self.log.flush() - return httplib.HTTPSConnection.request(self, method, url, - body, headers) + return HTTPSConnection.request(self, method, url, body, headers) class LoggingHTTPConnection(LoggingConnection, httplib.HTTPConnection): """ @@ -315,8 +345,8 @@ class ConnectionKey(object): # with upstream Python (see http://bugs.python.org/issue1589 for details) # and not with libcloud. - #conn_classes = (httplib.LoggingHTTPConnection, LoggingHTTPSConnection) - conn_classes = (httplib.HTTPConnection, httplib.HTTPSConnection) + #conn_classes = (LoggingHTTPSConnection) + conn_classes = (httplib.HTTPConnection, HTTPSConnection) responseCls = Response connection = None @@ -355,7 +385,17 @@ class ConnectionKey(object): host = host or self.host port = port or self.port[self.secure] - connection = self.conn_classes[self.secure](host, port) + kwargs = {'host': host, 'port': port} + if self.secure: + if VERIFY_SSL_CERT and M2CRYPTO: + ssl_context = SSL.Context() + ssl_context.load_verify_info(cafile = CA_CERTS_FILE_PATH) + ssl_context.set_verify(SSL.verify_peer | + SSL.verify_fail_if_no_peer_cert | + SSL.verify_client_once, 20) + kwargs['ssl_context'] = ssl_context + + connection = self.conn_classes[self.secure](**kwargs) # You can uncoment this line, if you setup a reverse proxy server # which proxies to your endpoint, and lets you easily capture # connections in cleartext when you setup the proxy to do SSL @@ -439,8 +479,12 @@ class ConnectionKey(object): # Removed terrible hack...this a less-bad hack that doesn't execute a # request twice, but it's still a hack. self.connect() - self.connection.request(method=method, url=url, body=data, - headers=headers) + try: + self.connection.request(method=method, url=url, body=data, + headers=headers) + except (SSLError, ssl.SSLError), e: + raise ssl.SSLError(str(e)) + response = self.responseCls(self.connection.getresponse()) response.connection = self return response @@ -631,7 +675,7 @@ class NodeDriver(object): or returning a generated password. This function may raise a L{DeplyomentException}, if a create_node - call was successful, but there is a later error (like SSH failing or + call was successful, but there is a later error (like SSH failing or timing out). This exception includes a Node object which you may want to destroy if incomplete deployments are not desirable. diff --git a/libcloud/httplib_ssl.py b/libcloud/httplib_ssl.py new file mode 100644 index 0000000..c176efc --- /dev/null +++ b/libcloud/httplib_ssl.py @@ -0,0 +1,80 @@ +# -*- coding: utf-8 -*- +# Copyright (c) 2010, Tomaž Muraus +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are met: +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# * Neither the name of the Tomaž Muraus nor the +# names of its contributors may be used to endorse or promote products +# derived from this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER BE LIABLE FOR ANY +# DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND +# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +# Based on example from post "HTTPS Certificate Verification in Python With urllib2" - +# http://www.muchtooscrawled.com/2010/03/https-certificate-verification-in-python-with-urllib2/ + +import socket +import ssl +import httplib +import urllib2 + +class VerifiedHTTPSConnection(httplib.HTTPSConnection): + def connect(self): + from libcloud.base import CA_CERTS_FILE_PATH + + sock = socket.create_connection((self.host, self.port), + self.timeout) + if self._tunnel_host: + self.sock = sock + self._tunnel() + + self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file, \ + cert_reqs = ssl.CERT_REQUIRED, \ + ca_certs = CA_CERTS_FILE_PATH, \ + ssl_version = ssl.PROTOCOL_TLSv1) + + cert = self.sock.getpeercert() + if not self._verify_hostname(self.host, cert): + raise ssl.SSLError('Failed to verify hostname') + + def _verify_hostname(self, hostname, cert): + common_name = self._get_commonName(cert) + alt_names = self._get_subjectAltName(cert) + + if (hostname == common_name) or hostname in alt_names: + return True + + return False + + def _get_subjectAltName(self, cert): + if not cert.has_key('subjectAltName'): + return None + + alt_names = [] + for value in cert['subjectAltName']: + if value[0].lower() == 'dns': + alt_names.append(value[0]) + + return alt_names + + def _get_commonName(self, cert): + if not cert.has_key('subject'): + return None + + for value in cert['subject']: + if value[0][0].lower() == 'commonname': + return value[0][1] + return None -- 1.7.3.2
