I just wanted to let you know that we also needed support for HTTP proxy for some things we are working on at DivvyCloud so I have pushed a preliminary support to trunk - https://github.com/apache/libcloud/commit/f3f600028384bc19c0f7ff051306a401d57afd35
There are some know limitations (doesn't support with Python 2.6 and it doesn't work with proxies which require an authentication), but besides that, I have tested it with 2.7, PyPy, 3.1, 3.2, 3.3 and 3.4 and everything appears to be working fine. You can find some documentation on how to use it at https://libcloud.readthedocs.org/en/latest/other/using-http-proxy.html On Wed, Jul 16, 2014 at 2:39 PM, Cedric Lebrun <[email protected]> wrote: > I have also installed CNTLM to bypass proxy authentication. > Not the best solution, but a temporary trick to be able to progress on my > prototype :-) > > Cheers > Cedric > > -----Original Message----- > From: [email protected] [mailto:[email protected]] > Sent: mercredi 16 juillet 2014 14:33 > To: [email protected] > Subject: Re: [dev] Libcloud behind ISA Proxy > > Hi Cedric, > > That's great news. > > I thought you'd mentioned that the proxy you were connecting through also > needed authentication credentials. - Is that the case and if so what > additional changes did you need to make to the code? > > Cheers, > Phil > > On 16 Jul 2014, at 13:29, Cedric Lebrun <[email protected]> wrote: > > > With Python 3.4 (on Windows), I have finally succeeded using your patch. > > > > I have just had to replace > > > > self._set_hostport(proxy_host, None) > > > > by > > > > self.host = proxyHost > > self.port = proxyPort > > > > > > With the appropriate values for proxyHost and proxyPort. > > > > > > Cedric > > > > -----Original Message----- > > From: Cedric Lebrun [mailto:[email protected]] > > Sent: mercredi 16 juillet 2014 08:22 > > To: [email protected] > > Subject: RE: [dev] Libcloud behind ISA Proxy > > > > Hi Phil, > > > > Sorry for my late reply because of vacations :-) I've tried with Python > 2.7 on a CentOS environment as well as with Python 3.4 on a Windows 7 > environment. But in both case, no success so far. > > Will continue investigating. > > > > Thank you. > > Cedric > > > > -----Original Message----- > > From: [email protected] [mailto:[email protected]] > > Sent: mardi 8 juillet 2014 15:29 > > To: [email protected] > > Subject: Re: [dev] Libcloud behind ISA Proxy > > > > Hi Cedric, > > > > I'm using Python 2.7.2. There seems to be differences between the two. > I would adapt your code to do the job of _set_hostport and set host and > port separately. They are public attributes in any case. > > > > Cheers, > > Phil > > > > On 8 Jul 2014, at 13:50, Cedric Lebrun <[email protected]> wrote: > > > >> Hi Philip, > >> > >> I'm using Python 2.7 as well (2.7.7). > >> And Libcloud 0.15.1 > >> > >> Cedric > >> > >> -----Original Message----- > >> From: [email protected] [mailto:[email protected]] > >> Sent: mardi 8 juillet 2014 12:03 > >> To: [email protected] > >> Subject: Re: [dev] Libcloud behind ISA Proxy > >> > >> Hi Cedric, > >> > >> What version of Python are you using? > >> > >> In my Python 2.7 installation, _set_hostport is a method of > httplib.HTTPConnection. It basically does some format checking and assigns > host and port number. > >> > >> Clearly some more work will be needed for a generic solution across > >> supported versions of Python :) > >> > >> Cheers, > >> Phil > >> > >> > >> def _set_hostport(self, host, port): > >> if port is None: > >> i = host.rfind(':') > >> j = host.rfind(']') # ipv6 addresses have [...] > >> if i > j: > >> try: > >> port = int(host[i+1:]) > >> except ValueError: > >> raise InvalidURL("nonnumeric port: '%s'" % host[i+1:]) > >> host = host[:i] > >> else: > >> port = self.default_port > >> if host and host[0] == '[' and host[-1] == ']': > >> host = host[1:-1] > >> self.host = host > >> self.port = port > >> > >> > >> On 8 Jul 2014, at 10:45, Cedric Lebrun <[email protected]> wrote: > >> > >>> Hi Philip, > >>> > >>> Thanks for your help. > >>> But after patching httplib_ssl, I fall into now another error: > >>> > >>> AttributeError: LibcloudHTTPSConnection instance has no attribute > '_set_hostport' > >>> > >>> > >>> Cedric > >>> > >>> -----Original Message----- > >>> From: [email protected] [mailto:[email protected]] > >>> Sent: mardi 8 juillet 2014 09:56 > >>> To: [email protected] > >>> Subject: Re: [dev] Libcloud behind ISA Proxy > >>> > >>> Hi Cedric, > >>> > >>> I ran into the same issue and patched libcloud.httplib_ssl. I've > included the code snippets below. The proxy I use doesn't require > authentication but hopefully you can adapt to make it do what you need. I > tested with Python 2.7 - may need tweaking for Python 3. > >>> > >>> Cheers, > >>> Phil > >>> > >>> > >>> I modified the LibcloudHTTPSConnection.__init__ to pick up the proxy > settings from the environment: > >>> > >>> > >>> class LibcloudHTTPSConnection(httplib.HTTPSConnection): > >>> """ > >>> LibcloudHTTPSConnection > >>> > >>> Subclass of HTTPSConnection which verifies certificate names if > >>> and only if CA certificates are available. > >>> """ > >>> verify = True # verify by default > >>> ca_cert = None # no default CA Certificate > >>> > >>> def __init__(self, *args, **kwargs): > >>> """ > >>> Constructor > >>> """ > >>> self._setup_verify() > >>> httplib.HTTPSConnection.__init__(self, *args, **kwargs) > >>> > >>> # Support for HTTPS Proxy > >>> if 'https_proxy' in os.environ: > >>> from urlparse import urlparse > >>> > >>> self.set_tunnel(self.host, port=self.port) > >>> > >>> proxy_host = urlparse(os.environ['https_proxy']).netloc > >>> self._set_hostport(proxy_host, None) > >>> > >>> . > >>> . > >>> . > >>> > >>> And then modified the connect call to use the tunnel: > >>> > >>> > >>> def connect(self): > >>> """ > >>> Connect > >>> > >>> Checks if verification is toggled; if not, just call > >>> httplib.HTTPSConnection's connect > >>> """ > >>> if not self.verify: > >>> return httplib.HTTPSConnection.connect(self) > >>> > >>> # otherwise, create a connection and verify the hostname > >>> # use socket.create_connection (in 2.6+) if possible > >>> if getattr(socket, 'create_connection', None): > >>> sock = socket.create_connection((self.host, self.port), > >>> self.timeout) > >>> else: > >>> sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) > >>> sock.connect((self.host, self.port)) > >>> > >>> # Support for HTTPS Proxy > >>> if self._tunnel_host: > >>> self.sock = sock > >>> self._tunnel() > >>> > >>> self.sock = ssl.wrap_socket(sock, > >>> self.key_file, > >>> self.cert_file, > >>> cert_reqs=ssl.CERT_REQUIRED, > >>> ca_certs=self.ca_cert, > >>> ssl_version=ssl.PROTOCOL_TLSv1) > >>> cert = self.sock.getpeercert() > >>> if not self._verify_hostname(self.host, cert): > >>> raise ssl.SSLError('Failed to verify hostname') > >>> > >>> > >>> On 8 Jul 2014, at 07:23, Cedric Lebrun <[email protected]> wrote: > >>> > >>>> Hi All, > >>>> > >>>> I try to use Libcloud to connect to EC2. > >>>> My environments (CentOS 6.5 and Ubuntu 14.04) are behind a proxy > server that requires authentication (MS ISA Proxy Server). > >>>> I tried with defining environment variables HTTP_PROXY and > HTTPS_PROXY, but still doesn't work. > >>>> Is there a way to set a proxy and proxy credentials directly using > Libcloud ? Or should I have to patch the http_lib to implement UrlLib2 > ProxyHandler ? > >>>> > >>>> Thanks for your help, > >>>> CL > >>> > >>> -- > >>> Scanned by iCritical. > >> > >> -- > >> Scanned by iCritical. > > > > -- > > Scanned by iCritical. > > -- > Scanned by iCritical. >
