Re: [dev] disable SSL verify for OpenStack

2017-07-12 Thread Markos Gogoulos 
Hi Anthony,

I didn't have the chance to try the recently merged changes to be honest!
We have diverged a bit from the OpenStack driver on mist.io, and took a
slightly different approach to create our own OpenStackNetwork and
OpenStackSubnet objects, asking neutron endpoints for
listing/creation/deletion of subnets and networks.  A lot of these things
are too on libcloud trunk now, but not the ability to ask neutron endpoints
-instead of nova-  and we are seeking ways to merge our work with the
existing libcloud trunk code and open PRs for the added functionality.

Regards
Markos




On Wed, Jul 12, 2017 at 12:27 PM, anthony shaw 
wrote:

> Have you seen the recently merged changes to the openstack driver Markos?
>
> I'm still trying to clean it up, but it's going to take a while
>
> On Tue, Jul 11, 2017 at 5:41 PM, Markos Gogoulos 
> wrote:
>
> > Thanks for the replies!
> >
> > Anthony, this method works well for other providers (we use it for OnApp
> > and vCloud) but unfortunately not for OpenStack. Code is more complex
> > regarding the connection and somehow/somewhere whatever I set as the
> > verification gets lost/reset
> >
> > Jay, I was hoping to avoid this, but ends up it's the only way to get
> > around! Thanks for the snippet, I'll perform a similar patch, since code
> is
> > different for the new libcloud version.
> >
> > Regards,
> > Markos
> >
> >
> >
> > On Mon, Jul 10, 2017 at 10:38 PM, Jay Rolette 
> wrote:
> >
> > > On Mon, Jul 10, 2017 at 2:34 PM, Jay Rolette 
> > wrote:
> > >
> > > >
> > > > On Mon, Jul 10, 2017 at 9:37 AM, Markos Gogoulos 
> > > > wrote:
> > > >
> > > >> Hi all,
> > > >>
> > > >> I'm trying to disable SSL verification via an optional switch, for
> > > >> OpenStack connections. I cannot use the 'global' VERIFY_SSL_CERT in
> my
> > > >> case, because multiple OpenStack connections take place at the same
> > > time,
> > > >> some of them should perform SSL verification while others shouldn't.
> > > >>
> > > >> That is, by instantiating a libcloud connection driver, I'd like to
> > > >> specify
> > > >> a verify False/True switch, and this should affect  the resulting
> > > requests
> > > >> call.
> > > >>
> > > >> As far as I know there's not a directional relationship between a
> > > >> connection and driver, so this makes things harder and although I
> can
> > > >> 'store' the switch on the driver object, connection cannot read it.
> > > >>
> > > >> Do you have any ideas or suggestions what would be a simple way of
> > > >> achieving the above?
> > > >>
> > > >> Regards,
> > > >> Markos
> > > >>
> > > >
> > > > It's not quite as simple as you'd hope due to the connection vs.
> driver
> > > > split, but here are diffs for how we did it. This should be relative
> > > > to apache-libcloud-1.0.0-pre1.
> > > >
> > > > Jay
> > > >
> > > > diff -r 17df54435983 -r 28d56440b52b libcloud/common/aws.py
> > > > --- a/libcloud/common/aws.pyTue Mar 22 22:56:44 2016 +
> > > > +++ b/libcloud/common/aws.pyWed Mar 23 22:46:52 2016 +
> > > > @@ -134,15 +134,19 @@
> > > >
> > > >  class AWSTokenConnection(ConnectionUserAndKey):
> > > >  def __init__(self, user_id, key, secure=True,
> > > > - host=None, port=None, url=None, timeout=None,
> > > > proxy_url=None,
> > > > - token=None, retry_delay=None, backoff=None):
> > > > -self.token = token
> > > > + host=None, port=None, url=None, timeout=None,
> > > **kwargs):
> > > > + # -redacted-
> > > > + # add kwargs to the various Libcloud Connection
> > classes
> > > > + # so we can extend parameters
> > > > + #
> > > > + # proxy_url=None, token=None, retry_delay=None,
> > > > backoff=None):
> > > > +self.token = kwargs.get('token', None)
> > > >  super(AWSTokenConnection, self).__init__(user_id, key,
> > > > secure=secure,
> > > >   host=host,
> port=port,
> > > > url=url,
> > > > - timeout=timeout,
> > > > -
> > >  retry_delay=retry_delay,
> > > > - backoff=backoff,
> > > > -
>  proxy_url=proxy_url)
> > > > + timeout=timeout,
> > > > **kwargs)
> > > > + #
> > > > retry_delay=retry_delay,
> > > > + # backoff=backoff,
> > > > + #
> > proxy_url=proxy_url)
> > > >
> > > >  def add_default_params(self, params):
> > > >  # Even though we are adding it to the headers, we need it
> here
> > > too
> > > > diff -r 17df54435983 -r 28d56440b52b libcloud/common/base.py
> > > > --- a/libcloud/common/base.py   Tue Mar 22 22:56:44 2016 +
>

Re: [dev] disable SSL verify for OpenStack

2017-07-12 Thread anthony shaw 
Have you seen the recently merged changes to the openstack driver Markos?

I'm still trying to clean it up, but it's going to take a while

On Tue, Jul 11, 2017 at 5:41 PM, Markos Gogoulos  wrote:

> Thanks for the replies!
>
> Anthony, this method works well for other providers (we use it for OnApp
> and vCloud) but unfortunately not for OpenStack. Code is more complex
> regarding the connection and somehow/somewhere whatever I set as the
> verification gets lost/reset
>
> Jay, I was hoping to avoid this, but ends up it's the only way to get
> around! Thanks for the snippet, I'll perform a similar patch, since code is
> different for the new libcloud version.
>
> Regards,
> Markos
>
>
>
> On Mon, Jul 10, 2017 at 10:38 PM, Jay Rolette  wrote:
>
> > On Mon, Jul 10, 2017 at 2:34 PM, Jay Rolette 
> wrote:
> >
> > >
> > > On Mon, Jul 10, 2017 at 9:37 AM, Markos Gogoulos 
> > > wrote:
> > >
> > >> Hi all,
> > >>
> > >> I'm trying to disable SSL verification via an optional switch, for
> > >> OpenStack connections. I cannot use the 'global' VERIFY_SSL_CERT in my
> > >> case, because multiple OpenStack connections take place at the same
> > time,
> > >> some of them should perform SSL verification while others shouldn't.
> > >>
> > >> That is, by instantiating a libcloud connection driver, I'd like to
> > >> specify
> > >> a verify False/True switch, and this should affect  the resulting
> > requests
> > >> call.
> > >>
> > >> As far as I know there's not a directional relationship between a
> > >> connection and driver, so this makes things harder and although I can
> > >> 'store' the switch on the driver object, connection cannot read it.
> > >>
> > >> Do you have any ideas or suggestions what would be a simple way of
> > >> achieving the above?
> > >>
> > >> Regards,
> > >> Markos
> > >>
> > >
> > > It's not quite as simple as you'd hope due to the connection vs. driver
> > > split, but here are diffs for how we did it. This should be relative
> > > to apache-libcloud-1.0.0-pre1.
> > >
> > > Jay
> > >
> > > diff -r 17df54435983 -r 28d56440b52b libcloud/common/aws.py
> > > --- a/libcloud/common/aws.pyTue Mar 22 22:56:44 2016 +
> > > +++ b/libcloud/common/aws.pyWed Mar 23 22:46:52 2016 +
> > > @@ -134,15 +134,19 @@
> > >
> > >  class AWSTokenConnection(ConnectionUserAndKey):
> > >  def __init__(self, user_id, key, secure=True,
> > > - host=None, port=None, url=None, timeout=None,
> > > proxy_url=None,
> > > - token=None, retry_delay=None, backoff=None):
> > > -self.token = token
> > > + host=None, port=None, url=None, timeout=None,
> > **kwargs):
> > > + # -redacted-
> > > + # add kwargs to the various Libcloud Connection
> classes
> > > + # so we can extend parameters
> > > + #
> > > + # proxy_url=None, token=None, retry_delay=None,
> > > backoff=None):
> > > +self.token = kwargs.get('token', None)
> > >  super(AWSTokenConnection, self).__init__(user_id, key,
> > > secure=secure,
> > >   host=host, port=port,
> > > url=url,
> > > - timeout=timeout,
> > > -
> >  retry_delay=retry_delay,
> > > - backoff=backoff,
> > > - proxy_url=proxy_url)
> > > + timeout=timeout,
> > > **kwargs)
> > > + #
> > > retry_delay=retry_delay,
> > > + # backoff=backoff,
> > > + #
> proxy_url=proxy_url)
> > >
> > >  def add_default_params(self, params):
> > >  # Even though we are adding it to the headers, we need it here
> > too
> > > diff -r 17df54435983 -r 28d56440b52b libcloud/common/base.py
> > > --- a/libcloud/common/base.py   Tue Mar 22 22:56:44 2016 +
> > > +++ b/libcloud/common/base.py   Wed Mar 23 22:46:52 2016 +
> > > @@ -528,7 +528,12 @@
> > >  allow_insecure = True
> > >
> > >  def __init__(self, secure=True, host=None, port=None, url=None,
> > > - timeout=None, proxy_url=None, retry_delay=None,
> > > backoff=None):
> > > + timeout=None, **kwargs):
> > > + # -redacted-
> > > + # add kwargs to the various Libcloud Connection
> classes
> > > + # so we can extend parameters
> > > + #
> > > + # timeout=None, proxy_url=None, retry_delay=None,
> > > backoff=None):
> > >  self.secure = secure and 1 or 0
> > >  self.ua = []
> > >  self.context = {}
> > > @@ -557,9 +562,12 @@
> > >   self.request_path) = self._tuple_from_url(url)
> > >

Re: [dev] disable SSL verify for OpenStack

2017-07-10 Thread Jay Rolette 
On Mon, Jul 10, 2017 at 9:37 AM, Markos Gogoulos  wrote:

> Hi all,
>
> I'm trying to disable SSL verification via an optional switch, for
> OpenStack connections. I cannot use the 'global' VERIFY_SSL_CERT in my
> case, because multiple OpenStack connections take place at the same time,
> some of them should perform SSL verification while others shouldn't.
>
> That is, by instantiating a libcloud connection driver, I'd like to specify
> a verify False/True switch, and this should affect  the resulting requests
> call.
>
> As far as I know there's not a directional relationship between a
> connection and driver, so this makes things harder and although I can
> 'store' the switch on the driver object, connection cannot read it.
>
> Do you have any ideas or suggestions what would be a simple way of
> achieving the above?
>
> Regards,
> Markos
>

It's not quite as simple as you'd hope due to the connection vs. driver
split, but here are diffs for how we did it. This should be relative
to apache-libcloud-1.0.0-pre1.

Jay

diff -r 17df54435983 -r 28d56440b52b libcloud/common/aws.py
--- a/libcloud/common/aws.pyTue Mar 22 22:56:44 2016 +
+++ b/libcloud/common/aws.pyWed Mar 23 22:46:52 2016 +
@@ -134,15 +134,19 @@

 class AWSTokenConnection(ConnectionUserAndKey):
 def __init__(self, user_id, key, secure=True,
- host=None, port=None, url=None, timeout=None,
proxy_url=None,
- token=None, retry_delay=None, backoff=None):
-self.token = token
+ host=None, port=None, url=None, timeout=None, **kwargs):
+ # -redacted-
+ # add kwargs to the various Libcloud Connection classes
+ # so we can extend parameters
+ #
+ # proxy_url=None, token=None, retry_delay=None,
backoff=None):
+self.token = kwargs.get('token', None)
 super(AWSTokenConnection, self).__init__(user_id, key,
secure=secure,
  host=host, port=port,
url=url,
- timeout=timeout,
- retry_delay=retry_delay,
- backoff=backoff,
- proxy_url=proxy_url)
+ timeout=timeout, **kwargs)
+ # retry_delay=retry_delay,
+ # backoff=backoff,
+ # proxy_url=proxy_url)

 def add_default_params(self, params):
 # Even though we are adding it to the headers, we need it here too
diff -r 17df54435983 -r 28d56440b52b libcloud/common/base.py
--- a/libcloud/common/base.py   Tue Mar 22 22:56:44 2016 +
+++ b/libcloud/common/base.py   Wed Mar 23 22:46:52 2016 +
@@ -528,7 +528,12 @@
 allow_insecure = True

 def __init__(self, secure=True, host=None, port=None, url=None,
- timeout=None, proxy_url=None, retry_delay=None,
backoff=None):
+ timeout=None, **kwargs):
+ # -redacted-
+ # add kwargs to the various Libcloud Connection classes
+ # so we can extend parameters
+ #
+ # timeout=None, proxy_url=None, retry_delay=None,
backoff=None):
 self.secure = secure and 1 or 0
 self.ua = []
 self.context = {}
@@ -557,9 +562,12 @@
  self.request_path) = self._tuple_from_url(url)

 self.timeout = timeout or self.timeout
-self.retry_delay = retry_delay
-self.backoff = backoff
-self.proxy_url = proxy_url
+self.retry_delay = kwargs.get('retry_delay', None)
+self.backoff = kwargs.get('backoff', None)
+self.proxy_url = kwargs.get('proxy_url', None)
+
+# -redacted-
+self.verify_ssl_cert = kwargs.get('verify_ssl_cert', None)

 def set_http_proxy(self, proxy_url):
 """
@@ -660,6 +668,10 @@
 if self.proxy_url:
 kwargs.update({'proxy_url': self.proxy_url})

+# -redacted-
+if self.verify_ssl_cert is not None:
+kwargs['verify_ssl_cert'] = self.verify_ssl_cert
+
 connection = self.conn_classes[secure](**kwargs)
 # You can uncoment this line, if you setup a reverse proxy server
 # which proxies to your endpoint, and lets you easily capture
@@ -1023,7 +1035,9 @@
 Base connection class which accepts a single ``key`` argument.
 """
 def __init__(self, key, secure=True, host=None, port=None, url=None,
- timeout=None, proxy_url=None, backoff=None,
retry_delay=None):
+ timeout=None, **kwargs):
+ # -redacted-
+ # timeout=None, proxy_url=None, backoff=None,
retry_delay=None):
 """
 Initialize `user_id` and