[libcloud] Git Push Summary

2017-04-21 Thread anthonyshaw 
Repository: libcloud
Updated Tags:  refs/tags/v2.0.0-tentative [created] 241b438d1

[2/2] libcloud git commit: prep for 2.0.0

2017-04-21 Thread anthonyshaw 
prep for 2.0.0


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/611035a3
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/611035a3
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/611035a3

Branch: refs/heads/trunk
Commit: 611035a3380c16a22373efb33e34a376abe9008c
Parents: b1bbb7f
Author: Anthony Shaw 
Authored: Sat Apr 22 12:35:58 2017 +1000
Committer: Anthony Shaw 
Committed: Sat Apr 22 12:35:58 2017 +1000

--
 libcloud/__init__.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/libcloud/blob/611035a3/libcloud/__init__.py
--
diff --git a/libcloud/__init__.py b/libcloud/__init__.py
index c2c3c9c..eb91c5c 100644
--- a/libcloud/__init__.py
+++ b/libcloud/__init__.py
@@ -36,7 +36,7 @@ __all__ = [
 '__version__',
 'enable_debug'
 ]
-__version__ = '2.0.0rc2'
+__version__ = '2.0.0'
 
 
 def enable_debug(fo):

[1/2] libcloud git commit: prep for 2.0.0

2017-04-21 Thread anthonyshaw 
Repository: libcloud
Updated Branches:
  refs/heads/trunk 741032969 -> 611035a33
Updated Tags:  refs/tags/2.0.0-tentative [created] 0e2879bca


prep for 2.0.0


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/b1bbb7f1
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/b1bbb7f1
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/b1bbb7f1

Branch: refs/heads/trunk
Commit: b1bbb7f1bd16a69a49fb3c2bbd38ea056faada7b
Parents: 7410329
Author: Anthony Shaw 
Authored: Sat Apr 22 12:35:50 2017 +1000
Committer: Anthony Shaw 
Committed: Sat Apr 22 12:35:50 2017 +1000

--
 CHANGES.rst | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/libcloud/blob/b1bbb7f1/CHANGES.rst
--
diff --git a/CHANGES.rst b/CHANGES.rst
index a13e34c..661ed6d 100644
--- a/CHANGES.rst
+++ b/CHANGES.rst
@@ -1,8 +1,8 @@
 Changelog
 =
 
-Changes in current version of Apache Libcloud
--
+Changes in Apache Libcloud 2.0.0
+
 
 Common
 ~~

[jira] [Commented] (LIBCLOUD-912) [python] Requests tries to connect to 127.0.0.1:443

2017-04-21 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/LIBCLOUD-912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15979722#comment-15979722
 ] 

ASF GitHub Bot commented on LIBCLOUD-912:
-

Github user asfgit closed the pull request at:

https://github.com/apache/libcloud/pull/1037


> [python] Requests tries to connect to 127.0.0.1:443
> ---
>
> Key: LIBCLOUD-912
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-912
> Project: Libcloud
>  Issue Type: Bug
> Environment: python 2.7
> apache-libcloud==2.0.0rc2
>Reporter: Alexis Lesieur
>  Labels: newbie, python
>
> Hi!
> Sorry if this story is not created properly.
> I'm not that familiar with how you guys manage your Jira. Feel free to give 
> me directions or just modify the ticket :-)
> So we have been trying to upgrade the version of our libcloud python package, 
> but it breaks for our rackspace/openstack environment.
> I have been digging for a while and I couldn't find where the difference was 
> between 1.x.x and 2.0.0rc2.
> Observed behavior:
> 1.x.x:
> 3 connections are created:
> - to 127.0.0.1:443
> - to host:3000 (ex_force_auth_url)
> - to host:8774/v2/... (don't know where it comes from)
> 2.0.0rc2:
> 2 connections
> - to 127.0.0.1:443
> - to host:3000 (ex_force_auth_url)
> and the last connection is done using 127.0.0.1:443 again, using the same 
> connector created in the first one
> I don't understand the change in behavior between the two versions.
> I tried use the ex_force_base_url flag, and while it does look like the path 
> of that url is respected, the host is always replaced by 127.0.0.1.
> Any idea of what is happening?
> Thanks!



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (LIBCLOUD-912) [python] Requests tries to connect to 127.0.0.1:443

2017-04-21 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/LIBCLOUD-912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15979721#comment-15979721
 ] 

ASF subversion and git services commented on LIBCLOUD-912:
--

Commit b8ba6c10accdf6d9ce8a40787e2300a594437ebe in libcloud's branch 
refs/heads/trunk from [~anthonypjshaw]
[ https://git-wip-us.apache.org/repos/asf?p=libcloud.git;h=b8ba6c1 ]

issues with LIBCLOUD-912


> [python] Requests tries to connect to 127.0.0.1:443
> ---
>
> Key: LIBCLOUD-912
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-912
> Project: Libcloud
>  Issue Type: Bug
> Environment: python 2.7
> apache-libcloud==2.0.0rc2
>Reporter: Alexis Lesieur
>  Labels: newbie, python
>
> Hi!
> Sorry if this story is not created properly.
> I'm not that familiar with how you guys manage your Jira. Feel free to give 
> me directions or just modify the ticket :-)
> So we have been trying to upgrade the version of our libcloud python package, 
> but it breaks for our rackspace/openstack environment.
> I have been digging for a while and I couldn't find where the difference was 
> between 1.x.x and 2.0.0rc2.
> Observed behavior:
> 1.x.x:
> 3 connections are created:
> - to 127.0.0.1:443
> - to host:3000 (ex_force_auth_url)
> - to host:8774/v2/... (don't know where it comes from)
> 2.0.0rc2:
> 2 connections
> - to 127.0.0.1:443
> - to host:3000 (ex_force_auth_url)
> and the last connection is done using 127.0.0.1:443 again, using the same 
> connector created in the first one
> I don't understand the change in behavior between the two versions.
> I tried use the ex_force_base_url flag, and while it does look like the path 
> of that url is respected, the host is always replaced by 127.0.0.1.
> Any idea of what is happening?
> Thanks!



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[5/9] libcloud git commit: simplify the test

2017-04-21 Thread anthonyshaw 
simplify the test


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/8560d207
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/8560d207
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/8560d207

Branch: refs/heads/trunk
Commit: 8560d2077873049a06e5fe62899d2197584c9d43
Parents: daaac0d
Author: Anthony Shaw 
Authored: Sat Apr 22 08:30:49 2017 +1000
Committer: Anthony Shaw 
Committed: Sat Apr 22 08:30:49 2017 +1000

--
 libcloud/test/compute/test_openstack.py | 6 ++
 1 file changed, 2 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/libcloud/blob/8560d207/libcloud/test/compute/test_openstack.py
--
diff --git a/libcloud/test/compute/test_openstack.py 
b/libcloud/test/compute/test_openstack.py
index 3351db9..029db10 100644
--- a/libcloud/test/compute/test_openstack.py
+++ b/libcloud/test/compute/test_openstack.py
@@ -69,15 +69,13 @@ class OpenStackAuthTests(unittest.TestCase):
 ex_force_auth_url='http://x.y.z.y:5000',
 ex_tenant_name='admin')
 self.assertEqual(d._ex_force_auth_url, forced_auth)
+
 with requests_mock.Mocker() as mock:
-body1 = 
ComputeFileFixtures('openstack').load('v1_slug_servers_ips.xml')
 body2 = ComputeFileFixtures('openstack').load('_v2_0__auth.json')
 
-mock.register_uri('GET', 
'https://test_endpoint.com/v2/1337/servers/test/ips', text=body1,
-  headers={'content-type': 'application/xml; 
charset=UTF-8'})
 mock.register_uri('POST', 'http://x.y.z.y:5000/v2.0/tokens', 
text=body2,
   headers={'content-type': 'application/json; 
charset=UTF-8'})
-d.ex_list_ip_addresses('test')
+d.connection._populate_hosts_and_request_paths()
 self.assertEqual(d.connection.host, 'test_endpoint.com')

[3/9] libcloud git commit: refactor the test to not call internal methods

2017-04-21 Thread anthonyshaw 
refactor the test to not call internal methods


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/eaaf6990
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/eaaf6990
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/eaaf6990

Branch: refs/heads/trunk
Commit: eaaf69900b59c98ba4e91123869a35f20af0d349
Parents: 3fd07eb
Author: Anthony Shaw 
Authored: Sat Apr 22 08:05:26 2017 +1000
Committer: Anthony Shaw 
Committed: Sat Apr 22 08:05:26 2017 +1000

--
 libcloud/test/compute/test_openstack.py | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/libcloud/blob/eaaf6990/libcloud/test/compute/test_openstack.py
--
diff --git a/libcloud/test/compute/test_openstack.py 
b/libcloud/test/compute/test_openstack.py
index b42f43d..d00a467 100644
--- a/libcloud/test/compute/test_openstack.py
+++ b/libcloud/test/compute/test_openstack.py
@@ -69,10 +69,14 @@ class OpenStackAuthTests(unittest.TestCase):
 ex_tenant_name='admin')
 self.assertEqual(d._ex_force_auth_url, forced_auth)
 with requests_mock.Mocker() as mock:
+body1 = 
ComputeFileFixtures('openstack').load('v1_slug_servers_ips.xml')
 body2 = ComputeFileFixtures('openstack').load('_v2_0__auth.json')
+
+mock.register_uri('GET', 
'https://test_endpoint.com/v2/1337/servers/test/ips', text=body1,
+  headers={'content-type': 'application/xml; 
charset=UTF-8'})
 mock.register_uri('POST', 'http://x.y.z.y:5000/v2.0/tokens', 
text=body2,
   headers={'content-type': 'application/json; 
charset=UTF-8'})
-d.connection._populate_hosts_and_request_paths()
+d.ex_list_ip_addresses('test')
 self.assertEqual(d.connection.host, 'test_endpoint.com')

[2/9] libcloud git commit: update test to validate custom host servce endpoint and fix issue

2017-04-21 Thread anthonyshaw 
update test to validate custom host servce endpoint and fix issue


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/3fd07eb5
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/3fd07eb5
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/3fd07eb5

Branch: refs/heads/trunk
Commit: 3fd07eb590f1cf5898a329a9649df881a6c0f12f
Parents: b8ba6c1
Author: Anthony Shaw 
Authored: Fri Apr 21 20:00:33 2017 +1000
Committer: Anthony Shaw 
Committed: Fri Apr 21 20:00:33 2017 +1000

--
 libcloud/common/openstack.py|  1 +
 libcloud/test/compute/test_openstack.py | 12 +---
 2 files changed, 6 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/libcloud/blob/3fd07eb5/libcloud/common/openstack.py
--
diff --git a/libcloud/common/openstack.py b/libcloud/common/openstack.py
index 07796b5..1350205 100644
--- a/libcloud/common/openstack.py
+++ b/libcloud/common/openstack.py
@@ -297,6 +297,7 @@ class OpenStackBaseConnection(ConnectionUserAndKey):
 def _set_up_connection_info(self, url):
 result = self._tuple_from_url(url)
 (self.host, self.port, self.secure, self.request_path) = result
+self.connect()
 
 def _populate_hosts_and_request_paths(self):
 """

http://git-wip-us.apache.org/repos/asf/libcloud/blob/3fd07eb5/libcloud/test/compute/test_openstack.py
--
diff --git a/libcloud/test/compute/test_openstack.py 
b/libcloud/test/compute/test_openstack.py
index 9dcfb53..b42f43d 100644
--- a/libcloud/test/compute/test_openstack.py
+++ b/libcloud/test/compute/test_openstack.py
@@ -63,19 +63,17 @@ class OpenStackAuthTests(unittest.TestCase):
 def test_auth_host_passed(self):
 forced_auth = 'http://x.y.z.y:5000'
 d = OpenStack_1_0_NodeDriver(
-'user', 'correct_password', 
-ex_force_auth_version='2.0_password', 
-ex_force_auth_url='http://x.y.z.y:5000', 
+'user', 'correct_password',
+ex_force_auth_version='2.0_password',
+ex_force_auth_url='http://x.y.z.y:5000',
 ex_tenant_name='admin')
 self.assertEqual(d._ex_force_auth_url, forced_auth)
 with requests_mock.Mocker() as mock:
-body1 = "[]"
 body2 = ComputeFileFixtures('openstack').load('_v2_0__auth.json')
-mock.register_uri('GET', 
'https://test_endpoint.com/v2/1337/servers/detail', text=body1,
-  headers={'content-type': 'application/json; 
charset=UTF-8'})
 mock.register_uri('POST', 'http://x.y.z.y:5000/v2.0/tokens', 
text=body2,
   headers={'content-type': 'application/json; 
charset=UTF-8'})
-d.list_nodes()
+d.connection._populate_hosts_and_request_paths()
+self.assertEqual(d.connection.host, 'test_endpoint.com')
 
 
 class OpenStack_1_0_Tests(TestCaseMixin):

[6/9] libcloud git commit: remove empty line

2017-04-21 Thread anthonyshaw 
remove empty line


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/a28a0799
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/a28a0799
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/a28a0799

Branch: refs/heads/trunk
Commit: a28a0799298cebcb2a102acc4a35465e50e02688
Parents: 8560d20
Author: Anthony Shaw 
Authored: Sat Apr 22 08:34:53 2017 +1000
Committer: Anthony Shaw 
Committed: Sat Apr 22 08:34:53 2017 +1000

--
 libcloud/test/compute/test_openstack.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/libcloud/blob/a28a0799/libcloud/test/compute/test_openstack.py
--
diff --git a/libcloud/test/compute/test_openstack.py 
b/libcloud/test/compute/test_openstack.py
index 029db10..b6b6917 100644
--- a/libcloud/test/compute/test_openstack.py
+++ b/libcloud/test/compute/test_openstack.py
@@ -69,7 +69,7 @@ class OpenStackAuthTests(unittest.TestCase):
 ex_force_auth_url='http://x.y.z.y:5000',
 ex_tenant_name='admin')
 self.assertEqual(d._ex_force_auth_url, forced_auth)
-
+
 with requests_mock.Mocker() as mock:
 body2 = ComputeFileFixtures('openstack').load('_v2_0__auth.json')

[GitHub] libcloud pull request #1037: Fix issues with OpenStack auth raised in LIBCLO...

2017-04-21 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/libcloud/pull/1037


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[8/9] libcloud git commit: Merge branch 'openstack_auth_passing' into trunk Closes #1037

2017-04-21 Thread anthonyshaw 
Merge branch 'openstack_auth_passing' into trunk
Closes #1037


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/3fd20259
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/3fd20259
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/3fd20259

Branch: refs/heads/trunk
Commit: 3fd20259dad72964048299395563908cae100f09
Parents: 660a144 9be9f9f
Author: Anthony Shaw 
Authored: Sat Apr 22 12:30:00 2017 +1000
Committer: Anthony Shaw 
Committed: Sat Apr 22 12:30:00 2017 +1000

--
 libcloud/common/openstack.py|  1 +
 libcloud/test/common/test_openstack.py  |  7 +++--
 .../compute/fixtures/openstack/_v2_0__auth.json |  6 ++---
 libcloud/test/compute/test_openstack.py | 28 +++-
 4 files changed, 36 insertions(+), 6 deletions(-)
--

[9/9] libcloud git commit: changes for #1037

2017-04-21 Thread anthonyshaw 
changes for #1037


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/74103296
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/74103296
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/74103296

Branch: refs/heads/trunk
Commit: 74103296985e66a9abe0126a05b20fb22362a280
Parents: 3fd2025
Author: Anthony Shaw 
Authored: Sat Apr 22 12:33:42 2017 +1000
Committer: Anthony Shaw 
Committed: Sat Apr 22 12:33:42 2017 +1000

--
 CHANGES.rst | 5 +
 1 file changed, 5 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/libcloud/blob/74103296/CHANGES.rst
--
diff --git a/CHANGES.rst b/CHANGES.rst
index 03d8990..a13e34c 100644
--- a/CHANGES.rst
+++ b/CHANGES.rst
@@ -7,6 +7,11 @@ Changes in current version of Apache Libcloud
 Common
 ~~
 
+- Fix OpenStack drivers not correctly setting URLs when used with identity 
API, would default to 127.0.0.1 and service
+  catalog URLs were not adhered to.
+  [GITHUB-1037, LIBCLOUD-912, LIBCLOUD-904]
+  (Anthony Shaw) 
+
 - Fix Aliyun ECS, Load balancer and storage adapters when using unicode UTF-8 
characters in the names of resources
   in 2.0.0rc2 < it would fail as a MalformedResponseError, Python 2.7 element 
tree was raising a unicode error
   [GITHUB-1032] [GITHUB-994]

[4/9] libcloud git commit: allow the tests to be called in another sequence

2017-04-21 Thread anthonyshaw 
allow the tests to be called in another sequence


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/daaac0d6
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/daaac0d6
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/daaac0d6

Branch: refs/heads/trunk
Commit: daaac0d61a5f9ce2afe2cc9b805c25fe774d57f9
Parents: eaaf699
Author: Anthony Shaw 
Authored: Sat Apr 22 08:09:58 2017 +1000
Committer: Anthony Shaw 
Committed: Sat Apr 22 08:09:58 2017 +1000

--
 libcloud/test/compute/test_openstack.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/libcloud/blob/daaac0d6/libcloud/test/compute/test_openstack.py
--
diff --git a/libcloud/test/compute/test_openstack.py 
b/libcloud/test/compute/test_openstack.py
index d00a467..3351db9 100644
--- a/libcloud/test/compute/test_openstack.py
+++ b/libcloud/test/compute/test_openstack.py
@@ -42,7 +42,8 @@ from libcloud.compute.drivers.openstack import (
 OpenStack_1_0_NodeDriver,
 OpenStack_1_1_NodeDriver, OpenStackSecurityGroup,
 OpenStackSecurityGroupRule, OpenStack_1_1_FloatingIpPool,
-OpenStack_1_1_FloatingIpAddress, OpenStackKeyPair
+OpenStack_1_1_FloatingIpAddress, OpenStackKeyPair,
+OpenStack_1_0_Connection
 )
 from libcloud.compute.base import Node, NodeImage, NodeSize
 from libcloud.pricing import set_pricing, clear_pricing_data
@@ -58,7 +59,7 @@ BASE_DIR = os.path.abspath(os.path.split(__file__)[0])
 
 class OpenStackAuthTests(unittest.TestCase):
 def setUp(self):
-pass
+OpenStack_1_0_NodeDriver.connectionCls = OpenStack_1_0_Connection
 
 def test_auth_host_passed(self):
 forced_auth = 'http://x.y.z.y:5000'

[7/9] libcloud git commit: write teardowns for openstack tests

2017-04-21 Thread anthonyshaw 
write teardowns for openstack tests


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/9be9f9f2
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/9be9f9f2
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/9be9f9f2

Branch: refs/heads/trunk
Commit: 9be9f9f22c8f7714afc62ba0cb02d13dd044e687
Parents: a28a079
Author: Anthony Shaw 
Authored: Sat Apr 22 12:20:06 2017 +1000
Committer: Anthony Shaw 
Committed: Sat Apr 22 12:20:06 2017 +1000

--
 libcloud/test/common/test_openstack.py  | 5 -
 libcloud/test/compute/test_openstack.py | 2 ++
 2 files changed, 6 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/libcloud/blob/9be9f9f2/libcloud/test/common/test_openstack.py
--
diff --git a/libcloud/test/common/test_openstack.py 
b/libcloud/test/common/test_openstack.py
index 6780aa6..b5e9341 100644
--- a/libcloud/test/common/test_openstack.py
+++ b/libcloud/test/common/test_openstack.py
@@ -17,7 +17,7 @@ import sys
 import unittest
 
 from mock import Mock
-
+from libcloud.common.base import LibcloudConnection
 from libcloud.common.openstack import OpenStackBaseConnection
 from libcloud.utils.py3 import PY25
 
@@ -32,6 +32,9 @@ class OpenStackBaseConnectionTest(unittest.TestCase):
 self.connection.driver = Mock()
 self.connection.driver.name = 'OpenStackDriver'
 
+def tearDown(self):
+OpenStackBaseConnection.conn_class = LibcloudConnection
+
 def test_base_connection_timeout(self):
 self.connection.connect()
 self.assertEqual(self.connection.timeout, self.timeout)

http://git-wip-us.apache.org/repos/asf/libcloud/blob/9be9f9f2/libcloud/test/compute/test_openstack.py
--
diff --git a/libcloud/test/compute/test_openstack.py 
b/libcloud/test/compute/test_openstack.py
index b6b6917..77d35e6 100644
--- a/libcloud/test/compute/test_openstack.py
+++ b/libcloud/test/compute/test_openstack.py
@@ -33,6 +33,7 @@ from libcloud.utils.py3 import httplib
 from libcloud.utils.py3 import method_type
 from libcloud.utils.py3 import u
 
+from libcloud.common.base import LibcloudConnection
 from libcloud.common.types import InvalidCredsError, MalformedResponseError, \
 LibcloudError
 from libcloud.compute.types import Provider, KeyPairDoesNotExistError, 
StorageVolumeState, \
@@ -60,6 +61,7 @@ BASE_DIR = os.path.abspath(os.path.split(__file__)[0])
 class OpenStackAuthTests(unittest.TestCase):
 def setUp(self):
 OpenStack_1_0_NodeDriver.connectionCls = OpenStack_1_0_Connection
+OpenStack_1_0_NodeDriver.connectionCls.conn_class = LibcloudConnection
 
 def test_auth_host_passed(self):
 forced_auth = 'http://x.y.z.y:5000'

[5/9] libcloud git commit: Propagate cloud_environment to Connection class.

2017-04-21 Thread anthonyshaw 
Propagate cloud_environment to Connection class.


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/018e09f6
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/018e09f6
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/018e09f6

Branch: refs/heads/trunk
Commit: 018e09f62d02ba930639e230f08c562229f63eee
Parents: a5654cb
Author: Peter Amstutz 
Authored: Fri Jan 13 16:58:29 2017 -0500
Committer: Peter Amstutz 
Committed: Thu Apr 20 14:46:22 2017 -0400

--
 libcloud/common/azure_arm.py  | 5 +++--
 libcloud/compute/drivers/azure_arm.py | 2 ++
 2 files changed, 5 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/libcloud/blob/018e09f6/libcloud/common/azure_arm.py
--
diff --git a/libcloud/common/azure_arm.py b/libcloud/common/azure_arm.py
index a73a905..1dcf817 100644
--- a/libcloud/common/azure_arm.py
+++ b/libcloud/common/azure_arm.py
@@ -143,10 +143,11 @@ class 
AzureResourceManagementConnection(ConnectionUserAndKey):
 rawResponseCls = RawResponse
 
 def __init__(self, key, secret, secure=True, tenant_id=None,
- subscription_id=None, **kwargs):
+ subscription_id=None, cloud_environment=None, **kwargs):
 super(AzureResourceManagementConnection, self) \
 .__init__(key, secret, **kwargs)
-cloud_environment = kwargs.get("cloud_environment", "default")
+if not cloud_environment:
+cloud_environment = "default"
 if isinstance(cloud_environment, basestring):
 cloud_environment = publicEnvironments[cloud_environment]
 if not isinstance(cloud_environment, dict):

http://git-wip-us.apache.org/repos/asf/libcloud/blob/018e09f6/libcloud/compute/drivers/azure_arm.py
--
diff --git a/libcloud/compute/drivers/azure_arm.py 
b/libcloud/compute/drivers/azure_arm.py
index 6fb5684..2048daf 100644
--- a/libcloud/compute/drivers/azure_arm.py
+++ b/libcloud/compute/drivers/azure_arm.py
@@ -191,6 +191,7 @@ class AzureNodeDriver(NodeDriver):
  api_version=None, region=None, **kwargs):
 self.tenant_id = tenant_id
 self.subscription_id = subscription_id
+self.cloud_environment = kwargs.get("cloud_environment")
 super(AzureNodeDriver, self).__init__(key=key, secret=secret,
   secure=secure,
   host=host, port=port,
@@ -1351,6 +1352,7 @@ class AzureNodeDriver(NodeDriver):
 kwargs = super(AzureNodeDriver, self)._ex_connection_class_kwargs()
 kwargs['tenant_id'] = self.tenant_id
 kwargs['subscription_id'] = self.subscription_id
+kwargs["cloud_environment"] = self.cloud_environment
 return kwargs
 
 def _to_node(self, data, fetch_nic=True):

[7/9] libcloud git commit: Use compatability shim for urlparse. Don't use dict comprehension. Fix linting.

2017-04-21 Thread anthonyshaw 
Use compatability shim for urlparse.  Don't use dict comprehension.  Fix
linting.


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/16a2611a
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/16a2611a
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/16a2611a

Branch: refs/heads/trunk
Commit: 16a2611a62af61aa1690d559a02d4a8c6af4bb62
Parents: 018e09f
Author: Peter Amstutz 
Authored: Thu Apr 20 14:47:48 2017 -0400
Committer: Peter Amstutz 
Committed: Thu Apr 20 15:29:24 2017 -0400

--
 libcloud/common/azure_arm.py  | 171 -
 libcloud/compute/drivers/azure_arm.py |   8 +-
 2 files changed, 99 insertions(+), 80 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/libcloud/blob/16a2611a/libcloud/common/azure_arm.py
--
diff --git a/libcloud/common/azure_arm.py b/libcloud/common/azure_arm.py
index 1dcf817..064e07e 100644
--- a/libcloud/common/azure_arm.py
+++ b/libcloud/common/azure_arm.py
@@ -19,7 +19,7 @@ except ImportError:
 import json
 
 import time
-from urlparse import urlparse
+from libcloud.utils.py3 import urlparse
 
 from libcloud.common.base import (ConnectionUserAndKey,
   JsonResponse,
@@ -56,80 +56,95 @@ class AzureAuthJsonResponse(JsonResponse):
 else:
 return str(b)
 
-# Based on 
https://github.com/Azure/azure-xplat-cli/blob/master/lib/util/profile/environment.js
-publicEnvironments = {v['name']: v for v in [
-  {
-'name': 'default',
-'portalUrl': 'http://go.microsoft.com/fwlink/?LinkId=254433',
-'publishingProfileUrl': 'http://go.microsoft.com/fwlink/?LinkId=254432',
-'managementEndpointUrl': 'https://management.core.windows.net',
-'resourceManagerEndpointUrl': 'https://management.azure.com/',
-'sqlManagementEndpointUrl': 'https://management.core.windows.net:8443/',
-'sqlServerHostnameSuffix': '.database.windows.net',
-'galleryEndpointUrl': 'https://gallery.azure.com/',
-'activeDirectoryEndpointUrl': 'https://login.microsoftonline.com',
-'activeDirectoryResourceId': 'https://management.core.windows.net/',
-'activeDirectoryGraphResourceId': 'https://graph.windows.net/',
-'activeDirectoryGraphApiVersion': '2013-04-05',
-'storageEndpointSuffix': '.core.windows.net',
-'keyVaultDnsSuffix': '.vault.azure.net',
-'azureDataLakeStoreFileSystemEndpointSuffix': 'azuredatalakestore.net',
-'azureDataLakeAnalyticsCatalogAndJobEndpointSuffix': 
'azuredatalakeanalytics.net'
-  },
-  {
-'name': 'AzureChinaCloud',
-'portalUrl': 'http://go.microsoft.com/fwlink/?LinkId=301902',
-'publishingProfileUrl': 'http://go.microsoft.com/fwlink/?LinkID=301774',
-'managementEndpointUrl': 'https://management.core.chinacloudapi.cn',
-'resourceManagerEndpointUrl': 'https://management.chinacloudapi.cn',
-'sqlManagementEndpointUrl': 
'https://management.core.chinacloudapi.cn:8443/',
-'sqlServerHostnameSuffix': '.database.chinacloudapi.cn',
-'galleryEndpointUrl': 'https://gallery.chinacloudapi.cn/',
-'activeDirectoryEndpointUrl': 'https://login.chinacloudapi.cn',
-'activeDirectoryResourceId': 'https://management.core.chinacloudapi.cn/',
-'activeDirectoryGraphResourceId': 'https://graph.chinacloudapi.cn/',
-'activeDirectoryGraphApiVersion': '2013-04-05',
-'storageEndpointSuffix': '.core.chinacloudapi.cn',
-'keyVaultDnsSuffix': '.vault.azure.cn',
-'azureDataLakeStoreFileSystemEndpointSuffix': 'N/A',
-'azureDataLakeAnalyticsCatalogAndJobEndpointSuffix': 'N/A'
-  },
-  {
-'name': 'AzureUSGovernment',
-'portalUrl': 'https://manage.windowsazure.us',
-'publishingProfileUrl': 
'https://manage.windowsazure.us/publishsettings/index',
-'managementEndpointUrl': 'https://management.core.usgovcloudapi.net',
-'resourceManagerEndpointUrl': 'https://management.usgovcloudapi.net',
-'sqlManagementEndpointUrl': 
'https://management.core.usgovcloudapi.net:8443/',
-'sqlServerHostnameSuffix': '.database.usgovcloudapi.net',
-'galleryEndpointUrl': 'https://gallery.usgovcloudapi.net/',
-'activeDirectoryEndpointUrl': 'https://login-us.microsoftonline.com',
-'activeDirectoryResourceId': 'https://management.core.usgovcloudapi.net/',
-'activeDirectoryGraphResourceId': 'https://graph.windows.net/',
-'activeDirectoryGraphApiVersion': '2013-04-05',
-'storageEndpointSuffix': '.core.usgovcloudapi.net',
-'keyVaultDnsSuffix': '.vault.usgovcloudapi.net',
-'azureDataLakeStoreFileSystemEndpointSuffix': 'N/A',
-'azureDataLakeAnalyticsCatalogAndJobEndpointSuffix': 'N/A'
-  },
-  {
-'name': 'AzureGermanCloud',
-'portalUrl':

[2/9] libcloud git commit: Add support for selecting alternate cloud_environments AzureChinaCloud, AzureUSGovernment, and AzureGermanCloud.

2017-04-21 Thread anthonyshaw 
Add support for selecting alternate cloud_environments AzureChinaCloud,
AzureUSGovernment, and AzureGermanCloud.


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/59153c37
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/59153c37
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/59153c37

Branch: refs/heads/trunk
Commit: 59153c375918f925400714a82d29d69725308060
Parents: 1a07215
Author: Peter Amstutz 
Authored: Thu Jan 5 16:42:33 2017 -0500
Committer: Peter Amstutz 
Committed: Thu Apr 20 14:46:21 2017 -0400

--
 libcloud/common/azure_arm.py | 82 +--
 1 file changed, 79 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/libcloud/blob/59153c37/libcloud/common/azure_arm.py
--
diff --git a/libcloud/common/azure_arm.py b/libcloud/common/azure_arm.py
index d8c07c1..48031a5 100644
--- a/libcloud/common/azure_arm.py
+++ b/libcloud/common/azure_arm.py
@@ -19,6 +19,7 @@ except ImportError:
 import json
 
 import time
+from urlparse import urlparse
 
 from libcloud.common.base import (ConnectionUserAndKey,
   JsonResponse,
@@ -55,6 +56,80 @@ class AzureAuthJsonResponse(JsonResponse):
 else:
 return str(b)
 
+# Based on 
https://github.com/Azure/azure-xplat-cli/blob/master/lib/util/profile/environment.js
+publicEnvironments = [
+  {
+'name': 'default',
+'portalUrl': 'http://go.microsoft.com/fwlink/?LinkId=254433',
+'publishingProfileUrl': 'http://go.microsoft.com/fwlink/?LinkId=254432',
+'managementEndpointUrl': 'https://management.core.windows.net',
+'resourceManagerEndpointUrl': 'https://management.azure.com/',
+'sqlManagementEndpointUrl': 'https://management.core.windows.net:8443/',
+'sqlServerHostnameSuffix': '.database.windows.net',
+'galleryEndpointUrl': 'https://gallery.azure.com/',
+'activeDirectoryEndpointUrl': 'https://login.microsoftonline.com',
+'activeDirectoryResourceId': 'https://management.core.windows.net/',
+'activeDirectoryGraphResourceId': 'https://graph.windows.net/',
+'activeDirectoryGraphApiVersion': '2013-04-05',
+'storageEndpointSuffix': '.core.windows.net',
+'keyVaultDnsSuffix': '.vault.azure.net',
+'azureDataLakeStoreFileSystemEndpointSuffix': 'azuredatalakestore.net',
+'azureDataLakeAnalyticsCatalogAndJobEndpointSuffix': 
'azuredatalakeanalytics.net'
+  },
+  {
+'name': 'AzureChinaCloud',
+'portalUrl': 'http://go.microsoft.com/fwlink/?LinkId=301902',
+'publishingProfileUrl': 'http://go.microsoft.com/fwlink/?LinkID=301774',
+'managementEndpointUrl': 'https://management.core.chinacloudapi.cn',
+'resourceManagerEndpointUrl': 'https://management.chinacloudapi.cn',
+'sqlManagementEndpointUrl': 
'https://management.core.chinacloudapi.cn:8443/',
+'sqlServerHostnameSuffix': '.database.chinacloudapi.cn',
+'galleryEndpointUrl': 'https://gallery.chinacloudapi.cn/',
+'activeDirectoryEndpointUrl': 'https://login.chinacloudapi.cn',
+'activeDirectoryResourceId': 'https://management.core.chinacloudapi.cn/',
+'activeDirectoryGraphResourceId': 'https://graph.chinacloudapi.cn/',
+'activeDirectoryGraphApiVersion': '2013-04-05',
+'storageEndpointSuffix': '.core.chinacloudapi.cn',
+'keyVaultDnsSuffix': '.vault.azure.cn',
+'azureDataLakeStoreFileSystemEndpointSuffix': 'N/A',
+'azureDataLakeAnalyticsCatalogAndJobEndpointSuffix': 'N/A'
+  },
+  {
+'name': 'AzureUSGovernment',
+'portalUrl': 'https://manage.windowsazure.us',
+'publishingProfileUrl': 
'https://manage.windowsazure.us/publishsettings/index',
+'managementEndpointUrl': 'https://management.core.usgovcloudapi.net',
+'resourceManagerEndpointUrl': 'https://management.usgovcloudapi.net',
+'sqlManagementEndpointUrl': 
'https://management.core.usgovcloudapi.net:8443/',
+'sqlServerHostnameSuffix': '.database.usgovcloudapi.net',
+'galleryEndpointUrl': 'https://gallery.usgovcloudapi.net/',
+'activeDirectoryEndpointUrl': 'https://login-us.microsoftonline.com',
+'activeDirectoryResourceId': 'https://management.core.usgovcloudapi.net/',
+'activeDirectoryGraphResourceId': 'https://graph.windows.net/',
+'activeDirectoryGraphApiVersion': '2013-04-05',
+'storageEndpointSuffix': '.core.usgovcloudapi.net',
+'keyVaultDnsSuffix': '.vault.usgovcloudapi.net',
+'azureDataLakeStoreFileSystemEndpointSuffix': 'N/A',
+'azureDataLakeAnalyticsCatalogAndJobEndpointSuffix': 'N/A'
+  },
+  {
+'name': 'AzureGermanCloud',
+'portalUrl': 'http://portal.microsoftazure.de/',
+'publishingProfileUrl':

[9/9] libcloud git commit: changes for #969

2017-04-21 Thread anthonyshaw 
changes for #969


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/660a144d
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/660a144d
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/660a144d

Branch: refs/heads/trunk
Commit: 660a144da2a063e1a52f8e18a4157c6217e09ccc
Parents: a5f4c98
Author: Anthony Shaw 
Authored: Sat Apr 22 08:12:30 2017 +1000
Committer: Anthony Shaw 
Committed: Sat Apr 22 08:12:30 2017 +1000

--
 CHANGES.rst | 4 
 1 file changed, 4 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/libcloud/blob/660a144d/CHANGES.rst
--
diff --git a/CHANGES.rst b/CHANGES.rst
index d7d51f2..03d8990 100644
--- a/CHANGES.rst
+++ b/CHANGES.rst
@@ -25,6 +25,10 @@ Common
 Compute
 ~~~
 
+- [ARM] Add support for Azure Cloud Environments as well as Locations
+  [GITHUB-969]
+  (Peter Amstutz)
+
 - [EC2] Add support for ModifyVolume and DescribeVolumesModifications
   [GITHUB-1036]
   (Hennadii Stas)

[1/9] libcloud git commit: Add documentation. Add support for passing dict into cloud_environments to supply explicit endpoints.

2017-04-21 Thread anthonyshaw 
Repository: libcloud
Updated Branches:
  refs/heads/trunk 55e5e82da -> 660a144da


Add documentation.  Add support for passing dict into cloud_environments to
supply explicit endpoints.


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/6aa7bb47
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/6aa7bb47
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/6aa7bb47

Branch: refs/heads/trunk
Commit: 6aa7bb47aa4fd883595afc988d70042d46b557cd
Parents: 59153c3
Author: Peter Amstutz 
Authored: Fri Jan 6 10:34:06 2017 -0500
Committer: Peter Amstutz 
Committed: Thu Apr 20 14:46:21 2017 -0400

--
 docs/compute/drivers/azure_arm.rst | 10 ++
 libcloud/common/azure_arm.py   | 14 +++---
 2 files changed, 21 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/libcloud/blob/6aa7bb47/docs/compute/drivers/azure_arm.rst
--
diff --git a/docs/compute/drivers/azure_arm.rst 
b/docs/compute/drivers/azure_arm.rst
index 6506d57..ff974ab 100644
--- a/docs/compute/drivers/azure_arm.rst
+++ b/docs/compute/drivers/azure_arm.rst
@@ -42,6 +42,16 @@ password ("secret"), you can create an AzureNodeDriver:
 .. literalinclude:: /examples/compute/azure_arm/instantiate.py
:language: python
 
+Alternate Cloud Environments
+
+
+You can select an alternate cloud environment using the "cloud_environment"
+parameter to AzureNodeDriver constructor.  Available alternate cloud
+environments are 'AzureChinaCloud', 'AzureUSGovernment' and 'AzureGermanCloud'.
+You can also supply explicit endpoints by providing a dict with the keys
+'resourceManagerEndpointUrl', 'activeDirectoryEndpointUrl' and
+'activeDirectoryResourceId'.
+
 API Docs
 
 

http://git-wip-us.apache.org/repos/asf/libcloud/blob/6aa7bb47/libcloud/common/azure_arm.py
--
diff --git a/libcloud/common/azure_arm.py b/libcloud/common/azure_arm.py
index 48031a5..cbd42f8 100644
--- a/libcloud/common/azure_arm.py
+++ b/libcloud/common/azure_arm.py
@@ -147,9 +147,17 @@ class 
AzureResourceManagementConnection(ConnectionUserAndKey):
 super(AzureResourceManagementConnection, self) \
 .__init__(key, secret, **kwargs)
 cloud_environment = kwargs.get("cloud_environment", "default")
-self.host = 
urlparse(publicEnvironments[cloud_environment]['resourceManagerEndpointUrl']).hostname
-self.login_host = 
urlparse(publicEnvironments[cloud_environment]['activeDirectoryEndpointUrl']).hostname
-self.login_resource = 
publicEnvironments[cloud_environment]['activeDirectoryResourceId']
+if isinstance(cloud_environment, basestring):
+cloud_environment = publicEnvironments[cloud_environment]
+if not isinstance(cloud_environment, dict):
+raise Exception("cloud_environment must be one of '%s' or a dict "
+"containing keys 'resourceManagerEndpointUrl', "
+"'activeDirectoryEndpointUrl', "
+"'activeDirectoryResourceId'" % (
+"', '".join(publicEnvironments.keys(
+self.host = 
urlparse(cloud_environment['resourceManagerEndpointUrl']).hostname
+self.login_host = 
urlparse(cloud_environment['activeDirectoryEndpointUrl']).hostname
+self.login_resource = cloud_environment['activeDirectoryResourceId']
 self.tenant_id = tenant_id
 self.subscription_id = subscription_id

[3/9] libcloud git commit: Explicit host provided to AzureBlobsStorageDriver needs to include storage account.

2017-04-21 Thread anthonyshaw 
Explicit host provided to AzureBlobsStorageDriver needs to include storage 
account.


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/e1cc5c7a
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/e1cc5c7a
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/e1cc5c7a

Branch: refs/heads/trunk
Commit: e1cc5c7ad796645d2e5688acb1fd28059140f841
Parents: 37e709c
Author: Peter Amstutz 
Authored: Fri Jan 6 11:08:25 2017 -0500
Committer: Peter Amstutz 
Committed: Thu Apr 20 14:46:22 2017 -0400

--
 libcloud/compute/drivers/azure_arm.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/libcloud/blob/e1cc5c7a/libcloud/compute/drivers/azure_arm.py
--
diff --git a/libcloud/compute/drivers/azure_arm.py 
b/libcloud/compute/drivers/azure_arm.py
index d261d20..6fb5684 100644
--- a/libcloud/compute/drivers/azure_arm.py
+++ b/libcloud/compute/drivers/azure_arm.py
@@ -1340,7 +1340,7 @@ class AzureNodeDriver(NodeDriver):
 storageAccount)
 blobdriver = AzureBlobsStorageDriver(storageAccount,
  keys["key1"],
- host="blob%s" % 
(self.connection.storage_suffix))
+ host="%s.blob%s" % 
(storageAccount, self.connection.storage_suffix))
 blobdriver.delete_object(blobdriver.get_object(blobContainer,
blob))
 return True

[6/9] libcloud git commit: Fix publicEnvironments to be a dict as intended.

2017-04-21 Thread anthonyshaw 
Fix publicEnvironments to be a dict as intended.


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/a5654cb4
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/a5654cb4
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/a5654cb4

Branch: refs/heads/trunk
Commit: a5654cb4d0f2b1d4c727fe22e0e791b6f10b71c4
Parents: e1cc5c7
Author: Peter Amstutz 
Authored: Fri Jan 13 16:08:22 2017 -0500
Committer: Peter Amstutz 
Committed: Thu Apr 20 14:46:22 2017 -0400

--
 libcloud/common/azure_arm.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/libcloud/blob/a5654cb4/libcloud/common/azure_arm.py
--
diff --git a/libcloud/common/azure_arm.py b/libcloud/common/azure_arm.py
index d4984a5..a73a905 100644
--- a/libcloud/common/azure_arm.py
+++ b/libcloud/common/azure_arm.py
@@ -57,7 +57,7 @@ class AzureAuthJsonResponse(JsonResponse):
 return str(b)
 
 # Based on 
https://github.com/Azure/azure-xplat-cli/blob/master/lib/util/profile/environment.js
-publicEnvironments = [
+publicEnvironments = {v['name']: v for v in [
   {
 'name': 'default',
 'portalUrl': 'http://go.microsoft.com/fwlink/?LinkId=254433',
@@ -129,7 +129,7 @@ publicEnvironments = [
 'keyVaultDnsSuffix': '.vault.microsoftazure.de',
 'azureDataLakeStoreFileSystemEndpointSuffix': 'N/A',
 'azureDataLakeAnalyticsCatalogAndJobEndpointSuffix': 'N/A'
-  }]
+  }]}
 
 class AzureResourceManagementConnection(ConnectionUserAndKey):
 """

[8/9] libcloud git commit: Merge branch 'github-969' into trunk Closes #969 Signed-off-by: Anthony Shaw <anthonys...@apache.org>

2017-04-21 Thread anthonyshaw 
Merge branch 'github-969' into trunk
Closes #969
Signed-off-by: Anthony Shaw 


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/a5f4c98b
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/a5f4c98b
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/a5f4c98b

Branch: refs/heads/trunk
Commit: a5f4c98b2f9d991ad4e81bae310793bc3b8bb261
Parents: 55e5e82 16a2611
Author: Anthony Shaw 
Authored: Sat Apr 22 08:11:11 2017 +1000
Committer: Anthony Shaw 
Committed: Sat Apr 22 08:11:11 2017 +1000

--
 docs/compute/drivers/azure_arm.rst|  10 +++
 libcloud/common/azure_arm.py  | 112 +++--
 libcloud/compute/drivers/azure_arm.py |  19 +++--
 3 files changed, 131 insertions(+), 10 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/libcloud/blob/a5f4c98b/libcloud/compute/drivers/azure_arm.py
--

[GitHub] libcloud pull request #969: Add support for selecting alternate cloud_enviro...

2017-04-21 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/libcloud/pull/969


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[4/9] libcloud git commit: Compute driver should correctly access blob storage in alternate cloud environments.

2017-04-21 Thread anthonyshaw 
Compute driver should correctly access blob storage in alternate cloud
environments.


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/37e709c9
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/37e709c9
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/37e709c9

Branch: refs/heads/trunk
Commit: 37e709c98d8db059673d437f9a5e49e003afc427
Parents: 6aa7bb4
Author: Peter Amstutz 
Authored: Fri Jan 6 11:03:00 2017 -0500
Committer: Peter Amstutz 
Committed: Thu Apr 20 14:46:22 2017 -0400

--
 docs/compute/drivers/azure_arm.rst|  4 ++--
 libcloud/common/azure_arm.py  |  4 +++-
 libcloud/compute/drivers/azure_arm.py | 13 -
 3 files changed, 13 insertions(+), 8 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/libcloud/blob/37e709c9/docs/compute/drivers/azure_arm.rst
--
diff --git a/docs/compute/drivers/azure_arm.rst 
b/docs/compute/drivers/azure_arm.rst
index ff974ab..74ecf72 100644
--- a/docs/compute/drivers/azure_arm.rst
+++ b/docs/compute/drivers/azure_arm.rst
@@ -49,8 +49,8 @@ You can select an alternate cloud environment using the 
"cloud_environment"
 parameter to AzureNodeDriver constructor.  Available alternate cloud
 environments are 'AzureChinaCloud', 'AzureUSGovernment' and 'AzureGermanCloud'.
 You can also supply explicit endpoints by providing a dict with the keys
-'resourceManagerEndpointUrl', 'activeDirectoryEndpointUrl' and
-'activeDirectoryResourceId'.
+'resourceManagerEndpointUrl', 'activeDirectoryEndpointUrl',
+'activeDirectoryResourceId' and 'storageEndpointSuffix'.
 
 API Docs
 

http://git-wip-us.apache.org/repos/asf/libcloud/blob/37e709c9/libcloud/common/azure_arm.py
--
diff --git a/libcloud/common/azure_arm.py b/libcloud/common/azure_arm.py
index cbd42f8..d4984a5 100644
--- a/libcloud/common/azure_arm.py
+++ b/libcloud/common/azure_arm.py
@@ -153,11 +153,13 @@ class 
AzureResourceManagementConnection(ConnectionUserAndKey):
 raise Exception("cloud_environment must be one of '%s' or a dict "
 "containing keys 'resourceManagerEndpointUrl', "
 "'activeDirectoryEndpointUrl', "
-"'activeDirectoryResourceId'" % (
+"'activeDirectoryResourceId', "
+"'storageEndpointSuffix'" % (
 "', '".join(publicEnvironments.keys(
 self.host = 
urlparse(cloud_environment['resourceManagerEndpointUrl']).hostname
 self.login_host = 
urlparse(cloud_environment['activeDirectoryEndpointUrl']).hostname
 self.login_resource = cloud_environment['activeDirectoryResourceId']
+self.storage_suffix = cloud_environment['storageEndpointSuffix']
 self.tenant_id = tenant_id
 self.subscription_id = subscription_id
 

http://git-wip-us.apache.org/repos/asf/libcloud/blob/37e709c9/libcloud/compute/drivers/azure_arm.py
--
diff --git a/libcloud/compute/drivers/azure_arm.py 
b/libcloud/compute/drivers/azure_arm.py
index c7bed5e..d261d20 100644
--- a/libcloud/compute/drivers/azure_arm.py
+++ b/libcloud/compute/drivers/azure_arm.py
@@ -61,9 +61,10 @@ class AzureVhdImage(NodeImage):
 """Represents a VHD node image that an Azure VM can boot from."""
 
 def __init__(self, storage_account, blob_container, name, driver):
-urn = "https://%s.blob.core.windows.net/%s/%s; % (storage_account,
-  blob_container,
-  name)
+urn = "https://%s.blob%s/%s/%s; % (storage_account,
+   driver.connection.storage_suffix,
+   blob_container,
+   name)
 super(AzureVhdImage, self).__init__(urn, name, driver)
 
 def __repr__(self):
@@ -511,9 +512,10 @@ class AzureNodeDriver(NodeDriver):
 n = 0
 while True:
 try:
-instance_vhd = "https://%s.blob.core.windows.net; \
+instance_vhd = "https://%s.blob%s; \
"/%s/%s-os_%i.vhd" \
% (ex_storage_account,
+  self.connection.storage_suffix,
   ex_blob_container,
   name,
   n)
@@ -1337,7 +1339,8 @@ class AzureNodeDriver(NodeDriver):
 keys =

[jira] [Updated] (LIBCLOUD-912) [python] Requests tries to connect to 127.0.0.1:443

2017-04-21 Thread Anthony Shaw (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LIBCLOUD-912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anthony Shaw updated LIBCLOUD-912:
--

Once the fix is reproducible in a solid test I wouldn't close it off until you 
or someone else can tested it against a proper OpenStack environment.
The next release should be coming soon and we wouldn't want to ship with it 
broken again.



> [python] Requests tries to connect to 127.0.0.1:443
> ---
>
> Key: LIBCLOUD-912
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-912
> Project: Libcloud
>  Issue Type: Bug
> Environment: python 2.7
> apache-libcloud==2.0.0rc2
>Reporter: Alexis Lesieur
>  Labels: newbie, python
>
> Hi!
> Sorry if this story is not created properly.
> I'm not that familiar with how you guys manage your Jira. Feel free to give 
> me directions or just modify the ticket :-)
> So we have been trying to upgrade the version of our libcloud python package, 
> but it breaks for our rackspace/openstack environment.
> I have been digging for a while and I couldn't find where the difference was 
> between 1.x.x and 2.0.0rc2.
> Observed behavior:
> 1.x.x:
> 3 connections are created:
> - to 127.0.0.1:443
> - to host:3000 (ex_force_auth_url)
> - to host:8774/v2/... (don't know where it comes from)
> 2.0.0rc2:
> 2 connections
> - to 127.0.0.1:443
> - to host:3000 (ex_force_auth_url)
> and the last connection is done using 127.0.0.1:443 again, using the same 
> connector created in the first one
> I don't understand the change in behavior between the two versions.
> I tried use the ex_force_base_url flag, and while it does look like the path 
> of that url is respected, the host is always replaced by 127.0.0.1.
> Any idea of what is happening?
> Thanks!



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] libcloud pull request #1038: Add i3 instance types for AWS

2017-04-21 Thread smullins7 
GitHub user smullins7 opened a pull request:

https://github.com/apache/libcloud/pull/1038

Add i3 instance types for AWS

Followed the r4 commit, adding i3 instances types for AWS. First time 
caller so let me know if I missed anything, please and thank you!

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/smullins7/libcloud aws-i3-instance-types

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/libcloud/pull/1038.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1038


commit 0d413236a96e150c2cf5124b475609fe6c1373bf
Author: Stephen Mullins 
Date:   2017-04-21T18:27:09Z

Add i3 instance types for AWS




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Reopened] (LIBCLOUD-912) [python] Requests tries to connect to 127.0.0.1:443

2017-04-21 Thread Alexis Lesieur (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LIBCLOUD-912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexis Lesieur reopened LIBCLOUD-912:
-

> [python] Requests tries to connect to 127.0.0.1:443
> ---
>
> Key: LIBCLOUD-912
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-912
> Project: Libcloud
>  Issue Type: Bug
> Environment: python 2.7
> apache-libcloud==2.0.0rc2
>Reporter: Alexis Lesieur
>  Labels: newbie, python
>
> Hi!
> Sorry if this story is not created properly.
> I'm not that familiar with how you guys manage your Jira. Feel free to give 
> me directions or just modify the ticket :-)
> So we have been trying to upgrade the version of our libcloud python package, 
> but it breaks for our rackspace/openstack environment.
> I have been digging for a while and I couldn't find where the difference was 
> between 1.x.x and 2.0.0rc2.
> Observed behavior:
> 1.x.x:
> 3 connections are created:
> - to 127.0.0.1:443
> - to host:3000 (ex_force_auth_url)
> - to host:8774/v2/... (don't know where it comes from)
> 2.0.0rc2:
> 2 connections
> - to 127.0.0.1:443
> - to host:3000 (ex_force_auth_url)
> and the last connection is done using 127.0.0.1:443 again, using the same 
> connector created in the first one
> I don't understand the change in behavior between the two versions.
> I tried use the ex_force_base_url flag, and while it does look like the path 
> of that url is respected, the host is always replaced by 127.0.0.1.
> Any idea of what is happening?
> Thanks!



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (LIBCLOUD-912) [python] Requests tries to connect to 127.0.0.1:443

2017-04-21 Thread Alexis Lesieur (JIRA) 

[ 
https://issues.apache.org/jira/browse/LIBCLOUD-912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15978790#comment-15978790
 ] 

Alexis Lesieur commented on LIBCLOUD-912:
-

Oh I'm dumb I didn't realize the PR was in response to my ticket... It's still 
too early in the morning ><

> [python] Requests tries to connect to 127.0.0.1:443
> ---
>
> Key: LIBCLOUD-912
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-912
> Project: Libcloud
>  Issue Type: Bug
> Environment: python 2.7
> apache-libcloud==2.0.0rc2
>Reporter: Alexis Lesieur
>  Labels: newbie, python
>
> Hi!
> Sorry if this story is not created properly.
> I'm not that familiar with how you guys manage your Jira. Feel free to give 
> me directions or just modify the ticket :-)
> So we have been trying to upgrade the version of our libcloud python package, 
> but it breaks for our rackspace/openstack environment.
> I have been digging for a while and I couldn't find where the difference was 
> between 1.x.x and 2.0.0rc2.
> Observed behavior:
> 1.x.x:
> 3 connections are created:
> - to 127.0.0.1:443
> - to host:3000 (ex_force_auth_url)
> - to host:8774/v2/... (don't know where it comes from)
> 2.0.0rc2:
> 2 connections
> - to 127.0.0.1:443
> - to host:3000 (ex_force_auth_url)
> and the last connection is done using 127.0.0.1:443 again, using the same 
> connector created in the first one
> I don't understand the change in behavior between the two versions.
> I tried use the ex_force_base_url flag, and while it does look like the path 
> of that url is respected, the host is always replaced by 127.0.0.1.
> Any idea of what is happening?
> Thanks!



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (LIBCLOUD-912) [python] Requests tries to connect to 127.0.0.1:443

2017-04-21 Thread Alexis Lesieur (JIRA) 

[ 
https://issues.apache.org/jira/browse/LIBCLOUD-912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15978785#comment-15978785
 ] 

Alexis Lesieur commented on LIBCLOUD-912:
-

How I couldn't find the PR during my search... Yeah the fix looks sound from my 
understanding of the issue and the little know-how I have of this code :-)
Well I guess I'll close this ticket then!

Thanks for the quick answers guys!

> [python] Requests tries to connect to 127.0.0.1:443
> ---
>
> Key: LIBCLOUD-912
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-912
> Project: Libcloud
>  Issue Type: Bug
> Environment: python 2.7
> apache-libcloud==2.0.0rc2
>Reporter: Alexis Lesieur
>  Labels: newbie, python
>
> Hi!
> Sorry if this story is not created properly.
> I'm not that familiar with how you guys manage your Jira. Feel free to give 
> me directions or just modify the ticket :-)
> So we have been trying to upgrade the version of our libcloud python package, 
> but it breaks for our rackspace/openstack environment.
> I have been digging for a while and I couldn't find where the difference was 
> between 1.x.x and 2.0.0rc2.
> Observed behavior:
> 1.x.x:
> 3 connections are created:
> - to 127.0.0.1:443
> - to host:3000 (ex_force_auth_url)
> - to host:8774/v2/... (don't know where it comes from)
> 2.0.0rc2:
> 2 connections
> - to 127.0.0.1:443
> - to host:3000 (ex_force_auth_url)
> and the last connection is done using 127.0.0.1:443 again, using the same 
> connector created in the first one
> I don't understand the change in behavior between the two versions.
> I tried use the ex_force_base_url flag, and while it does look like the path 
> of that url is respected, the host is always replaced by 127.0.0.1.
> Any idea of what is happening?
> Thanks!



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (LIBCLOUD-912) [python] Requests tries to connect to 127.0.0.1:443

2017-04-21 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/LIBCLOUD-912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15978219#comment-15978219
 ] 

ASF GitHub Bot commented on LIBCLOUD-912:
-

GitHub user tonybaloney opened a pull request:

https://github.com/apache/libcloud/pull/1037

[WIP] issues with OpenStack auth raised in LIBCLOUD-912 and LIBCLOUD-904

Noticed so far:

The existing test fixture returns the host as 127.0.0.1 which also happens 
to be the default host for a connection class. 

The test I just added fails to demonstrate the issue with

requests_mock.exceptions.NoMockAddress: No mock address: GET 
https://127.0.0.1/v2/1337/servers/detail



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tonybaloney/libcloud openstack_auth_passing

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/libcloud/pull/1037.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1037


commit b8ba6c10accdf6d9ce8a40787e2300a594437ebe
Author: Anthony Shaw 
Date:   2017-04-21T07:21:55Z

issues with LIBCLOUD-912




> [python] Requests tries to connect to 127.0.0.1:443
> ---
>
> Key: LIBCLOUD-912
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-912
> Project: Libcloud
>  Issue Type: Bug
> Environment: python 2.7
> apache-libcloud==2.0.0rc2
>Reporter: Alexis Lesieur
>  Labels: newbie, python
>
> Hi!
> Sorry if this story is not created properly.
> I'm not that familiar with how you guys manage your Jira. Feel free to give 
> me directions or just modify the ticket :-)
> So we have been trying to upgrade the version of our libcloud python package, 
> but it breaks for our rackspace/openstack environment.
> I have been digging for a while and I couldn't find where the difference was 
> between 1.x.x and 2.0.0rc2.
> Observed behavior:
> 1.x.x:
> 3 connections are created:
> - to 127.0.0.1:443
> - to host:3000 (ex_force_auth_url)
> - to host:8774/v2/... (don't know where it comes from)
> 2.0.0rc2:
> 2 connections
> - to 127.0.0.1:443
> - to host:3000 (ex_force_auth_url)
> and the last connection is done using 127.0.0.1:443 again, using the same 
> connector created in the first one
> I don't understand the change in behavior between the two versions.
> I tried use the ex_force_base_url flag, and while it does look like the path 
> of that url is respected, the host is always replaced by 127.0.0.1.
> Any idea of what is happening?
> Thanks!



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] libcloud pull request #1037: [WIP] issues with OpenStack auth raised in LIBC...

2017-04-21 Thread tonybaloney 
GitHub user tonybaloney opened a pull request:

https://github.com/apache/libcloud/pull/1037

[WIP] issues with OpenStack auth raised in LIBCLOUD-912 and LIBCLOUD-904

Noticed so far:

The existing test fixture returns the host as 127.0.0.1 which also happens 
to be the default host for a connection class. 

The test I just added fails to demonstrate the issue with

requests_mock.exceptions.NoMockAddress: No mock address: GET 
https://127.0.0.1/v2/1337/servers/detail



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tonybaloney/libcloud openstack_auth_passing

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/libcloud/pull/1037.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1037


commit b8ba6c10accdf6d9ce8a40787e2300a594437ebe
Author: Anthony Shaw 
Date:   2017-04-21T07:21:55Z

issues with LIBCLOUD-912




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (LIBCLOUD-903) AWS S3 upload_object_via_stream fails on non-file iterable due to missing Content-Length header

2017-04-21 Thread Anthony Shaw (JIRA) 

[ 
https://issues.apache.org/jira/browse/LIBCLOUD-903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15978150#comment-15978150
 ] 

Anthony Shaw commented on LIBCLOUD-903:
---

which version of Libcloud was this using?

> AWS S3 upload_object_via_stream fails on non-file iterable due to missing 
> Content-Length header
> ---
>
> Key: LIBCLOUD-903
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-903
> Project: Libcloud
>  Issue Type: Bug
>Reporter: Richard Xia
>
> The issue I am seeing appears to be due to the incorrect integration of 4 
> separate libraries, but I believe the real problem is here in libcloud, in 
> the {{upload_object_via_stream()}} method on the S3 storage driver.
>   
>
> I am using Python 3.5.1 and the the four libraries I am using are:
>
>   
>
> * Django 1.10.6   
>
> * django-storages 1.5.2   
>
> * libcloud v2.0.0rc1-tentative
>
> * requests 2.13.0 
>
>   
>
> Specifically, when I try to use a Django 
> [ContentFile|https://docs.djangoproject.com/en/1.10/ref/files/file/#django.core.files.base.ContentFile],
>  Django's own file-like wrapper for strings, to save a new file to S3 via the 
> Libcloud backend of django-storages, I get the following error:
>   
>
> {code:xml}
>
>  encoding="UTF-8"?>\nNotImplementedA header you 
> provided implies functionality that is not 
> implementedTransfer-EncodingA2FC4D5109083076K9WGhd18iqQHyIyv+GxWcxHexvapVSidTtHzSqujtT9nT5LhmIEygMKOfR/7F0v7ujnlE/CoYiM=
> {code}
>
>   
>
> The reason this happens is because Libcloud is generating an HTTP request to 
> AWS S3 that is missing the {{Content-Length}} header. AWS S3 requires the 
> {{Content-Length}} header for file uploads *unless* if it is a multi-part 
> upload. This is why this used to work on the 1.5.0 release of {{libcloud}}, 
> because even single-part uploads were done as a one-part multi-part upload.
>   
>
> I've traced my bug down through all four libraries and have determined 
> exactly why the {{Content-Length}} header is missing in my particular use 
> case. The {{upload_object_via_stream()}} has an {{iterator}} argument that 
> should yield the content body data, and it eventually passes that argument 
> directly to the {{requests}} library. The {{requests}} library will actually 
> [try very hard to add the {{Content-Length}} 
> header|https://github.com/kennethreitz/requests/blob/c43fefa7ed535c41ba7d58021f0f16ed5ba1d584/requests/models.py#L471],
>  even for certain types of iterator streams. In particular it can determine 
> the length of file-like objects which support stat operations and it can 
> handle StringIO/BytesIO objects. However, the Django {{ContentFile}} is 
> neither, and {{requests}} cannot extract the length of the stream without 
> consuming the iterator, so it does not try.
>   
>
>   
>
> Here's some (Python 3) code to demonstrate the bug:   
>
>   
>
> {code:python} 
>
> from io import BytesIO
>
>   
>
> class MyWrapper(object):  
>
> """A contrived wrapper that acts similar to BytesIO."""   
>
> def __init__(self, content):  
>
> self.content = BytesIO(content)   
>
>   
>
> def __iter__(self):   
>
>

[jira] [Resolved] (LIBCLOUD-901) 4xx and 5xx HTTP status codes are silently masked as 200

2017-04-21 Thread Anthony Shaw (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LIBCLOUD-901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anthony Shaw resolved LIBCLOUD-901.
---
Resolution: Fixed

> 4xx and 5xx HTTP status codes are silently masked as 200
> 
>
> Key: LIBCLOUD-901
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-901
> Project: Libcloud
>  Issue Type: Bug
>Reporter: Richard Xia
>
> I'm seeing this on the latest trunk as well as v2.0.0rc1-tentative. When I 
> try to upload an object to AWS S3 using the {{upload_object_via_stream}} 
> method and there is an error (I'll create a separate ticket for that, which 
> I've also determined the cause of), {{libcloud}} will ignore the 5xx error 
> response from {{requests}} and set the status to the default of 200.
> The reason this occurs is because the constructor of {{RawResponse}} has an 
> optional argument {{response}} which has a default value of {{None}}. The 
> body of the constructor checks [{{if 
> response}}|https://github.com/apache/libcloud/blob/516018d050a5470afe4a07d33d1fcb26d0e1df18/libcloud/common/base.py#L285]
>  to check if a response was passed into the constructor, but it turns out 
> that the {{requests}} library [overloads the {{__nonzero__}}/{{__bool__}} 
> methods|https://github.com/kennethreitz/requests/issues/2002] to return 
> {{False}} if the response status code is 4xx or 5xx. This [has been 
> fixed|https://github.com/kennethreitz/requests/pull/2587] in the 3.0 branch 
> of {{requests}}, but version 3.0 hasn't been released yet, so this currently 
> affects all versions of requests.
> I will submit a pull request with the small change to fix this.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (LIBCLOUD-904) OpenStack not working on latest trunk

2017-04-21 Thread Anthony Shaw (JIRA) 

[ 
https://issues.apache.org/jira/browse/LIBCLOUD-904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15978147#comment-15978147
 ] 

Anthony Shaw commented on LIBCLOUD-904:
---

related to LIBCLOUD-912

> OpenStack not working on latest trunk
> -
>
> Key: LIBCLOUD-904
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-904
> Project: Libcloud
>  Issue Type: Bug
>  Components: Compute
>Reporter: Markos Gogoulos
>
> Althought authentication works with keystone, it doesn't try to connect to 
> the compute endpoint. At that point I see self.connection.host is 
> https://127.0.0.1 but self.host and self.port are correct. 
> This seems related with common/base.py and libcloud/httplib_ssl.py
> from libcloud.compute.providers import get_driver; from 
> libcloud.compute.types import Provider; driver = get_driver('openstack')
> c=driver('user', 'correct_password', ex_force_auth_version='2.0_password', 
> ex_force_auth_url='http://x.y.z.y:5000', ex_tenant_name='admin')
> c.list_nodes()
> ---
> ConnectionError   Traceback (most recent call last)
> ConnectionError: HTTPSConnectionPool(host='127.0.0.1', port=443): Max retries 
> exceeded with url: /v2.1/d0ff206af73c459bad15a33cba35e90d/servers/detail 
> (Caused by 
> NewConnectionError('  object at 0x7ffb36a85a10>: Failed to establish a new connection: [Errno 111] 
> Connection refused',))



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (LIBCLOUD-906) Public IP not assigned when creating NIC on Azure ARM

2017-04-21 Thread Anthony Shaw (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LIBCLOUD-906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anthony Shaw resolved LIBCLOUD-906.
---
Resolution: Fixed

> Public IP not assigned when creating NIC on Azure ARM
> -
>
> Key: LIBCLOUD-906
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-906
> Project: Libcloud
>  Issue Type: Bug
>Reporter: Simone Ripamonti
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (LIBCLOUD-909) Add key pair methods for OnApp

2017-04-21 Thread Anthony Shaw (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LIBCLOUD-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anthony Shaw resolved LIBCLOUD-909.
---
Resolution: Fixed

> Add key pair methods for OnApp
> --
>
> Key: LIBCLOUD-909
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-909
> Project: Libcloud
>  Issue Type: Improvement
>  Components: Compute
>Reporter: Tinu Cleatus
>Priority: Minor
>
> Add ability to list, get, import and delete key pairs for an OnApp cloud



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (LIBCLOUD-910) Not a Gzipped file when using LIBCLOUD_DEBUG

2017-04-21 Thread Anthony Shaw (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LIBCLOUD-910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anthony Shaw closed LIBCLOUD-910.
-

> Not a Gzipped file when using LIBCLOUD_DEBUG
> 
>
> Key: LIBCLOUD-910
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-910
> Project: Libcloud
>  Issue Type: Bug
>  Components: Core
> Environment: python 2.6, debian jessie
>Reporter: Tom Melendez
>
> ## Summary
> When LIBCLOUD_DEBUG is set, receive a gzip error.  If the call to 
> decompress_data is commented out, things progress fine. 
> Offending line:
>   File "./libcloud/utils/loggingconnection.py", line 71, in _log_response
> body = decompress_data('gzip', body)
> ## Stacktrace
> (libcloud-testing) 
> supertom@supertom:~/virts/libcloud-testing/code/apache-libcloud-2.0.0rc2$ 
> demos/gce_demo.py --compute
> => Compute demo/test start time: 2017-04-05 16:22:58.309187
> DEBUG:requests.packages.urllib3.connectionpool:Starting new HTTPS connection 
> (1): www.googleapis.com
> DEBUG:requests.packages.urllib3.connectionpool:https://www.googleapis.com:443 
> "GET /compute/v1/projects/supertom-graphite/zones HTTP/1.1" 200 None
> /usr/lib/python2.7/gzip.py:196: UnicodeWarning: Unicode unequal comparison 
> failed to convert both arguments to Unicode - interpreting them as being 
> unequal
>   if magic != '\037\213':
> Traceback (most recent call last):
>   File "demos/gce_demo.py", line 957, in 
> main_compute()
>   File "demos/gce_demo.py", line 342, in main_compute
> gce = get_gce_driver()
>   File "demos/gce_demo.py", line 111, in get_gce_driver
> driver = get_driver(Provider.GCE)(*args, **kwargs)
>   File "./libcloud/compute/drivers/gce.py", line 1803, in __init__
> self.zone_list = self.ex_list_zones()
>   File "./libcloud/compute/drivers/gce.py", line 2757, in ex_list_zones
> response = self.connection.request(request, method='GET').object
>   File "./libcloud/compute/drivers/gce.py", line 121, in request
> response = super(GCEConnection, self).request(*args, **kwargs)
>   File "./libcloud/common/google.py", line 806, in request
> *args, **kwargs)
>   File "./libcloud/common/base.py", line 664, in request
> 'response': self.connection.getresponse()}
>   File "./libcloud/utils/loggingconnection.py", line 141, in getresponse
> rv = self._log_response(HttpLibResponseProxy(original_response))
>   File "./libcloud/utils/loggingconnection.py", line 71, in _log_response
> body = decompress_data('gzip', body)
>   File "./libcloud/utils/compression.py", line 39, in decompress_data
> return gzip.GzipFile(fileobj=cls(data)).read()
>   File "/usr/lib/python2.7/gzip.py", line 261, in read
> self._read(readsize)
>   File "/usr/lib/python2.7/gzip.py", line 303, in _read
> self._read_gzip_header()
>   File "/usr/lib/python2.7/gzip.py", line 197, in _read_gzip_header
> raise IOError, 'Not a gzipped file'
> IOError: Not a gzipped file
> ## Steps to reproduce
> export LIBCLOUD_DEBUG=/tmp/gce-libcloud.log
> (set values in demos/secrets.py)
> demos/gce_demo.py --compute



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (LIBCLOUD-910) Not a Gzipped file when using LIBCLOUD_DEBUG

2017-04-21 Thread Anthony Shaw (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LIBCLOUD-910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anthony Shaw resolved LIBCLOUD-910.
---
Resolution: Fixed

> Not a Gzipped file when using LIBCLOUD_DEBUG
> 
>
> Key: LIBCLOUD-910
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-910
> Project: Libcloud
>  Issue Type: Bug
>  Components: Core
> Environment: python 2.6, debian jessie
>Reporter: Tom Melendez
>
> ## Summary
> When LIBCLOUD_DEBUG is set, receive a gzip error.  If the call to 
> decompress_data is commented out, things progress fine. 
> Offending line:
>   File "./libcloud/utils/loggingconnection.py", line 71, in _log_response
> body = decompress_data('gzip', body)
> ## Stacktrace
> (libcloud-testing) 
> supertom@supertom:~/virts/libcloud-testing/code/apache-libcloud-2.0.0rc2$ 
> demos/gce_demo.py --compute
> => Compute demo/test start time: 2017-04-05 16:22:58.309187
> DEBUG:requests.packages.urllib3.connectionpool:Starting new HTTPS connection 
> (1): www.googleapis.com
> DEBUG:requests.packages.urllib3.connectionpool:https://www.googleapis.com:443 
> "GET /compute/v1/projects/supertom-graphite/zones HTTP/1.1" 200 None
> /usr/lib/python2.7/gzip.py:196: UnicodeWarning: Unicode unequal comparison 
> failed to convert both arguments to Unicode - interpreting them as being 
> unequal
>   if magic != '\037\213':
> Traceback (most recent call last):
>   File "demos/gce_demo.py", line 957, in 
> main_compute()
>   File "demos/gce_demo.py", line 342, in main_compute
> gce = get_gce_driver()
>   File "demos/gce_demo.py", line 111, in get_gce_driver
> driver = get_driver(Provider.GCE)(*args, **kwargs)
>   File "./libcloud/compute/drivers/gce.py", line 1803, in __init__
> self.zone_list = self.ex_list_zones()
>   File "./libcloud/compute/drivers/gce.py", line 2757, in ex_list_zones
> response = self.connection.request(request, method='GET').object
>   File "./libcloud/compute/drivers/gce.py", line 121, in request
> response = super(GCEConnection, self).request(*args, **kwargs)
>   File "./libcloud/common/google.py", line 806, in request
> *args, **kwargs)
>   File "./libcloud/common/base.py", line 664, in request
> 'response': self.connection.getresponse()}
>   File "./libcloud/utils/loggingconnection.py", line 141, in getresponse
> rv = self._log_response(HttpLibResponseProxy(original_response))
>   File "./libcloud/utils/loggingconnection.py", line 71, in _log_response
> body = decompress_data('gzip', body)
>   File "./libcloud/utils/compression.py", line 39, in decompress_data
> return gzip.GzipFile(fileobj=cls(data)).read()
>   File "/usr/lib/python2.7/gzip.py", line 261, in read
> self._read(readsize)
>   File "/usr/lib/python2.7/gzip.py", line 303, in _read
> self._read_gzip_header()
>   File "/usr/lib/python2.7/gzip.py", line 197, in _read_gzip_header
> raise IOError, 'Not a gzipped file'
> IOError: Not a gzipped file
> ## Steps to reproduce
> export LIBCLOUD_DEBUG=/tmp/gce-libcloud.log
> (set values in demos/secrets.py)
> demos/gce_demo.py --compute



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (LIBCLOUD-795) Drop support for Python 2.5

2017-04-21 Thread Anthony Shaw (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LIBCLOUD-795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anthony Shaw closed LIBCLOUD-795.
-
Resolution: Fixed

> Drop support for Python 2.5
> ---
>
> Key: LIBCLOUD-795
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-795
> Project: Libcloud
>  Issue Type: Sub-task
>Reporter: Tomaz Muraus
>Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (LIBCLOUD-902) Invalid credentials with the provider error

2017-04-21 Thread Anthony Shaw (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LIBCLOUD-902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anthony Shaw closed LIBCLOUD-902.
-
Resolution: Fixed

> Invalid credentials with the provider  error
> 
>
> Key: LIBCLOUD-902
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-902
> Project: Libcloud
>  Issue Type: Bug
>  Components: Compute
>Reporter: sathyamoorthy
>Priority: Blocker
>
> I have some issues with libcloud library when trying to retrieve the 
> openstack instance details. Please let me know if you have any idea on the 
> below error. This error is not seen in kilo setup. 
> --
> >>> driver.list_nodes()
> Traceback (most recent call last):
> File "/home/s.viswanathan/GV/lib/cloudlib.py", line 94, in show_instances
> ERROR 2017-03-07 04:04:20 nodes = sess.list_nodes()
> ERROR 2017-03-07 04:04:20   File 
> "/usr/local/lib/python2.7/dist-packages/libcloud/compute/drivers/openstack.py",
>  line 177, in list_nodes
> ERROR 2017-03-07 04:04:20 self.connection.request('/servers/detail', 
> params=params).object)
> ERROR 2017-03-07 04:04:20   File 
> "/usr/local/lib/python2.7/dist-packages/libcloud/common/openstack.py", line 
> 227, in request
> ERROR 2017-03-07 04:04:20 raw=raw)
> ERROR 2017-03-07 04:04:20   File 
> "/usr/local/lib/python2.7/dist-packages/libcloud/common/base.py", line 757, 
> in request
> ERROR 2017-03-07 04:04:20 action = self.morph_action_hook(action)
> ERROR 2017-03-07 04:04:20   File 
> "/usr/local/lib/python2.7/dist-packages/libcloud/common/openstack.py", line 
> 294, in morph_action_hook
> ERROR 2017-03-07 04:04:20 self._populate_hosts_and_request_paths()
> ERROR 2017-03-07 04:04:20   File 
> "/usr/local/lib/python2.7/dist-packages/libcloud/common/openstack.py", line 
> 327, in _populate_hosts_and_request_paths
> ERROR 2017-03-07 04:04:20 osa = osa.authenticate(**kwargs)  # may throw 
> InvalidCreds
> ERROR 2017-03-07 04:04:20   File 
> "/usr/local/lib/python2.7/dist-packages/libcloud/common/openstack_identity.py",
>  line 855, in authenticate
> ERROR 2017-03-07 04:04:20 return self._authenticate_2_0_with_password()
> ERROR 2017-03-07 04:04:20   File 
> "/usr/local/lib/python2.7/dist-packages/libcloud/common/openstack_identity.py",
>  line 880, in _authenticate_2_0_with_password
> ERROR 2017-03-07 04:04:20 return self._authenticate_2_0_with_body(reqbody)
> ERROR 2017-03-07 04:04:20   File 
> "/usr/local/lib/python2.7/dist-packages/libcloud/common/openstack_identity.py",
>  line 888, in _authenticate_2_0_with_body
> ERROR 2017-03-07 04:04:20 raise InvalidCredsError()
> ERROR 2017-03-07 04:04:20 InvalidCredsError: 'Invalid credentials with the 
> provider' 
> --
> I have tried from python interpreter manually also, seen same problem.
> from libcloud.compute.types import Provider
> from libcloud.compute.providers import get_driver
> import libcloud.security
> OpenStack = get_driver(Provider.OPENSTACK)
> driver = OpenStack('xxx', 
> yyy',ex_force_auth_url='http://x.x.x.x:5000',ex_force_auth_version='2.0_password')
> driver.list_nodes()
> Thanks in advance,
> Sathya



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Closed] (LIBCLOUD-834) AWS Storage S3 multipart uploads fail when using v4 authentication

2017-04-21 Thread Anthony Shaw (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LIBCLOUD-834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anthony Shaw closed LIBCLOUD-834.
-
Resolution: Fixed

> AWS Storage S3 multipart uploads fail when using v4 authentication 
> ---
>
> Key: LIBCLOUD-834
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-834
> Project: Libcloud
>  Issue Type: Bug
>  Components: Storage
> Environment: Production
>Reporter: Luke Morfitt
>Priority: Blocker
>
> When using v4 authentication and attempting to perform a object upload using 
> "upload_object_via_stream" the upload fails with the following error.
> I suspect the issue relates to the PUT or POST method with v4 authentication, 
> however I have not tracked it down.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (LIBCLOUD-911) Create 1&1 Compute Driver

2017-04-21 Thread Anthony Shaw (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LIBCLOUD-911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anthony Shaw resolved LIBCLOUD-911.
---
Resolution: Delivered

> Create 1&1 Compute Driver
> -
>
> Key: LIBCLOUD-911
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-911
> Project: Libcloud
>  Issue Type: New Feature
>  Components: Compute
> Environment: Linux, Windows, MacOs
>Reporter: Jasmin Gacic
>
> Creating compute driver for the 1&1 platform



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (LIBCLOUD-912) [python] Requests tries to connect to 127.0.0.1:443

2017-04-21 Thread Anthony Shaw (JIRA) 

[ 
https://issues.apache.org/jira/browse/LIBCLOUD-912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15978131#comment-15978131
 ] 

Anthony Shaw commented on LIBCLOUD-912:
---

thanks for raising this. We need to get it fixed.

Can you give me a code snippet about how you instantiate the driver and what 
arguments you use so I can reproduce it?

When you say Rackspace/openstack are you using the rackspace driver or the 
openstack one directly?

The behaviour you're seeing is because the OpenStack driver creates 2 
connections, one to the API and one to the authentication endpoint, the second 
is created after instantiation of the first. If you see 127.0.0.1, thats the 
default host address (although it should be none or something more obvious that 
it's broken).
I assume what's happening is the host isn't being passed to the second 
connection class. 

> [python] Requests tries to connect to 127.0.0.1:443
> ---
>
> Key: LIBCLOUD-912
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-912
> Project: Libcloud
>  Issue Type: Bug
> Environment: python 2.7
> apache-libcloud==2.0.0rc2
>Reporter: Alexis Lesieur
>  Labels: newbie, python
>
> Hi!
> Sorry if this story is not created properly.
> I'm not that familiar with how you guys manage your Jira. Feel free to give 
> me directions or just modify the ticket :-)
> So we have been trying to upgrade the version of our libcloud python package, 
> but it breaks for our rackspace/openstack environment.
> I have been digging for a while and I couldn't find where the difference was 
> between 1.x.x and 2.0.0rc2.
> Observed behavior:
> 1.x.x:
> 3 connections are created:
> - to 127.0.0.1:443
> - to host:3000 (ex_force_auth_url)
> - to host:8774/v2/... (don't know where it comes from)
> 2.0.0rc2:
> 2 connections
> - to 127.0.0.1:443
> - to host:3000 (ex_force_auth_url)
> and the last connection is done using 127.0.0.1:443 again, using the same 
> connector created in the first one
> I don't understand the change in behavior between the two versions.
> I tried use the ex_force_base_url flag, and while it does look like the path 
> of that url is respected, the host is always replaced by 127.0.0.1.
> Any idea of what is happening?
> Thanks!



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)