[GitHub] libcloud pull request #1131: remove unused digitalocean v1 fixtures

2017-10-16 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/libcloud/pull/1131


---

libcloud git commit: remove unused digitalocean v1 fixtures

2017-10-16 Thread quentinp 
Repository: libcloud
Updated Branches:
  refs/heads/trunk 29c8eab97 -> 7ea3646c0


remove unused digitalocean v1 fixtures

the fixtures digitalocean/ex_change_kernel.json and
digitalocean/ex_hard_reboot.json are not used for the v1 digitalocean
driver.

Closes #1131

Signed-off-by: Quentin Pradet 


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/7ea3646c
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/7ea3646c
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/7ea3646c

Branch: refs/heads/trunk
Commit: 7ea3646c03fc286c9124a14743592bbde955a0f6
Parents: 29c8eab
Author: Rick van de Loo 
Authored: Mon Oct 16 09:58:16 2017 +0200
Committer: Quentin Pradet 
Committed: Tue Oct 17 08:45:20 2017 +0400

--
 .../compute/fixtures/digitalocean/ex_change_kernel.json | 12 
 .../compute/fixtures/digitalocean/ex_hard_reboot.json   | 12 
 2 files changed, 24 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/libcloud/blob/7ea3646c/libcloud/test/compute/fixtures/digitalocean/ex_change_kernel.json
--
diff --git a/libcloud/test/compute/fixtures/digitalocean/ex_change_kernel.json 
b/libcloud/test/compute/fixtures/digitalocean/ex_change_kernel.json
deleted file mode 100644
index 561101f..000
--- a/libcloud/test/compute/fixtures/digitalocean/ex_change_kernel.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
-  "action": {
-"id": 36077295,
-"status": "in-progress",
-"type": "kernel_change",
-"started_at": "2014-11-04T17:08:03Z",
-"completed_at": null,
-"resource_id": 3067650,
-"resource_type": "droplet",
-"region": "ams2"
-  }
-}

http://git-wip-us.apache.org/repos/asf/libcloud/blob/7ea3646c/libcloud/test/compute/fixtures/digitalocean/ex_hard_reboot.json
--
diff --git a/libcloud/test/compute/fixtures/digitalocean/ex_hard_reboot.json 
b/libcloud/test/compute/fixtures/digitalocean/ex_hard_reboot.json
deleted file mode 100644
index 891b32e..000
--- a/libcloud/test/compute/fixtures/digitalocean/ex_hard_reboot.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
-  "action": {
-"id": 36077294,
-"status": "in-progress",
-"type": "power_cycle",
-"started_at": "2014-11-04T17:08:03Z",
-"completed_at": null,
-"resource_id": 3067651,
-"resource_type": "droplet",
-"region": "ams3"
-  }
-}

[GitHub] libcloud pull request #1134: Limit number of retries in Azure ARM destroy_no...

2017-10-16 Thread tetron 
GitHub user tetron opened a pull request:

https://github.com/apache/libcloud/pull/1134

Limit number of retries in Azure ARM destroy_node.

Limit number of retries in Azure ARM destroy_node.

### Description

Azure ARM has several retry loops related to deleting the NIC and VHD as 
part of node cleanup.  This bounds those loops so they don't retry forever.

### Status

- done, ready for review

### Checklist (tick everything that applies)

- [ ] [Code 
linting](http://libcloud.readthedocs.org/en/latest/development.html#code-style-guide)
 (required, can be done after the PR checks)
- [ ] Documentation
- [ ] 
[ICLA](http://libcloud.readthedocs.org/en/latest/development.html#contributing-bigger-changes)
 (required for bigger changes)


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/curoverse/libcloud destroy-node-limit-retries

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/libcloud/pull/1134.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1134


commit da58796454c693354b5eb6632fb522faf4c47356
Author: Peter Amstutz 
Date:   2017-10-16T20:12:45Z

Limit number of retries in destroy_node.




---

[jira] [Resolved] (LIBCLOUD-857) DigitalOcean API v1 has been EOLed

2017-10-16 Thread Andrew Starr-Bochicchio (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LIBCLOUD-857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Starr-Bochicchio resolved LIBCLOUD-857.
--
Resolution: Fixed

> DigitalOcean API v1 has been EOLed
> --
>
> Key: LIBCLOUD-857
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-857
> Project: Libcloud
>  Issue Type: Bug
>  Components: Compute
>Reporter: Andrew Starr-Bochicchio
>Priority: Minor
>  Labels: digitalocean
>
> DigitalOcean API v1 reached EOL November 9, 2015. It's code paths can/should 
> be removed.
> https://developers.digitalocean.com/documentation/changelog/api-v1/sunsetting-api-v1/
> I intend to submit a patch for this.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (LIBCLOUD-955) Add support for DigitalOcean Spaces

2017-10-16 Thread Andrew Starr-Bochicchio (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LIBCLOUD-955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Starr-Bochicchio updated LIBCLOUD-955:
-
Labels: digitalocean  (was: )

> Add support for DigitalOcean Spaces
> ---
>
> Key: LIBCLOUD-955
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-955
> Project: Libcloud
>  Issue Type: New Feature
>  Components: Storage
>Reporter: Andrew Starr-Bochicchio
>Priority: Minor
>  Labels: digitalocean
>
> DigitalOcean Spaces is an S3 compatible object storage service.
> https://developers.digitalocean.com/documentation/spaces/
> PR incoming.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (LIBCLOUD-955) Add support for DigitalOcean Spaces

2017-10-16 Thread Andrew Starr-Bochicchio (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LIBCLOUD-955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Starr-Bochicchio resolved LIBCLOUD-955.
--
Resolution: Fixed

> Add support for DigitalOcean Spaces
> ---
>
> Key: LIBCLOUD-955
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-955
> Project: Libcloud
>  Issue Type: New Feature
>  Components: Storage
>Reporter: Andrew Starr-Bochicchio
>Priority: Minor
>
> DigitalOcean Spaces is an S3 compatible object storage service.
> https://developers.digitalocean.com/documentation/spaces/
> PR incoming.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] libcloud pull request #1133: Fix improper getresponse() uses

2017-10-16 Thread pquentin 
GitHub user pquentin opened a pull request:

https://github.com/apache/libcloud/pull/1133

Fix improper getresponse() uses

## Fix improper getresponse() uses

### Description

During the migration from httplib to requests, a lot of driver code did not 
change because they use the high-level libcloud API which hid the change from 
httplib to requests with the HttpLibResponseProxy adapter. However, in some 
cases, driver code needs to access the actual response, and does this by 
calling getresponse().

While this used to return an httlib.HTTPResponse instance, this now returns 
a requests.Response instance. All code covered by tests was fixed, but OVH and 
Brightbox common code is not fully covered. I don't have a subscription to 
either service and can't test the change, but the resulting code can only be 
more correct since it fixes the `getresponse() result comes from httplib` 
assumption.

### Status

- done, ready for review

### Checklist (tick everything that applies)

- [x] [Code 
linting](http://libcloud.readthedocs.org/en/latest/development.html#code-style-guide)
 (required, can be done after the PR checks)
- [ ] Documentation
- [ ] [Tests](http://libcloud.readthedocs.org/en/latest/testing.html)


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/pquentin/libcloud getresponse-improper-usage

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/libcloud/pull/1133.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1133


commit b10395fe21768264c79118155be34d11979de9fa
Author: Quentin Pradet 
Date:   2017-10-16T10:11:50Z

Fix improper getresponse() uses

During the migration from httplib to requests, a lot of driver code did
not change because they use the high-level libcloud API which hid the
change from httplib to requests with the HttpLibResponseProxy adapter.
However, in some cases, driver code needs to access the actual response,
and does this by calling getresponse().

While this used to return an httlib.HTTPResponse instance, this now
returns a requests.Response instance. All code covered by tests was
fixed, but OVH and Brightbox common code is not fully covered. I don't
have a subscription to either service and can't test the change, but the
resulting code can only be more correct since it fixes the
`getresponse() result comes from httplib` assumption.




---

[GitHub] libcloud pull request #1132: Ensure RawResponse downloads don't consume RAM

2017-10-16 Thread pquentin 
GitHub user pquentin opened a pull request:

https://github.com/apache/libcloud/pull/1132

Ensure RawResponse downloads don't consume RAM


## Ensure RawResponse downloads don't consume RAM

The fix is general but the cause is very specific: since libcloud 2.0, 
`download_object()` in the S3 and GCS drivers first places the item in RAM, 
then writes it to a file. The main reason it broke is that accessing `body` in 
a requests Response consumes it entirely and loads it in RAM. To fix this, I 
moved `body` to a property: it is still accessibly, but loaded only if 
requested.

Tested this change by downloading a 38M file using this code:

```
from libcloud.storage.drivers.google_storage import GoogleStorageDriver

driver = GoogleStorageDriver(key=..., secret=...)
obj = driver.get_object('container_name', 'object_name')
driver.download_object(obj, '/path/to/file')
```

The RAM usage before:


![before](https://user-images.githubusercontent.com/42327/31609057-b172a3f0-b283-11e7-9108-2964978bbcd2.png)

The RAM usage after:


![after](https://user-images.githubusercontent.com/42327/31609063-b846db1a-b283-11e7-97ed-1e6ce365a41b.png)


### Status

- done, ready for review

### Checklist (tick everything that applies)

- [x] [Code 
linting](http://libcloud.readthedocs.org/en/latest/development.html#code-style-guide)
 (required, can be done after the PR checks)
- [ ] Documentation
- [ ] [Tests](http://libcloud.readthedocs.org/en/latest/testing.html)

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/pquentin/libcloud streaming-s3

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/libcloud/pull/1132.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1132


commit 374cb88c64e4d77c338a9d1b0269d8c01ffc15b8
Author: Quentin Pradet 
Date:   2017-10-16T10:58:45Z

Ensure RawResponse downloads don't consume RAM

The fix is general but the cause is very specific: since libcloud 2.0,
`download_object()` in the S3 and GCS drivers first places the item in
RAM, then writes it to a file. The reason it broke is that accessing
`body` in a requests Response consumes it entirely and loads it in RAM.
To fix this, I moved `body` to a property: it is still accessibly, but
loaded only if requested.




---

[GitHub] libcloud pull request #1131: remove unused digitalocean v1 fixtures

2017-10-16 Thread vdloo 
GitHub user vdloo opened a pull request:

https://github.com/apache/libcloud/pull/1131

remove unused digitalocean v1 fixtures

the fixtures digitalocean/ex_change_kernel.json and 
digitalocean/ex_hard_reboot.json are not used for the v1 digitalocean driver.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/vdloo/libcloud 
remove-unused-digitalocean-v1-fixtures

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/libcloud/pull/1131.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1131


commit 144622ec04a9cf5b8338c49def76637135dd900b
Author: Rick van de Loo 
Date:   2017-10-16T07:58:16Z

remove unused digitalocean v1 fixtures

the fixtures digitalocean/ex_change_kernel.json and
digitalocean/ex_hard_reboot.json are not used for the v1 digitalocean
driver.




---

[GitHub] libcloud pull request #1130: add ex_enable_ipv6 in DigitalOcean_v2 driver

2017-10-16 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/libcloud/pull/1130


---

[1/2] libcloud git commit: add ex_enable_ipv6 in DigitalOcean_v2 driver

2017-10-16 Thread quentinp 
Repository: libcloud
Updated Branches:
  refs/heads/trunk ef35d3467 -> 29c8eab97


add ex_enable_ipv6 in DigitalOcean_v2 driver

> To enable IPv6 networking on an existing Droplet
> (within a region that has IPv6 available), send
> a POST request to /v2/droplets/$DROPLET_ID/actions.
> Set the "type" attribute to enable_ipv6.

https://developers.digitalocean.com/documentation/v2/

Signed-off-by: Quentin Pradet 


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/dd98507d
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/dd98507d
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/dd98507d

Branch: refs/heads/trunk
Commit: dd98507d783d8a4d3a33a3d2c970377b0b73f34a
Parents: ef35d34
Author: Rick van de Loo 
Authored: Sun Oct 15 15:01:02 2017 +0200
Committer: Quentin Pradet 
Committed: Mon Oct 16 11:44:45 2017 +0400

--
 libcloud/compute/drivers/digitalocean.py   |  6 ++
 .../fixtures/digitalocean_v2/ex_enable_ipv6.json   | 13 +
 libcloud/test/compute/test_digitalocean_v2.py  | 11 +++
 3 files changed, 30 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/libcloud/blob/dd98507d/libcloud/compute/drivers/digitalocean.py
--
diff --git a/libcloud/compute/drivers/digitalocean.py 
b/libcloud/compute/drivers/digitalocean.py
index a9b1e4f..4063c28 100644
--- a/libcloud/compute/drivers/digitalocean.py
+++ b/libcloud/compute/drivers/digitalocean.py
@@ -250,6 +250,12 @@ class 
DigitalOcean_v2_NodeDriver(DigitalOcean_v2_BaseDriver,
   data=json.dumps(attr), method='POST')
 return res.status == httplib.CREATED
 
+def ex_enable_ipv6(self, node):
+attr = {'type': 'enable_ipv6'}
+res = self.connection.request('/v2/droplets/%s/actions' % (node.id),
+  data=json.dumps(attr), method='POST')
+return res.status == httplib.CREATED
+
 def ex_rename_node(self, node, name):
 attr = {'type': 'rename', 'name': name}
 res = self.connection.request('/v2/droplets/%s/actions' % (node.id),

http://git-wip-us.apache.org/repos/asf/libcloud/blob/dd98507d/libcloud/test/compute/fixtures/digitalocean_v2/ex_enable_ipv6.json
--
diff --git a/libcloud/test/compute/fixtures/digitalocean_v2/ex_enable_ipv6.json 
b/libcloud/test/compute/fixtures/digitalocean_v2/ex_enable_ipv6.json
new file mode 100644
index 000..9386b97
--- /dev/null
+++ b/libcloud/test/compute/fixtures/digitalocean_v2/ex_enable_ipv6.json
@@ -0,0 +1,13 @@
+{
+  "action": {
+"id": 36804954,
+"status": "in-progress",
+"type": "enable_ipv6",
+"started_at": "2014-11-14T16:34:24Z",
+"completed_at": null,
+"resource_id": 3164450,
+"resource_type": "droplet",
+"region": "ams2",
+"region_slug": "ams2"
+  }
+}

http://git-wip-us.apache.org/repos/asf/libcloud/blob/dd98507d/libcloud/test/compute/test_digitalocean_v2.py
--
diff --git a/libcloud/test/compute/test_digitalocean_v2.py 
b/libcloud/test/compute/test_digitalocean_v2.py
index c15dbe7..8589571 100644
--- a/libcloud/test/compute/test_digitalocean_v2.py
+++ b/libcloud/test/compute/test_digitalocean_v2.py
@@ -166,6 +166,12 @@ class DigitalOcean_v2_Tests(LibcloudTestCase):
 result = self.driver.ex_change_kernel(node, 7515)
 self.assertTrue(result)
 
+def test_ex_enable_ipv6_success(self):
+node = self.driver.list_nodes()[0]
+DigitalOceanMockHttp.type = 'ENABLEIPV6'
+result = self.driver.ex_enable_ipv6(node)
+self.assertTrue(result)
+
 def test_ex_rename_node_success(self):
 node = self.driver.list_nodes()[0]
 DigitalOceanMockHttp.type = 'RENAME'
@@ -312,6 +318,11 @@ class DigitalOceanMockHttp(MockHttp):
 body = self.fixtures.load('ex_change_kernel.json')
 return (httplib.CREATED, body, {}, httplib.responses[httplib.CREATED])
 
+def _v2_droplets_316_actions_ENABLEIPV6(self, method, url, body, 
headers):
+# enable_ipv6
+body = self.fixtures.load('ex_enable_ipv6.json')
+return (httplib.CREATED, body, {}, httplib.responses[httplib.CREATED])
+
 def _v2_droplets_316_actions_RENAME(self, method, url, body, headers):
 # rename_node
 body = self.fixtures.load('ex_rename_node.json')

[2/2] libcloud git commit: Add changes for #1130

2017-10-16 Thread quentinp 
Add changes for #1130

Closes #1130


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/29c8eab9
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/29c8eab9
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/29c8eab9

Branch: refs/heads/trunk
Commit: 29c8eab97eb63bd2e7f19760958465ebf92260be
Parents: dd98507
Author: Quentin Pradet 
Authored: Mon Oct 16 11:47:05 2017 +0400
Committer: Quentin Pradet 
Committed: Mon Oct 16 11:47:05 2017 +0400

--
 CHANGES.rst | 4 
 1 file changed, 4 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/libcloud/blob/29c8eab9/CHANGES.rst
--
diff --git a/CHANGES.rst b/CHANGES.rst
index 3653a4d..d8d7191 100644
--- a/CHANGES.rst
+++ b/CHANGES.rst
@@ -66,6 +66,10 @@ Compute
   (LIBCLOUD-954, GITHUB-1127)
   [Quentin Pradet]
 
+- [Digital Ocean] Add ex_enable_ipv6 in DigitalOcean_v2 driver
+  (GITHUB-1130)
+  [Rick van de Loo]
+
 Storage
 ~~~

[jira] [Commented] (LIBCLOUD-955) Add support for DigitalOcean Spaces

2017-10-16 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/LIBCLOUD-955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205474#comment-16205474
 ] 

ASF GitHub Bot commented on LIBCLOUD-955:
-

Github user asfgit closed the pull request at:

https://github.com/apache/libcloud/pull/1129


> Add support for DigitalOcean Spaces
> ---
>
> Key: LIBCLOUD-955
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-955
> Project: Libcloud
>  Issue Type: New Feature
>  Components: Storage
>Reporter: Andrew Starr-Bochicchio
>Priority: Minor
>
> DigitalOcean Spaces is an S3 compatible object storage service.
> https://developers.digitalocean.com/documentation/spaces/
> PR incoming.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] libcloud pull request #1129: [LIBCLOUD-955] Add DigitalOcean Spaces support.

2017-10-16 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/libcloud/pull/1129


---

libcloud git commit: Add changes for #1129

2017-10-16 Thread quentinp 
Repository: libcloud
Updated Branches:
  refs/heads/trunk be3d1e18a -> ef35d3467


Add changes for #1129

Closes #1129


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/ef35d346
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/ef35d346
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/ef35d346

Branch: refs/heads/trunk
Commit: ef35d34672929b6b48c3a2af3e8af96692021580
Parents: be3d1e1
Author: Quentin Pradet 
Authored: Mon Oct 16 10:36:05 2017 +0400
Committer: Quentin Pradet 
Committed: Mon Oct 16 10:36:05 2017 +0400

--
 CHANGES.rst | 6 ++
 1 file changed, 6 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/libcloud/blob/ef35d346/CHANGES.rst
--
diff --git a/CHANGES.rst b/CHANGES.rst
index 98fc44f..3653a4d 100644
--- a/CHANGES.rst
+++ b/CHANGES.rst
@@ -66,6 +66,12 @@ Compute
   (LIBCLOUD-954, GITHUB-1127)
   [Quentin Pradet]
 
+Storage
+~~~
+
+- Added Digital Ocean Spaces driver (LIBCLOUD-955, GITHUB-1129)
+  [Andrew Starr-Bochicchio]
+
 
 Changes in Apache Libcloud 2.2.1

[1/5] libcloud git commit: Document changing signature version.

2017-10-16 Thread quentinp 
Repository: libcloud
Updated Branches:
  refs/heads/trunk 820ff6f70 -> be3d1e18a


Document changing signature version.

Signed-off-by: Quentin Pradet 


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/be3d1e18
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/be3d1e18
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/be3d1e18

Branch: refs/heads/trunk
Commit: be3d1e18ac18a694a453a004d3f72ac7e9867c4e
Parents: 6d01dda
Author: Andrew Starr-Bochicchio 
Authored: Sun Oct 15 13:06:06 2017 -0400
Committer: Quentin Pradet 
Committed: Mon Oct 16 10:24:58 2017 +0400

--
 docs/examples/storage/digitalocean_spaces/v4sig.py | 8 
 docs/storage/drivers/digitalocean_spaces.rst   | 7 +++
 2 files changed, 15 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/libcloud/blob/be3d1e18/docs/examples/storage/digitalocean_spaces/v4sig.py
--
diff --git a/docs/examples/storage/digitalocean_spaces/v4sig.py 
b/docs/examples/storage/digitalocean_spaces/v4sig.py
new file mode 100644
index 000..d370b33
--- /dev/null
+++ b/docs/examples/storage/digitalocean_spaces/v4sig.py
@@ -0,0 +1,8 @@
+from libcloud.storage.types import Provider
+from libcloud.storage.providers import get_driver
+
+cls = get_driver(Provider.DIGITALOCEAN_SPACES)
+
+driver = cls(key='DO_ACCESS_KEY',
+ secret='DO_SECRET_KEY',
+ signature_version='4')

http://git-wip-us.apache.org/repos/asf/libcloud/blob/be3d1e18/docs/storage/drivers/digitalocean_spaces.rst
--
diff --git a/docs/storage/drivers/digitalocean_spaces.rst 
b/docs/storage/drivers/digitalocean_spaces.rst
index 919cdc8..b8739fc 100644
--- a/docs/storage/drivers/digitalocean_spaces.rst
+++ b/docs/storage/drivers/digitalocean_spaces.rst
@@ -20,6 +20,13 @@ below.
 .. literalinclude:: /examples/storage/digitalocean_spaces/instantiate.py
:language: python
 
+Spaces supports both the v2 and v4 AWS signature types. By default, this driver
+will use v2. You can configure it to use v4 by passing the 
``signature_version``
+argument when instantiating the driver as shown below.
+
+.. literalinclude:: /examples/storage/digitalocean_spaces/v4sig.py
+   :language: python
+
 Specifying canned ACLs
 ~~

[jira] [Commented] (LIBCLOUD-955) Add support for DigitalOcean Spaces

2017-10-16 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/LIBCLOUD-955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205471#comment-16205471
 ] 

ASF subversion and git services commented on LIBCLOUD-955:
--

Commit 79052a080f87a436f27701db7429fa7dcb11d433 in libcloud's branch 
refs/heads/trunk from [~andrewsomething]
[ https://git-wip-us.apache.org/repos/asf?p=libcloud.git;h=79052a0 ]

[LIBCLOUD-955] Add DigitalOcean Spaces support.

Signed-off-by: Quentin Pradet 


> Add support for DigitalOcean Spaces
> ---
>
> Key: LIBCLOUD-955
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-955
> Project: Libcloud
>  Issue Type: New Feature
>  Components: Storage
>Reporter: Andrew Starr-Bochicchio
>Priority: Minor
>
> DigitalOcean Spaces is an S3 compatible object storage service.
> https://developers.digitalocean.com/documentation/spaces/
> PR incoming.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[3/5] libcloud git commit: [LIBCLOUD-955] Add DigitalOcean Spaces support.

2017-10-16 Thread quentinp 
[LIBCLOUD-955] Add DigitalOcean Spaces support.

Signed-off-by: Quentin Pradet 


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/79052a08
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/79052a08
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/79052a08

Branch: refs/heads/trunk
Commit: 79052a080f87a436f27701db7429fa7dcb11d433
Parents: 820ff6f
Author: Andrew Starr-Bochicchio 
Authored: Fri Oct 6 15:59:48 2017 -0400
Committer: Quentin Pradet 
Committed: Mon Oct 16 10:24:58 2017 +0400

--
 libcloud/storage/drivers/digitalocean_spaces.py | 101 +++
 libcloud/storage/providers.py   |   3 +
 libcloud/storage/types.py   |   1 +
 .../test/storage/test_digitalocean_spaces.py| 122 +++
 4 files changed, 227 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/libcloud/blob/79052a08/libcloud/storage/drivers/digitalocean_spaces.py
--
diff --git a/libcloud/storage/drivers/digitalocean_spaces.py 
b/libcloud/storage/drivers/digitalocean_spaces.py
new file mode 100644
index 000..1808b4d
--- /dev/null
+++ b/libcloud/storage/drivers/digitalocean_spaces.py
@@ -0,0 +1,101 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from libcloud.common.types import LibcloudError
+from libcloud.common.aws import SignedAWSConnection, DEFAULT_SIGNATURE_VERSION
+from libcloud.storage.drivers.s3 import BaseS3Connection, S3Connection
+from libcloud.storage.drivers.s3 import S3StorageDriver, API_VERSION
+
+__all__ = [
+'DigitalOceanSpacesStorageDriver'
+]
+
+DO_SPACES_HOSTS_BY_REGION = {'nyc3': 'nyc3.digitaloceanspaces.com'}
+
+DO_SPACES_DEFAULT_REGION = 'nyc3'
+
+
+class DOSpacesConnectionAWS4(SignedAWSConnection, BaseS3Connection):
+service_name = 's3'
+version = API_VERSION
+
+def __init__(self, user_id, key, secure=True, host=None, port=None,
+ url=None, timeout=None, proxy_url=None, token=None,
+ retry_delay=None, backoff=None, **kwargs):
+
+super(DOSpacesConnectionAWS4, self).__init__(user_id, key,
+ secure, host,
+ port, url,
+ timeout,
+ proxy_url, token,
+ retry_delay,
+ backoff,
+ 4)  # force aws4
+
+
+class DOSpacesConnectionAWS2(S3Connection):
+
+def __init__(self, user_id, key, secure=True, host=None, port=None,
+ url=None, timeout=None, proxy_url=None, token=None,
+ retry_delay=None, backoff=None, **kwargs):
+
+super(DOSpacesConnectionAWS2, self).__init__(user_id, key,
+ secure, host,
+ port, url,
+ timeout,
+ proxy_url, token,
+ retry_delay,
+ backoff)
+
+
+class DigitalOceanSpacesStorageDriver(S3StorageDriver):
+name = 'DigitalOcean Spaces'
+website = 'https://www.digitalocean.com/products/object-storage/'
+supports_chunked_encoding = False
+supports_s3_multipart_upload = True
+
+def __init__(self, key, secret=None, secure=True, host=None, port=None,
+ api_version=None, region=DO_SPACES_DEFAULT_REGION, **kwargs):
+
+if region not in DO_SPACES_HOSTS_BY_REGION:
+raise LibcloudError('Unknown region (%s)' % (region), driver=self)
+
+host = DO_SPACES_HOSTS_BY_REGION[region]
+self.name = 'DigitalOcean Spaces (%s)' % (region)
+
+

[4/5] libcloud git commit: Define contants and test for error states.

2017-10-16 Thread quentinp 
Define contants and test for error states.

Signed-off-by: Quentin Pradet 


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/6d01dda5
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/6d01dda5
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/6d01dda5

Branch: refs/heads/trunk
Commit: 6d01dda509b0df827252f3106f8bfb7e933c3ad7
Parents: 24584e6
Author: Andrew Starr-Bochicchio 
Authored: Sun Oct 15 13:05:22 2017 -0400
Committer: Quentin Pradet 
Committed: Mon Oct 16 10:24:58 2017 +0400

--
 libcloud/storage/drivers/digitalocean_spaces.py   | 18 +-
 libcloud/test/storage/test_digitalocean_spaces.py | 14 ++
 2 files changed, 23 insertions(+), 9 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/libcloud/blob/6d01dda5/libcloud/storage/drivers/digitalocean_spaces.py
--
diff --git a/libcloud/storage/drivers/digitalocean_spaces.py 
b/libcloud/storage/drivers/digitalocean_spaces.py
index 1808b4d..9f6768c 100644
--- a/libcloud/storage/drivers/digitalocean_spaces.py
+++ b/libcloud/storage/drivers/digitalocean_spaces.py
@@ -14,22 +14,23 @@
 # limitations under the License.
 
 from libcloud.common.types import LibcloudError
-from libcloud.common.aws import SignedAWSConnection, DEFAULT_SIGNATURE_VERSION
+from libcloud.common.aws import SignedAWSConnection
 from libcloud.storage.drivers.s3 import BaseS3Connection, S3Connection
-from libcloud.storage.drivers.s3 import S3StorageDriver, API_VERSION
+from libcloud.storage.drivers.s3 import S3StorageDriver
 
 __all__ = [
 'DigitalOceanSpacesStorageDriver'
 ]
 
 DO_SPACES_HOSTS_BY_REGION = {'nyc3': 'nyc3.digitaloceanspaces.com'}
-
 DO_SPACES_DEFAULT_REGION = 'nyc3'
+DEFAULT_SIGNATURE_VERSION = '2'
+S3_API_VERSION = '2006-03-01'
 
 
 class DOSpacesConnectionAWS4(SignedAWSConnection, BaseS3Connection):
 service_name = 's3'
-version = API_VERSION
+version = S3_API_VERSION
 
 def __init__(self, user_id, key, secure=True, host=None, port=None,
  url=None, timeout=None, proxy_url=None, token=None,
@@ -42,7 +43,7 @@ class DOSpacesConnectionAWS4(SignedAWSConnection, 
BaseS3Connection):
  proxy_url, token,
  retry_delay,
  backoff,
- 4)  # force aws4
+ signature_version=4)
 
 
 class DOSpacesConnectionAWS2(S3Connection):
@@ -79,14 +80,13 @@ class DigitalOceanSpacesStorageDriver(S3StorageDriver):
 self.signature_version = str(kwargs.pop('signature_version',
 DEFAULT_SIGNATURE_VERSION))
 
-if self.signature_version not in ['2', '4']:
-raise ValueError('Invalid signature_version: %s' %
- (self.signature_version))
-
 if self.signature_version == '2':
 self.connectionCls = DOSpacesConnectionAWS2
 elif self.signature_version == '4':
 self.connectionCls = DOSpacesConnectionAWS4
+else:
+raise ValueError('Invalid signature_version: %s' %
+ (self.signature_version))
 self.connectionCls.host = host
 
 super(DigitalOceanSpacesStorageDriver,

http://git-wip-us.apache.org/repos/asf/libcloud/blob/6d01dda5/libcloud/test/storage/test_digitalocean_spaces.py
--
diff --git a/libcloud/test/storage/test_digitalocean_spaces.py 
b/libcloud/test/storage/test_digitalocean_spaces.py
index 47f4779..7c32deb 100644
--- a/libcloud/test/storage/test_digitalocean_spaces.py
+++ b/libcloud/test/storage/test_digitalocean_spaces.py
@@ -16,6 +16,8 @@
 import sys
 import unittest
 
+from libcloud.common.types import LibcloudError
+
 from libcloud.storage.base import Container, Object
 from libcloud.storage.drivers.digitalocean_spaces import (
 DigitalOceanSpacesStorageDriver,
@@ -67,6 +69,18 @@ class DigitalOceanSpacesTests(LibcloudTestCase):
 with self.assertRaises(NotImplementedError):
 self.object.get_cdn_url()
 
+def test_invalid_signature_version(self):
+with self.assertRaises(ValueError):
+self.driver_type(*self.driver_args,
+ signature_version='3',
+ host=self.default_host)
+
+def test_invalid_region(self):
+with self.assertRaises(LibcloudError):
+self.driver_type(*self.driver_args,
+ region='atlantis',
+

[5/5] libcloud git commit: Update supported providers table

2017-10-16 Thread quentinp 
Update supported providers table

Signed-off-by: Quentin Pradet 


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/1bb5bed7
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/1bb5bed7
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/1bb5bed7

Branch: refs/heads/trunk
Commit: 1bb5bed7edcdd37f2df42c71948ebae9b0903fb5
Parents: 79052a0
Author: Andrew Starr-Bochicchio 
Authored: Mon Oct 16 10:19:45 2017 +0400
Committer: Quentin Pradet 
Committed: Mon Oct 16 10:24:58 2017 +0400

--
 docs/storage/_supported_providers.rst | 68 +++---
 1 file changed, 34 insertions(+), 34 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/libcloud/blob/1bb5bed7/docs/storage/_supported_providers.rst
--
diff --git a/docs/storage/_supported_providers.rst 
b/docs/storage/_supported_providers.rst
index 8f7f51e..482d49a 100644
--- a/docs/storage/_supported_providers.rst
+++ b/docs/storage/_supported_providers.rst
@@ -1,39 +1,39 @@
 .. NOTE: This file has been generated automatically using 
generate_provider_feature_matrix_table.py script, don't manually edit it
 
-= === 
=  
== 

-Provider  Documentation   
Provider Constant Supported RegionsModule   
  Class Name  
-= === 
=  
== 

-`Aliyun OSS`_ 
ALIYUN_OSSsingle region driver 
:mod:`libcloud.storage.drivers.oss`:class:`OSSStorageDriver`
   
-`PCextreme AuroraObjects`_:doc:`Click `   
AURORAOBJECTS single region driver 
:mod:`libcloud.storage.drivers.auroraobjects`  
:class:`AuroraObjectsStorageDriver` 
-`Microsoft Azure (blobs)`_:doc:`Click ` 
AZURE_BLOBS   single region driver 
:mod:`libcloud.storage.drivers.azure_blobs`:class:`AzureBlobsStorageDriver` 
   
-`Backblaze B2`_   :doc:`Click `
BACKBLAZE_B2  single region driver 
:mod:`libcloud.storage.drivers.backblaze_b2`   
:class:`BackblazeB2StorageDriver`   
-`CloudFiles`_ 
CLOUDFILESdfw, hkg, iad, lon, ord, syd 
:mod:`libcloud.storage.drivers.cloudfiles` :class:`CloudFilesStorageDriver` 
   
-`Google Cloud Storage`_   :doc:`Click `  
GOOGLE_STORAGEsingle region driver 
:mod:`libcloud.storage.drivers.google_storage` :class:`GoogleStorageDriver` 
   
-`KTUCloud Storage`_   
KTUCLOUD  dfw, hkg, iad, lon, ord, syd 
:mod:`libcloud.storage.drivers.ktucloud`   :class:`KTUCloudStorageDriver`   
   
-`Local Storage`_  
LOCAL single region driver 
:mod:`libcloud.storage.drivers.local`  :class:`LocalStorageDriver`  
   
-`Nimbus.io`_  
NIMBUSsingle region driver 
:mod:`libcloud.storage.drivers.nimbus` :class:`NimbusStorageDriver` 
   
-`Ninefold`_   
NINEFOLD  single region driver 
:mod:`libcloud.storage.drivers.ninefold`   :class:`NinefoldStorageDriver`   
   
-`OpenStack Swift`_:doc:`Click ` 
OPENSTACK_SWIFT   dfw, hkg, iad, lon, ord, syd 
:mod:`libcloud.storage.drivers.cloudfiles` 
:class:`OpenStackSwiftStorageDriver`
-`Amazon S3 (us-east-1)`_  :doc:`Click `  
S3single region driver 
:mod:`libcloud.storage.drivers.s3` :class:`S3StorageDriver` 
   
-`Amazon S3 (ap-northeast-1)`_ 
S3_AP_NORTHEAST   single region driver 
:mod:`libcloud.storage.drivers.s3` :class:`S3APNE1StorageDriver`
   
-`Amazon S3 (ap-northeast-1)`_ 
S3_AP_NORTHEAST1  single region driver 
:mod:`libcloud.storage.drivers.s3` :class:`S3APNE1StorageDriver`
   
-`Amazon S3 (ap-northeast-2)`_ 
S3_AP_NORTHEAST2  single region driver

[2/5] libcloud git commit: Add docs for DigitalOcean Spaces.

2017-10-16 Thread quentinp 
Add docs for DigitalOcean Spaces.

Signed-off-by: Quentin Pradet 


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/24584e6d
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/24584e6d
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/24584e6d

Branch: refs/heads/trunk
Commit: 24584e6d6aff0a8edbda39e47a75d038669c8dc1
Parents: 1bb5bed
Author: Andrew Starr-Bochicchio 
Authored: Mon Oct 16 10:21:04 2017 +0400
Committer: Quentin Pradet 
Committed: Mon Oct 16 10:24:58 2017 +0400

--
 .../storage/digitalocean_spaces/__init__.py |  0
 .../storage/digitalocean_spaces/instantiate.py  |  6 +++
 .../digitalocean_spaces/upload_object_acls.py   | 19 
 docs/storage/_supported_methods_main.rst|  2 +
 docs/storage/_supported_providers.rst   |  2 +
 docs/storage/drivers/digitalocean_spaces.rst| 49 
 6 files changed, 78 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/libcloud/blob/24584e6d/docs/examples/storage/digitalocean_spaces/__init__.py
--
diff --git a/docs/examples/storage/digitalocean_spaces/__init__.py 
b/docs/examples/storage/digitalocean_spaces/__init__.py
new file mode 100644
index 000..e69de29

http://git-wip-us.apache.org/repos/asf/libcloud/blob/24584e6d/docs/examples/storage/digitalocean_spaces/instantiate.py
--
diff --git a/docs/examples/storage/digitalocean_spaces/instantiate.py 
b/docs/examples/storage/digitalocean_spaces/instantiate.py
new file mode 100644
index 000..c882596
--- /dev/null
+++ b/docs/examples/storage/digitalocean_spaces/instantiate.py
@@ -0,0 +1,6 @@
+from libcloud.storage.types import Provider
+from libcloud.storage.providers import get_driver
+
+cls = get_driver(Provider.DIGITALOCEAN_SPACES)
+
+driver = cls(key='DO_ACCESS_KEY', secret='DO_SECRET_KEY')

http://git-wip-us.apache.org/repos/asf/libcloud/blob/24584e6d/docs/examples/storage/digitalocean_spaces/upload_object_acls.py
--
diff --git a/docs/examples/storage/digitalocean_spaces/upload_object_acls.py 
b/docs/examples/storage/digitalocean_spaces/upload_object_acls.py
new file mode 100644
index 000..e5fe0bd
--- /dev/null
+++ b/docs/examples/storage/digitalocean_spaces/upload_object_acls.py
@@ -0,0 +1,19 @@
+from libcloud.storage.types import Provider
+from libcloud.storage.providers import get_driver
+
+FILE_PATH = '/home/user/myfile.tar.gz'
+
+cls = get_driver(Provider.DIGITALOCEAN_SPACES)
+driver = cls('api key', 'api secret key')
+
+container = driver.get_container(container_name='my-backups-12345')
+
+# This method blocks until all the parts have been uploaded.
+extra = {'content_type': 'application/octet-stream',
+ 'acl': 'public-read'}
+
+with open(FILE_PATH, 'rb') as iterator:
+obj = driver.upload_object_via_stream(iterator=iterator,
+  container=container,
+  object_name='backup.tar.gz',
+  extra=extra)

http://git-wip-us.apache.org/repos/asf/libcloud/blob/24584e6d/docs/storage/_supported_methods_main.rst
--
diff --git a/docs/storage/_supported_methods_main.rst 
b/docs/storage/_supported_methods_main.rst
index 4f7eb96..0e377a2 100644
--- a/docs/storage/_supported_methods_main.rst
+++ b/docs/storage/_supported_methods_main.rst
@@ -8,6 +8,7 @@ Provider  list containers list objects 
create container dele
 `Microsoft Azure (blobs)`_yes yes  yes  
yes  yes   yes yes yes  
 yes  
 `Backblaze B2`_   yes yes  yes  
yes  yes   yes yes yes  
 yes  
 `CloudFiles`_ yes yes  yes  
yes  yes   yes yes yes  
 yes  
+`DigitalOcean Spaces`_yes yes  yes  
yes  yes   yes yes yes  
 yes  
 `Google Cloud Storage`_   yes yes  yes  
yes  yes   yes yes yes  
 yes  
 `KTUCloud Storage`_   yes yes  yes  
yes  yes   yes