subject:"\[jira\] \[Commented\] \(LIBCLOUD\-986\) List \/ Iterate container with prefix for Azure Blob service \(This support exists in the other drivers already\)"

[jira] [Commented] (LIBCLOUD-986) List / Iterate container with prefix for Azure Blob service (This support exists in the other drivers already)

2018-03-29 Thread Quentin Pradet (JIRA)


[ 
https://issues.apache.org/jira/browse/LIBCLOUD-986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16420173#comment-16420173
 ] 

Quentin Pradet commented on LIBCLOUD-986:
-

Thanks for the fix! I can't close this issue myself, it would be nice if you 
could do it.

> List / Iterate container with prefix for Azure Blob service (This support 
> exists in the other drivers already)
> --
>
> Key: LIBCLOUD-986
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-986
> Project: Libcloud
>  Issue Type: Improvement
>  Components: Storage
>Reporter: Joshua Hawkinson
>Priority: Major
>
> While testing out the libcloud storage abstraction I've come across a curious 
> omission in driver support. It seems that for many drivers there is 
> additional support for the {{list_container_objects()}} and 
> {{iterate_container_objects()}} to support a prefix (or {{ex_prefix)}}. 
> This functionality is not present in the Azure driver for some reason.   I've 
> gone and checked the list_blobs API and it also supports this functionality 
> as seen here:
> [https://docs.microsoft.com/en-us/rest/api/storageservices/list-blobs]
>  
> I've attempted a quick and dirty patch that seems to work as well.
> {code}
> azure_blobs.py Line 384
> -def iterate_container_objects(self, container):
> +def iterate_container_objects(self, container, ex_prefix=None):
> """
> @inherits: :class:`StorageDriver.iterate_container_objects`
> """
> params = {'restype': 'container',
>   'comp': 'list',
> +  'prefix': ex_prefix,
>   'maxresults': RESPONSES_PER_REQUEST,
>   'include': 'metadata'}
> Line 419 (Add new function)
> def list_container_objects(self, container, ex_prefix=None):
> """
> Return a list of objects for the given container.
> 
> :param container: Container instance.
> :type container: :class:`Container`
> 
> :param ex_prefix: Only return objects starting with ex_prefix
> :type ex_prefix: ``str``
> 
> :return: A list of Object instances.
> :rtype: ``list`` of :class:`Object`
> """
> return list(self.iterate_container_objects(container,
>ex_prefix=ex_prefix))
> 
>  {code}
>  
> I've just worked around this at a higher level for now. There isn't any 
> urgency on my end,  but it would be nice if the abstraction was more 
> consistent so I could remove the workaround code.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (LIBCLOUD-986) List / Iterate container with prefix for Azure Blob service (This support exists in the other drivers already)

2018-03-29 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/LIBCLOUD-986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16420172#comment-16420172
 ] 

ASF subversion and git services commented on LIBCLOUD-986:
--

Commit ed0406043e438e0eeeb7e94dbc7aa0d65c0b8720 in libcloud's branch 
refs/heads/trunk from jhawk
[ https://git-wip-us.apache.org/repos/asf?p=libcloud.git;h=ed04060 ]

LIBCLOUD-986: Fix S3 iteration without prefix

Closes #1194

Signed-off-by: Quentin Pradet 


> List / Iterate container with prefix for Azure Blob service (This support 
> exists in the other drivers already)
> --
>
> Key: LIBCLOUD-986
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-986
> Project: Libcloud
>  Issue Type: Improvement
>  Components: Storage
>Reporter: Joshua Hawkinson
>Priority: Major
>
> While testing out the libcloud storage abstraction I've come across a curious 
> omission in driver support. It seems that for many drivers there is 
> additional support for the {{list_container_objects()}} and 
> {{iterate_container_objects()}} to support a prefix (or {{ex_prefix)}}. 
> This functionality is not present in the Azure driver for some reason.   I've 
> gone and checked the list_blobs API and it also supports this functionality 
> as seen here:
> [https://docs.microsoft.com/en-us/rest/api/storageservices/list-blobs]
>  
> I've attempted a quick and dirty patch that seems to work as well.
> {code}
> azure_blobs.py Line 384
> -def iterate_container_objects(self, container):
> +def iterate_container_objects(self, container, ex_prefix=None):
> """
> @inherits: :class:`StorageDriver.iterate_container_objects`
> """
> params = {'restype': 'container',
>   'comp': 'list',
> +  'prefix': ex_prefix,
>   'maxresults': RESPONSES_PER_REQUEST,
>   'include': 'metadata'}
> Line 419 (Add new function)
> def list_container_objects(self, container, ex_prefix=None):
> """
> Return a list of objects for the given container.
> 
> :param container: Container instance.
> :type container: :class:`Container`
> 
> :param ex_prefix: Only return objects starting with ex_prefix
> :type ex_prefix: ``str``
> 
> :return: A list of Object instances.
> :rtype: ``list`` of :class:`Object`
> """
> return list(self.iterate_container_objects(container,
>ex_prefix=ex_prefix))
> 
>  {code}
>  
> I've just worked around this at a higher level for now. There isn't any 
> urgency on my end,  but it would be nice if the abstraction was more 
> consistent so I could remove the workaround code.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (LIBCLOUD-986) List / Iterate container with prefix for Azure Blob service (This support exists in the other drivers already)

2018-03-22 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/LIBCLOUD-986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410830#comment-16410830
 ] 

ASF GitHub Bot commented on LIBCLOUD-986:
-

Github user asfgit closed the pull request at:

https://github.com/apache/libcloud/pull/1193


> List / Iterate container with prefix for Azure Blob service (This support 
> exists in the other drivers already)
> --
>
> Key: LIBCLOUD-986
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-986
> Project: Libcloud
>  Issue Type: Improvement
>  Components: Storage
>Reporter: Joshua Hawkinson
>Priority: Major
>
> While testing out the libcloud storage abstraction I've come across a curious 
> omission in driver support. It seems that for many drivers there is 
> additional support for the {{list_container_objects()}} and 
> {{iterate_container_objects()}} to support a prefix (or {{ex_prefix)}}. 
> This functionality is not present in the Azure driver for some reason.   I've 
> gone and checked the list_blobs API and it also supports this functionality 
> as seen here:
> [https://docs.microsoft.com/en-us/rest/api/storageservices/list-blobs]
>  
> I've attempted a quick and dirty patch that seems to work as well.
> {code}
> azure_blobs.py Line 384
> -def iterate_container_objects(self, container):
> +def iterate_container_objects(self, container, ex_prefix=None):
> """
> @inherits: :class:`StorageDriver.iterate_container_objects`
> """
> params = {'restype': 'container',
>   'comp': 'list',
> +  'prefix': ex_prefix,
>   'maxresults': RESPONSES_PER_REQUEST,
>   'include': 'metadata'}
> Line 419 (Add new function)
> def list_container_objects(self, container, ex_prefix=None):
> """
> Return a list of objects for the given container.
> 
> :param container: Container instance.
> :type container: :class:`Container`
> 
> :param ex_prefix: Only return objects starting with ex_prefix
> :type ex_prefix: ``str``
> 
> :return: A list of Object instances.
> :rtype: ``list`` of :class:`Object`
> """
> return list(self.iterate_container_objects(container,
>ex_prefix=ex_prefix))
> 
>  {code}
>  
> I've just worked around this at a higher level for now. There isn't any 
> urgency on my end,  but it would be nice if the abstraction was more 
> consistent so I could remove the workaround code.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (LIBCLOUD-986) List / Iterate container with prefix for Azure Blob service (This support exists in the other drivers already)

2018-03-22 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/LIBCLOUD-986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410829#comment-16410829
 ] 

ASF subversion and git services commented on LIBCLOUD-986:
--

Commit 1af5caf92a97092f7f11221ca881127e90807cb5 in libcloud's branch 
refs/heads/trunk from jhawk
[ https://git-wip-us.apache.org/repos/asf?p=libcloud.git;h=1af5caf ]

LIBCLOUD-986: Add prefix option to azure blob list and iterate container methods

Signed-off-by: Quentin Pradet 


> List / Iterate container with prefix for Azure Blob service (This support 
> exists in the other drivers already)
> --
>
> Key: LIBCLOUD-986
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-986
> Project: Libcloud
>  Issue Type: Improvement
>  Components: Storage
>Reporter: Joshua Hawkinson
>Priority: Major
>
> While testing out the libcloud storage abstraction I've come across a curious 
> omission in driver support. It seems that for many drivers there is 
> additional support for the {{list_container_objects()}} and 
> {{iterate_container_objects()}} to support a prefix (or {{ex_prefix)}}. 
> This functionality is not present in the Azure driver for some reason.   I've 
> gone and checked the list_blobs API and it also supports this functionality 
> as seen here:
> [https://docs.microsoft.com/en-us/rest/api/storageservices/list-blobs]
>  
> I've attempted a quick and dirty patch that seems to work as well.
> {code}
> azure_blobs.py Line 384
> -def iterate_container_objects(self, container):
> +def iterate_container_objects(self, container, ex_prefix=None):
> """
> @inherits: :class:`StorageDriver.iterate_container_objects`
> """
> params = {'restype': 'container',
>   'comp': 'list',
> +  'prefix': ex_prefix,
>   'maxresults': RESPONSES_PER_REQUEST,
>   'include': 'metadata'}
> Line 419 (Add new function)
> def list_container_objects(self, container, ex_prefix=None):
> """
> Return a list of objects for the given container.
> 
> :param container: Container instance.
> :type container: :class:`Container`
> 
> :param ex_prefix: Only return objects starting with ex_prefix
> :type ex_prefix: ``str``
> 
> :return: A list of Object instances.
> :rtype: ``list`` of :class:`Object`
> """
> return list(self.iterate_container_objects(container,
>ex_prefix=ex_prefix))
> 
>  {code}
>  
> I've just worked around this at a higher level for now. There isn't any 
> urgency on my end,  but it would be nice if the abstraction was more 
> consistent so I could remove the workaround code.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (LIBCLOUD-986) List / Iterate container with prefix for Azure Blob service (This support exists in the other drivers already)

2018-03-21 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/LIBCLOUD-986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408465#comment-16408465
 ] 

ASF GitHub Bot commented on LIBCLOUD-986:
-

GitHub user jhawkinson opened a pull request:

https://github.com/apache/libcloud/pull/1193

LIBCLOUD-986: Add prefix option to azure blob list and iterate containers

## LIBCLOUD-986: Add prefix filter for list and iterate containers to Azure 
Blob Storage Driver

### Description
While testing out the libcloud storage abstraction I've come across a 
curious omission in driver support. It seems that for many drivers there is 
additional support for the list_container_objects() and 
iterate_container_objects() to support a prefix (or ex_prefix). 

This functionality is not present in the Azure driver for some reason.   
I've gone and checked the list_blobs API and it also supports this 
functionality as seen here:

https://docs.microsoft.com/en-us/rest/api/storageservices/list-blobs

I've added the support to the AzureBlobsStorageDriver with these changes.

### Status
- done, ready for review

### Checklist
- [x] [Code 
linting](http://libcloud.readthedocs.org/en/latest/development.html#code-style-guide)
 (required, can be done after the PR checks)
- [ ] Documentation
- [x] [Tests](http://libcloud.readthedocs.org/en/latest/testing.html)
- [ ] 
[ICLA](http://libcloud.readthedocs.org/en/latest/development.html#contributing-bigger-changes)


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jhawkinson/libcloud LIBCLOUD-986

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/libcloud/pull/1193.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1193


commit 735c9e645a067da48192f4846502fd50e1feef0b
Author: jhawk 
Date:   2018-03-21T19:08:02Z

LIBCLOUD-986: Add prefix option to azure blob list and iterate container 
methods




> List / Iterate container with prefix for Azure Blob service (This support 
> exists in the other drivers already)
> --
>
> Key: LIBCLOUD-986
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-986
> Project: Libcloud
>  Issue Type: Improvement
>  Components: Storage
>Reporter: Joshua Hawkinson
>Priority: Major
>
> While testing out the libcloud storage abstraction I've come across a curious 
> omission in driver support. It seems that for many drivers there is 
> additional support for the {{list_container_objects()}} and 
> {{iterate_container_objects()}} to support a prefix (or {{ex_prefix)}}. 
> This functionality is not present in the Azure driver for some reason.   I've 
> gone and checked the list_blobs API and it also supports this functionality 
> as seen here:
> [https://docs.microsoft.com/en-us/rest/api/storageservices/list-blobs]
>  
> I've attempted a quick and dirty patch that seems to work as well.
> {code}
> azure_blobs.py Line 384
> -def iterate_container_objects(self, container):
> +def iterate_container_objects(self, container, ex_prefix=None):
> """
> @inherits: :class:`StorageDriver.iterate_container_objects`
> """
> params = {'restype': 'container',
>   'comp': 'list',
> +  'prefix': ex_prefix,
>   'maxresults': RESPONSES_PER_REQUEST,
>   'include': 'metadata'}
> Line 419 (Add new function)
> def list_container_objects(self, container, ex_prefix=None):
> """
> Return a list of objects for the given container.
> 
> :param container: Container instance.
> :type container: :class:`Container`
> 
> :param ex_prefix: Only return objects starting with ex_prefix
> :type ex_prefix: ``str``
> 
> :return: A list of Object instances.
> :rtype: ``list`` of :class:`Object`
> """
> return list(self.iterate_container_objects(container,
>ex_prefix=ex_prefix))
> 
>  {code}
>  
> I've just worked around this at a higher level for now. There isn't any 
> urgency on my end,  but it would be nice if the abstraction was more 
> consistent so I could remove the workaround code.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (LIBCLOUD-986) List / Iterate container with prefix for Azure Blob service (This support exists in the other drivers already)

2018-03-21 Thread Quentin Pradet (JIRA)


[ 
https://issues.apache.org/jira/browse/LIBCLOUD-986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408317#comment-16408317
 ] 

Quentin Pradet commented on LIBCLOUD-986:
-

Great! Can you please open a pull request on GitHub? 
https://github.com/apache/libcloud/

> List / Iterate container with prefix for Azure Blob service (This support 
> exists in the other drivers already)
> --
>
> Key: LIBCLOUD-986
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-986
> Project: Libcloud
>  Issue Type: Improvement
>  Components: Storage
>Reporter: Joshua Hawkinson
>Priority: Major
>
> While testing out the libcloud storage abstraction I've come across a curious 
> omission in driver support. It seems that for many drivers there is 
> additional support for the {{list_container_objects()}} and 
> {{iterate_container_objects()}} to support a prefix (or {{ex_prefix)}}. 
> This functionality is not present in the Azure driver for some reason.   I've 
> gone and checked the list_blobs API and it also supports this functionality 
> as seen here:
> [https://docs.microsoft.com/en-us/rest/api/storageservices/list-blobs]
>  
> I've attempted a quick and dirty patch that seems to work as well.
> {code}
> azure_blobs.py Line 384
> -def iterate_container_objects(self, container):
> +def iterate_container_objects(self, container, ex_prefix=None):
> """
> @inherits: :class:`StorageDriver.iterate_container_objects`
> """
> params = {'restype': 'container',
>   'comp': 'list',
> +  'prefix': ex_prefix,
>   'maxresults': RESPONSES_PER_REQUEST,
>   'include': 'metadata'}
> Line 419 (Add new function)
> def list_container_objects(self, container, ex_prefix=None):
> """
> Return a list of objects for the given container.
> 
> :param container: Container instance.
> :type container: :class:`Container`
> 
> :param ex_prefix: Only return objects starting with ex_prefix
> :type ex_prefix: ``str``
> 
> :return: A list of Object instances.
> :rtype: ``list`` of :class:`Object`
> """
> return list(self.iterate_container_objects(container,
>ex_prefix=ex_prefix))
> 
>  {code}
>  
> I've just worked around this at a higher level for now. There isn't any 
> urgency on my end,  but it would be nice if the abstraction was more 
> consistent so I could remove the workaround code.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (LIBCLOUD-986) List / Iterate container with prefix for Azure Blob service (This support exists in the other drivers already)

2018-03-21 Thread Joshua Hawkinson (JIRA)


[ 
https://issues.apache.org/jira/browse/LIBCLOUD-986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408315#comment-16408315
 ] 

Joshua Hawkinson commented on LIBCLOUD-986:
---

Okay, done... Here are the diffs for your review:
{code:java}
diff --git a/libcloud/storage/drivers/azure_blobs.py 
b/libcloud/storage/drivers/azure_blobs.py
index 82ee03a..2475793 100644
--- a/libcloud/storage/drivers/azure_blobs.py
+++ b/libcloud/storage/drivers/azure_blobs.py
@@ -381,11 +381,12 @@ class AzureBlobsStorageDriver(StorageDriver):
if not params['marker']:
break

- def iterate_container_objects(self, container):
+ def iterate_container_objects(self, container, ex_prefix=None):
"""
@inherits: :class:`StorageDriver.iterate_container_objects`
"""
-   params = {'restype': 'container',
+   params = {'prefix': ex_prefix,
+ 'restype': 'container',
  'comp': 'list',
  'maxresults': RESPONSES_PER_REQUEST,
  'include': 'metadata'}
@@ -416,6 +417,22 @@ class AzureBlobsStorageDriver(StorageDriver):
   if not params['marker']:
   break

+def list_container_objects(self, container, ex_prefix=None):
+  """
+  Return a list of objects for the given container.
+
+  :param container: Container instance.
+  :type container: :class:`Container`
+
+  :param ex_prefix: Only return objects starting with ex_prefix
+  :type ex_prefix: ``str``
+
+  :return: A list of Object instances.
+  :rtype: ``list`` of :class:`Object`
+  """
+  return list(self.iterate_container_objects(container,
+ ex_prefix=ex_prefix))
+
def get_container(self, container_name):
"""
@inherits: :class:`StorageDriver.get_container`
diff --git a/libcloud/test/storage/test_azure_blobs.py 
b/libcloud/test/storage/test_azure_blobs.py
index 95ab3f1..69bdebb 100644
--- a/libcloud/test/storage/test_azure_blobs.py
+++ b/libcloud/test/storage/test_azure_blobs.py
@@ -447,6 +447,28 @@ class AzureBlobsTests(unittest.TestCase):
 self.assertTrue('content_encoding' in obj.extra)
 self.assertTrue('content_language' in obj.extra)

+def test_list_container_objects_with_prefix(self):
+self.mock_response_klass.type = None
+AzureBlobsStorageDriver.RESPONSES_PER_REQUEST = 2
+
+container = Container(name='test_container', extra={},
+  driver=self.driver)
+objects = self.driver.list_container_objects(container=container,
+ex_prefix='test_prefix')
+self.assertEqual(len(objects), 4)
+
+obj = objects[1]
+self.assertEqual(obj.name, 'object2.txt')
+self.assertEqual(obj.hash, '0x8CFB90F1BA8CD8F')
+self.assertEqual(obj.size, 1048576)
+self.assertEqual(obj.container.name, 'test_container')
+self.assertTrue('meta1' in obj.meta_data)
+self.assertTrue('meta2' in obj.meta_data)
+self.assertTrue('last_modified' in obj.extra)
+self.assertTrue('content_type' in obj.extra)
+self.assertTrue('content_encoding' in obj.extra)
+self.assertTrue('content_language' in obj.extra)
+
{code}
 

And the before and after test results:
{code:java}
[root@localhost libcloud]# python3 -m pytest -k azure_blobs
=== test 
session starts 

platform linux -- Python 3.4.3, pytest-3.4.2, py-1.5.2, pluggy-0.6.0
rootdir: /usr/libcloud, inifile: pytest.ini
collected 9116 items 

libcloud/test/storage/test_azure_blobs.py F 
[100%]

= 
FAILURES 
=
_ 
AzureBlobsTests.test_list_container_objects_with_prefix 
__

self = 

def test_list_container_objects_with_prefix(self):
self.mock_response_klass.type = None
AzureBlobsStorageDriver.RESPONSES_PER_REQUEST = 2

container = Container(name='test_container', extra={},
driver=self.driver)
objects = self.driver.list_container_objects(container=container,
> ex_prefix='test_prefix')
E TypeError: list_container_objects() got an unexpected keyword argument 
'ex_prefix'

libcloud/test/storage/test_azure_blobs.py:457: TypeError
== 9079 tests 
deselected ===
=== 1 failed, 36 passed, 9079 
deselected in 3.39 seconds ===
[root@localhost libcloud]# python3 -m pytest -k azure_blobs

[jira] [Commented] (LIBCLOUD-986) List / Iterate container with prefix for Azure Blob service (This support exists in the other drivers already)

2018-03-21 Thread Quentin Pradet (JIRA)


[ 
https://issues.apache.org/jira/browse/LIBCLOUD-986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407536#comment-16407536
 ] 

Quentin Pradet commented on LIBCLOUD-986:
-

The first thing to do would be to run the tests locally:

 * git clone https://github.com/apache/libcloud.git
 * cd libcloud
 * pip install -r requirements-test.txt
 * cp libcloud/test/secrets.py-dist libcloud/test/secrets.py
 * pytest

When you'll be working on the Azure Blobs tests, running `pytest -k 
azure_blobs` will be faster.

The next step is to understand how the test fixtures work. Here's an example 
for another driver: 
https://github.com/apache/libcloud/pull/1157#issuecomment-355883772

Next we'll need to write a test that fails without your changes, and succeeds 
with your changes. Try to come up with a way to do this.

If you're stuck, make sure to ask questions!

> List / Iterate container with prefix for Azure Blob service (This support 
> exists in the other drivers already)
> --
>
> Key: LIBCLOUD-986
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-986
> Project: Libcloud
>  Issue Type: Improvement
>  Components: Storage
>Reporter: Joshua Hawkinson
>Priority: Major
>
> While testing out the libcloud storage abstraction I've come across a curious 
> omission in driver support. It seems that for many drivers there is 
> additional support for the {{list_container_objects()}} and 
> {{iterate_container_objects()}} to support a prefix (or {{ex_prefix)}}. 
> This functionality is not present in the Azure driver for some reason.   I've 
> gone and checked the list_blobs API and it also supports this functionality 
> as seen here:
> [https://docs.microsoft.com/en-us/rest/api/storageservices/list-blobs]
>  
> I've attempted a quick and dirty patch that seems to work as well.
> {code}
> azure_blobs.py Line 384
> -def iterate_container_objects(self, container):
> +def iterate_container_objects(self, container, ex_prefix=None):
> """
> @inherits: :class:`StorageDriver.iterate_container_objects`
> """
> params = {'restype': 'container',
>   'comp': 'list',
> +  'prefix': ex_prefix,
>   'maxresults': RESPONSES_PER_REQUEST,
>   'include': 'metadata'}
> Line 419 (Add new function)
> def list_container_objects(self, container, ex_prefix=None):
> """
> Return a list of objects for the given container.
> 
> :param container: Container instance.
> :type container: :class:`Container`
> 
> :param ex_prefix: Only return objects starting with ex_prefix
> :type ex_prefix: ``str``
> 
> :return: A list of Object instances.
> :rtype: ``list`` of :class:`Object`
> """
> return list(self.iterate_container_objects(container,
>ex_prefix=ex_prefix))
> 
>  {code}
>  
> I've just worked around this at a higher level for now. There isn't any 
> urgency on my end,  but it would be nice if the abstraction was more 
> consistent so I could remove the workaround code.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (LIBCLOUD-986) List / Iterate container with prefix for Azure Blob service (This support exists in the other drivers already)

2018-03-20 Thread Joshua Hawkinson (JIRA)


[ 
https://issues.apache.org/jira/browse/LIBCLOUD-986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16406931#comment-16406931
 ] 

Joshua Hawkinson commented on LIBCLOUD-986:
---

Sure,  I don't mind getting my hands dirty so to speak.  How would you like me 
to proceed?

> List / Iterate container with prefix for Azure Blob service (This support 
> exists in the other drivers already)
> --
>
> Key: LIBCLOUD-986
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-986
> Project: Libcloud
>  Issue Type: Improvement
>  Components: Storage
>Reporter: Joshua Hawkinson
>Priority: Major
>
> While testing out the libcloud storage abstraction I've come across a curious 
> omission in driver support. It seems that for many drivers there is 
> additional support for the {{list_container_objects()}} and 
> {{iterate_container_objects()}} to support a prefix (or {{ex_prefix)}}. 
> This functionality is not present in the Azure driver for some reason.   I've 
> gone and checked the list_blobs API and it also supports this functionality 
> as seen here:
> [https://docs.microsoft.com/en-us/rest/api/storageservices/list-blobs]
>  
> I've attempted a quick and dirty patch that seems to work as well.
> {code}
> azure_blobs.py Line 384
> -def iterate_container_objects(self, container):
> +def iterate_container_objects(self, container, ex_prefix=None):
> """
> @inherits: :class:`StorageDriver.iterate_container_objects`
> """
> params = {'restype': 'container',
>   'comp': 'list',
> +  'prefix': ex_prefix,
>   'maxresults': RESPONSES_PER_REQUEST,
>   'include': 'metadata'}
> Line 419 (Add new function)
> def list_container_objects(self, container, ex_prefix=None):
> """
> Return a list of objects for the given container.
> 
> :param container: Container instance.
> :type container: :class:`Container`
> 
> :param ex_prefix: Only return objects starting with ex_prefix
> :type ex_prefix: ``str``
> 
> :return: A list of Object instances.
> :rtype: ``list`` of :class:`Object`
> """
> return list(self.iterate_container_objects(container,
>ex_prefix=ex_prefix))
> 
>  {code}
>  
> I've just worked around this at a higher level for now. There isn't any 
> urgency on my end,  but it would be nice if the abstraction was more 
> consistent so I could remove the workaround code.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (LIBCLOUD-986) List / Iterate container with prefix for Azure Blob service (This support exists in the other drivers already)

2018-03-15 Thread Quentin Pradet (JIRA)


[ 
https://issues.apache.org/jira/browse/LIBCLOUD-986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16401454#comment-16401454
 ] 

Quentin Pradet commented on LIBCLOUD-986:
-

Hello! We unfortunately have a number of inconsistencies, and would be happy to 
fix this one. Would you like to work on it? I'd be happy to show you what needs 
to be done.

> List / Iterate container with prefix for Azure Blob service (This support 
> exists in the other drivers already)
> --
>
> Key: LIBCLOUD-986
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-986
> Project: Libcloud
>  Issue Type: Improvement
>  Components: Storage
>Reporter: Joshua Hawkinson
>Priority: Major
>
> While testing out the libcloud storage abstraction I've come across a curious 
> omission in driver support. It seems that for many drivers there is 
> additional support for the {{list_container_objects()}} and 
> {{iterate_container_objects()}} to support a prefix (or {{ex_prefix)}}. 
> This functionality is not present in the Azure driver for some reason.   I've 
> gone and checked the list_blobs API and it also supports this functionality 
> as seen here:
> [https://docs.microsoft.com/en-us/rest/api/storageservices/list-blobs]
>  
> I've attempted a quick and dirty patch that seems to work as well.
> {code}
> azure_blobs.py Line 384
> -def iterate_container_objects(self, container):
> +def iterate_container_objects(self, container, ex_prefix=None):
> """
> @inherits: :class:`StorageDriver.iterate_container_objects`
> """
> params = {'restype': 'container',
>   'comp': 'list',
> +  'prefix': ex_prefix,
>   'maxresults': RESPONSES_PER_REQUEST,
>   'include': 'metadata'}
> Line 419 (Add new function)
> def list_container_objects(self, container, ex_prefix=None):
> """
> Return a list of objects for the given container.
> 
> :param container: Container instance.
> :type container: :class:`Container`
> 
> :param ex_prefix: Only return objects starting with ex_prefix
> :type ex_prefix: ``str``
> 
> :return: A list of Object instances.
> :rtype: ``list`` of :class:`Object`
> """
> return list(self.iterate_container_objects(container,
>ex_prefix=ex_prefix))
> 
>  {code}
>  
> I've just worked around this at a higher level for now. There isn't any 
> urgency on my end,  but it would be nice if the abstraction was more 
> consistent so I could remove the workaround code.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (LIBCLOUD-986) List / Iterate container with prefix for Azure Blob service (This support exists in the other drivers already)

[jira] [Commented] (LIBCLOUD-986) List / Iterate container with prefix for Azure Blob service (This support exists in the other drivers already)

[jira] [Commented] (LIBCLOUD-986) List / Iterate container with prefix for Azure Blob service (This support exists in the other drivers already)

[jira] [Commented] (LIBCLOUD-986) List / Iterate container with prefix for Azure Blob service (This support exists in the other drivers already)

[jira] [Commented] (LIBCLOUD-986) List / Iterate container with prefix for Azure Blob service (This support exists in the other drivers already)

[jira] [Commented] (LIBCLOUD-986) List / Iterate container with prefix for Azure Blob service (This support exists in the other drivers already)

[jira] [Commented] (LIBCLOUD-986) List / Iterate container with prefix for Azure Blob service (This support exists in the other drivers already)

[jira] [Commented] (LIBCLOUD-986) List / Iterate container with prefix for Azure Blob service (This support exists in the other drivers already)

[jira] [Commented] (LIBCLOUD-986) List / Iterate container with prefix for Azure Blob service (This support exists in the other drivers already)

[jira] [Commented] (LIBCLOUD-986) List / Iterate container with prefix for Azure Blob service (This support exists in the other drivers already)

10 matches

Site Navigation

Mail list logo

Footer information