[
https://issues.apache.org/jira/browse/LIBCLOUD-851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16440126#comment-16440126
]
ASF GitHub Bot commented on LIBCLOUD-851:
-----------------------------------------
GitHub user daviskirk opened a pull request:
https://github.com/apache/libcloud/pull/1202
Fix hash value in azure blob store
## Upgrade azure blob storage api
### Description
* Upgrade to minimum api version allowed by azure (see )
* Return actual file (md5) hash instead of etag ('"0xFFFFFFFFFFFFFFF"')
The current trunk version of libcloud throws a Libcloud error also
described in (https://issues.apache.org/jira/browse/LIBCLOUD-851)
```
Traceback (most recent call last):
File "...", line 12, in <module>
c = d.list_containers()[1]
File "libcloud/libcloud/storage/base.py", line 209, in list_containers
return list(self.iterate_containers())
File "libcloud/libcloud/storage/drivers/azure_blobs.py", line 371, in
iterate_containers
(response.status), driver=self)
libcloud.common.types.LibcloudError: <LibcloudError in
<libcloud.storage.drivers.azure_blobs.AzureBlobsStorageDriver object at
0x7f7389bf1080> 'Unexpected status code: 400'>
```
and the corresponing response body of the failing request is:
```
response.body
'<?xml version="1.0"
encoding="utf-8"?><Error><Code>InvalidHeaderValue</Code><Message>The value for
one of the HTTP headers is not in the correct
format.\nRequestId:dd262993-501e-0030-6cd1-d58682000000\nTime:2018-04-16T22:24:39.6566784Z</Message><HeaderName>x-ms-version</HeaderName><HeaderValue>2012-02-12</HeaderValue></Error>'
```
From looking at:
https://docs.microsoft.com/en-us/rest/api/storageservices/versioning-for-the-azure-storage-services#for-blob-storage-accounts
the easiest fix is to upgrade the api version to the minumum supported
version (2014-02-12).
In addition, the other cloud drivers all provide the actual md5 hash when
uploading or getting an object. This allows the user to check if a file has
changed online compared to locally. The current master version provides the
etag value as the "hash" attribute (interestingly with literal quotes around
the tag: `'"0xFFFFFFFFFFFFFFF"'`) which is not compatible with the other
drivers.
Replace this with the PR description (mention the changes you have made, why
you have made them, provide some background and any references to the
provider
documentation if needed, etc.).
I am assuming that these are both bugs and not design decisions (if so feel
free to close this request).
### Status
- work in progress
### Checklist (tick everything that applies)
- [ ] [Code
linting](http://libcloud.readthedocs.org/en/latest/development.html#code-style-guide)
(required, can be done after the PR checks)
- [ ] Documentation
- [ ] [Tests](http://libcloud.readthedocs.org/en/latest/testing.html)
- [ ]
[ICLA](http://libcloud.readthedocs.org/en/latest/development.html#contributing-bigger-changes)
(required for bigger changes)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/daviskirk/libcloud
851_fix-azure-blob-storage-api-version
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/libcloud/pull/1202.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1202
----
commit c73beb6879353ff9689f23580a5ee3499782de05
Author: Davis Kirkendall <davis.e.kirkendall@...>
Date: 2018-04-16T22:05:02Z
Fix hash value in azure blob store
- Upgrade to minimum api version allowed by azure (see
https://docs.microsoft.com/en-us/rest/api/storageservices/versioning-for-the-azure-storage-services#for-blob-storage-accounts)
- Return actual file (md5) hash instead of etag ('"0xFFFFFFFFFFFFFFF"')
----
> Azure blob storage - Unable to list blobs of a container
> --------------------------------------------------------
>
> Key: LIBCLOUD-851
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-851
> Project: Libcloud
> Issue Type: Bug
> Components: Storage
> Environment: Debian GNU/Linux
> Reporter: Antoine R. Dumont
> Priority: Major
> Labels: blobstore, storage_drivers
>
> Hello,
> Trying to list the blobs of a container in an azure blob storage, the
> instanciation step of a container systematically breaks with a 400 bad
> request.
> I have not much details in the error message:
> {code:title=output|borderStyle=solid}
> Traceback (most recent call last):
> File "./bin/basic-scenario", line 20, in <module>
> container = driver.get_container(container_name=container_name)
> File
> "/usr/lib/python3/dist-packages/libcloud/storage/drivers/azure_blobs.py",
> line 441, in get_container
> (response.status), driver=self)
> libcloud.common.types.LibcloudError: <LibcloudError in
> <libcloud.storage.drivers.azure_blobs.AzureBlobsStorageDriver object at
> 0x7f674c6b67f0> 'Unexpected status code: 400'
> {code}
> Here is the code of ./bin/basic-scenario:
> {code:title=basic-scenario|language=python|borderStyle=solid}
> #!/usr/bin/env python3
> from libcloud.storage.types import Provider
> from libcloud.storage.providers import get_driver
> if __name__ == '__main__':
> # setup
> account_name = 'account-name'
> key = 'secret-key'
> container_name = 'container-name'
> # libcloud setup
> cls = get_driver(Provider.AZURE_BLOBS)
> # this instanciates ok
> driver = cls(key=account_name, secret=key)
> # this explodes
> container = driver.get_container(container_name=container_name)
> for c in driver.iterate_container_objects(container):
> print(c)
> {code}
> Trying to dig in the error, using LIBCLOUD_DEBUG:
> {code}
> LIBCLOUD_DEBUG=error-output ./bin/basic-scenario
> {code}
> I have some more input, here is the content of error-output file:
> {code}
> # -------- begin 140359483662744 request ----------
> curl -i --head -H 'Authorization: SharedKey
> account-name:blabla/blabla/blabla=' -H 'x-ms-version: 2012-02-12' -H
> 'User-Agent: libcloud/0.20.0 (Microsoft Azure (blobs)) ' -H 'Accept-Encoding:
> gzip,deflate' -H 'x-ms-date: Fri, 09 Sep 2016 10:19:36 GMT' -H
> 'X-LC-Request-ID: 140340383662744' --compress
> 'https://account-name.blob.core.windows.net:443/container-name?restype=container'
> # -------- begin 140359483662744:140359483745056 response ----------
> HTTP/1.1 400 The value for one of the HTTP headers is not in the correct
> format.
> Transfer-Encoding: chunked
> Server: Microsoft-HTTPAPI/2.0
> X-Ms-Request-Id: ad3456f8-0002-0076-2f83-0ab6c2888888
> Date: Fri, 09 Sep 2016 10:19:36 GMT
> 0
> 0
> # -------- end 140359483662744:140359483745056 response ----------
> {code}
> As my package is a debian one with what seems an older version (0.20.0, seen
> in CHANGES.rst and not in the propose jira list), I tried with the current
> git repository (1.1.0).
> Procedure:
> 1. purged python3-libcloud from my machine
> 2. `python3 setup.py install` from git repository.
> I have the same error:
> {code}
> # -------- begin 139921458559016 request ----------
> curl -i --head -H 'x-ms-version: 2012-02-12' -H 'Authorization: SharedKey
> account-name:blabloblu=' -H 'X-LC-Request-ID: 139921458559432' -H
> 'User-Agent: libcloud/1.1.0 (Microsoft Azure (blobs)) ' -H 'Accept-Encoding:
> gzip,deflate' -H 'x-ms-date: Fri, 09 Sep 2016 10:51:53 GMT' --compress
> 'https://account-name.blob.core.windows.net:443/container-name?restype=container'
> # -------- begin 139921458559016:139921458672360 response ----------
> HTTP/1.1 400 The value for one of the HTTP headers is not in the correct
> format.
> Transfer-Encoding: chunked
> Server: Microsoft-HTTPAPI/2.0
> X-Ms-Request-Id: 10947125-0001-005e-7788-0ac27a888888
> Date: Fri, 09 Sep 2016 10:51:52 GMT
> 0
> 0
> # -------- end 139921458559016:139921458672360 response ----------
> {code}
> I'm stuck there since i don't see anything wrong with the headers.
> Do you please have some hint?
> Thanks for your time
> Note:
> - I'm confident in the azure blob storage setup and credentials since i got
> it working with another library (azure-storage via pip3)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
