Hello community,
here is the log from the commit of package python-glanceclient for
openSUSE:Factory checked in at 2019-02-11 21:24:53
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Comparing /work/SRC/openSUSE:Factory/python-glanceclient (Old)
and /work/SRC/openSUSE:Factory/.python-glanceclient.new.28833 (New)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "python-glanceclient"
Mon Feb 11 21:24:53 2019 rev:28 rq:672978 version:2.13.1
Changes:
--------
--- /work/SRC/openSUSE:Factory/python-glanceclient/python-glanceclient.changes
2018-09-07 15:37:15.326674606 +0200
+++
/work/SRC/openSUSE:Factory/.python-glanceclient.new.28833/python-glanceclient.changes
2019-02-11 21:24:56.427094944 +0100
@@ -1,0 +2,9 @@
+Wed Feb 6 15:00:28 UTC 2019 - [email protected]
+
+- update to version 2.13.1
+ - Refactor periodic "tips" jobs
+ - import zuul job settings from project-config
+ - Use "multihash" for data download validation
+ - Don't quote colon in HTTP headers
+
+-------------------------------------------------------------------
Old:
----
python-glanceclient-2.12.1.tar.gz
New:
----
python-glanceclient-2.13.1.tar.gz
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Other differences:
------------------
++++++ python-glanceclient.spec ++++++
--- /var/tmp/diff_new_pack.ziIYFL/_old 2019-02-11 21:24:56.899094690 +0100
+++ /var/tmp/diff_new_pack.ziIYFL/_new 2019-02-11 21:24:56.899094690 +0100
@@ -1,7 +1,7 @@
#
# spec file for package python-glanceclient
#
-# Copyright (c) 2018 SUSE LINUX GmbH, Nuernberg, Germany.
+# Copyright (c) 2019 SUSE LINUX GmbH, Nuernberg, Germany.
#
# All modifications and additions to the file contributed by third parties
# remain the property of their copyright owners, unless otherwise agreed
@@ -12,18 +12,18 @@
# license that conforms to the Open Source Definition (Version 1.9)
# published by the Open Source Initiative.
-# Please submit bugfixes or comments via http://bugs.opensuse.org/
+# Please submit bugfixes or comments via https://bugs.opensuse.org/
#
Name: python-glanceclient
-Version: 2.12.1
+Version: 2.13.1
Release: 0
Summary: Python API and CLI for OpenStack Glance
License: Apache-2.0
Group: Development/Languages/Python
URL: https://launchpad.net/python-glanceclient
-Source0:
https://files.pythonhosted.org/packages/source/p/python-glanceclient/python-glanceclient-2.12.1.tar.gz
+Source0:
https://files.pythonhosted.org/packages/source/p/python-glanceclient/python-glanceclient-2.13.1.tar.gz
BuildRequires: openstack-macros
BuildRequires: python-devel
BuildRequires: python2-PrettyTable >= 0.7.1
@@ -94,7 +94,7 @@
This package contains auto-generated documentation.
%prep
-%autosetup -p1 -n python-glanceclient-2.12.1
+%autosetup -p1 -n python-glanceclient-2.13.1
%py_req_cleanup
sed -i 's/^warning-is-error.*/warning-is-error = 0/g' setup.cfg
@@ -102,8 +102,8 @@
%python_build
# generate html docs
-PBR_VERSION=2.12.1 sphinx-build -b html doc/source doc/build/html
-PBR_VERSION=2.12.1 sphinx-build -b man doc/source doc/build/man
+PBR_VERSION=2.13.1 sphinx-build -b html doc/source doc/build/html
+PBR_VERSION=2.13.1 sphinx-build -b man doc/source doc/build/man
# remove the sphinx-build leftovers
rm -rf doc/build/html/.{doctrees,buildinfo}
++++++ python-glanceclient-2.12.1.tar.gz -> python-glanceclient-2.13.1.tar.gz
++++++
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/python-glanceclient-2.12.1/.zuul.yaml
new/python-glanceclient-2.13.1/.zuul.yaml
--- old/python-glanceclient-2.12.1/.zuul.yaml 2018-07-26 23:20:46.000000000
+0200
+++ new/python-glanceclient-2.13.1/.zuul.yaml 2018-12-12 04:54:05.000000000
+0100
@@ -70,6 +70,7 @@
- job:
name: glanceclient-tox-keystone-tips-base
parent: tox
+ abstract: true
description: Abstract job for glanceclient vs. keystone
required-projects:
- name: openstack/keystoneauth
@@ -93,6 +94,7 @@
- job:
name: glanceclient-tox-oslo-tips-base
parent: tox
+ abstract: true
description: Abstract job for glanceclient vs. oslo
required-projects:
- name: openstack/oslo.i18n
@@ -122,6 +124,13 @@
USE_PYTHON3: true
- project:
+ templates:
+ - openstack-python-jobs
+ - openstack-python35-jobs
+ - release-notes-jobs
+ - publish-openstack-sphinx-docs
+ - check-requirements
+ - lib-forward-testing
check:
jobs:
- glanceclient-dsvm-functional-v1
@@ -134,10 +143,29 @@
- openstack-tox-lower-constraints
periodic:
jobs:
- - glanceclient-tox-py27-keystone-tips
- - glanceclient-tox-py35-keystone-tips
- - glanceclient-tox-py27-oslo-tips
- - glanceclient-tox-py35-oslo-tips
+ # NOTE(rosmaita): we only want the "tips" jobs to be run against
+ # master, hence the 'branches' qualifiers below. Without them, when
+ # a stable branch is cut, the tests would be run against the stable
+ # branch as well, which is pointless because these libraries are
+ # frozen (more or less) in the stable branches.
+ #
+ # The "tips" jobs can be removed from the stable branch .zuul.yaml
+ # files if someone is so inclined, but that would require manual
+ # maintenance, so we do not do it by default. Another option is
+ # to define these jobs in the openstack-infra/project-config repo.
+ # That would make us less agile in adjusting these tests, so we
+ # aren't doing that either.
+ - glanceclient-tox-py27-keystone-tips:
+ branches: master
+ - glanceclient-tox-py35-keystone-tips:
+ branches: master
+ - glanceclient-tox-py27-oslo-tips:
+ branches: master
+ - glanceclient-tox-py35-oslo-tips:
+ branches: master
experimental:
jobs:
- glanceclient-dsvm-functional-py3
+ post:
+ jobs:
+ - openstack-tox-cover
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/python-glanceclient-2.12.1/AUTHORS
new/python-glanceclient-2.13.1/AUTHORS
--- old/python-glanceclient-2.12.1/AUTHORS 2018-07-26 23:24:34.000000000
+0200
+++ new/python-glanceclient-2.13.1/AUTHORS 2018-12-12 04:56:32.000000000
+0100
@@ -213,7 +213,6 @@
Zhi Yan Liu <[email protected]>
ZhiQiang Fan <[email protected]>
Zhiqiang Fan <[email protected]>
-Zuul <[email protected]>
amalaba <[email protected]>
bhagyashris <[email protected]>
caishan <[email protected]>
@@ -224,6 +223,7 @@
haobing1 <[email protected]>
iccha-sethi <[email protected]>
iccha.sethi <[email protected]>
+imacdonn <[email protected]>
isethi <[email protected]>
jaypipes <[email protected]>
ji-xuepeng <[email protected]>
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/python-glanceclient-2.12.1/ChangeLog
new/python-glanceclient-2.13.1/ChangeLog
--- old/python-glanceclient-2.12.1/ChangeLog 2018-07-26 23:24:34.000000000
+0200
+++ new/python-glanceclient-2.13.1/ChangeLog 2018-12-12 04:56:32.000000000
+0100
@@ -1,6 +1,18 @@
CHANGES
=======
+2.13.1
+------
+
+* Don't quote colon in HTTP headers
+
+2.13.0
+------
+
+* Use "multihash" for data download validation
+* Refactor periodic "tips" jobs
+* import zuul job settings from project-config
+
2.12.1
------
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/python-glanceclient-2.12.1/PKG-INFO
new/python-glanceclient-2.13.1/PKG-INFO
--- old/python-glanceclient-2.12.1/PKG-INFO 2018-07-26 23:24:35.000000000
+0200
+++ new/python-glanceclient-2.13.1/PKG-INFO 2018-12-12 04:56:32.000000000
+0100
@@ -1,6 +1,6 @@
Metadata-Version: 1.1
Name: python-glanceclient
-Version: 2.12.1
+Version: 2.13.1
Summary: OpenStack Image API Client Library
Home-page: https://docs.openstack.org/python-glanceclient/latest/
Author: OpenStack
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/python-glanceclient-2.12.1/glanceclient/common/http.py
new/python-glanceclient-2.13.1/glanceclient/common/http.py
--- old/python-glanceclient-2.12.1/glanceclient/common/http.py 2018-07-26
23:20:46.000000000 +0200
+++ new/python-glanceclient-2.13.1/glanceclient/common/http.py 2018-12-12
04:53:59.000000000 +0100
@@ -66,7 +66,11 @@
for h, v in headers.items():
if v is not None:
# if the item is token, do not quote '+' as well.
- safe = '=+/' if h in TOKEN_HEADERS else '/'
+ # NOTE(imacdonn): urlparse.quote() is intended for quoting the
+ # path part of a URL, but headers like x-image-meta-location
+ # include an entire URL. We should avoid encoding the colon in
+ # this case (bug #1788942)
+ safe = '=+/' if h in TOKEN_HEADERS else '/:'
if six.PY2:
# incoming items may be unicode, so get them into something
# the py2 version of urllib can handle before percent encoding
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/python-glanceclient-2.12.1/glanceclient/common/utils.py
new/python-glanceclient-2.13.1/glanceclient/common/utils.py
--- old/python-glanceclient-2.12.1/glanceclient/common/utils.py 2018-07-26
23:20:46.000000000 +0200
+++ new/python-glanceclient-2.13.1/glanceclient/common/utils.py 2018-12-12
04:53:59.000000000 +0100
@@ -449,6 +449,26 @@
(md5sum, checksum))
+def serious_integrity_iter(iter, hasher, hash_value):
+ """Check image data integrity using the Glance "multihash".
+
+ :param iter: iterable containing the image data
+ :param hasher: a hashlib object
+ :param hash_value: hexdigest of the image data
+ :raises: IOError if the hashdigest of the data is not hash_value
+ """
+ for chunk in iter:
+ yield chunk
+ if isinstance(chunk, six.string_types):
+ chunk = six.b(chunk)
+ hasher.update(chunk)
+ computed = hasher.hexdigest()
+ if computed != hash_value:
+ raise IOError(errno.EPIPE,
+ 'Corrupt image download. Hash was %s expected %s' %
+ (computed, hash_value))
+
+
def memoized_property(fn):
attr_name = '_lazy_once_' + fn.__name__
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/python-glanceclient-2.12.1/glanceclient/tests/unit/test_http.py
new/python-glanceclient-2.13.1/glanceclient/tests/unit/test_http.py
--- old/python-glanceclient-2.12.1/glanceclient/tests/unit/test_http.py
2018-07-26 23:20:46.000000000 +0200
+++ new/python-glanceclient-2.13.1/glanceclient/tests/unit/test_http.py
2018-12-12 04:53:59.000000000 +0100
@@ -216,7 +216,11 @@
def test_headers_encoding(self):
value = u'ni\xf1o'
- headers = {"test": value, "none-val": None, "Name": "value"}
+ fake_location = b'http://web_server:80/images/fake.img'
+ headers = {"test": value,
+ "none-val": None,
+ "Name": "value",
+ "x-image-meta-location": fake_location}
encoded = http.encode_headers(headers)
# Bug #1766235: According to RFC 8187, headers must be
# encoded as 7-bit ASCII, so expect to see only displayable
@@ -225,6 +229,8 @@
self.assertNotIn("none-val", encoded)
self.assertNotIn(b"none-val", encoded)
self.assertEqual(b"value", encoded[b"Name"])
+ # Bug #1788942: Colons in URL should not get percent-encoded
+ self.assertEqual(fake_location, encoded[b"x-image-meta-location"])
@mock.patch('keystoneauth1.adapter.Adapter.request')
def test_http_duplicate_content_type_headers(self, mock_ksarq):
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/python-glanceclient-2.12.1/glanceclient/tests/unit/test_shell.py
new/python-glanceclient-2.13.1/glanceclient/tests/unit/test_shell.py
--- old/python-glanceclient-2.12.1/glanceclient/tests/unit/test_shell.py
2018-07-26 23:20:46.000000000 +0200
+++ new/python-glanceclient-2.13.1/glanceclient/tests/unit/test_shell.py
2018-12-12 04:53:59.000000000 +0100
@@ -965,7 +965,9 @@
self.requests = self.useFixture(rm_fixture.Fixture())
self.requests.get('http://example.com/v2/images/%s/file' % id,
headers=headers, raw=fake)
-
+ self.requests.get('http://example.com/v2/images/%s' % id,
+ headers={'Content-type': 'application/json'},
+ json=image_show_fixture)
shell = openstack_shell.OpenStackImagesShell()
argstr = ('--os-image-api-version 2 --os-auth-token faketoken '
'--os-image-url http://example.com '
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/python-glanceclient-2.12.1/glanceclient/tests/unit/v2/fixtures.py
new/python-glanceclient-2.13.1/glanceclient/tests/unit/v2/fixtures.py
--- old/python-glanceclient-2.12.1/glanceclient/tests/unit/v2/fixtures.py
2018-07-26 23:20:46.000000000 +0200
+++ new/python-glanceclient-2.13.1/glanceclient/tests/unit/v2/fixtures.py
2018-12-12 04:53:59.000000000 +0100
@@ -14,6 +14,9 @@
# License for the specific language governing permissions and limitations
# under the License.
+import hashlib
+
+
UUID = "3fc2ba62-9a02-433e-b565-d493ffc69034"
image_list_fixture = {
@@ -65,7 +68,9 @@
"tags": [],
"updated_at": "2015-07-24T12:18:13Z",
"virtual_size": "null",
- "visibility": "shared"
+ "visibility": "shared",
+ "os_hash_algo": "sha384",
+ "os_hash_value": hashlib.sha384(b'DATA').hexdigest()
}
image_create_fixture = {
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/python-glanceclient-2.12.1/glanceclient/tests/unit/v2/test_images.py
new/python-glanceclient-2.13.1/glanceclient/tests/unit/v2/test_images.py
--- old/python-glanceclient-2.12.1/glanceclient/tests/unit/v2/test_images.py
2018-07-26 23:20:46.000000000 +0200
+++ new/python-glanceclient-2.13.1/glanceclient/tests/unit/v2/test_images.py
2018-12-12 04:53:59.000000000 +0100
@@ -14,6 +14,7 @@
# under the License.
import errno
+import hashlib
import mock
import testtools
@@ -193,7 +194,43 @@
'A',
),
},
- '/v2/images/66fb18d6-db27-11e1-a1eb-080027cbe205/file': {
+ '/v2/images/5cc4bebc-db27-11e1-a1eb-080027cbe205': {
+ 'GET': (
+ {},
+ {},
+ ),
+ },
+ '/v2/images/headeronly-db27-11e1-a1eb-080027cbe205/file': {
+ 'GET': (
+ {
+ 'content-md5': 'wrong'
+ },
+ 'BB',
+ ),
+ },
+ '/v2/images/headeronly-db27-11e1-a1eb-080027cbe205': {
+ 'GET': (
+ {},
+ {},
+ ),
+ },
+ '/v2/images/chkonly-db27-11e1-a1eb-080027cbe205/file': {
+ 'GET': (
+ {
+ 'content-md5': 'wrong'
+ },
+ 'BB',
+ ),
+ },
+ '/v2/images/chkonly-db27-11e1-a1eb-080027cbe205': {
+ 'GET': (
+ {},
+ {
+ 'checksum': 'wrong',
+ },
+ ),
+ },
+ '/v2/images/multihash-db27-11e1-a1eb-080027cbe205/file': {
'GET': (
{
'content-md5': 'wrong'
@@ -201,7 +238,67 @@
'BB',
),
},
- '/v2/images/1b1c6366-dd57-11e1-af0f-02163e68b1d8/file': {
+ '/v2/images/multihash-db27-11e1-a1eb-080027cbe205': {
+ 'GET': (
+ {},
+ {
+ 'checksum': 'wrong',
+ 'os_hash_algo': 'md5',
+ 'os_hash_value': 'junk'
+ },
+ ),
+ },
+ '/v2/images/badalgo-db27-11e1-a1eb-080027cbe205/file': {
+ 'GET': (
+ {
+ 'content-md5': hashlib.md5(b'BB').hexdigest()
+ },
+ 'BB',
+ ),
+ },
+ '/v2/images/badalgo-db27-11e1-a1eb-080027cbe205': {
+ 'GET': (
+ {},
+ {
+ 'checksum': hashlib.md5(b'BB').hexdigest(),
+ 'os_hash_algo': 'not_an_algo',
+ 'os_hash_value': 'whatever'
+ },
+ ),
+ },
+ '/v2/images/bad-multihash-value-good-checksum/file': {
+ 'GET': (
+ {
+ 'content-md5': hashlib.md5(b'GOODCHECKSUM').hexdigest()
+ },
+ 'GOODCHECKSUM',
+ ),
+ },
+ '/v2/images/bad-multihash-value-good-checksum': {
+ 'GET': (
+ {},
+ {
+ 'checksum': hashlib.md5(b'GOODCHECKSUM').hexdigest(),
+ 'os_hash_algo': 'sha512',
+ 'os_hash_value': 'badmultihashvalue'
+ },
+ ),
+ },
+ '/v2/images/headeronly-dd57-11e1-af0f-02163e68b1d8/file': {
+ 'GET': (
+ {
+ 'content-md5': 'defb99e69a9f1f6e06f15006b1f166ae'
+ },
+ 'CCC',
+ ),
+ },
+ '/v2/images/headeronly-dd57-11e1-af0f-02163e68b1d8': {
+ 'GET': (
+ {},
+ {},
+ ),
+ },
+ '/v2/images/chkonly-dd57-11e1-af0f-02163e68b1d8/file': {
'GET': (
{
'content-md5': 'defb99e69a9f1f6e06f15006b1f166ae'
@@ -209,6 +306,32 @@
'CCC',
),
},
+ '/v2/images/chkonly-dd57-11e1-af0f-02163e68b1d8': {
+ 'GET': (
+ {},
+ {
+ 'checksum': 'defb99e69a9f1f6e06f15006b1f166ae',
+ },
+ ),
+ },
+ '/v2/images/multihash-dd57-11e1-af0f-02163e68b1d8/file': {
+ 'GET': (
+ {
+ 'content-md5': 'defb99e69a9f1f6e06f15006b1f166ae'
+ },
+ 'CCC',
+ ),
+ },
+ '/v2/images/multihash-dd57-11e1-af0f-02163e68b1d8': {
+ 'GET': (
+ {},
+ {
+ 'checksum': 'defb99e69a9f1f6e06f15006b1f166ae',
+ 'os_hash_algo': 'sha384',
+ 'os_hash_value': hashlib.sha384(b'CCC').hexdigest()
+ },
+ ),
+ },
'/v2/images/87b634c1-f893-33c9-28a9-e5673c99239a/actions/reactivate': {
'POST': ({}, None)
},
@@ -846,12 +969,24 @@
self.assertEqual('A', body)
def test_data_with_wrong_checksum(self):
- body = self.controller.data('66fb18d6-db27-11e1-a1eb-080027cbe205',
+ body = self.controller.data('headeronly-db27-11e1-a1eb-080027cbe205',
do_checksum=False)
body = ''.join([b for b in body])
self.assertEqual('BB', body)
+ body = self.controller.data('headeronly-db27-11e1-a1eb-080027cbe205')
+ try:
+ body = ''.join([b for b in body])
+ self.fail('data did not raise an error.')
+ except IOError as e:
+ self.assertEqual(errno.EPIPE, e.errno)
+ msg = 'was 9d3d9048db16a7eee539e93e3618cbe7 expected wrong'
+ self.assertIn(msg, str(e))
- body = self.controller.data('66fb18d6-db27-11e1-a1eb-080027cbe205')
+ body = self.controller.data('chkonly-db27-11e1-a1eb-080027cbe205',
+ do_checksum=False)
+ body = ''.join([b for b in body])
+ self.assertEqual('BB', body)
+ body = self.controller.data('chkonly-db27-11e1-a1eb-080027cbe205')
try:
body = ''.join([b for b in body])
self.fail('data did not raise an error.')
@@ -860,15 +995,103 @@
msg = 'was 9d3d9048db16a7eee539e93e3618cbe7 expected wrong'
self.assertIn(msg, str(e))
- def test_data_with_checksum(self):
- body = self.controller.data('1b1c6366-dd57-11e1-af0f-02163e68b1d8',
+ body = self.controller.data('multihash-db27-11e1-a1eb-080027cbe205',
do_checksum=False)
body = ''.join([b for b in body])
- self.assertEqual('CCC', body)
+ self.assertEqual('BB', body)
+ body = self.controller.data('multihash-db27-11e1-a1eb-080027cbe205')
+ try:
+ body = ''.join([b for b in body])
+ self.fail('data did not raise an error.')
+ except IOError as e:
+ self.assertEqual(errno.EPIPE, e.errno)
+ msg = 'was 9d3d9048db16a7eee539e93e3618cbe7 expected junk'
+ self.assertIn(msg, str(e))
+
+ body = self.controller.data('badalgo-db27-11e1-a1eb-080027cbe205',
+ do_checksum=False)
+ body = ''.join([b for b in body])
+ self.assertEqual('BB', body)
+ try:
+ body = self.controller.data('badalgo-db27-11e1-a1eb-080027cbe205')
+ self.fail('bad os_hash_algo did not raise an error.')
+ except ValueError as e:
+ msg = 'unsupported hash type not_an_algo'
+ self.assertIn(msg, str(e))
+
+ def test_data_with_checksum(self):
+ for prefix in ['headeronly', 'chkonly', 'multihash']:
+ body = self.controller.data(prefix +
+ '-dd57-11e1-af0f-02163e68b1d8',
+ do_checksum=False)
+ body = ''.join([b for b in body])
+ self.assertEqual('CCC', body)
+
+ body = self.controller.data(prefix +
+ '-dd57-11e1-af0f-02163e68b1d8')
+ body = ''.join([b for b in body])
+ self.assertEqual('CCC', body)
+
+ def test_data_with_checksum_and_fallback(self):
+ # make sure the allow_md5_fallback option does not cause any
+ # incorrect behavior when fallback is not needed
+ for prefix in ['headeronly', 'chkonly', 'multihash']:
+ body = self.controller.data(prefix +
+ '-dd57-11e1-af0f-02163e68b1d8',
+ do_checksum=False,
+ allow_md5_fallback=True)
+ body = ''.join([b for b in body])
+ self.assertEqual('CCC', body)
- body = self.controller.data('1b1c6366-dd57-11e1-af0f-02163e68b1d8')
+ body = self.controller.data(prefix +
+ '-dd57-11e1-af0f-02163e68b1d8',
+ allow_md5_fallback=True)
+ body = ''.join([b for b in body])
+ self.assertEqual('CCC', body)
+
+ def test_data_with_bad_hash_algo_and_fallback(self):
+ # shouldn't matter when do_checksum is False
+ body = self.controller.data('badalgo-db27-11e1-a1eb-080027cbe205',
+ do_checksum=False,
+ allow_md5_fallback=True)
+ body = ''.join([b for b in body])
+ self.assertEqual('BB', body)
+
+ # default value for do_checksum is True
+ body = self.controller.data('badalgo-db27-11e1-a1eb-080027cbe205',
+ allow_md5_fallback=True)
+ body = ''.join([b for b in body])
+ self.assertEqual('BB', body)
+
+ def test_neg_data_with_bad_hash_value_and_fallback_enabled(self):
+ # make sure download fails when good hash_algo but bad hash_value
+ # even when correct checksum is present regardless of
+ # allow_md5_fallback setting
+ body = self.controller.data('bad-multihash-value-good-checksum',
+ allow_md5_fallback=False)
+ try:
+ body = ''.join([b for b in body])
+ self.fail('bad os_hash_value did not raise an error.')
+ except IOError as e:
+ self.assertEqual(errno.EPIPE, e.errno)
+ msg = 'expected badmultihashvalue'
+ self.assertIn(msg, str(e))
+
+ body = self.controller.data('bad-multihash-value-good-checksum',
+ allow_md5_fallback=True)
+ try:
+ body = ''.join([b for b in body])
+ self.fail('bad os_hash_value did not raise an error.')
+ except IOError as e:
+ self.assertEqual(errno.EPIPE, e.errno)
+ msg = 'expected badmultihashvalue'
+ self.assertIn(msg, str(e))
+
+ # download should succeed when do_checksum is off, though
+ body = self.controller.data('bad-multihash-value-good-checksum',
+ do_checksum=False)
body = ''.join([b for b in body])
- self.assertEqual('CCC', body)
+ self.assertEqual('GOODCHECKSUM', body)
def test_image_import(self):
uri = 'http://example.com/image.qcow'
@@ -883,7 +1106,7 @@
def test_download_no_data(self):
resp = utils.FakeResponse(headers={}, status_code=204)
self.controller.controller.http_client.get = mock.Mock(
- return_value=(resp, None))
+ return_value=(resp, {}))
self.controller.data('image_id')
def test_download_forbidden(self):
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/python-glanceclient-2.12.1/glanceclient/tests/unit/v2/test_shell_v2.py
new/python-glanceclient-2.13.1/glanceclient/tests/unit/v2/test_shell_v2.py
--- old/python-glanceclient-2.12.1/glanceclient/tests/unit/v2/test_shell_v2.py
2018-07-26 23:20:46.000000000 +0200
+++ new/python-glanceclient-2.13.1/glanceclient/tests/unit/v2/test_shell_v2.py
2018-12-12 04:54:05.000000000 +0100
@@ -1729,7 +1729,8 @@
def test_image_download(self):
args = self._make_args(
- {'id': 'IMG-01', 'file': 'test', 'progress': True})
+ {'id': 'IMG-01', 'file': 'test', 'progress': True,
+ 'allow_md5_fallback': False})
with mock.patch.object(self.gc.images, 'data') as mocked_data, \
mock.patch.object(utils, '_extract_request_id'):
@@ -1737,14 +1738,27 @@
[c for c in 'abcdef'])
test_shell.do_image_download(self.gc, args)
- mocked_data.assert_called_once_with('IMG-01')
+ mocked_data.assert_called_once_with('IMG-01',
+ allow_md5_fallback=False)
+
+ # check that non-default value is being passed correctly
+ args.allow_md5_fallback = True
+ with mock.patch.object(self.gc.images, 'data') as mocked_data, \
+ mock.patch.object(utils, '_extract_request_id'):
+ mocked_data.return_value = utils.RequestIdProxy(
+ [c for c in 'abcdef'])
+
+ test_shell.do_image_download(self.gc, args)
+ mocked_data.assert_called_once_with('IMG-01',
+ allow_md5_fallback=True)
@mock.patch.object(utils, 'exit')
@mock.patch('sys.stdout', autospec=True)
def test_image_download_no_file_arg(self, mocked_stdout,
mocked_utils_exit):
# Indicate that no file name was given as command line argument
- args = self._make_args({'id': '1234', 'file': None, 'progress': False})
+ args = self._make_args({'id': '1234', 'file': None, 'progress': False,
+ 'allow_md5_fallback': False})
# Indicate that no file is specified for output redirection
mocked_stdout.isatty = lambda: True
test_shell.do_image_download(self.gc, args)
@@ -1835,7 +1849,8 @@
def test_do_image_download_with_forbidden_id(self, mocked_print_err,
mocked_stdout):
args = self._make_args({'id': 'IMG-01', 'file': None,
- 'progress': False})
+ 'progress': False,
+ 'allow_md5_fallback': False})
mocked_stdout.isatty = lambda: False
with mock.patch.object(self.gc.images, 'data') as mocked_data:
mocked_data.side_effect = exc.HTTPForbidden
@@ -1852,7 +1867,8 @@
@mock.patch.object(utils, 'print_err')
def test_do_image_download_with_500(self, mocked_print_err, mocked_stdout):
args = self._make_args({'id': 'IMG-01', 'file': None,
- 'progress': False})
+ 'progress': False,
+ 'allow_md5_fallback': False})
mocked_stdout.isatty = lambda: False
with mock.patch.object(self.gc.images, 'data') as mocked_data:
mocked_data.side_effect = exc.HTTPInternalServerError
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/python-glanceclient-2.12.1/glanceclient/v2/images.py
new/python-glanceclient-2.13.1/glanceclient/v2/images.py
--- old/python-glanceclient-2.12.1/glanceclient/v2/images.py 2018-07-26
23:20:46.000000000 +0200
+++ new/python-glanceclient-2.13.1/glanceclient/v2/images.py 2018-12-12
04:53:59.000000000 +0100
@@ -13,6 +13,7 @@
# License for the specific language governing permissions and limitations
# under the License.
+import hashlib
import json
from oslo_utils import encodeutils
from requests import codes
@@ -197,13 +198,39 @@
return self._get(image_id)
@utils.add_req_id_to_object()
- def data(self, image_id, do_checksum=True):
+ def data(self, image_id, do_checksum=True, allow_md5_fallback=False):
"""Retrieve data of an image.
- :param image_id: ID of the image to download.
- :param do_checksum: Enable/disable checksum validation.
- :returns: An iterable body or None
+ When do_checksum is enabled, validation proceeds as follows:
+
+ 1. if the image has a 'os_hash_value' property, the algorithm
+ specified in the image's 'os_hash_algo' property will be used
+ to validate against the 'os_hash_value' value. If the
+ specified hash algorithm is not available AND allow_md5_fallback
+ is True, then continue to step #2
+ 2. else if the image has a checksum property, MD5 is used to
+ validate against the 'checksum' value
+ 3. else if the download response has a 'content-md5' header, MD5
+ is used to validate against the header value
+ 4. if none of 1-3 obtain, the data is **not validated** (this is
+ compatible with legacy behavior)
+
+ :param image_id: ID of the image to download
+ :param do_checksum: Enable/disable checksum validation
+ :param allow_md5_fallback:
+ Use the MD5 checksum for validation if the algorithm specified by
+ the image's 'os_hash_algo' property is not available
+ :returns: An iterable body or ``None``
"""
+ if do_checksum:
+ # doing this first to prevent race condition if image record
+ # is deleted during the image download
+ url = '/v2/images/%s' % image_id
+ resp, image_meta = self.http_client.get(url)
+ meta_checksum = image_meta.get('checksum', None)
+ meta_hash_value = image_meta.get('os_hash_value', None)
+ meta_hash_algo = image_meta.get('os_hash_algo', None)
+
url = '/v2/images/%s/file' % image_id
resp, body = self.http_client.get(url)
if resp.status_code == codes.no_content:
@@ -212,8 +239,32 @@
checksum = resp.headers.get('content-md5', None)
content_length = int(resp.headers.get('content-length', 0))
- if do_checksum and checksum is not None:
- body = utils.integrity_iter(body, checksum)
+ check_md5sum = do_checksum
+ if do_checksum and meta_hash_value is not None:
+ try:
+ hasher = hashlib.new(str(meta_hash_algo))
+ body = utils.serious_integrity_iter(body,
+ hasher,
+ meta_hash_value)
+ check_md5sum = False
+ except ValueError as ve:
+ if (str(ve).startswith('unsupported hash type') and
+ allow_md5_fallback):
+ check_md5sum = True
+ else:
+ raise
+
+ if do_checksum and check_md5sum:
+ if meta_checksum is not None:
+ body = utils.integrity_iter(body, meta_checksum)
+ elif checksum is not None:
+ body = utils.integrity_iter(body, checksum)
+ else:
+ # NOTE(rosmaita): this preserves legacy behavior to return the
+ # image data when checksumming is requested but there's no
+ # 'content-md5' header in the response. Just want to make it
+ # clear that we're doing this on purpose.
+ pass
return utils.IterableWithLength(body, content_length), resp
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/python-glanceclient-2.12.1/glanceclient/v2/shell.py
new/python-glanceclient-2.13.1/glanceclient/v2/shell.py
--- old/python-glanceclient-2.12.1/glanceclient/v2/shell.py 2018-07-26
23:20:46.000000000 +0200
+++ new/python-glanceclient-2.13.1/glanceclient/v2/shell.py 2018-12-12
04:54:05.000000000 +0100
@@ -490,6 +490,17 @@
utils.print_dict(stores_info)
[email protected]('--allow-md5-fallback', action='store_true',
+ default=utils.env('OS_IMAGE_ALLOW_MD5_FALLBACK', default=False),
+ help=_('If os_hash_algo and os_hash_value properties are available '
+ 'on the image, they will be used to validate the downloaded '
+ 'image data. If the indicated secure hash algorithm is not '
+ 'available on the client, the download will fail. Use this '
+ 'flag to indicate that in such a case the legacy MD5 image '
+ 'checksum should be used to validate the downloaded data. '
+ 'You can also set the enviroment variable '
+ 'OS_IMAGE_ALLOW_MD5_FALLBACK to any value to activate this '
+ 'option.'))
@utils.arg('--file', metavar='<FILE>',
help=_('Local file to save downloaded image data to. '
'If this is not specified and there is no redirection '
@@ -506,7 +517,8 @@
utils.exit(msg)
try:
- body = gc.images.data(args.id)
+ body = gc.images.data(args.id,
+ allow_md5_fallback=args.allow_md5_fallback)
except (exc.HTTPForbidden, exc.HTTPException) as e:
msg = "Unable to download image '%s'. (%s)" % (args.id, e)
utils.exit(msg)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/python-glanceclient-2.12.1/python_glanceclient.egg-info/PKG-INFO
new/python-glanceclient-2.13.1/python_glanceclient.egg-info/PKG-INFO
--- old/python-glanceclient-2.12.1/python_glanceclient.egg-info/PKG-INFO
2018-07-26 23:24:34.000000000 +0200
+++ new/python-glanceclient-2.13.1/python_glanceclient.egg-info/PKG-INFO
2018-12-12 04:56:32.000000000 +0100
@@ -1,6 +1,6 @@
Metadata-Version: 1.1
Name: python-glanceclient
-Version: 2.12.1
+Version: 2.13.1
Summary: OpenStack Image API Client Library
Home-page: https://docs.openstack.org/python-glanceclient/latest/
Author: OpenStack
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/python-glanceclient-2.12.1/python_glanceclient.egg-info/SOURCES.txt
new/python-glanceclient-2.13.1/python_glanceclient.egg-info/SOURCES.txt
--- old/python-glanceclient-2.12.1/python_glanceclient.egg-info/SOURCES.txt
2018-07-26 23:24:35.000000000 +0200
+++ new/python-glanceclient-2.13.1/python_glanceclient.egg-info/SOURCES.txt
2018-12-12 04:56:32.000000000 +0100
@@ -121,6 +121,7 @@
releasenotes/notes/http-headers-per-rfc-8187-aafa3199f863be81.yaml
releasenotes/notes/log-request-id-e7f67a23a0ed5c7b.yaml
releasenotes/notes/multi-store-support-acc7ad0e7e8b6f99.yaml
+releasenotes/notes/multihash-download-verification-596e91bf7b68e7db.yaml
releasenotes/notes/multihash-support-f1474590cf3ef5cf.yaml
releasenotes/notes/pike-relnote-2c77b01aa8799f35.yaml
releasenotes/notes/return-request-id-to-caller-47f4c0a684b1d88e.yaml
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/python-glanceclient-2.12.1/python_glanceclient.egg-info/pbr.json
new/python-glanceclient-2.13.1/python_glanceclient.egg-info/pbr.json
--- old/python-glanceclient-2.12.1/python_glanceclient.egg-info/pbr.json
2018-07-26 23:24:34.000000000 +0200
+++ new/python-glanceclient-2.13.1/python_glanceclient.egg-info/pbr.json
2018-12-12 04:56:32.000000000 +0100
@@ -1 +1 @@
-{"git_version": "b79154c", "is_release": true}
\ No newline at end of file
+{"git_version": "e0673a1", "is_release": true}
\ No newline at end of file
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/python-glanceclient-2.12.1/releasenotes/notes/multihash-download-verification-596e91bf7b68e7db.yaml
new/python-glanceclient-2.13.1/releasenotes/notes/multihash-download-verification-596e91bf7b68e7db.yaml
---
old/python-glanceclient-2.12.1/releasenotes/notes/multihash-download-verification-596e91bf7b68e7db.yaml
1970-01-01 01:00:00.000000000 +0100
+++
new/python-glanceclient-2.13.1/releasenotes/notes/multihash-download-verification-596e91bf7b68e7db.yaml
2018-12-12 04:53:59.000000000 +0100
@@ -0,0 +1,41 @@
+---
+features:
+ - |
+ This release adds verification of image data downloads using the Glance
+ "multihash" feature introduced in the OpenStack Rocky release. When
+ the ``os_hash_value`` is populated on an image, the glanceclient will
+ verify this value by computing the hexdigest of the downloaded data
+ using the algorithm specified by the image's ``os_hash_algo`` property.
+
+ Because the secure hash algorithm specified is determined by the cloud
+ provider, it is possible that the ``os_hash_algo`` may identify an
+ algorithm not available in the version of the Python ``hashlib`` library
+ used by the client. In such a case the download will fail due to an
+ unsupported hash type. In the event this occurs, a new option,
+ ``--allow-md5-fallback``, is introduced to the ``image-download`` command.
+ When present, this option will allow the glanceclient to use the legacy
+ MD5 checksum to verify the downloaded data if the secure hash algorithm
+ specified by the ``os_hash_algo`` image property is not supported.
+
+ Note that the fallback is *not* used in the case where the algorithm is
+ supported but the hexdigest of the downloaded data does not match the
+ ``os_hash_value``. In that case the download fails regardless of whether
+ the option is present or not.
+
+ Whether using the ``--allow-md5-fallback`` option is a good idea depends
+ upon the user's expectations for the verification. MD5 is an insecure
+ hashing algorithm, so if you are interested in making sure that the
+ downloaded image data has not been replaced by a datastream carefully
+ crafted to have the same MD5 checksum, then you should not use the
+ fallback. If, however, you are using Glance in a trusted environment
+ and your interest is simply to verify that no bits have flipped during
+ the data transfer, the MD5 fallback is sufficient for that purpose. That
+ being said, it is our recommendation that the multihash should be used
+ whenever possible.
+security:
+ - |
+ This release of the glanceclient uses the Glance "multihash" feature,
+ introduced in Rocky, to use a secure hashing algorithm to verify the
+ integrity of downloaded data. Legacy images without the "multihash"
+ image properties (``os_hash_algo`` and ``os_hash_value``) are verified
+ using the MD5 ``checksum`` image property.
