Hello community, here is the log from the commit of package python-tooz for openSUSE:Factory checked in at 2017-11-29 10:52:48 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/python-tooz (Old) and /work/SRC/openSUSE:Factory/.python-tooz.new (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "python-tooz" Wed Nov 29 10:52:48 2017 rev:4 rq:545730 version:1.59.0 Changes: -------- --- /work/SRC/openSUSE:Factory/python-tooz/python-tooz.changes 2017-08-28 15:31:59.887637444 +0200 +++ /work/SRC/openSUSE:Factory/.python-tooz.new/python-tooz.changes 2017-11-29 10:53:02.227829171 +0100 @@ -1,0 +2,11 @@ +Sun Nov 26 16:07:34 UTC 2017 - [email protected] + +- update to version 1.59.0 + - redis: always remove lock from acquired lock when release()ing + - Update reno for stable/pike + - Add etcd3 group support + - Use the same default timeout for async result + - redis: log an error on release failure + - Acquire fails with "ToozError: Not found" + +------------------------------------------------------------------- Old: ---- tooz-1.58.0.tar.gz New: ---- tooz-1.59.0.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ python-tooz.spec ++++++ --- /var/tmp/diff_new_pack.31bLQ7/_old 2017-11-29 10:53:02.851806532 +0100 +++ /var/tmp/diff_new_pack.31bLQ7/_new 2017-11-29 10:53:02.855806388 +0100 @@ -18,23 +18,28 @@ %global pypi_name tooz Name: python-tooz -Version: 1.58.0 +Version: 1.59.0 Release: 0 Summary: Coordination library for distributed systems License: Apache-2.0 Group: Development/Languages/Python Url: http://docs.openstack.org/developer/tooz/ -Source0: https://pypi.io/packages/source/t/%{pypi_name}/%{pypi_name}-%{version}.tar.gz +Source0: https://files.pythonhosted.org/packages/source/t/%{pypi_name}/%{pypi_name}-%{version}.tar.gz +BuildRequires: memcached BuildRequires: openstack-macros BuildRequires: python-devel BuildRequires: python-enum34 BuildRequires: python-fasteners >= 0.7 +BuildRequires: python-fixtures >= 3.0.0 BuildRequires: python-futures >= 3.0 BuildRequires: python-futurist >= 0.11.0 BuildRequires: python-iso8601 >= 0.1.11 +BuildRequires: python-mock >= 2.0 BuildRequires: python-oslo.serialization >= 1.10.0 BuildRequires: python-oslo.utils >= 3.20.0 BuildRequires: python-pbr >= 2.0.0 +BuildRequires: python-pifpaf >= 0.10.0 +BuildRequires: python-pymemcache >= 1.2.9 BuildRequires: python-setuptools >= 16.0 BuildRequires: python-stevedore >= 1.20.0 BuildRequires: python-tenacity >= 3.2.1 @@ -66,6 +71,7 @@ Group: Documentation BuildRequires: python-Sphinx BuildRequires: python-openstackdocstheme >= 1.11.0 +BuildRequires: python-redis >= 2.10.0 %description doc The Tooz project aims at centralizing the most common distributed primitives @@ -88,7 +94,9 @@ rm -rf html/.{doctrees,buildinfo} %check -%{__python2} setup.py test +export TOOZ_TEST_DRIVERS="memcached" +export PATH=%{_prefix}/sbin:$PATH +bash run-tests.sh %install %py2_install ++++++ _service ++++++ --- /var/tmp/diff_new_pack.31bLQ7/_old 2017-11-29 10:53:02.891805082 +0100 +++ /var/tmp/diff_new_pack.31bLQ7/_new 2017-11-29 10:53:02.891805082 +0100 @@ -1,8 +1,8 @@ <services> <service mode="disabled" name="renderspec"> - <param name="input-template">https://raw.githubusercontent.com/openstack/rpm-packaging/stable/pike/openstack/tooz/tooz.spec.j2</param> + <param name="input-template">https://raw.githubusercontent.com/openstack/rpm-packaging/master/openstack/tooz/tooz.spec.j2</param> <param name="output-name">python-tooz.spec</param> - <param name="requirements">https://raw.githubusercontent.com/openstack/rpm-packaging/stable/pike/global-requirements.txt</param> + <param name="requirements">https://raw.githubusercontent.com/openstack/rpm-packaging/master/requirements.txt</param> <param name="changelog-email">[email protected]</param> <param name="changelog-provider">gh,openstack,tooz</param> </service> ++++++ tooz-1.58.0.tar.gz -> tooz-1.59.0.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/tooz-1.58.0/AUTHORS new/tooz-1.59.0/AUTHORS --- old/tooz-1.58.0/AUTHORS 2017-07-20 13:09:21.000000000 +0200 +++ new/tooz-1.59.0/AUTHORS 2017-09-06 10:31:50.000000000 +0200 @@ -11,6 +11,7 @@ Eoghan Glynn <[email protected]> Gary Kotton <[email protected]> Gorka Eguileor <[email protected]> +Imran Ansari <[email protected]> Jay Clark <[email protected]> Jay Pipes <[email protected]> Jeremy Stanley <[email protected]> diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/tooz-1.58.0/ChangeLog new/tooz-1.59.0/ChangeLog --- old/tooz-1.58.0/ChangeLog 2017-07-20 13:09:21.000000000 +0200 +++ new/tooz-1.59.0/ChangeLog 2017-09-06 10:31:50.000000000 +0200 @@ -1,6 +1,15 @@ CHANGES ======= +1.59.0 +------ + +* Acquire fails with "ToozError: Not found" +* redis: always remove lock from acquired lock when release()ing +* redis: log an error on release failure +* Use the same default timeout for async result +* Update reno for stable/pike + 1.58.0 ------ @@ -13,6 +22,7 @@ * Switch from oslosphinx to openstackdocstheme * Turn on warning-is-error in doc build +* Add etcd3 group support 1.57.3 ------ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/tooz-1.58.0/PKG-INFO new/tooz-1.59.0/PKG-INFO --- old/tooz-1.58.0/PKG-INFO 2017-07-20 13:09:21.000000000 +0200 +++ new/tooz-1.59.0/PKG-INFO 2017-09-06 10:31:51.000000000 +0200 @@ -1,11 +1,12 @@ Metadata-Version: 1.1 Name: tooz -Version: 1.58.0 +Version: 1.59.0 Summary: Coordination library for distributed systems. Home-page: https://docs.openstack.org/tooz/latest/ Author: OpenStack Author-email: [email protected] License: Apache-2 +Description-Content-Type: UNKNOWN Description: Tooz ==== diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/tooz-1.58.0/releasenotes/notes/etcd3-group-support-b039cf19f4a268a3.yaml new/tooz-1.59.0/releasenotes/notes/etcd3-group-support-b039cf19f4a268a3.yaml --- old/tooz-1.58.0/releasenotes/notes/etcd3-group-support-b039cf19f4a268a3.yaml 1970-01-01 01:00:00.000000000 +0100 +++ new/tooz-1.59.0/releasenotes/notes/etcd3-group-support-b039cf19f4a268a3.yaml 2017-09-06 10:30:35.000000000 +0200 @@ -0,0 +1,4 @@ +--- +features: + - | + The etcd3 driver now supports the group membership API. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/tooz-1.58.0/releasenotes/source/index.rst new/tooz-1.59.0/releasenotes/source/index.rst --- old/tooz-1.58.0/releasenotes/source/index.rst 2017-07-20 13:08:30.000000000 +0200 +++ new/tooz-1.59.0/releasenotes/source/index.rst 2017-09-06 10:30:35.000000000 +0200 @@ -6,4 +6,5 @@ :maxdepth: 1 unreleased + pike ocata diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/tooz-1.58.0/releasenotes/source/pike.rst new/tooz-1.59.0/releasenotes/source/pike.rst --- old/tooz-1.58.0/releasenotes/source/pike.rst 1970-01-01 01:00:00.000000000 +0100 +++ new/tooz-1.59.0/releasenotes/source/pike.rst 2017-09-06 10:30:35.000000000 +0200 @@ -0,0 +1,6 @@ +=================================== + Pike Series Release Notes +=================================== + +.. release-notes:: + :branch: stable/pike diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/tooz-1.58.0/tooz/coordination.py new/tooz-1.59.0/tooz/coordination.py --- old/tooz-1.58.0/tooz/coordination.py 2017-07-20 13:08:30.000000000 +0200 +++ new/tooz-1.59.0/tooz/coordination.py 2017-09-06 10:30:35.000000000 +0200 @@ -636,7 +636,7 @@ """ @abc.abstractmethod - def get(self, timeout=10): + def get(self, timeout=None): """Retrieve the result of the corresponding asynchronous call. :param timeout: block until the timeout expire. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/tooz-1.58.0/tooz/drivers/etcd3.py new/tooz-1.59.0/tooz/drivers/etcd3.py --- old/tooz-1.58.0/tooz/drivers/etcd3.py 2017-07-20 13:08:30.000000000 +0200 +++ new/tooz-1.59.0/tooz/drivers/etcd3.py 2017-09-06 10:30:35.000000000 +0200 @@ -13,6 +13,8 @@ # under the License. from __future__ import absolute_import +import contextlib +import functools import threading import etcd3 @@ -21,30 +23,37 @@ import six import tooz +from tooz import _retry from tooz import coordination from tooz import locking from tooz import utils -def _translate_failures(func): [email protected] +def _failure_translator(): """Translates common requests exceptions into tooz exceptions.""" + try: + yield + except etcd3_exc.ConnectionFailedError as e: + utils.raise_with_cause(coordination.ToozConnectionError, + encodeutils.exception_to_unicode(e), + cause=e) + except etcd3_exc.ConnectionTimeoutError as e: + utils.raise_with_cause(coordination.OperationTimedOut, + encodeutils.exception_to_unicode(e), + cause=e) + except etcd3_exc.Etcd3Exception as e: + utils.raise_with_cause(coordination.ToozError, + encodeutils.exception_to_unicode(e), + cause=e) + + +def _translate_failures(func): @six.wraps(func) def wrapper(*args, **kwargs): - try: + with _failure_translator(): return func(*args, **kwargs) - except etcd3_exc.ConnectionFailedError as e: - utils.raise_with_cause(coordination.ToozConnectionError, - encodeutils.exception_to_unicode(e), - cause=e) - except etcd3_exc.ConnectionTimeoutError as e: - utils.raise_with_cause(coordination.OperationTimedOut, - encodeutils.exception_to_unicode(e), - cause=e) - except etcd3_exc.Etcd3Exception as e: - utils.raise_with_cause(coordination.ToozError, - encodeutils.exception_to_unicode(e), - cause=e) return wrapper @@ -100,7 +109,8 @@ return False -class Etcd3Driver(coordination.CoordinationDriver): +class Etcd3Driver(coordination.CoordinationDriverCachedRunWatchers, + coordination.CoordinationDriverWithExecutor): """An etcd based driver. This driver uses etcd provide the coordination driver semantics and @@ -124,8 +134,18 @@ timeout = int(options.get('timeout', self.DEFAULT_TIMEOUT)) self.client = etcd3.client(host=host, port=port, timeout=timeout) self.lock_timeout = int(options.get('lock_timeout', timeout)) + self.membership_timeout = int(options.get( + 'membership_timeout', timeout)) self._acquired_locks = set() + def _start(self): + super(Etcd3Driver, self)._start() + self._membership_lease = self.client.lease(self.membership_timeout) + + def _stop(self): + super(Etcd3Driver, self)._stop() + self._membership_lease.revoke() + def get_lock(self, name): return Etcd3Lock(self, name, self.lock_timeout) @@ -133,16 +153,191 @@ # NOTE(jaypipes): Copying because set can mutate during iteration for lock in self._acquired_locks.copy(): lock.heartbeat() - return self.lock_timeout + # TODO(jd) use the same lease for locks? + self._membership_lease.refresh() + return min(self.lock_timeout, self.membership_timeout) + + GROUP_PREFIX = b"tooz/groups/" + + def _encode_group_id(self, group_id): + return self.GROUP_PREFIX + group_id + b"/" + + def _encode_group_member_id(self, group_id, member_id): + return self._encode_group_id(group_id) + member_id + + def create_group(self, group_id): + encoded_group = self._encode_group_id(group_id) + + @_translate_failures + def _create_group(): + status, results = self.client.transaction( + compare=[ + self.client.transactions.version( + encoded_group) == 0 + ], + success=[ + self.client.transactions.put(encoded_group, b"") + ], + failure=[], + ) + if not status: + raise coordination.GroupAlreadyExist(group_id) + + return EtcdFutureResult(self._executor.submit(_create_group)) + + def _destroy_group(self, group_id): + self.client.delete(self._encode_group_id(group_id)) + + def delete_group(self, group_id): + encoded_group = self._encode_group_id(group_id) + + @_translate_failures + def _delete_group(): + members = list(self.client.get_prefix(encoded_group)) + if len(members) > 1: + raise coordination.GroupNotEmpty(group_id) + + # Warning: as of this writing python-etcd3 does not support the + # NOT_EQUAL operator so we use the EQUAL operator and will retry on + # success, hihi + status, results = self.client.transaction( + compare=[ + self.client.transactions.version(encoded_group) == 0 + ], + success=[], + failure=[ + self.client.transactions.delete(encoded_group) + ], + ) + if status: + raise coordination.GroupNotCreated(group_id) + + return EtcdFutureResult(self._executor.submit(_delete_group)) + + def join_group(self, group_id, capabilities=b""): + encoded_group = self._encode_group_id(group_id) + + @_retry.retry() + @_translate_failures + def _join_group(): + members = list(self.client.get_prefix(encoded_group)) + + encoded_member = self._encode_group_member_id( + group_id, self._member_id) + + group_metadata = None + for cap, metadata in members: + if metadata.key == encoded_member: + raise coordination.MemberAlreadyExist(group_id, + self._member_id) + if metadata.key == encoded_group: + group_metadata = metadata + + if group_metadata is None: + raise coordination.GroupNotCreated(group_id) + + status, results = self.client.transaction( + # This comparison makes sure the group has not been deleted in + # the mean time + compare=[ + self.client.transactions.version(encoded_group) == + group_metadata.version + ], + success=[ + self.client.transactions.put(encoded_member, + utils.dumps(capabilities), + lease=self._membership_lease) + ], + failure=[], + ) + if not status: + # TODO(jd) There's a small optimization doable by getting the + # current range on failure and passing it to this function as + # the first arg when retrying to avoid redoing a get_prefix() + raise _retry.TryAgain + + return EtcdFutureResult(self._executor.submit(_join_group)) + + def leave_group(self, group_id): + encoded_group = self._encode_group_id(group_id) + + @_translate_failures + def _leave_group(): + members = list(self.client.get_prefix(encoded_group)) + + encoded_member = self._encode_group_member_id( + group_id, self._member_id) + + for capabilities, metadata in members: + if metadata.key == encoded_member: + break + else: + raise coordination.MemberNotJoined(group_id, + self._member_id) + + self.client.delete(encoded_member) + + return EtcdFutureResult(self._executor.submit(_leave_group)) + + def get_members(self, group_id): + encoded_group = self._encode_group_id(group_id) + + @_translate_failures + def _get_members(): + members = set() + group_found = False + + for cap, metadata in self.client.get_prefix(encoded_group): + if metadata.key == encoded_group: + group_found = True + else: + members.add(metadata.key[len(encoded_group):]) + + if not group_found: + raise coordination.GroupNotCreated(group_id) + + return members + + return EtcdFutureResult(self._executor.submit(_get_members)) + + def get_member_capabilities(self, group_id, member_id): + encoded_member = self._encode_group_member_id( + group_id, member_id) + + @_translate_failures + def _get_member_capabilities(): + capabilities, metadata = self.client.get(encoded_member) + if capabilities is None: + raise coordination.MemberNotJoined(group_id, member_id) + return utils.loads(capabilities) + + return EtcdFutureResult( + self._executor.submit(_get_member_capabilities)) + + def update_capabilities(self, group_id, capabilities): + encoded_member = self._encode_group_member_id( + group_id, self._member_id) + + @_translate_failures + def _update_capabilities(): + cap, metadata = self.client.get(encoded_member) + if cap is None: + raise coordination.MemberNotJoined(group_id, self._member_id) - def watch_join_group(self, group_id, callback): - raise tooz.NotImplemented + self.client.put(encoded_member, utils.dumps(capabilities), + lease=self._membership_lease) - def unwatch_join_group(self, group_id, callback): - raise tooz.NotImplemented + return EtcdFutureResult( + self._executor.submit(_update_capabilities)) - def watch_leave_group(self, group_id, callback): + @staticmethod + def watch_elected_as_leader(group_id, callback): raise tooz.NotImplemented - def unwatch_leave_group(self, group_id, callback): + @staticmethod + def unwatch_elected_as_leader(group_id, callback): raise tooz.NotImplemented + + +EtcdFutureResult = functools.partial(coordination.CoordinatorResult, + failure_translator=_failure_translator) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/tooz-1.58.0/tooz/drivers/etcd3gw.py new/tooz-1.59.0/tooz/drivers/etcd3gw.py --- old/tooz-1.58.0/tooz/drivers/etcd3gw.py 2017-07-20 13:08:30.000000000 +0200 +++ new/tooz-1.59.0/tooz/drivers/etcd3gw.py 2017-09-06 10:30:35.000000000 +0200 @@ -68,7 +68,6 @@ self._key = self.LOCK_PREFIX + name self._key_b64 = base64.b64encode(self._key).decode("ascii") self._uuid = base64.b64encode(uuid.uuid4().bytes).decode("ascii") - self._lease = self._coord.client.lease(self._timeout) self._exclusive_access = threading.Lock() @_translate_failures @@ -80,6 +79,7 @@ def _acquire(): # TODO(jd): save the created revision so we can check it later to # make sure we still have the lock + self._lease = self._coord.client.lease(self._timeout) txn = { 'compare': [{ 'key': self._key_b64, diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/tooz-1.58.0/tooz/drivers/redis.py new/tooz-1.59.0/tooz/drivers/redis.py --- old/tooz-1.58.0/tooz/drivers/redis.py 2017-07-20 13:08:30.000000000 +0200 +++ new/tooz-1.59.0/tooz/drivers/redis.py 2017-09-06 10:30:35.000000000 +0200 @@ -99,9 +99,11 @@ with _translate_failures(): try: self._lock.release() - except exceptions.LockError: + except exceptions.LockError as e: + LOG.error("Unable to release lock '%r': %s", self, e) return False - self._coord._acquired_locks.discard(self) + finally: + self._coord._acquired_locks.discard(self) return True def heartbeat(self): diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/tooz-1.58.0/tooz/drivers/zookeeper.py new/tooz-1.59.0/tooz/drivers/zookeeper.py --- old/tooz-1.58.0/tooz/drivers/zookeeper.py 2017-07-20 13:08:30.000000000 +0200 +++ new/tooz-1.59.0/tooz/drivers/zookeeper.py 2017-09-06 10:30:35.000000000 +0200 @@ -540,7 +540,7 @@ self._handler = handler self._kwargs = kwargs - def get(self, timeout=10): + def get(self, timeout=None): return self._handler(self._kazoo_async_result, timeout, **self._kwargs) def done(self): diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/tooz-1.58.0/tooz.egg-info/PKG-INFO new/tooz-1.59.0/tooz.egg-info/PKG-INFO --- old/tooz-1.58.0/tooz.egg-info/PKG-INFO 2017-07-20 13:09:21.000000000 +0200 +++ new/tooz-1.59.0/tooz.egg-info/PKG-INFO 2017-09-06 10:31:50.000000000 +0200 @@ -1,11 +1,12 @@ Metadata-Version: 1.1 Name: tooz -Version: 1.58.0 +Version: 1.59.0 Summary: Coordination library for distributed systems. Home-page: https://docs.openstack.org/tooz/latest/ Author: OpenStack Author-email: [email protected] License: Apache-2 +Description-Content-Type: UNKNOWN Description: Tooz ==== diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/tooz-1.58.0/tooz.egg-info/SOURCES.txt new/tooz-1.59.0/tooz.egg-info/SOURCES.txt --- old/tooz-1.58.0/tooz.egg-info/SOURCES.txt 2017-07-20 13:09:21.000000000 +0200 +++ new/tooz-1.59.0/tooz.egg-info/SOURCES.txt 2017-09-06 10:31:51.000000000 +0200 @@ -38,12 +38,14 @@ examples/lock.py examples/partitioner.py releasenotes/notes/add-reno-996dd44974d53238.yaml +releasenotes/notes/etcd3-group-support-b039cf19f4a268a3.yaml releasenotes/notes/hashring-0470f9119ef63d49.yaml releasenotes/notes/join_group_create-5095ec02e20c7242.yaml releasenotes/notes/partitioner-4005767d287dc7c9.yaml releasenotes/source/conf.py releasenotes/source/index.rst releasenotes/source/ocata.rst +releasenotes/source/pike.rst releasenotes/source/unreleased.rst releasenotes/source/_static/.placeholder releasenotes/source/_templates/.placeholder diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/tooz-1.58.0/tooz.egg-info/pbr.json new/tooz-1.59.0/tooz.egg-info/pbr.json --- old/tooz-1.58.0/tooz.egg-info/pbr.json 2017-07-20 13:09:21.000000000 +0200 +++ new/tooz-1.59.0/tooz.egg-info/pbr.json 2017-09-06 10:31:50.000000000 +0200 @@ -1 +1 @@ -{"git_version": "c216782", "is_release": true} \ No newline at end of file +{"git_version": "29ec8eb", "is_release": true} \ No newline at end of file
