Hello community, here is the log from the commit of package python-queuelib for openSUSE:Factory checked in at 2018-11-18 23:32:37 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/python-queuelib (Old) and /work/SRC/openSUSE:Factory/.python-queuelib.new (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "python-queuelib" Sun Nov 18 23:32:37 2018 rev:2 rq:649921 version:1.5.0 Changes: -------- --- /work/SRC/openSUSE:Factory/python-queuelib/python-queuelib.changes 2018-01-10 23:32:34.335331708 +0100 +++ /work/SRC/openSUSE:Factory/.python-queuelib.new/python-queuelib.changes 2018-11-18 23:32:53.401430916 +0100 @@ -1,0 +2,8 @@ +Fri Nov 16 18:52:21 UTC 2018 - Todd R <[email protected]> + +- Update to version 1.5.0 + * rename Queue -> RRQueue + * start_keys -> start_domains + * Remove print + +------------------------------------------------------------------- Old: ---- queuelib-1.4.2.tar.gz New: ---- queuelib-1.5.0.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ python-queuelib.spec ++++++ --- /var/tmp/diff_new_pack.0pmNS1/_old 2018-11-18 23:32:55.225428725 +0100 +++ /var/tmp/diff_new_pack.0pmNS1/_new 2018-11-18 23:32:55.229428720 +0100 @@ -1,7 +1,7 @@ # # spec file for package python-queuelib # -# Copyright (c) 2017 SUSE LINUX GmbH, Nuernberg, Germany. +# Copyright (c) 2018 SUSE LINUX GmbH, Nuernberg, Germany. # # All modifications and additions to the file contributed by third parties # remain the property of their copyright owners, unless otherwise agreed @@ -18,19 +18,22 @@ %{?!python_module:%define python_module() python-%{**} python3-%{**}} Name: python-queuelib -Version: 1.4.2 +Version: 1.5.0 Release: 0 Summary: Collection of Persistent (Disk-Based) Queues License: BSD-2-Clause Group: Development/Languages/Python -Url: http://github.com/scrapy/queuelib +Url: https://github.com/scrapy/queuelib Source: https://files.pythonhosted.org/packages/source/q/queuelib/queuelib-%{version}.tar.gz BuildRequires: %{python_module devel} -BuildRequires: %{python_module pytest} BuildRequires: %{python_module setuptools} BuildRequires: fdupes BuildRequires: python-rpm-macros +# SECTION test requirements +BuildRequires: %{python_module pytest} +# /SECTION BuildArch: noarch + %python_subpackages %description @@ -53,7 +56,8 @@ %python_exec setup.py test %files %{python_files} -%doc LICENSE README.rst NEWS +%doc README.rst NEWS +%license LICENSE %{python_sitelib}/* %changelog ++++++ queuelib-1.4.2.tar.gz -> queuelib-1.5.0.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/queuelib-1.4.2/PKG-INFO new/queuelib-1.5.0/PKG-INFO --- old/queuelib-1.4.2/PKG-INFO 2015-09-09 20:58:56.000000000 +0200 +++ new/queuelib-1.5.0/PKG-INFO 2018-03-12 13:50:48.000000000 +0100 @@ -1,11 +1,12 @@ Metadata-Version: 1.1 Name: queuelib -Version: 1.4.2 +Version: 1.5.0 Summary: Collection of persistent (disk-based) queues Home-page: https://github.com/scrapy/queuelib Author: Scrapy project Author-email: [email protected] License: BSD +Description-Content-Type: UNKNOWN Description: ======== queuelib ======== @@ -23,6 +24,8 @@ Queuelib goals are speed and simplicity. It was originally part of the `Scrapy framework`_ and stripped out on its own library. + Note: Queuelib isn't thread-safe. + Requirements ============ @@ -105,6 +108,34 @@ >>> pq.pop() b'a' + RoundRobinQueue + =============== + + Has nearly the same interface and implementation as a Priority Queue except + that each element must be pushed with a (mandatory) key. Popping from the + queue cycles through the keys "round robin". + + Instantiate the Round Robin Queue similarly to the Priority Queue:: + + >>> from queuelib import RoundRobinQueue + >>> rr = RoundRobinQueue(qfactory) + + And use it:: + + >>> rr.push(b'a', '1') + >>> rr.push(b'b', '1') + >>> rr.push(b'c', '2') + >>> rr.push(b'd', '2') + >>> rr.pop() + b'a' + >>> rr.pop() + b'c' + >>> rr.pop() + b'b' + >>> rr.pop() + b'd' + + Mailing list ============ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/queuelib-1.4.2/README.rst new/queuelib-1.5.0/README.rst --- old/queuelib-1.4.2/README.rst 2015-09-09 20:58:27.000000000 +0200 +++ new/queuelib-1.5.0/README.rst 2018-03-12 13:49:52.000000000 +0100 @@ -15,6 +15,8 @@ Queuelib goals are speed and simplicity. It was originally part of the `Scrapy framework`_ and stripped out on its own library. +Note: Queuelib isn't thread-safe. + Requirements ============ @@ -97,6 +99,34 @@ >>> pq.pop() b'a' +RoundRobinQueue +=============== + +Has nearly the same interface and implementation as a Priority Queue except +that each element must be pushed with a (mandatory) key. Popping from the +queue cycles through the keys "round robin". + +Instantiate the Round Robin Queue similarly to the Priority Queue:: + + >>> from queuelib import RoundRobinQueue + >>> rr = RoundRobinQueue(qfactory) + +And use it:: + + >>> rr.push(b'a', '1') + >>> rr.push(b'b', '1') + >>> rr.push(b'c', '2') + >>> rr.push(b'd', '2') + >>> rr.pop() + b'a' + >>> rr.pop() + b'c' + >>> rr.pop() + b'b' + >>> rr.pop() + b'd' + + Mailing list ============ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/queuelib-1.4.2/queuelib/__init__.py new/queuelib-1.5.0/queuelib/__init__.py --- old/queuelib-1.4.2/queuelib/__init__.py 2015-09-09 20:58:27.000000000 +0200 +++ new/queuelib-1.5.0/queuelib/__init__.py 2018-03-12 13:49:52.000000000 +0100 @@ -1,2 +1,3 @@ from queuelib.queue import FifoDiskQueue, LifoDiskQueue from queuelib.pqueue import PriorityQueue +from queuelib.rrqueue import RoundRobinQueue \ No newline at end of file diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/queuelib-1.4.2/queuelib/rrqueue.py new/queuelib-1.5.0/queuelib/rrqueue.py --- old/queuelib-1.4.2/queuelib/rrqueue.py 1970-01-01 01:00:00.000000000 +0100 +++ new/queuelib-1.5.0/queuelib/rrqueue.py 2018-03-12 13:49:52.000000000 +0100 @@ -0,0 +1,69 @@ +from collections import deque + +class RoundRobinQueue(object): + """A round robin queue implemented using multiple internal queues (typically, + FIFO queues). The internal queue must implement the following methods: + * push(obj) + * pop() + * close() + * __len__() + The constructor receives a qfactory argument, which is a callable used to + instantiate a new (internal) queue when a new key is allocated. The + qfactory function is called with the key number as first and only + argument. + start_keys is a sequence of domains to start with. If the queue was + previously closed leaving some domain buckets non-empty, those domains + should be passed in start_keys. + + The queue maintains a fifo queue of keys. The key that went last is + poped first and the next queue for that key is then poped. This allows + for a round robin + """ + + def __init__(self, qfactory, start_domains=()): + self.queues = {} + self.qfactory = qfactory + for key in start_domains: + self.queues[key] = self.qfactory(key) + + self.key_queue = deque(start_domains) + + def push(self, obj, key): + if key not in self.key_queue: + self.queues[key] = self.qfactory(key) + self.key_queue.appendleft(key) # it's new, might as well pop first + + q = self.queues[key] + q.push(obj) # this may fail (eg. serialization error) + + def pop(self): + m = None + # pop until we find a valid object, closing necessary queues + while m is None: + try: + key = self.key_queue.pop() + except IndexError: + return + + q = self.queues[key] + m = q.pop() + + if len(q) == 0: + del self.queues[key] + q.close() + else: + self.key_queue.appendleft(key) + + if m: + return m + + def close(self): + active = [] + for k, q in self.queues.items(): + if len(q): + active.append(k) + q.close() + return active + + def __len__(self): + return sum(len(x) for x in self.queues.values()) if self.queues else 0 diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/queuelib-1.4.2/queuelib/tests/__init__.py new/queuelib-1.5.0/queuelib/tests/__init__.py --- old/queuelib-1.4.2/queuelib/tests/__init__.py 2015-09-09 20:58:27.000000000 +0200 +++ new/queuelib-1.5.0/queuelib/tests/__init__.py 2018-03-12 13:49:52.000000000 +0100 @@ -15,3 +15,19 @@ def mkdtemp(self): return tempfile.mkdtemp(dir=self.tmpdir) + + +def track_closed(cls): + """Wraps a queue class to track down if close() method was called""" + + class TrackingClosed(cls): + + def __init__(self, *a, **kw): + super(TrackingClosed, self).__init__(*a, **kw) + self.closed = False + + def close(self): + super(TrackingClosed, self).close() + self.closed = True + + return TrackingClosed diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/queuelib-1.4.2/queuelib/tests/test_pqueue.py new/queuelib-1.5.0/queuelib/tests/test_pqueue.py --- old/queuelib-1.4.2/queuelib/tests/test_pqueue.py 2015-09-09 20:58:27.000000000 +0200 +++ new/queuelib-1.5.0/queuelib/tests/test_pqueue.py 2018-03-12 13:49:52.000000000 +0100 @@ -4,23 +4,7 @@ FifoMemoryQueue, LifoMemoryQueue, FifoDiskQueue, LifoDiskQueue, FifoSQLiteQueue, LifoSQLiteQueue, ) -from queuelib.tests import QueuelibTestCase - - -def track_closed(cls): - """Wraps a queue class to track down if close() method was called""" - - class TrackingClosed(cls): - - def __init__(self, *a, **kw): - super(TrackingClosed, self).__init__(*a, **kw) - self.closed = False - - def close(self): - super(TrackingClosed, self).close() - self.closed = True - - return TrackingClosed +from queuelib.tests import (QueuelibTestCase, track_closed) # hack to prevent py.test from discovering base test class diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/queuelib-1.4.2/queuelib/tests/test_rrqueue.py new/queuelib-1.5.0/queuelib/tests/test_rrqueue.py --- old/queuelib-1.4.2/queuelib/tests/test_rrqueue.py 1970-01-01 01:00:00.000000000 +0100 +++ new/queuelib-1.5.0/queuelib/tests/test_rrqueue.py 2018-03-12 13:49:52.000000000 +0100 @@ -0,0 +1,144 @@ +import os +from queuelib.rrqueue import RoundRobinQueue +from queuelib.queue import ( + FifoMemoryQueue, LifoMemoryQueue, FifoDiskQueue, LifoDiskQueue, + FifoSQLiteQueue, LifoSQLiteQueue, +) +from queuelib.tests import (QueuelibTestCase, track_closed) + + +# hack to prevent py.test from discovering base test class +class base: + class RRQueueTestBase(QueuelibTestCase): + + def setUp(self): + QueuelibTestCase.setUp(self) + self.q = RoundRobinQueue(self.qfactory) + + def qfactory(self, key): + raise NotImplementedError + + def test_len_nonzero(self): + assert not self.q + self.assertEqual(len(self.q), 0) + self.q.push(b'a', '3') + assert self.q + self.q.push(b'b', '1') + self.q.push(b'c', '2') + self.q.push(b'd', '1') + self.assertEqual(len(self.q), 4) + self.q.pop() + self.q.pop() + self.q.pop() + self.q.pop() + assert not self.q + self.assertEqual(len(self.q), 0) + + def test_close(self): + self.q.push(b'a', '3') + self.q.push(b'b', '1') + self.q.push(b'c', '2') + self.q.push(b'd', '1') + iqueues = self.q.queues.values() + self.assertEqual(sorted(self.q.close()), ['1', '2', '3']) + assert all(q.closed for q in iqueues) + + def test_close_return_active(self): + self.q.push(b'b', '1') + self.q.push(b'c', '2') + self.q.push(b'a', '3') + self.q.pop() + self.assertEqual(sorted(self.q.close()), ['2', '3']) + + +class FifoTestMixin(object): + def test_push_pop_key(self): + self.q.push(b'a', '1') + self.q.push(b'b', '1') + self.q.push(b'c', '2') + self.q.push(b'd', '2') + self.assertEqual(self.q.pop(), b'a') + self.assertEqual(self.q.pop(), b'c') + self.assertEqual(self.q.pop(), b'b') + self.assertEqual(self.q.pop(), b'd') + self.assertEqual(self.q.pop(), None) + + +class LifoTestMixin(object): + + def test_push_pop_key(self): + self.q.push(b'a', '1') + self.q.push(b'b', '1') + self.q.push(b'c', '2') + self.q.push(b'd', '2') + self.assertEqual(self.q.pop(), b'b') + self.assertEqual(self.q.pop(), b'd') + self.assertEqual(self.q.pop(), b'a') + self.assertEqual(self.q.pop(), b'c') + self.assertEqual(self.q.pop(), None) + + +class FifoMemoryRRQueueTest(FifoTestMixin, base.RRQueueTestBase): + + def qfactory(self, key): + return track_closed(FifoMemoryQueue)() + + +class LifoMemoryRRQueueTest(LifoTestMixin, base.RRQueueTestBase): + + def qfactory(self, key): + return track_closed(LifoMemoryQueue)() + + +class DiskTestMixin(object): + + def test_nonserializable_object_one(self): + self.assertRaises(TypeError, self.q.push, lambda x: x, '0') + self.assertEqual(self.q.close(), []) + + def test_nonserializable_object_many_close(self): + self.q.push(b'a', '3') + self.q.push(b'b', '1') + self.assertRaises(TypeError, self.q.push, lambda x: x, '0') + self.q.push(b'c', '2') + self.assertEqual(self.q.pop(), b'a') + self.assertEqual(sorted(self.q.close()), ['1', '2']) + + def test_nonserializable_object_many_pop(self): + self.q.push(b'a', '3') + self.q.push(b'b', '1') + self.assertRaises(TypeError, self.q.push, lambda x: x, '0') + self.q.push(b'c', '2') + self.assertEqual(self.q.pop(), b'a') + self.assertEqual(self.q.pop(), b'b') + self.assertEqual(self.q.pop(), b'c') + self.assertEqual(self.q.pop(), None) + self.assertEqual(self.q.close(), []) + + +class FifoDiskRRQueueTest(FifoTestMixin, DiskTestMixin, base.RRQueueTestBase): + + def qfactory(self, key): + path = os.path.join(self.qdir, str(key)) + return track_closed(FifoDiskQueue)(path) + + +class LifoDiskRRQueueTest(LifoTestMixin, DiskTestMixin, base.RRQueueTestBase): + + def qfactory(self, key): + path = os.path.join(self.qdir, str(key)) + return track_closed(LifoDiskQueue)(path) + + +class FifoSQLiteRRQueueTest(FifoTestMixin, DiskTestMixin, base.RRQueueTestBase): + + def qfactory(self, key): + path = os.path.join(self.qdir, str(key)) + return track_closed(FifoSQLiteQueue)(path) + + +class LifoSQLiteRRQueueTest(LifoTestMixin, DiskTestMixin, base.RRQueueTestBase): + + def qfactory(self, key): + path = os.path.join(self.qdir, str(key)) + return track_closed(LifoSQLiteQueue)(path) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/queuelib-1.4.2/queuelib.egg-info/PKG-INFO new/queuelib-1.5.0/queuelib.egg-info/PKG-INFO --- old/queuelib-1.4.2/queuelib.egg-info/PKG-INFO 2015-09-09 20:58:56.000000000 +0200 +++ new/queuelib-1.5.0/queuelib.egg-info/PKG-INFO 2018-03-12 13:50:48.000000000 +0100 @@ -1,11 +1,12 @@ Metadata-Version: 1.1 Name: queuelib -Version: 1.4.2 +Version: 1.5.0 Summary: Collection of persistent (disk-based) queues Home-page: https://github.com/scrapy/queuelib Author: Scrapy project Author-email: [email protected] License: BSD +Description-Content-Type: UNKNOWN Description: ======== queuelib ======== @@ -23,6 +24,8 @@ Queuelib goals are speed and simplicity. It was originally part of the `Scrapy framework`_ and stripped out on its own library. + Note: Queuelib isn't thread-safe. + Requirements ============ @@ -105,6 +108,34 @@ >>> pq.pop() b'a' + RoundRobinQueue + =============== + + Has nearly the same interface and implementation as a Priority Queue except + that each element must be pushed with a (mandatory) key. Popping from the + queue cycles through the keys "round robin". + + Instantiate the Round Robin Queue similarly to the Priority Queue:: + + >>> from queuelib import RoundRobinQueue + >>> rr = RoundRobinQueue(qfactory) + + And use it:: + + >>> rr.push(b'a', '1') + >>> rr.push(b'b', '1') + >>> rr.push(b'c', '2') + >>> rr.push(b'd', '2') + >>> rr.pop() + b'a' + >>> rr.pop() + b'c' + >>> rr.pop() + b'b' + >>> rr.pop() + b'd' + + Mailing list ============ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/queuelib-1.4.2/queuelib.egg-info/SOURCES.txt new/queuelib-1.5.0/queuelib.egg-info/SOURCES.txt --- old/queuelib-1.4.2/queuelib.egg-info/SOURCES.txt 2015-09-09 20:58:56.000000000 +0200 +++ new/queuelib-1.5.0/queuelib.egg-info/SOURCES.txt 2018-03-12 13:50:48.000000000 +0100 @@ -7,10 +7,12 @@ queuelib/__init__.py queuelib/pqueue.py queuelib/queue.py +queuelib/rrqueue.py queuelib.egg-info/PKG-INFO queuelib.egg-info/SOURCES.txt queuelib.egg-info/dependency_links.txt queuelib.egg-info/top_level.txt queuelib/tests/__init__.py queuelib/tests/test_pqueue.py -queuelib/tests/test_queue.py \ No newline at end of file +queuelib/tests/test_queue.py +queuelib/tests/test_rrqueue.py \ No newline at end of file diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/queuelib-1.4.2/setup.cfg new/queuelib-1.5.0/setup.cfg --- old/queuelib-1.4.2/setup.cfg 2015-09-09 20:58:56.000000000 +0200 +++ new/queuelib-1.5.0/setup.cfg 2018-03-12 13:50:48.000000000 +0100 @@ -4,5 +4,4 @@ [egg_info] tag_build = tag_date = 0 -tag_svn_revision = 0 diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/queuelib-1.4.2/setup.py new/queuelib-1.5.0/setup.py --- old/queuelib-1.4.2/setup.py 2015-09-09 20:58:27.000000000 +0200 +++ new/queuelib-1.5.0/setup.py 2018-03-12 13:49:52.000000000 +0100 @@ -2,7 +2,7 @@ setup( name='queuelib', - version='1.4.2', + version='1.5.0', license='BSD', description='Collection of persistent (disk-based) queues', long_description=open('README.rst').read(),
