Hello community, here is the log from the commit of package python-numexpr for openSUSE:Factory checked in at 2018-09-04 22:56:26 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/python-numexpr (Old) and /work/SRC/openSUSE:Factory/.python-numexpr.new (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "python-numexpr" Tue Sep 4 22:56:26 2018 rev:9 rq:632816 version:2.6.8 Changes: -------- --- /work/SRC/openSUSE:Factory/python-numexpr/python-numexpr.changes 2018-08-07 09:44:25.625450852 +0200 +++ /work/SRC/openSUSE:Factory/.python-numexpr.new/python-numexpr.changes 2018-09-04 22:56:27.545060130 +0200 @@ -1,0 +2,23 @@ +Sun Sep 2 16:48:00 UTC 2018 - [email protected] + +- update to version 2.6.8: + * Add check to make sure that f_locals is not actually f_globals + when we do the f_locals clear to avoid the #310 memory leak issue. + * Compare NumPy versions using distutils.version.LooseVersion to + avoid issue #312 when working with NumPy development versions. + * As part of multibuild, wheels for Python 3.7 for Linux and MacOSX + are now available on PyPI. + +- changes from version 2.6.7: + * Thanks to Lehman Garrison for finding and fixing a bug that + exhibited memory leak-like behavior. The use in numexpr.evaluate + of sys._getframe combined with .f_locals from that frame object + results an extra refcount on objects in the frame that calls + numexpr.evaluate, and not evaluate's frame. So if the calling + frame remains in scope for a long time (such as a procedural + script where numexpr is called from the base frame) garbage + collection would never occur. + * Imports for the numexpr.test submodule were made lazy in the + numexpr module. + +------------------------------------------------------------------- Old: ---- numexpr-2.6.6.tar.gz New: ---- numexpr-2.6.8.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ python-numexpr.spec ++++++ --- /var/tmp/diff_new_pack.vRk2bc/_old 2018-09-04 22:56:28.165062247 +0200 +++ /var/tmp/diff_new_pack.vRk2bc/_new 2018-09-04 22:56:28.165062247 +0200 @@ -18,12 +18,12 @@ %{?!python_module:%define python_module() python-%{**} python3-%{**}} Name: python-numexpr -Version: 2.6.6 +Version: 2.6.8 Release: 0 -Url: https://github.com/pydata/numexpr/ Summary: Numerical expression evaluator for NumPy License: MIT Group: Development/Languages/Python +URL: https://github.com/pydata/numexpr/ Source: https://files.pythonhosted.org/packages/source/n/numexpr/numexpr-%{version}.tar.gz BuildRequires: %{python_module devel} BuildRequires: %{python_module numpy-devel >= 1.6} @@ -32,7 +32,6 @@ BuildRequires: gcc-c++ BuildRequires: python-rpm-macros Requires: python-numpy >= 1.6 - %python_subpackages %description ++++++ numexpr-2.6.6.tar.gz -> numexpr-2.6.8.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.6.6/ANNOUNCE.rst new/numexpr-2.6.8/ANNOUNCE.rst --- old/numexpr-2.6.6/ANNOUNCE.rst 2018-07-16 17:57:51.000000000 +0200 +++ new/numexpr-2.6.8/ANNOUNCE.rst 2018-08-17 05:48:20.000000000 +0200 @@ -1,21 +1,29 @@ ========================== - Announcing Numexpr 2.6.6 + Announcing Numexpr 2.6.8 ========================== Hi everyone, -This is a bug-fix release. Thanks to Mark Dickinson for a fix to the thread -barrier that occassionally suffered from spurious wakeups on MacOSX. +Our attempt to fix the memory leak in 2.6.7 had an unforseen consequence that +the `f_locals` from the top-most frame is actually `f_globals`, and clearing it +to fix the extra reference count deletes all global variables. Needless to say +this is undesired behavior. A check has been added to prevent clearing the +globals dict, tested against both `python` and `ipython`. As such, we recommend +skipping 2.6.7 and upgrading straight to 2.6.8 from 2.6.6. Project documentation is available at: http://numexpr.readthedocs.io/ -Changes from 2.6.5 to 2.6.6 +Changes from 2.6.7 to 2.6.8 --------------------------- -- Thanks to Mark Dickinson for a fix to the thread barrier that occassionally - suffered from spurious wakeups on MacOSX. +- Add check to make sure that `f_locals` is not actually `f_globals` when we + do the `f_locals` clear to avoid the #310 memory leak issue. +- Compare NumPy versions using `distutils.version.LooseVersion` to avoid issue + #312 when working with NumPy development versions. +- As part of `multibuild`, wheels for Python 3.7 for Linux and MacOSX are now + available on PyPI. What's Numexpr? --------------- diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.6.6/PKG-INFO new/numexpr-2.6.8/PKG-INFO --- old/numexpr-2.6.6/PKG-INFO 2018-07-16 21:37:09.000000000 +0200 +++ new/numexpr-2.6.8/PKG-INFO 2018-08-19 20:43:01.000000000 +0200 @@ -1,6 +1,6 @@ Metadata-Version: 1.0 Name: numexpr -Version: 2.6.6 +Version: 2.6.8 Summary: Fast numerical expression evaluator for NumPy Home-page: https://github.com/pydata/numexpr Author: David M. Cooke, Francesc Alted and others @@ -21,9 +21,9 @@ .. |travis| image:: https://travis-ci.org/pydata/numexpr.png?branch=master :target: https://travis-ci.org/pydata/numexpr - .. |appveyor| image:: https://ci.appveyor.com/api/projects/status/github/pydata/numexpr - :target: https://ci.appveyor.com/project/pydata/numexpr - .. |docs| image:: https://media.readthedocs.org/static/projects/badges/passing.svg + .. |appveyor| image:: https://ci.appveyor.com/api/projects/status/github/robbmcleod/numexpr + :target: https://ci.appveyor.com/project/robbmcleod/numexpr + .. |docs| image:: https://readthedocs.org/projects/numexpr/badge/?version=latest :target: http://numexpr.readthedocs.io/en/latest .. |version| image:: https://img.shields.io/pypi/v/numexpr.png :target: https://pypi.python.org/pypi/numexpr diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.6.6/README.rst new/numexpr-2.6.8/README.rst --- old/numexpr-2.6.6/README.rst 2018-07-12 19:33:23.000000000 +0200 +++ new/numexpr-2.6.8/README.rst 2018-08-12 18:59:27.000000000 +0200 @@ -13,9 +13,9 @@ .. |travis| image:: https://travis-ci.org/pydata/numexpr.png?branch=master :target: https://travis-ci.org/pydata/numexpr -.. |appveyor| image:: https://ci.appveyor.com/api/projects/status/github/pydata/numexpr - :target: https://ci.appveyor.com/project/pydata/numexpr -.. |docs| image:: https://media.readthedocs.org/static/projects/badges/passing.svg +.. |appveyor| image:: https://ci.appveyor.com/api/projects/status/github/robbmcleod/numexpr + :target: https://ci.appveyor.com/project/robbmcleod/numexpr +.. |docs| image:: https://readthedocs.org/projects/numexpr/badge/?version=latest :target: http://numexpr.readthedocs.io/en/latest .. |version| image:: https://img.shields.io/pypi/v/numexpr.png :target: https://pypi.python.org/pypi/numexpr diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.6.6/RELEASE_NOTES.rst new/numexpr-2.6.8/RELEASE_NOTES.rst --- old/numexpr-2.6.6/RELEASE_NOTES.rst 2018-07-12 19:35:59.000000000 +0200 +++ new/numexpr-2.6.8/RELEASE_NOTES.rst 2018-08-17 05:48:27.000000000 +0200 @@ -2,6 +2,28 @@ Release notes for Numexpr 2.6 series ===================================== +Changes from 2.6.7 to 2.6.8 +--------------------------- + +- Add check to make sure that `f_locals` is not actually `f_globals` when we + do the `f_locals` clear to avoid the #310 memory leak issue. +- Compare NumPy versions using `distutils.version.LooseVersion` to avoid issue + #312 when working with NumPy development versions. +- As part of `multibuild`, wheels for Python 3.7 for Linux and MacOSX are now + available on PyPI. + +Changes from 2.6.6 to 2.6.7 +--------------------------- + +- Thanks to Lehman Garrison for finding and fixing a bug that exhibited memory + leak-like behavior. The use in `numexpr.evaluate` of `sys._getframe` combined + with `.f_locals` from that frame object results an extra refcount on objects + in the frame that calls `numexpr.evaluate`, and not `evaluate`'s frame. So if + the calling frame remains in scope for a long time (such as a procedural + script where `numexpr` is called from the base frame) garbage collection would + never occur. +- Imports for the `numexpr.test` submodule were made lazy in the `numexpr` module. + Changes from 2.6.5 to 2.6.6 --------------------------- diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.6.6/numexpr/__init__.py new/numexpr-2.6.8/numexpr/__init__.py --- old/numexpr-2.6.6/numexpr/__init__.py 2018-07-12 19:33:23.000000000 +0200 +++ new/numexpr-2.6.8/numexpr/__init__.py 2018-07-19 23:15:31.000000000 +0200 @@ -37,7 +37,7 @@ import platform from numexpr.expressions import E from numexpr.necompiler import NumExpr, disassemble, evaluate, re_evaluate -from numexpr.tests import test, print_versions +# from numexpr.tests import test, print_versions from numexpr.interpreter import MAX_THREADS from numexpr.utils import ( get_vml_version, set_vml_accuracy_mode, set_vml_num_threads, @@ -68,3 +68,20 @@ __version__ = version.version +def print_versions(): + """Print the versions of software that numexpr relies on.""" + try: + import numexpr.tests + return numexpr.tests.print_versions() + except ImportError: + # To maintain Python 2.6 compatibility we have simple error handling + raise ImportError('`numexpr.tests` could not be imported, likely it was excluded from the distribution.') + +def test(verbosity=1): + """Run all the tests in the test suite.""" + try: + import numexpr.tests + return numexpr.tests.test(verbosity=verbosity) + except ImportError: + # To maintain Python 2.6 compatibility we have simple error handling + raise ImportError('`numexpr.tests` could not be imported, likely it was excluded from the distribution.') \ No newline at end of file diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.6.6/numexpr/expressions.py new/numexpr-2.6.8/numexpr/expressions.py --- old/numexpr-2.6.6/numexpr/expressions.py 2018-07-12 19:33:23.000000000 +0200 +++ new/numexpr-2.6.8/numexpr/expressions.py 2018-08-15 20:03:58.000000000 +0200 @@ -17,8 +17,8 @@ import numpy # numpy's behavoir sometimes changes with versioning, especially in regard as # to when ints are cast to floats. -# _np_version will be similar to ('1', '13', '1') for which we can use simple comparisons -_np_version = tuple( ver for ver in numpy.__version__.split('.') ) +from distutils.version import LooseVersion +_np_version_forbids_neg_powint = LooseVersion(numpy.__version__) >= LooseVersion('1.12.0b1') # Declare a double type that does not exist in Python space double = numpy.double @@ -283,7 +283,7 @@ @ophelper def pow_op(a, b): - if (_np_version >= ('1', '12', '0b1') and + if (_np_version_forbids_neg_powint and b.astKind in ('int', 'long') and a.astKind in ('int', 'long') and numpy.any(b.value < 0)): diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.6.6/numexpr/necompiler.py new/numexpr-2.6.8/numexpr/necompiler.py --- old/numexpr-2.6.6/numexpr/necompiler.py 2018-07-13 06:47:37.000000000 +0200 +++ new/numexpr-2.6.8/numexpr/necompiler.py 2018-08-16 03:01:37.000000000 +0200 @@ -724,18 +724,34 @@ def getArguments(names, local_dict=None, global_dict=None): """Get the arguments based on the names.""" call_frame = sys._getframe(2) + + clear_local_dict = False if local_dict is None: local_dict = call_frame.f_locals - if global_dict is None: - global_dict = call_frame.f_globals + clear_local_dict = True + try: + frame_globals = call_frame.f_globals + if global_dict is None: + global_dict = frame_globals + + # If `call_frame` is the top frame of the interpreter we can't clear its + # `local_dict`, because it is actually the `global_dict`. + clear_local_dict = clear_local_dict and not frame_globals is local_dict + + arguments = [] + for name in names: + try: + a = local_dict[name] + except KeyError: + a = global_dict[name] + arguments.append(numpy.asarray(a)) + finally: + # If we generated local_dict via an explicit reference to f_locals, + # clear the dict to prevent creating extra ref counts in the caller's scope + # See https://github.com/pydata/numexpr/issues/310 + if clear_local_dict: + local_dict.clear() - arguments = [] - for name in names: - try: - a = local_dict[name] - except KeyError: - a = global_dict[name] - arguments.append(numpy.asarray(a)) return arguments diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.6.6/numexpr/tests/test_numexpr.py new/numexpr-2.6.8/numexpr/tests/test_numexpr.py --- old/numexpr-2.6.6/numexpr/tests/test_numexpr.py 2018-07-12 19:33:23.000000000 +0200 +++ new/numexpr-2.6.8/numexpr/tests/test_numexpr.py 2018-08-16 19:26:59.000000000 +0200 @@ -39,8 +39,8 @@ long = int # Recommended minimum versions -minimum_numpy_version = ('1', '7', '0') - +from distutils.version import LooseVersion +minimum_numpy_version = LooseVersion('1.7.0') class test_numexpr(TestCase): """Testing with 1 thread""" @@ -307,6 +307,37 @@ res = evaluate('where(a, b, c)') assert_array_equal(res, c) + + def test_refcount(self): + # Regression test for issue #310 + a = array([1]) + assert sys.getrefcount(a) == 2 + evaluate('1') + assert sys.getrefcount(a) == 2 + + def test_locals_clears_globals(self): + # Check for issue #313, whereby clearing f_locals also clear f_globals + # if in the top-frame. This cannot be done inside `unittest` as it is always + # executing code in a child frame. + import subprocess + script = r';'.join([ + r"import numexpr as ne", + r"a=10", + r"ne.evaluate('1')", + r"a += 1", + r"ne.evaluate('2', local_dict={})", + r"a += 1", + r"ne.evaluate('3', global_dict={})", + r"a += 1", + r"ne.evaluate('4', local_dict={}, global_dict={})", + r"a += 1", + ]) + # Raises CalledProcessError on a non-normal exit + check = subprocess.check_call('{0} -c "{1}"'.format(sys.executable, script), shell=True) + # Ideally this test should also be done against ipython but it's not + # a requirement. + + class test_numexpr2(test_numexpr): """Testing with 2 threads""" @@ -636,8 +667,7 @@ neval, type(neval), shape(neval)) method.description = ('test_expressions(%s, test_scalar=%r, ' - 'dtype=%r, optimization=%r, exact=%r)') \ - % (expr, test_scalar, dtype.__name__, optimization, exact) + 'dtype=%r, optimization=%r, exact=%r)') % (expr, test_scalar, dtype.__name__, optimization, exact) test_no[0] += 1 method.__name__ = 'test_scalar%d_%s_%s_%s_%04d' % (test_scalar, dtype.__name__, @@ -1010,11 +1040,10 @@ from numexpr.cpuinfo import cpu import platform - np_version = tuple( ver for ver in np.__version__.split('.') ) + np_version = LooseVersion(np.__version__) if minimum_numpy_version < np_version: - print('*Warning*: NumPy version is lower than recommended: %s < %s' % \ - (np.__version__, minimum_numpy_version)) + print('*Warning*: NumPy version is lower than recommended: %s < %s' % (np_version, minimum_numpy_version)) print('-=' * 38) print('Numexpr version: %s' % numexpr.__version__) print('NumPy version: %s' % np.__version__) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.6.6/numexpr/version.py new/numexpr-2.6.8/numexpr/version.py --- old/numexpr-2.6.6/numexpr/version.py 2018-07-16 17:56:44.000000000 +0200 +++ new/numexpr-2.6.8/numexpr/version.py 2018-08-17 05:30:53.000000000 +0200 @@ -8,4 +8,4 @@ # rights to use. #################################################################### -version = '2.6.6' +version = '2.6.8' diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.6.6/numexpr.egg-info/PKG-INFO new/numexpr-2.6.8/numexpr.egg-info/PKG-INFO --- old/numexpr-2.6.6/numexpr.egg-info/PKG-INFO 2018-07-16 21:37:09.000000000 +0200 +++ new/numexpr-2.6.8/numexpr.egg-info/PKG-INFO 2018-08-19 20:43:01.000000000 +0200 @@ -1,6 +1,6 @@ Metadata-Version: 1.0 Name: numexpr -Version: 2.6.6 +Version: 2.6.8 Summary: Fast numerical expression evaluator for NumPy Home-page: https://github.com/pydata/numexpr Author: David M. Cooke, Francesc Alted and others @@ -21,9 +21,9 @@ .. |travis| image:: https://travis-ci.org/pydata/numexpr.png?branch=master :target: https://travis-ci.org/pydata/numexpr - .. |appveyor| image:: https://ci.appveyor.com/api/projects/status/github/pydata/numexpr - :target: https://ci.appveyor.com/project/pydata/numexpr - .. |docs| image:: https://media.readthedocs.org/static/projects/badges/passing.svg + .. |appveyor| image:: https://ci.appveyor.com/api/projects/status/github/robbmcleod/numexpr + :target: https://ci.appveyor.com/project/robbmcleod/numexpr + .. |docs| image:: https://readthedocs.org/projects/numexpr/badge/?version=latest :target: http://numexpr.readthedocs.io/en/latest .. |version| image:: https://img.shields.io/pypi/v/numexpr.png :target: https://pypi.python.org/pypi/numexpr
