Hello community, here is the log from the commit of package python-Cython for openSUSE:Factory checked in at 2020-07-15 12:03:48 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/python-Cython (Old) and /work/SRC/openSUSE:Factory/.python-Cython.new.3060 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "python-Cython" Wed Jul 15 12:03:48 2020 rev:57 rq:819616 version:0.29.21 Changes: -------- --- /work/SRC/openSUSE:Factory/python-Cython/python-Cython.changes 2020-06-21 19:02:41.928278581 +0200 +++ /work/SRC/openSUSE:Factory/.python-Cython.new.3060/python-Cython.changes 2020-07-15 12:04:43.468136133 +0200 @@ -1,0 +2,22 @@ +Wed Jul 8 22:36:13 UTC 2020 - Ondřej Súkup <mimi...@gmail.com> + +- Update TO 0.29.21 + * Fix a regression in 0.29.20 where __div__ failed to be found in extension types + * Fix a regression in 0.29.20 where a call inside of a finally clause could + fail to compile + * Zero-sized buffers could fail to validate as C/Fortran-contiguous. + * exec() did not allow recent Python syntax features in Py3.8+ + due to https://bugs.python.org/issue35975 + * Binding staticmethods of Cython functions were not behaving like + Python methods in Py3 + * Pythran calls to NumPy methods no longer generate useless method lookup code. + * The PyUnicode_GET_LENGTH() macro was missing from the cpython.* declarations + * The deprecated PyUnicode_*() C-API functions are no longer used, except + for Unicode strings that contain lone surrogates. Unicode strings that + contain non-BMP characters or surrogate pairs now generate different C + code on 16-bit Python 2.x Unicode deployments (such as MS-Windows). + Generating the C code on Python 3.x is recommended in this case + * Some template parameters were missing from the C++ std::unordered_map declaration + * Several internal code generation issues regarding temporary variables were resolved + +------------------------------------------------------------------- Old: ---- Cython-0.29.20.tar.gz New: ---- Cython-0.29.21.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ python-Cython.spec ++++++ --- /var/tmp/diff_new_pack.fExevC/_old 2020-07-15 12:04:45.648138540 +0200 +++ /var/tmp/diff_new_pack.fExevC/_new 2020-07-15 12:04:45.652138545 +0200 @@ -19,7 +19,7 @@ %{?!python_module:%define python_module() python-%{**} python3-%{**}} %define oldpython python Name: python-Cython -Version: 0.29.20 +Version: 0.29.21 Release: 0 Summary: The Cython compiler for writing C extensions for the Python language License: Apache-2.0 ++++++ Cython-0.29.20.tar.gz -> Cython-0.29.21.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/.gitrev new/Cython-0.29.21/.gitrev --- old/Cython-0.29.20/.gitrev 2020-06-10 15:13:39.000000000 +0200 +++ new/Cython-0.29.21/.gitrev 2020-07-08 23:45:30.000000000 +0200 @@ -1 +1 @@ -9bc3fc65e906c7b1f5e4e19c92cb726b0ef361bb +976f5483c6df8570f34076ef25af7e7512dd9347 diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/CHANGES.rst new/Cython-0.29.21/CHANGES.rst --- old/Cython-0.29.20/CHANGES.rst 2020-06-10 15:13:18.000000000 +0200 +++ new/Cython-0.29.21/CHANGES.rst 2020-07-08 23:44:39.000000000 +0200 @@ -2,6 +2,46 @@ Cython Changelog ================ +0.29.21 (2020-07-09) +==================== + +Bugs fixed +---------- + +* Fix a regression in 0.29.20 where ``__div__`` failed to be found in extension types. + (Github issue #3688) + +* Fix a regression in 0.29.20 where a call inside of a finally clause could fail to compile. + Patch by David Woods. (Github issue #3712) + +* Zero-sized buffers could fail to validate as C/Fortran-contiguous. + Patch by Clemens Hofreither. (Github issue #2093) + +* ``exec()`` did not allow recent Python syntax features in Py3.8+ due to + https://bugs.python.org/issue35975. + (Github issue #3695) + +* Binding staticmethods of Cython functions were not behaving like Python methods in Py3. + Patch by Jeroen Demeyer and Michał Górny. (Github issue #3106) + +* Pythran calls to NumPy methods no longer generate useless method lookup code. + +* The ``PyUnicode_GET_LENGTH()`` macro was missing from the ``cpython.*`` declarations. + Patch by Thomas Caswell. (Github issue #3692) + +* The deprecated ``PyUnicode_*()`` C-API functions are no longer used, except for Unicode + strings that contain lone surrogates. Unicode strings that contain non-BMP characters + or surrogate pairs now generate different C code on 16-bit Python 2.x Unicode deployments + (such as MS-Windows). Generating the C code on Python 3.x is recommended in this case. + Original patches by Inada Naoki and Victor Stinner. (Github issues #3677, #3721, #3697) + +* Some template parameters were missing from the C++ ``std::unordered_map`` declaration. + Patch by will. (Github issue #3685) + +* Several internal code generation issues regarding temporary variables were resolved. + (Github issue #3708) + + 0.29.20 (2020-06-10) ==================== diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/Cython/Compiler/Code.py new/Cython-0.29.21/Cython/Compiler/Code.py --- old/Cython-0.29.20/Cython/Compiler/Code.py 2020-06-10 15:13:18.000000000 +0200 +++ new/Cython-0.29.21/Cython/Compiler/Code.py 2020-07-08 23:44:39.000000000 +0200 @@ -718,9 +718,10 @@ self.can_trace = False self.gil_owned = True - self.temps_allocated = [] # of (name, type, manage_ref, static) - self.temps_free = {} # (type, manage_ref) -> list of free vars with same type/managed status - self.temps_used_type = {} # name -> (type, manage_ref) + self.temps_allocated = [] # of (name, type, manage_ref, static) + self.temps_free = {} # (type, manage_ref) -> list of free vars with same type/managed status + self.temps_used_type = {} # name -> (type, manage_ref) + self.zombie_temps = set() # temps that must not be reused after release self.temp_counter = 0 self.closure_temps = None @@ -735,6 +736,20 @@ self.should_declare_error_indicator = False self.uses_error_indicator = False + # safety checks + + def validate_exit(self): + # validate that all allocated temps have been freed + if self.temps_allocated: + leftovers = self.temps_in_use() + if leftovers: + msg = "TEMPGUARD: Temps left over at end of '%s': %s" % (self.scope.name, ', '.join([ + '%s [%s]' % (name, ctype) + for name, ctype, is_pytemp in sorted(leftovers)]), + ) + #print(msg) + raise RuntimeError(msg) + # labels def new_label(self, name=None): @@ -804,7 +819,7 @@ # temp handling - def allocate_temp(self, type, manage_ref, static=False): + def allocate_temp(self, type, manage_ref, static=False, reusable=True): """ Allocates a temporary (which may create a new one or get a previously allocated and released one of the same type). Type is simply registered @@ -823,6 +838,8 @@ This is only used when allocating backing store for a module-level C array literals. + if reusable=False, the temp will not be reused after release. + A C string referring to the variable is returned. """ if type.is_const and not type.is_reference: @@ -838,7 +855,7 @@ manage_ref = False freelist = self.temps_free.get((type, manage_ref)) - if freelist is not None and freelist[0]: + if reusable and freelist is not None and freelist[0]: result = freelist[0].pop() freelist[1].remove(result) else: @@ -847,9 +864,11 @@ result = "%s%d" % (Naming.codewriter_temp_prefix, self.temp_counter) if result not in self.names_taken: break self.temps_allocated.append((result, type, manage_ref, static)) + if not reusable: + self.zombie_temps.add(result) self.temps_used_type[result] = (type, manage_ref) if DebugFlags.debug_temp_code_comments: - self.owner.putln("/* %s allocated (%s) */" % (result, type)) + self.owner.putln("/* %s allocated (%s)%s */" % (result, type, "" if reusable else " - zombie")) if self.collect_temps_stack: self.collect_temps_stack[-1].add((result, type)) @@ -868,10 +887,12 @@ self.temps_free[(type, manage_ref)] = freelist if name in freelist[1]: raise RuntimeError("Temp %s freed twice!" % name) - freelist[0].append(name) + if name not in self.zombie_temps: + freelist[0].append(name) freelist[1].add(name) if DebugFlags.debug_temp_code_comments: - self.owner.putln("/* %s released */" % name) + self.owner.putln("/* %s released %s*/" % ( + name, " - zombie" if name in self.zombie_temps else "")) def temps_in_use(self): """Return a list of (cname,type,manage_ref) tuples of temp names and their type diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/Cython/Compiler/ExprNodes.py new/Cython-0.29.21/Cython/Compiler/ExprNodes.py --- old/Cython-0.29.20/Cython/Compiler/ExprNodes.py 2020-06-10 15:13:18.000000000 +0200 +++ new/Cython-0.29.21/Cython/Compiler/ExprNodes.py 2020-07-08 23:44:39.000000000 +0200 @@ -1623,17 +1623,23 @@ def generate_evaluation_code(self, code): if self.type.is_pyobject: - if self.contains_surrogates(): - # surrogates are not really portable and cannot be + # FIXME: this should go away entirely! + # Since string_contains_lone_surrogates() returns False for surrogate pairs in Py2/UCS2, + # Py2 can generate different code from Py3 here. Let's hope we get away with claiming that + # the processing of surrogate pairs in code was always ambiguous and lead to different results + # on P16/32bit Unicode platforms. + if StringEncoding.string_contains_lone_surrogates(self.value): + # lone (unpaired) surrogates are not really portable and cannot be # decoded by the UTF-8 codec in Py3.3 self.result_code = code.get_py_const(py_object_type, 'ustring') - data_cname = code.get_pyunicode_ptr_const(self.value) + data_cname = code.get_string_const( + StringEncoding.BytesLiteral(self.value.encode('unicode_escape'))) const_code = code.get_cached_constants_writer(self.result_code) if const_code is None: return # already initialised const_code.mark_pos(self.pos) const_code.putln( - "%s = PyUnicode_FromUnicode(%s, (sizeof(%s) / sizeof(Py_UNICODE))-1); %s" % ( + "%s = PyUnicode_DecodeUnicodeEscape(%s, sizeof(%s) - 1, NULL); %s" % ( self.result_code, data_cname, data_cname, @@ -4334,6 +4340,7 @@ pythran_indexing_code(self.indices), op, rhs.pythran_result())) + code.funcstate.release_temp(obj) return # Used from generate_assignment_code and InPlaceAssignmentNode @@ -4374,11 +4381,11 @@ code.putln("%s = (PyObject *) *%s;" % (self.result(), self.buffer_ptr_code)) code.putln("__Pyx_INCREF((PyObject*)%s);" % self.result()) - def free_temps(self, code): + def free_subexpr_temps(self, code): for temp in self.index_temps: code.funcstate.release_temp(temp) self.index_temps = () - super(BufferIndexNode, self).free_temps(code) + super(BufferIndexNode, self).free_subexpr_temps(code) class MemoryViewIndexNode(BufferIndexNode): @@ -4655,6 +4662,7 @@ self.dst.generate_evaluation_code(code) self._generate_assignment_code(rhs, code) self.dst.generate_disposal_code(code) + self.dst.free_temps(code) rhs.generate_disposal_code(code) rhs.free_temps(code) @@ -5558,7 +5566,7 @@ env.add_include_file(pythran_get_func_include_file(function)) return NumPyMethodCallNode.from_node( self, - function=function, + function_cname=pythran_functor(function), arg_tuple=self.arg_tuple, type=PythranExpr(pythran_func_type(function, self.arg_tuple.args)), ) @@ -5963,13 +5971,13 @@ code.funcstate.release_temp(self.opt_arg_struct) -class NumPyMethodCallNode(SimpleCallNode): +class NumPyMethodCallNode(ExprNode): # Pythran call to a NumPy function or method. # - # function ExprNode the function/method to call - # arg_tuple TupleNode the arguments as an args tuple + # function_cname string the function/method to call + # arg_tuple TupleNode the arguments as an args tuple - subexprs = ['function', 'arg_tuple'] + subexprs = ['arg_tuple'] is_temp = True may_return_none = True @@ -5977,7 +5985,6 @@ code.mark_pos(self.pos) self.allocate_temp_result(code) - self.function.generate_evaluation_code(code) assert self.arg_tuple.mult_factor is None args = self.arg_tuple.args for arg in args: @@ -5988,7 +5995,7 @@ code.putln("new (&%s) decltype(%s){%s{}(%s)};" % ( self.result(), self.result(), - pythran_functor(self.function), + self.function_cname, ", ".join(a.pythran_result() for a in args))) @@ -7277,6 +7284,8 @@ self.member.upper(), self.obj.result_as(self.obj.type), rhs.result_as(self.ctype()))) + rhs.generate_disposal_code(code) + rhs.free_temps(code) else: select_code = self.result() if self.type.is_pyobject and self.use_managed_ref: @@ -8127,20 +8136,18 @@ return t def allocate_temp_result(self, code): - if self.type.is_array and self.in_module_scope: - self.temp_code = code.funcstate.allocate_temp( - self.type, manage_ref=False, static=True) - else: - SequenceNode.allocate_temp_result(self, code) - - def release_temp_result(self, env): if self.type.is_array: - # To be valid C++, we must allocate the memory on the stack - # manually and be sure not to reuse it for something else. - # Yes, this means that we leak a temp array variable. - pass + if self.in_module_scope: + self.temp_code = code.funcstate.allocate_temp( + self.type, manage_ref=False, static=True, reusable=False) + else: + # To be valid C++, we must allocate the memory on the stack + # manually and be sure not to reuse it for something else. + # Yes, this means that we leak a temp array variable. + self.temp_code = code.funcstate.allocate_temp( + self.type, manage_ref=False, reusable=False) else: - SequenceNode.release_temp_result(self, env) + SequenceNode.allocate_temp_result(self, code) def calculate_constant_result(self): if self.mult_factor: @@ -13076,9 +13083,18 @@ def generate_post_assignment_code(self, code): self.arg.generate_post_assignment_code(code) + def allocate_temp_result(self, code): + pass + + def release_temp_result(self, code): + pass + def free_temps(self, code): self.arg.free_temps(code) + def free_subexpr_temps(self, code): + self.arg.free_subexpr_temps(code) + class NoneCheckNode(CoercionNode): # This node is used to check that a Python object is not None and diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/Cython/Compiler/FlowControl.py new/Cython-0.29.21/Cython/Compiler/FlowControl.py --- old/Cython-0.29.20/Cython/Compiler/FlowControl.py 2020-06-10 15:13:18.000000000 +0200 +++ new/Cython-0.29.21/Cython/Compiler/FlowControl.py 2020-07-08 23:44:39.000000000 +0200 @@ -1198,6 +1198,7 @@ if self.flow.loops: self.flow.loops[-1].exceptions.append(descr) self.flow.block = body_block + body_block.add_child(entry_point) self.flow.nextblock() self._visit(node.body) self.flow.exceptions.pop() diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/Cython/Compiler/FusedNode.py new/Cython-0.29.21/Cython/Compiler/FusedNode.py --- old/Cython-0.29.20/Cython/Compiler/FusedNode.py 2020-05-19 10:45:14.000000000 +0200 +++ new/Cython-0.29.21/Cython/Compiler/FusedNode.py 2020-07-08 23:44:39.000000000 +0200 @@ -878,17 +878,23 @@ (self.resulting_fused_function.result(), self.__signatures__.result())) code.put_giveref(self.__signatures__.result()) + self.__signatures__.generate_post_assignment_code(code) + self.__signatures__.free_temps(code) self.fused_func_assignment.generate_execution_code(code) # Dispose of results self.resulting_fused_function.generate_disposal_code(code) + self.resulting_fused_function.free_temps(code) self.defaults_tuple.generate_disposal_code(code) + self.defaults_tuple.free_temps(code) self.code_object.generate_disposal_code(code) + self.code_object.free_temps(code) for default in self.defaults: if default is not None: default.generate_disposal_code(code) + default.free_temps(code) def annotate(self, code): for stat in self.stats: diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/Cython/Compiler/ModuleNode.py new/Cython-0.29.21/Cython/Compiler/ModuleNode.py --- old/Cython-0.29.20/Cython/Compiler/ModuleNode.py 2020-06-10 15:13:18.000000000 +0200 +++ new/Cython-0.29.21/Cython/Compiler/ModuleNode.py 2020-07-08 23:44:39.000000000 +0200 @@ -29,7 +29,7 @@ from .Errors import error, warning from .PyrexTypes import py_object_type from ..Utils import open_new_file, replace_suffix, decode_filename, build_hex_version -from .Code import UtilityCode, IncludeCode, TempitaUtilityCode +from .Code import UtilityCode, IncludeCode from .StringEncoding import EncodedString from .Pythran import has_np_pythran @@ -1223,6 +1223,10 @@ type = entry.type scope = type.scope if scope: # could be None if there was an error + if not scope.directives['c_api_binop_methods']: + error(self.pos, + "The 'c_api_binop_methods' directive is only supported for forward compatibility" + " and must be True.") self.generate_exttype_vtable(scope, code) self.generate_new_function(scope, code, entry) self.generate_dealloc_function(scope, code) @@ -1255,9 +1259,6 @@ self.generate_dict_getter_function(scope, code) if scope.defines_any_special(TypeSlots.richcmp_special_methods): self.generate_richcmp_function(scope, code) - for slot in TypeSlots.PyNumberMethods: - if slot.is_binop and scope.defines_any_special(slot.user_methods): - self.generate_binop_function(scope, slot, code) self.generate_property_accessors(scope, code) self.generate_method_table(scope, code) self.generate_getset_table(scope, code) @@ -1897,64 +1898,6 @@ code.putln("}") # switch code.putln("}") - def generate_binop_function(self, scope, slot, code): - func_name = scope.mangle_internal(slot.slot_name) - if scope.directives['c_api_binop_methods']: - code.putln('#define %s %s' % (func_name, slot.left_slot.slot_code(scope))) - return - else: - error(self.pos, - "The 'c_api_binop_methods' directive is only supported for forward compatibility" - " and must be True.") - - code.putln() - preprocessor_guard = slot.preprocessor_guard_code() - if preprocessor_guard: - code.putln(preprocessor_guard) - - if slot.left_slot.signature == TypeSlots.binaryfunc: - slot_type = 'binaryfunc' - extra_arg = extra_arg_decl = '' - elif slot.left_slot.signature == TypeSlots.ternaryfunc: - slot_type = 'ternaryfunc' - extra_arg = ', extra_arg' - extra_arg_decl = ', PyObject* extra_arg' - else: - error(entry.pos, "Unexpected type lost signature: %s" % slot) - - def has_slot_method(method_name): - entry = scope.lookup(method_name) - return bool(entry and entry.is_special and entry.func_cname) - def call_slot_method(method_name, reverse): - entry = scope.lookup(method_name) - if entry and entry.is_special and entry.func_cname: - return "%s(%s%s)" % ( - entry.func_cname, - "right, left" if reverse else "left, right", - extra_arg) - else: - return '%s_maybe_call_slot(%s, left, right %s)' % ( - func_name, - 'Py_TYPE(right)->tp_base' if reverse else 'Py_TYPE(left)->tp_base', - extra_arg) - - code.putln( - TempitaUtilityCode.load_as_string( - "BinopSlot", "ExtensionTypes.c", - context={ - "func_name": func_name, - "slot_name": slot.slot_name, - "overloads_left": int(has_slot_method(slot.left_slot.method_name)), - "call_left": call_slot_method(slot.left_slot.method_name, reverse=False), - "call_right": call_slot_method(slot.right_slot.method_name, reverse=True), - "type_cname": scope.parent_type.typeptr_cname, - "slot_type": slot_type, - "extra_arg": extra_arg, - "extra_arg_decl": extra_arg_decl, - })[1]) - if preprocessor_guard: - code.putln("#endif") - def generate_getattro_function(self, scope, code): # First try to get the attribute using __getattribute__, if defined, or # PyObject_GenericGetAttr. @@ -2715,6 +2658,8 @@ code.putln('static void %s(CYTHON_UNUSED PyObject *self) {' % Naming.cleanup_cname) + code.enter_cfunc_scope(env) + if Options.generate_cleanup_code >= 2: code.putln("/*--- Global cleanup code ---*/") rev_entries = list(env.var_entries) @@ -3004,6 +2949,7 @@ module.qualified_name, temp, code.error_goto(self.pos))) + code.put_gotref(temp) for entry in entries: if env is module: cname = entry.cname @@ -3014,7 +2960,8 @@ 'if (__Pyx_ImportVoidPtr(%s, "%s", (void **)&%s, "%s") < 0) %s' % ( temp, entry.name, cname, signature, code.error_goto(self.pos))) - code.putln("Py_DECREF(%s); %s = 0;" % (temp, temp)) + code.put_decref_clear(temp, py_object_type) + code.funcstate.release_temp(temp) def generate_c_function_import_code_for_module(self, module, env, code): # Generate import code for all exported C functions in a cimported module. @@ -3032,6 +2979,7 @@ module.qualified_name, temp, code.error_goto(self.pos))) + code.put_gotref(temp) for entry in entries: code.putln( 'if (__Pyx_ImportFunction(%s, "%s", (void (**)(void))&%s, "%s") < 0) %s' % ( @@ -3040,7 +2988,8 @@ entry.cname, entry.type.signature_string(), code.error_goto(self.pos))) - code.putln("Py_DECREF(%s); %s = 0;" % (temp, temp)) + code.put_decref_clear(temp, py_object_type) + code.funcstate.release_temp(temp) def generate_type_init_code(self, env, code): # Generate type import code for extern extension types diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/Cython/Compiler/Nodes.py new/Cython-0.29.21/Cython/Compiler/Nodes.py --- old/Cython-0.29.20/Cython/Compiler/Nodes.py 2020-06-10 15:13:18.000000000 +0200 +++ new/Cython-0.29.21/Cython/Compiler/Nodes.py 2020-07-08 23:44:39.000000000 +0200 @@ -5866,6 +5866,7 @@ arg.generate_evaluation_code(code) code.putln("delete %s;" % arg.result()) arg.generate_disposal_code(code) + arg.free_temps(code) # else error reported earlier def annotate(self, code): @@ -6410,6 +6411,8 @@ # generate the switch statement, so shouldn't be bothered). code.putln("default: break;") code.putln("}") + self.test.generate_disposal_code(code) + self.test.free_temps(code) def generate_function_definitions(self, env, code): self.test.generate_function_definitions(env, code) @@ -7673,9 +7676,10 @@ if self.func_return_type.is_pyobject: code.putln("%s = 0;" % ret_temp) code.funcstate.release_temp(ret_temp) - ret_temp = None if self.in_generator: self.put_error_uncatcher(code, exc_vars) + for cname in exc_vars: + code.funcstate.release_temp(cname) if not self.finally_clause.is_terminator: code.put_goto(old_label) @@ -8772,6 +8776,11 @@ self.begin_of_parallel_control_block_point = None self.begin_of_parallel_control_block_point_after_decls = None + if self.num_threads is not None: + # FIXME: is it the right place? should not normally produce code. + self.num_threads.generate_disposal_code(code) + self.num_threads.free_temps(code) + # Firstly, always prefer errors over returning, continue or break if self.error_label_used: c.putln("const char *%s = NULL; int %s = 0, %s = 0;" % self.parallel_pos_info) @@ -9151,7 +9160,7 @@ # And finally, release our privates and write back any closure # variables - for temp in start_stop_step + (self.chunksize, self.num_threads): + for temp in start_stop_step + (self.chunksize,): if temp is not None: temp.generate_disposal_code(code) temp.free_temps(code) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/Cython/Compiler/Options.py new/Cython-0.29.21/Cython/Compiler/Options.py --- old/Cython-0.29.20/Cython/Compiler/Options.py 2020-06-10 15:13:18.000000000 +0200 +++ new/Cython-0.29.21/Cython/Compiler/Options.py 2020-07-08 23:44:39.000000000 +0200 @@ -178,7 +178,7 @@ 'auto_pickle': None, 'cdivision': False, # was True before 0.12 'cdivision_warnings': False, - 'c_api_binop_methods': True, # Change for 3.0 + 'c_api_binop_methods': True, 'overflowcheck': False, 'overflowcheck.fold': True, 'always_allow_keywords': False, diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/Cython/Compiler/StringEncoding.py new/Cython-0.29.21/Cython/Compiler/StringEncoding.py --- old/Cython-0.29.20/Cython/Compiler/StringEncoding.py 2020-05-19 10:45:14.000000000 +0200 +++ new/Cython-0.29.21/Cython/Compiler/StringEncoding.py 2020-07-08 23:44:39.000000000 +0200 @@ -154,6 +154,34 @@ return False +def string_contains_lone_surrogates(ustring): + """ + Check if the unicode string contains lone surrogate code points + on a CPython platform with wide (UCS-4) or narrow (UTF-16) + Unicode, i.e. characters that would be spelled as two + separate code units on a narrow platform, but that do not form a pair. + """ + last_was_start = False + unicode_uses_surrogate_encoding = sys.maxunicode == 65535 + for c in map(ord, ustring): + # surrogates tend to be rare + if c < 0xD800 or c > 0xDFFF: + if last_was_start: + return True + elif not unicode_uses_surrogate_encoding: + # on 32bit Unicode platforms, there is never a pair + return True + elif c <= 0xDBFF: + if last_was_start: + return True # lone start + last_was_start = True + else: + if not last_was_start: + return True # lone end + last_was_start = False + return last_was_start + + class BytesLiteral(_bytes): # bytes subclass that is compatible with EncodedString encoding = None diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/Cython/Compiler/Tests/TestStringEncoding.py new/Cython-0.29.21/Cython/Compiler/Tests/TestStringEncoding.py --- old/Cython-0.29.20/Cython/Compiler/Tests/TestStringEncoding.py 1970-01-01 01:00:00.000000000 +0100 +++ new/Cython-0.29.21/Cython/Compiler/Tests/TestStringEncoding.py 2020-07-08 23:44:39.000000000 +0200 @@ -0,0 +1,44 @@ +# -*- coding: utf-8 -*- + +import sys +import unittest + +import Cython.Compiler.StringEncoding as StringEncoding + + +class StringEncodingTest(unittest.TestCase): + """ + Test the StringEncoding module. + """ + def test_string_contains_lone_surrogates(self): + self.assertFalse(StringEncoding.string_contains_lone_surrogates(u"abc")) + self.assertFalse(StringEncoding.string_contains_lone_surrogates(u"\uABCD")) + self.assertFalse(StringEncoding.string_contains_lone_surrogates(u"\N{SNOWMAN}")) + + # This behaves differently in Py2 when freshly parsed and read from a .pyc file, + # but it seems to be a marshalling bug in Py2, which doesn't hurt us in Cython. + if sys.version_info[0] != 2: + self.assertTrue(StringEncoding.string_contains_lone_surrogates(u"\uD800\uDFFF")) + + # In Py2 with 16bit Unicode, the following is indistinguishable from the 32bit character. + obfuscated_surrogate_pair = (u"\uDFFF" + "\uD800")[::-1] + if sys.version_info[0] == 2 and sys.maxunicode == 65565: + self.assertFalse(StringEncoding.string_contains_lone_surrogates(obfuscated_surrogate_pair)) + else: + self.assertTrue(StringEncoding.string_contains_lone_surrogates(obfuscated_surrogate_pair)) + + self.assertTrue(StringEncoding.string_contains_lone_surrogates(u"\uD800")) + self.assertTrue(StringEncoding.string_contains_lone_surrogates(u"\uDFFF")) + self.assertTrue(StringEncoding.string_contains_lone_surrogates(u"\uDFFF\uD800")) + self.assertTrue(StringEncoding.string_contains_lone_surrogates(u"\uD800x\uDFFF")) + + def test_string_contains_surrogates(self): + self.assertFalse(StringEncoding.string_contains_surrogates(u"abc")) + self.assertFalse(StringEncoding.string_contains_surrogates(u"\uABCD")) + self.assertFalse(StringEncoding.string_contains_surrogates(u"\N{SNOWMAN}")) + + self.assertTrue(StringEncoding.string_contains_surrogates(u"\uD800")) + self.assertTrue(StringEncoding.string_contains_surrogates(u"\uDFFF")) + self.assertTrue(StringEncoding.string_contains_surrogates(u"\uD800\uDFFF")) + self.assertTrue(StringEncoding.string_contains_surrogates(u"\uDFFF\uD800")) + self.assertTrue(StringEncoding.string_contains_surrogates(u"\uD800x\uDFFF")) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/Cython/Compiler/TypeSlots.py new/Cython-0.29.21/Cython/Compiler/TypeSlots.py --- old/Cython-0.29.20/Cython/Compiler/TypeSlots.py 2020-06-10 15:13:18.000000000 +0200 +++ new/Cython-0.29.21/Cython/Compiler/TypeSlots.py 2020-07-08 23:44:39.000000000 +0200 @@ -180,14 +180,13 @@ # ifdef Full #ifdef string that slot is wrapped in. Using this causes py3, py2 and flags to be ignored.) def __init__(self, slot_name, dynamic=False, inherited=False, - py3=True, py2=True, ifdef=None, is_binop=False): + py3=True, py2=True, ifdef=None): self.slot_name = slot_name self.is_initialised_dynamically = dynamic self.is_inherited = inherited self.ifdef = ifdef self.py3 = py3 self.py2 = py2 - self.is_binop = is_binop def preprocessor_guard_code(self): ifdef = self.ifdef @@ -406,17 +405,6 @@ return self.default_value -class BinopSlot(SyntheticSlot): - def __init__(self, signature, slot_name, left_method, **kargs): - assert left_method.startswith('__') - right_method = '__r' + left_method[2:] - SyntheticSlot.__init__( - self, slot_name, [left_method, right_method], "0", is_binop=True, **kargs) - # MethodSlot causes special method registration. - self.left_slot = MethodSlot(signature, "", left_method) - self.right_slot = MethodSlot(signature, "", right_method) - - class RichcmpSlot(MethodSlot): def slot_code(self, scope): entry = scope.lookup_here(self.method_name) @@ -740,23 +728,23 @@ PyNumberMethods_Py3_GUARD = "PY_MAJOR_VERSION < 3 || (CYTHON_COMPILING_IN_PYPY && PY_VERSION_HEX < 0x03050000)" PyNumberMethods = ( - BinopSlot(binaryfunc, "nb_add", "__add__"), - BinopSlot(binaryfunc, "nb_subtract", "__sub__"), - BinopSlot(binaryfunc, "nb_multiply", "__mul__"), - BinopSlot(binaryfunc, "nb_divide", "__div__", ifdef = PyNumberMethods_Py3_GUARD), - BinopSlot(binaryfunc, "nb_remainder", "__mod__"), - BinopSlot(binaryfunc, "nb_divmod", "__divmod__"), - BinopSlot(ternaryfunc, "nb_power", "__pow__"), + MethodSlot(binaryfunc, "nb_add", "__add__"), + MethodSlot(binaryfunc, "nb_subtract", "__sub__"), + MethodSlot(binaryfunc, "nb_multiply", "__mul__"), + MethodSlot(binaryfunc, "nb_divide", "__div__", ifdef = PyNumberMethods_Py3_GUARD), + MethodSlot(binaryfunc, "nb_remainder", "__mod__"), + MethodSlot(binaryfunc, "nb_divmod", "__divmod__"), + MethodSlot(ternaryfunc, "nb_power", "__pow__"), MethodSlot(unaryfunc, "nb_negative", "__neg__"), MethodSlot(unaryfunc, "nb_positive", "__pos__"), MethodSlot(unaryfunc, "nb_absolute", "__abs__"), MethodSlot(inquiry, "nb_nonzero", "__nonzero__", py3 = ("nb_bool", "__bool__")), MethodSlot(unaryfunc, "nb_invert", "__invert__"), - BinopSlot(binaryfunc, "nb_lshift", "__lshift__"), - BinopSlot(binaryfunc, "nb_rshift", "__rshift__"), - BinopSlot(binaryfunc, "nb_and", "__and__"), - BinopSlot(binaryfunc, "nb_xor", "__xor__"), - BinopSlot(binaryfunc, "nb_or", "__or__"), + MethodSlot(binaryfunc, "nb_lshift", "__lshift__"), + MethodSlot(binaryfunc, "nb_rshift", "__rshift__"), + MethodSlot(binaryfunc, "nb_and", "__and__"), + MethodSlot(binaryfunc, "nb_xor", "__xor__"), + MethodSlot(binaryfunc, "nb_or", "__or__"), EmptySlot("nb_coerce", ifdef = PyNumberMethods_Py3_GUARD), MethodSlot(unaryfunc, "nb_int", "__int__", fallback="__long__"), MethodSlot(unaryfunc, "nb_long", "__long__", fallback="__int__", py3 = "<RESERVED>"), @@ -779,8 +767,8 @@ # Added in release 2.2 # The following require the Py_TPFLAGS_HAVE_CLASS flag - BinopSlot(binaryfunc, "nb_floor_divide", "__floordiv__"), - BinopSlot(binaryfunc, "nb_true_divide", "__truediv__"), + MethodSlot(binaryfunc, "nb_floor_divide", "__floordiv__"), + MethodSlot(binaryfunc, "nb_true_divide", "__truediv__"), MethodSlot(ibinaryfunc, "nb_inplace_floor_divide", "__ifloordiv__"), MethodSlot(ibinaryfunc, "nb_inplace_true_divide", "__itruediv__"), @@ -788,7 +776,7 @@ MethodSlot(unaryfunc, "nb_index", "__index__"), # Added in release 3.5 - BinopSlot(binaryfunc, "nb_matrix_multiply", "__matmul__", ifdef="PY_VERSION_HEX >= 0x03050000"), + MethodSlot(binaryfunc, "nb_matrix_multiply", "__matmul__", ifdef="PY_VERSION_HEX >= 0x03050000"), MethodSlot(ibinaryfunc, "nb_inplace_matrix_multiply", "__imatmul__", ifdef="PY_VERSION_HEX >= 0x03050000"), ) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/Cython/Includes/cpython/unicode.pxd new/Cython-0.29.21/Cython/Includes/cpython/unicode.pxd --- old/Cython-0.29.20/Cython/Includes/cpython/unicode.pxd 2020-05-19 10:45:14.000000000 +0200 +++ new/Cython-0.29.21/Cython/Includes/cpython/unicode.pxd 2020-07-08 23:44:39.000000000 +0200 @@ -10,8 +10,19 @@ # Return the size of the object. o has to be a PyUnicodeObject # (not checked). + # + # Deprecated since version 3.3, will be removed in version 3.10: + # Part of the old-style Unicode API, please migrate to using + # PyUnicode_GET_LENGTH(). Py_ssize_t PyUnicode_GET_SIZE(object o) + # Return the length of the Unicode string, in code points. o has + # to be a Unicode object in the “canonical” representation (not + # checked). + # + # New in version 3.3. + Py_ssize_t PyUnicode_GET_LENGTH(object o) + # Return the size of the object's internal buffer in bytes. o has # to be a PyUnicodeObject (not checked). Py_ssize_t PyUnicode_GET_DATA_SIZE(object o) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/Cython/Includes/libcpp/unordered_map.pxd new/Cython-0.29.21/Cython/Includes/libcpp/unordered_map.pxd --- old/Cython-0.29.20/Cython/Includes/libcpp/unordered_map.pxd 2020-04-19 11:26:03.000000000 +0200 +++ new/Cython-0.29.21/Cython/Includes/libcpp/unordered_map.pxd 2020-07-08 23:44:39.000000000 +0200 @@ -1,7 +1,7 @@ from .utility cimport pair cdef extern from "<unordered_map>" namespace "std" nogil: - cdef cppclass unordered_map[T, U]: + cdef cppclass unordered_map[T, U, HASH=*, PRED=*, ALLOCATOR=*]: ctypedef T key_type ctypedef U mapped_type ctypedef pair[const T, U] value_type diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/Cython/Shadow.py new/Cython-0.29.21/Cython/Shadow.py --- old/Cython-0.29.20/Cython/Shadow.py 2020-06-10 15:13:18.000000000 +0200 +++ new/Cython-0.29.21/Cython/Shadow.py 2020-07-08 23:44:39.000000000 +0200 @@ -1,7 +1,7 @@ # cython.* namespace for pure mode. from __future__ import absolute_import -__version__ = "0.29.20" +__version__ = "0.29.21" try: from __builtin__ import basestring diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/Cython/Utility/Builtins.c new/Cython-0.29.21/Cython/Utility/Builtins.c --- old/Cython-0.29.20/Cython/Utility/Builtins.c 2020-06-10 15:13:18.000000000 +0200 +++ new/Cython-0.29.21/Cython/Utility/Builtins.c 2020-07-08 23:44:39.000000000 +0200 @@ -128,6 +128,9 @@ } else { PyCompilerFlags cf; cf.cf_flags = 0; +#if PY_VERSION_HEX >= 0x030800A3 + cf.cf_feature_version = PY_MINOR_VERSION; +#endif if (PyUnicode_Check(o)) { cf.cf_flags = PyCF_SOURCE_IS_UTF8; s = PyUnicode_AsUTF8String(o); diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/Cython/Utility/CythonFunction.c new/Cython-0.29.21/Cython/Utility/CythonFunction.c --- old/Cython-0.29.20/Cython/Utility/CythonFunction.c 2020-06-10 15:13:18.000000000 +0200 +++ new/Cython-0.29.21/Cython/Utility/CythonFunction.c 2020-07-08 23:44:39.000000000 +0200 @@ -548,6 +548,7 @@ static PyObject *__Pyx_CyFunction_descr_get(PyObject *func, PyObject *obj, PyObject *type) { +#if PY_MAJOR_VERSION < 3 __pyx_CyFunctionObject *m = (__pyx_CyFunctionObject *) func; if (m->flags & __Pyx_CYFUNCTION_STATICMETHOD) { @@ -563,6 +564,7 @@ if (obj == Py_None) obj = NULL; +#endif return __Pyx_PyMethod_New(func, obj, type); } diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/Cython/Utility/ExtensionTypes.c new/Cython-0.29.21/Cython/Utility/ExtensionTypes.c --- old/Cython-0.29.20/Cython/Utility/ExtensionTypes.c 2020-06-10 15:13:18.000000000 +0200 +++ new/Cython-0.29.21/Cython/Utility/ExtensionTypes.c 2020-07-08 23:44:39.000000000 +0200 @@ -278,56 +278,3 @@ Py_XDECREF(setstate_cython); return ret; } - -/////////////// BinopSlot /////////////// - -static CYTHON_INLINE PyObject *{{func_name}}_maybe_call_slot(PyTypeObject* type, PyObject *left, PyObject *right {{extra_arg_decl}}) { - {{slot_type}} slot; -#if CYTHON_USE_TYPE_SLOTS - slot = type->tp_as_number ? type->tp_as_number->{{slot_name}} : NULL; -#else - slot = ({{slot_type}}) PyType_GetSlot(type, Py_{{slot_name}}); -#endif - return slot ? slot(left, right {{extra_arg}}) : __Pyx_NewRef(Py_NotImplemented); -} - -static PyObject *{{func_name}}(PyObject *left, PyObject *right {{extra_arg_decl}}) { - PyObject *res; - int maybe_self_is_left, maybe_self_is_right = 0; - maybe_self_is_left = Py_TYPE(left) == Py_TYPE(right) -#if CYTHON_USE_TYPE_SLOTS - || (Py_TYPE(left)->tp_as_number && Py_TYPE(left)->tp_as_number->{{slot_name}} == &{{func_name}}) -#endif - || __Pyx_TypeCheck(left, {{type_cname}}); - // Optimize for the common case where the left operation is defined (and successful). - if (!{{overloads_left}}) { - maybe_self_is_right = Py_TYPE(left) == Py_TYPE(right) -#if CYTHON_USE_TYPE_SLOTS - || (Py_TYPE(right)->tp_as_number && Py_TYPE(right)->tp_as_number->{{slot_name}} == &{{func_name}}) -#endif - || __Pyx_TypeCheck(right, {{type_cname}}); - } - if (maybe_self_is_left) { - if (maybe_self_is_right && !{{overloads_left}}) { - res = {{call_right}}; - if (res != Py_NotImplemented) return res; - Py_DECREF(res); - // Don't bother calling it again. - maybe_self_is_right = 0; - } - res = {{call_left}}; - if (res != Py_NotImplemented) return res; - Py_DECREF(res); - } - if ({{overloads_left}}) { - maybe_self_is_right = Py_TYPE(left) == Py_TYPE(right) -#if CYTHON_USE_TYPE_SLOTS - || (Py_TYPE(right)->tp_as_number && Py_TYPE(right)->tp_as_number->{{slot_name}} == &{{func_name}}) -#endif - || PyType_IsSubtype(Py_TYPE(right), {{type_cname}}); - } - if (maybe_self_is_right) { - return {{call_right}}; - } - return __Pyx_NewRef(Py_NotImplemented); -} diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/Cython/Utility/ImportExport.c new/Cython-0.29.21/Cython/Utility/ImportExport.c --- old/Cython-0.29.20/Cython/Utility/ImportExport.c 2020-05-19 10:45:14.000000000 +0200 +++ new/Cython-0.29.21/Cython/Utility/ImportExport.c 2020-07-08 23:44:39.000000000 +0200 @@ -152,11 +152,12 @@ } if (skip_leading_underscores && #if PY_MAJOR_VERSION < 3 - PyString_Check(name) && + likely(PyString_Check(name)) && PyString_AS_STRING(name)[0] == '_') #else - PyUnicode_Check(name) && - PyUnicode_AS_UNICODE(name)[0] == '_') + likely(PyUnicode_Check(name)) && + likely(__Pyx_PyUnicode_GET_LENGTH(name)) && + __Pyx_PyUnicode_READ_CHAR(name, 0) == '_') #endif { Py_DECREF(name); diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/Cython/Utility/MemoryView_C.c new/Cython-0.29.21/Cython/Utility/MemoryView_C.c --- old/Cython-0.29.20/Cython/Utility/MemoryView_C.c 2020-05-19 10:45:14.000000000 +0200 +++ new/Cython-0.29.21/Cython/Utility/MemoryView_C.c 2020-07-08 23:44:39.000000000 +0200 @@ -347,18 +347,22 @@ } /* Check axes */ - for (i = 0; i < ndim; i++) { - spec = axes_specs[i]; - if (unlikely(!__pyx_check_strides(buf, i, ndim, spec))) - goto fail; - if (unlikely(!__pyx_check_suboffsets(buf, i, ndim, spec))) + if (buf->len > 0) { + // 0-sized arrays do not undergo these checks since their strides are + // irrelevant and they are always both C- and F-contiguous. + for (i = 0; i < ndim; i++) { + spec = axes_specs[i]; + if (unlikely(!__pyx_check_strides(buf, i, ndim, spec))) + goto fail; + if (unlikely(!__pyx_check_suboffsets(buf, i, ndim, spec))) + goto fail; + } + + /* Check contiguity */ + if (unlikely(buf->strides && !__pyx_verify_contig(buf, ndim, c_or_f_flag))) goto fail; } - /* Check contiguity */ - if (unlikely(buf->strides && !__pyx_verify_contig(buf, ndim, c_or_f_flag))) - goto fail; - /* Initialize */ if (unlikely(__Pyx_init_memviewslice(memview, ndim, memviewslice, new_memview != NULL) == -1)) { diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/Cython/Utility/ModuleSetupCode.c new/Cython-0.29.21/Cython/Utility/ModuleSetupCode.c --- old/Cython-0.29.20/Cython/Utility/ModuleSetupCode.c 2020-06-10 15:13:18.000000000 +0200 +++ new/Cython-0.29.21/Cython/Utility/ModuleSetupCode.c 2020-07-08 23:44:39.000000000 +0200 @@ -532,7 +532,11 @@ #define __Pyx_PyUnicode_DATA(u) PyUnicode_DATA(u) #define __Pyx_PyUnicode_READ(k, d, i) PyUnicode_READ(k, d, i) #define __Pyx_PyUnicode_WRITE(k, d, i, ch) PyUnicode_WRITE(k, d, i, ch) + #if defined(PyUnicode_IS_READY) && defined(PyUnicode_GET_SIZE) #define __Pyx_PyUnicode_IS_TRUE(u) (0 != (likely(PyUnicode_IS_READY(u)) ? PyUnicode_GET_LENGTH(u) : PyUnicode_GET_SIZE(u))) + #else + #define __Pyx_PyUnicode_IS_TRUE(u) (0 != PyUnicode_GET_LENGTH(u)) + #endif #else #define CYTHON_PEP393_ENABLED 0 #define PyUnicode_1BYTE_KIND 1 @@ -663,7 +667,7 @@ #endif #if PY_MAJOR_VERSION >= 3 - #define __Pyx_PyMethod_New(func, self, klass) ((self) ? PyMethod_New(func, self) : (Py_INCREF(func), func)) + #define __Pyx_PyMethod_New(func, self, klass) ((self) ? ((void)(klass), PyMethod_New(func, self)) : __Pyx_NewRef(func)) #else #define __Pyx_PyMethod_New(func, self, klass) PyMethod_New(func, self, klass) #endif diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/Cython/Utility/StringTools.c new/Cython-0.29.21/Cython/Utility/StringTools.c --- old/Cython-0.29.20/Cython/Utility/StringTools.c 2020-05-19 10:45:14.000000000 +0200 +++ new/Cython-0.29.21/Cython/Utility/StringTools.c 2020-07-08 23:44:39.000000000 +0200 @@ -66,6 +66,8 @@ //////////////////// PyUCS4InUnicode //////////////////// +#if PY_VERSION_HEX < 0x03090000 +#if Py_UNICODE_SIZE == 2 static int __Pyx_PyUnicodeBufferContainsUCS4_SP(Py_UNICODE* buffer, Py_ssize_t length, Py_UCS4 character) { /* handle surrogate pairs for Py_UNICODE buffers in 16bit Unicode builds */ Py_UNICODE high_val, low_val; @@ -77,6 +79,7 @@ } return 0; } +#endif static int __Pyx_PyUnicodeBufferContainsUCS4_BMP(Py_UNICODE* buffer, Py_ssize_t length, Py_UCS4 character) { Py_UNICODE uchar; @@ -87,6 +90,7 @@ } return 0; } +#endif static CYTHON_INLINE int __Pyx_UnicodeContainsUCS4(PyObject* unicode, Py_UCS4 character) { #if CYTHON_PEP393_ENABLED @@ -100,19 +104,26 @@ } return 0; } +#elif PY_VERSION_HEX >= 0x03090000 + #error Cannot use "UChar in Unicode" in Python 3.9 without PEP-393 unicode strings. #endif - if (Py_UNICODE_SIZE == 2 && unlikely(character > 65535)) { +#if PY_VERSION_HEX < 0x03090000 +#if Py_UNICODE_SIZE == 2 + if (unlikely(character > 65535)) { return __Pyx_PyUnicodeBufferContainsUCS4_SP( PyUnicode_AS_UNICODE(unicode), PyUnicode_GET_SIZE(unicode), character); - } else { + } else +#endif + { return __Pyx_PyUnicodeBufferContainsUCS4_BMP( PyUnicode_AS_UNICODE(unicode), PyUnicode_GET_SIZE(unicode), character); } +#endif } @@ -443,6 +454,7 @@ /////////////// decode_c_string /////////////// //@requires: IncludeStringH //@requires: decode_c_string_utf16 +//@substitute: naming /* duplicate code to avoid calling strlen() if start >= 0 and stop >= 0 */ static CYTHON_INLINE PyObject* __Pyx_decode_c_string( @@ -467,7 +479,7 @@ stop += length; } if (unlikely(stop <= start)) - return PyUnicode_FromUnicode(NULL, 0); + return __Pyx_NewRef($empty_unicode); length = stop - start; cstring += start; if (decode_func) { @@ -486,6 +498,7 @@ /////////////// decode_c_bytes /////////////// //@requires: decode_c_string_utf16 +//@substitute: naming static CYTHON_INLINE PyObject* __Pyx_decode_c_bytes( const char* cstring, Py_ssize_t length, Py_ssize_t start, Py_ssize_t stop, @@ -503,7 +516,7 @@ if (stop > length) stop = length; if (unlikely(stop <= start)) - return PyUnicode_FromUnicode(NULL, 0); + return __Pyx_NewRef($empty_unicode); length = stop - start; cstring += start; if (decode_func) { @@ -543,6 +556,7 @@ PyObject* text, Py_ssize_t start, Py_ssize_t stop); /////////////// PyUnicode_Substring /////////////// +//@substitute: naming static CYTHON_INLINE PyObject* __Pyx_PyUnicode_Substring( PyObject* text, Py_ssize_t start, Py_ssize_t stop) { @@ -559,7 +573,7 @@ else if (stop > length) stop = length; if (stop <= start) - return PyUnicode_FromUnicode(NULL, 0); + return __Pyx_NewRef($empty_unicode); #if CYTHON_PEP393_ENABLED return PyUnicode_FromKindAndData(PyUnicode_KIND(text), PyUnicode_1BYTE_DATA(text) + start*PyUnicode_KIND(text), stop-start); diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/PKG-INFO new/Cython-0.29.21/PKG-INFO --- old/Cython-0.29.20/PKG-INFO 2020-06-10 15:13:46.000000000 +0200 +++ new/Cython-0.29.21/PKG-INFO 2020-07-08 23:45:37.000000000 +0200 @@ -1,6 +1,6 @@ Metadata-Version: 1.2 Name: Cython -Version: 0.29.20 +Version: 0.29.21 Summary: The Cython compiler for writing C extensions for the Python language. Home-page: http://cython.org/ Author: Robert Bradshaw, Stefan Behnel, Dag Seljebotn, Greg Ewing, et al. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/runtests.py new/Cython-0.29.21/runtests.py --- old/Cython-0.29.20/runtests.py 2020-06-10 15:13:18.000000000 +0200 +++ new/Cython-0.29.21/runtests.py 2020-07-08 23:44:39.000000000 +0200 @@ -1747,6 +1747,8 @@ out.append(_out) err.append(_err) res = p.returncode + if res == 0 and b'REFNANNY: ' in _out: + res = -1 if res != 0: for c, o, e in zip(cmd, out, err): sys.stderr.write("%s\n%s\n%s\n\n" % ( diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/tests/compile/buildenv.pyx new/Cython-0.29.21/tests/compile/buildenv.pyx --- old/Cython-0.29.20/tests/compile/buildenv.pyx 2020-05-19 10:45:14.000000000 +0200 +++ new/Cython-0.29.21/tests/compile/buildenv.pyx 2020-07-08 23:44:39.000000000 +0200 @@ -121,4 +121,12 @@ CFLAGS (env) = {get_env('CFLAGS', '')} LINKCC (distutils) = {config_var('LINKCC')} LINKCC (env) = {get_env('LINKCC', '')} + +Encodings: +sys maxunicode = {sys.maxunicode} +LANG (env) = {get_env('LANG', '')} +PYTHONIOENCODING (env) = {get_env('PYTHONIOENCODING', '')} +sys stdout encoding = {sys.stdout.encoding} +sys default encoding = {sys.getdefaultencoding()} +sys FS encoding = {sys.getfilesystemencoding()} """) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/tests/compile/tryfinally.pyx new/Cython-0.29.21/tests/compile/tryfinally.pyx --- old/Cython-0.29.20/tests/compile/tryfinally.pyx 2015-06-22 14:53:11.000000000 +0200 +++ new/Cython-0.29.21/tests/compile/tryfinally.pyx 2020-07-08 23:44:39.000000000 +0200 @@ -18,4 +18,11 @@ finally: i = 42 +def use_name_in_finally(name): + # GH3712 + try: + [] + finally: + name() + diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/tests/memoryview/relaxed_strides.pyx new/Cython-0.29.21/tests/memoryview/relaxed_strides.pyx --- old/Cython-0.29.20/tests/memoryview/relaxed_strides.pyx 2020-05-19 10:45:14.000000000 +0200 +++ new/Cython-0.29.21/tests/memoryview/relaxed_strides.pyx 2020-07-08 23:44:39.000000000 +0200 @@ -37,6 +37,13 @@ np.ones((10, 1), order="C").flags.f_contiguous) +def not_py26(f): + import sys + if sys.version_info < (2, 7): + return lambda a: None + return f + + def test_one_sized(array): """ >>> contig = np.ascontiguousarray(np.arange(10, dtype=np.double)[::100]) @@ -62,3 +69,26 @@ """ cdef double[::1] a = array return a + +def test_zero_sized_multidim_ccontig(array): + """ + >>> contig = np.ascontiguousarray(np.zeros((4,4,4))[::2, 2:2, ::2]) + >>> _ = test_zero_sized_multidim_ccontig(contig) + + >>> a = np.zeros((4,4,4))[::2, 2:2, ::2] + >>> if NUMPY_HAS_RELAXED_STRIDES: _ = test_zero_sized_multidim_ccontig(a) + """ + cdef double[:, :, ::1] a = array + return a + +@not_py26 +def test_zero_sized_multidim_fcontig(array): + """ + >>> contig = np.ascontiguousarray(np.zeros((4,4,4))[::2, 2:2, ::2]) + >>> _ = test_zero_sized_multidim_fcontig(contig) + + >>> a = np.zeros((4,4,4))[::2, 2:2, ::2] + >>> if NUMPY_HAS_RELAXED_STRIDES: _ = test_zero_sized_multidim_fcontig(a) + """ + cdef double[::1, :, :] a = array + return a diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/tests/run/cpp_classes.pyx new/Cython-0.29.21/tests/run/cpp_classes.pyx --- old/Cython-0.29.20/tests/run/cpp_classes.pyx 2018-09-22 16:18:56.000000000 +0200 +++ new/Cython-0.29.21/tests/run/cpp_classes.pyx 2020-07-08 23:44:39.000000000 +0200 @@ -212,6 +212,7 @@ cdef int f(int x): return x + def test_nested_del(): """ >>> test_nested_del() @@ -220,3 +221,15 @@ v.push_back(new vector[int]()) del v[0] del create_to_delete()[f(f(0))] + + +def test_nested_del_repeat(): + """ + >>> test_nested_del_repeat() + """ + cdef vector[vector_int_ptr] v + v.push_back(new vector[int]()) + del v[0] + del create_to_delete()[f(f(0))] + del create_to_delete()[f(f(0))] + del create_to_delete()[f(f(0))] diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/tests/run/cpp_stl_cpp11.pyx new/Cython-0.29.21/tests/run/cpp_stl_cpp11.pyx --- old/Cython-0.29.20/tests/run/cpp_stl_cpp11.pyx 2020-04-19 11:26:03.000000000 +0200 +++ new/Cython-0.29.21/tests/run/cpp_stl_cpp11.pyx 2020-07-08 23:44:39.000000000 +0200 @@ -140,6 +140,11 @@ return "pass" +cdef extern from "cpp_unordered_map_helper.h": + cdef cppclass IntVectorHash: + pass + + def test_unordered_map_functionality(): """ >>> test_unordered_map_functionality() @@ -153,6 +158,8 @@ unordered_map[int, int] int_map2 unordered_map[int, int*] intptr_map const int* intptr + unordered_map[vector[int], int, IntVectorHash] int_vector_map + vector[int] intvec assert int_map[1] == 2 assert int_map.size() == 1 assert int_map.erase(1) == 1 # returns number of elements erased @@ -183,4 +190,12 @@ intptr_map[0] = NULL intptr = intptr_map.const_at(0) + + intvec = [1, 2] + int_vector_map[intvec] = 3 + intvec = [4, 5] + int_vector_map[intvec] = 6 + assert int_vector_map[intvec] == 6 + intvec = [1, 2] + assert int_vector_map[intvec] == 3 return "pass" diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/tests/run/cpp_unordered_map_helper.h new/Cython-0.29.21/tests/run/cpp_unordered_map_helper.h --- old/Cython-0.29.20/tests/run/cpp_unordered_map_helper.h 1970-01-01 01:00:00.000000000 +0100 +++ new/Cython-0.29.21/tests/run/cpp_unordered_map_helper.h 2020-07-08 23:44:39.000000000 +0200 @@ -0,0 +1,13 @@ +#include <functional> +#include <vector> + +struct IntVectorHash { + size_t operator()(const std::vector<int>& v) const { + std::hash<int> hasher; + size_t seed = 0; + for (int i : v) { + seed ^= hasher(i) + 0x9e3779b9 + (seed << 6) + (seed >> 2); + } + return seed; + } +}; diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/tests/run/exectest.pyx new/Cython-0.29.21/tests/run/exectest.pyx --- old/Cython-0.29.20/tests/run/exectest.pyx 2015-06-22 14:53:11.000000000 +0200 +++ new/Cython-0.29.21/tests/run/exectest.pyx 2020-07-08 23:44:39.000000000 +0200 @@ -145,3 +145,18 @@ TypeError: exec: arg 1 must be string, bytes or code object, got int """ exec x in {} + + +def exec_with_new_features(s, d): + """ + >>> import sys + >>> pyversion = sys.version_info[:2] + + >>> d = {} + >>> exec_with_new_features('print(123)', d) + 123 + >>> if pyversion == (2, 7): exec_with_new_features('exec "123"', d) + >>> if pyversion >= (3, 6): exec_with_new_features('f = f"abc"', d) + >>> if pyversion >= (3, 8): exec_with_new_features('a = (b := 1)', d) + """ + exec s in d diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/Cython-0.29.20/tests/run/unicodeliterals.pyx new/Cython-0.29.21/tests/run/unicodeliterals.pyx --- old/Cython-0.29.20/tests/run/unicodeliterals.pyx 2020-05-19 10:45:14.000000000 +0200 +++ new/Cython-0.29.21/tests/run/unicodeliterals.pyx 2020-07-08 23:44:39.000000000 +0200 @@ -21,6 +21,13 @@ u'\udc00' >>> h u'\ud800' + >>> q + u'\udc00\ud800' + + # The output of surrogate pairs differs between 16/32bit Unicode runtimes. + #>>> p + #u'\ud800\udc00' + >>> add u'S\xf8k ik\xfc\xd6\xe4abc' >>> null @@ -44,6 +51,10 @@ 1 >>> len(h) 1 + >>> len(q) + 2 + >>> len(q) + 2 >>> len(add) 12 >>> len(null) @@ -75,11 +86,15 @@ True >>> h == u'\\ud800' # unescaped by Python (required by doctest) True - >>> k == u'\\N{SNOWMAN}' == u'\\u2603' + >>> p == (u'\\ud800\\udc00' if sys.maxunicode == 1114111 else u'\\U00010000') or p # unescaped by Python (required by doctest) + True + >>> q == u'\\udc00\\ud800' or q # unescaped by Python (required by doctest) + True + >>> k == u'\\N{SNOWMAN}' == u'\\u2603' or k True - >>> m == u'abc\\\\xf8\\\\t\\u00f8\\U000000f8' # unescaped by Python (required by doctest) + >>> m == u'abc\\\\xf8\\\\t\\u00f8\\U000000f8' or m # unescaped by Python (required by doctest) True - >>> add == u'Søk ik' + u'üÖä' + 'abc' + >>> add == u'Søk ik' + u'üÖä' + 'abc' or add True >>> null == u'\\x00' # unescaped by Python (required by doctest) True @@ -115,6 +130,8 @@ h = u'\ud800' # lone lead surrogate k = u'\N{SNOWMAN}' m = ur'abc\xf8\t\u00f8\U000000f8' +p = u'\ud800\udc00' # surrogate pair +q = u'\udc00\ud800' # reversed surrogate pair add = u'Søk ik' + u'üÖä' + u'abc' null = u'\x00'
