Thanks for the careful analysis. See
https://github.com/cython/cython/commit/e8b54dec4cdc94764abda4a107bc37dcd58e6f13
On Tue, Sep 10, 2013 at 3:26 PM, Dénes Vadász <pyth...@dvadasz.hu> wrote:
> Hello,
>
> according to valgrind the following Cython fragment causes a heap corruption
> (with Python 2.7.5 and Cython 0.19.1 on Ubuntu):
>
> cdef class MetaClass(type):
> cdef int i
>
> class MyClass(object):
> __metaclass__ = MetaClass
>
>
>
> Please find below the result of a many hour investigation (originally
> triggered by random crashes and heap corruptions of a bigger Cython program
> that uses metaclasses).
>
> MetaClass is compiled to a C structure that extends PyTypeObject (a cc. 200
> byte structure on a 32-bit architecture):
>
> struct __pyx_obj_4meta_MetaClass {
> PyTypeObject __pyx_base;
> int i;
> };
>
>
> Instances of PyTypeObject are supposed to be allocated statically when
> initializing extension modules, because the structure does not support
> (among others) garbage collection. However, when MyClass is created, an
> instance of struct __pyx_obj_4meta_MetaClass (cc. 200 + 4 bytes) is
> dynamically allocated by the Python memory management machinery. The
> machinery then tries to initialize the allocated memory. The problem is that
> it expects that the first member of struct __pyx_obj_4meta_MetaClass is of
> type PyHeapTypeObject (a cc. 500 byte structure) and after a type cast it
> writes to the tail members of the assumed PyHeapTypeObject, i.e. way beyond
> the allocated cc. 200 + 4 bytes, corrupting the object heap. This corruption
> is nicely reported by valgrind.
>
> The proposed fix is to make __pyx_obj_4meta_MetaClass extend
> PyHeapTypeObject:
>
> struct __pyx_obj_4meta_MetaClass {
> PyHeapTypeObject __pyx_base;
> int i;
> };
>
>
> This can be achieved by adding the below 2 lines to Compiler/Builtin.py:
>
> 382a383,384
>> elif name == 'type':
>> objstruct_cname = 'PyHeapTypeObject'
>
>
> So that the init_builtin_types function becomes:
>
> def init_builtin_types():
> global builtin_types
> for name, cname, methods in builtin_types_table:
> utility = builtin_utility_code.get(name)
> if name == 'frozenset':
> objstruct_cname = 'PySetObject'
> elif name == 'bool':
> objstruct_cname = None
> elif name == 'type':
> objstruct_cname = 'PyHeapTypeObject'
> else:
> objstruct_cname = 'Py%sObject' % name.capitalize()
> the_type = builtin_scope.declare_builtin_type(name, cname, utility,
> objstruct_cname)
> builtin_types[name] = the_type
> for method in methods:
> method.declare_in_type(the_type)
>
>
> After patching my Cython installation with the above, valgrind stopped to
> complain and there were no more crashes.
>
> Please consider adding this fix in the next release of Cython.
>
> Regards
>
> Dénes Vadász
>
> _______________________________________________
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
>
_______________________________________________
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel
