On 03/20/2013 10:46 AM, Alex Leach wrote:
Dear list,
I've started using Boost.Python to wrap a 3rd party C++ library, and
whilst running my unit tests, the Python interpreter segfault's during
garbage collection.
With explicit object deletion:
$ gdb -q --args python -c 'import mylib; obj = mylib.MyObj(); del(obj)'
...
*** Error in `/usr/bin/python': free(): invalid next size (fast):
0x0000000000806fd0 ***
======= Backtrace: =========
/usr/lib/libc.so.6(+0x7ab06)[0x7ffff74d3b06]
/usr/lib/libc.so.6(+0x7b883)[0x7ffff74d4883]
/usr/lib/libboost_python.so.1.53.0(_ZN5boost6python15instance_holder10deallocateEP7_objectPv+0x15)[0x7ffff004f555]
/usr/lib/libboost_python.so.1.53.0(+0x265a1)[0x7ffff004f5a1]
...
======= Memory map: ========
...
or, leaving it to the garbage collector:-
$ gdb -q --args python -c 'import mylib; obj = mylib.obj() '
...
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7b2b0b0 in visit_decref () from /usr/lib/libpython2.7.so.1.0
(gdb)
Quick question: What do I need to do to fix this?
It looks very much like you're deleting an object twice. I'd guess
that's probably because something went wrong with Boost.Python's
understanding of ownership, but that's just a guess. It might be due to
threading problems.
==========================
--- The wrapper class ---
The wrapper class I've written is fairly simple; I haven't explicitly
added a 'PyObject* _self' attribute, nor done any of the wiki
recommendations[3] for managing ownership, as I don't see anywhere in
the tutorial that recommends any of these methods.
If you are returning objects by pointer or [const] reference anywhere,
you should double-check your use of call policies, to ensure you've told
Python about ownership correctly. That'd be the first place I'd look
for a problem.
The tutorial is very incomplete, and should not be used to rule out more
advanced features you may need when wrapping more advanced libraries (I
wish I could point you at better documentation for those features, but
it doesn't really exist beyond what it looks like you've already found).
------------------------------------
--- Using boost::python::wrapper ---
The wrapped class (in 3rd party library) does have one virtual method
(that is protected and I haven't exposed to Python) and a virtual
destructor. I gather the right thing to do with virtual classes is to
also inherit from 'boost::python::wrapper<Base>', but this doesn't
affect the seg-fault. I guess I don't need to inherit from
wrapper<Base>, as I am not exposing any virtual methods, nor using
'this->get_override(...)' anywhere in the class definition.
If you do not need to override virtual methods in Python, there's no
need at all to use wrapper. In that case, you're better off without it,
as this will speed up your method calls and avoid complexity that could
help hide your problem.
Another very similar class is in the same source file though (virtual
destructor and one unexposed virtual method), but this one has a public
copy constructor. If I inherit from 'wrapper<Base>', I get a compilation
error regarding an argument mismatch when Boost.Python passes the
derived class to the copy constructor of the base class. Of course I can
add noncopyable, to the class_<..> registration, but I'd like the
exposed class to stay copyable, and I might like to use some of the
wrapper template class' functionality. Should I wrap the copy
constructor in the wrapper class too? Would it be possible to add this
functionality to the wrapper<> template, when the class is not noncopyable?
I don't think this could be added to the wrapper<> template in C++98, as
I think it would require inheriting a constructor. It may not even be
possible in C++11, because it may need to add a PyObject* argument; I
don't exactly recall how constructors for wrapper<> work. In any case,
it sounds like you don't need wrapper<> at all, so I'd remove it, and
that should make this a non-issue.
------------------------------------
--- Calling thread-safe methods ---
I don't think this is directly relevant, but the classes in question use
the 3rd party library's own mutex implementations to ensure
thread-safety. I've just got these working by writing a simple utility
class that releases the GIL in the constructor and reacquires it in its
destructor, using the C-Python API PyGILState_* functions[1].
The general tactic was described on the boost.python wiki[2], but the
PyEval_(Save|Restore)Thread functions described caused the same
assertion errors that I was getting earlier, with the pthreads mutex.
This happened when the 3rd party library tried to lock the global mutex.
Although this is mentioned in the C-Python manual, I would have found it
useful if thread-safety was specifically mentioned in the tutorial or
reference manual somewhere... I attach the solution I wrote to this
problem, in case someone would want to add it and document it in
Boost.Python somewhere.
http://www.boost.org/doc/libs/1_53_0/libs/python/doc/v2/faq.html#threadsupport
There are other people on this list who know a lot more about this than
I do, but my understanding has always been that it you use Boost.Python
with threading on the C++ side, all bets are off, unless the Python
interaction is strictly limited to one thread.
Jim
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig
