On Wed, 20 Mar 2013 19:28:35 -0000, Jeffrey Van Voorst <vanv0...@umn.edu>
wrote:
In short, the boost smart pointers, boost spirit, etc. implementations
vary depending on the compile time options. I don't know if
boost.python has options.
Hi again,
So I've just rebuilt everything to be compatible with Python's
'--with-pydebug' configure option. Was quite a bit of work, as a debug
version of libpython is not ABI compatible with a normal build, so I also
had to rebuild libboost_python and gdb..
I've got those all working from /usr/local now, and am getting much more
informative backtraces, but I'm still none the wiser as to why this
problem arises.
I also overlooked the fact that Base class constructors are always called
in the initialiser list, whether there is one specified or not. As the
base class' constructors go on to lock the mutex, I made a few changes to
release and reacquire the GIL around the base class constructor. Code
demonstrating this is below...
-----------------
--- Backtrace ---
Hopefully this might make sense to someone here.. Apologies for the
inherent bulk of posting backtraces and code..
$ /usr/local/bin/gdb -q --args python2-dbg -c 'import gc;
gc.set_debug(gc.DEBUG_LEAK); import mylib; env =
Environment(["foo=bar"]); print "done" '
... (no output from my program, only gdb stuff) ...
Program received signal SIGSEGV, Segmentation fault.
boost::python::detail::construct_pointee<boost::python::list,
boost::python::detail::borrowed_reference_t* const>
(storage=storage@entry=0x7fffffffd000, x=@0x7fffffffcff8: 0x2) at
/usr/include/boost/python/detail/construct.hpp:19
19 new (storage) T(x);
(gdb) bt
#0 boost::python::detail::construct_pointee<boost::python::list,
boost::python::detail::borrowed_reference_t* const>
(storage=storage@entry=0x7fffffffd000, x=@0x7fffffffcff8: 0x2) at
/usr/include/boost/python/detail/construct.hpp:19
#1 0x00007ffff4b23e1e in
boost::python::detail::construct_referent_impl<boost::python::list const,
boost::python::detail::borrowed_reference_t* const>
(storage=storage@entry=0x7fffffffd000, x=@0x7fffffffcff8: 0x2)
at /usr/include/boost/python/detail/construct.hpp:25
#2 0x00007ffff4b23e2c in
boost::python::detail::construct_referent<boost::python::list const&,
boost::python::detail::borrowed_reference_t*>
(storage=storage@entry=0x7fffffffd000, x=@0x7fffffffcff8: 0x2,
tag=tag@entry=0x0)
at /usr/include/boost/python/detail/construct.hpp:31
#3 0x00007ffff4b23e6c in object_manager_ref_arg_from_python (x=<optimized
out>, this=0x7fffffffd000) at
/usr/include/boost/python/converter/obj_mgr_arg_from_python.hpp:86
#4 arg_from_python (source=<optimized out>, this=0x7fffffffd000) at
/usr/include/boost/python/arg_from_python.hpp:70
#5 operator() (args_=<optimized out>, this=0x8d0078) at
/usr/include/boost/preprocessor/iteration/detail/local.hpp:37
#6
boost::python::objects::caller_py_function_impl<boost::python::detail::caller<void
(*)(_object*, boost::python::list const&),
boost::python::default_call_policies, boost::mpl::vector3<void, _object*,
boost::python::list const&> > >::operator() (
this=0x8d0070, args=<optimized out>, kw=<optimized out>) at
/usr/include/boost/python/object/py_function.hpp:38
#7 0x00007ffff010dacb in operator() (kw=0x0, args=<optimized out>,
this=0x8d00b0) at ./boost/python/object/py_function.hpp:143
#8 boost::python::objects::function::call (this=0x8d0090, args=0x8048d0,
keywords=0x0) at libs/python/src/object/function.cpp:226
#9 0x00007ffff010de38 in operator() (this=<optimized out>) at
libs/python/src/object/function.cpp:585
#10
boost::detail::function::void_function_ref_invoker0<boost::python::objects::{anonymous}::bind_return,
void>::invoke(boost::detail::function::function_buffer &)
(function_obj_ptr=...) at ./boost/function/function_template.hpp:188
#11 0x00007ffff0118f23 in operator() (this=<optimized out>) at
./boost/function/function_template.hpp:767
... lots of exception handlers ...
#97 0x00007ffff010ba65 in
handle_exception<boost::python::objects::{anonymous}::bind_return> (f=...)
at ./boost/python/errors.hpp:29
#98 boost::python::objects::function_call (func=<optimized out>,
args=<optimized out>, kw=<optimized out>) at
libs/python/src/object/function.cpp:626
#99 0x00007ffff7a36978 in PyObject_Call (func=func@entry=0x8d0090,
arg=arg@entry=0x8048d0, kw=kw@entry=0x0) at Objects/abstract.c:2529
#100 0x00007ffff7a455ed in instancemethod_call (func=0x8d0090,
arg=0x8048d0, kw=0x0) at Objects/classobject.c:2578
#101 0x00007ffff7a36978 in PyObject_Call (func=func@entry=0x8144e0,
arg=arg@entry=0xa79060, kw=kw@entry=0x0) at Objects/abstract.c:2529
#102 0x00007ffff7a9bf06 in slot_tp_init (self=<optimized out>,
args=0xa79060, kwds=0x0) at Objects/typeobject.c:5663
#103 0x00007ffff7a9bb4e in type_call (type=<optimized out>, args=0xa79060,
kwds=0x0) at Objects/typeobject.c:737
#104 0x00007ffff7a36978 in PyObject_Call (func=func@entry=0x8cfa70,
arg=arg@entry=0xa79060, kw=kw@entry=0x0) at Objects/abstract.c:2529
#105 0x00007ffff7aec3a8 in do_call (nk=0, na=1, pp_stack=0x7fffffffd8d8,
func=0x8cfa70) at Python/ceval.c:4239
#106 call_function (oparg=<optimized out>, pp_stack=0x7fffffffd8d8) at
Python/ceval.c:4044
#107 PyEval_EvalFrameEx (f=f@entry=0x7214b0, throwflag=throwflag@entry=0)
at Python/ceval.c:2666
#108 0x00007ffff7aef539 in PyEval_EvalCodeEx (co=co@entry=0x94b880,
globals=globals@entry=0x684770, locals=locals@entry=0x684770,
args=args@entry=0x0, argcount=argcount@entry=0, kws=kws@entry=0x0,
kwcount=kwcount@entry=0, defs=defs@entry=0x0,
defcount=defcount@entry=0, closure=closure@entry=0x0) at
Python/ceval.c:3253
#109 0x00007ffff7aef631 in PyEval_EvalCode (co=co@entry=0x94b880,
globals=globals@entry=0x684770, locals=locals@entry=0x684770) at
Python/ceval.c:667
#110 0x00007ffff7b0cf62 in run_mod (mod=<optimized out>,
filename=filename@entry=0x7ffff7b38836 "<string>",
globals=globals@entry=0x684770, locals=locals@entry=0x684770,
flags=flags@entry=0x7fffffffdb20, arena=arena@entry=0x66f540) at
Python/pythonrun.c:1353
#111 0x00007ffff7b0dc11 in PyRun_StringFlags (str=str@entry=0x602010
"import gc; gc.set_debug(gc.DEBUG_UNCOLLECTABLE); import mylib; env =
mylib.Environment([\"fo=bar\"]); print \"done\" \n",
start=start@entry=257, globals=0x684770,
locals=0x684770, flags=flags@entry=0x7fffffffdb20) at
Python/pythonrun.c:1316
#112 0x00007ffff7b0e730 in PyRun_SimpleStringFlags
(command=command@entry=0x602010 "import gc;
gc.set_debug(gc.DEBUG_UNCOLLECTABLE); import mylib; env =
mylib.options.Environment([\"fo=bar\"]); print \"done\" \n",
flags=flags@entry=0x7fffffffdb20) at Python/pythonrun.c:969
#113 0x00007ffff7b20e45 in Py_Main (argc=3, argv=0x7fffffffdce8) at
Modules/main.c:583
#114 0x0000000000400805 in main (argc=<optimized out>, argv=<optimized
out>) at ./Modules/python.c:23
-----------------------
-- My wrapper class ---
// in pyenv.hpp
class PyEnv
: public library::Environment
{
public:
PyEnv(void);
/// Construct from a list or a dict.
PyEnv(const boost::python::list& envp);
// Constructor helpers to release the GIL before calling
base-constructor
InitPyEnv(const boost::python::list& envp);
// ...
private:
MakeThreadSafe * m_gil_state;
};
// in pyenv.cpp
PyEnv::PyEnv(void)
: Environment(InitPyEnv())
{
delete m_gil_state;
}
PyEnv::PyEnv(const boost::python::list& envp)
: Environment( InitPyEnv(envp) )
{
delete m_gil_state;
}
// Constructor initialiser list helpers
const char* const* InitPyEnv(void)
{
m_gil_state = new MakeThreadSafe();
return environ; // declared as 'extern char**', somewhere
}
const char* const* InitPyEnv(const boost::python::list& envp)
{
int envc = boost::python::len(envp);
const char* const* penvp = ConvertPyListToCharArrayPointer(envc, envp);
m_gilstate = new MakeThreadsafe();
return penvp;
}
// other member functions ...
void RegisterEnv(void)
{
boost::python::class_<PyEnv, boost::noncopyable>
("Environment", boost::python::init<>() )
.def("boost::python::init<const boost::python::list&>() )
// ...
;
}
// converters.hpp
const char* const* ConvertPyListToCharArrayPointer(
int argc, const boost::python::list& list)
{
const char** newarr = new const char*[argc+1];
boost::python::stl_input_iterator<std::string> begin(list), end;
int i=0;
while (begin != end)
{
newarr[i++] = (*begin).c_str();
begin++;
}
newarr[i] = '\0';
const char* const* parr = &newarr[0];
delete[] newarr;
return parr;
}
----------------------
--- Interpretation ---
Having dug around in the debugger a bit, ('print *this' from branch-point
#6), the seg-fault seems to arise when registering the following class
initialiser:-
.def("boost::python::init<const boost::python::list&>() )
I thought this might mean there's an issue with using a const reference to
a derived type of boost::python::object in 'init<const bp::list&>', but
there are still problems if I comment out those '.def' lines, and instead
use the default constructor..
-------------------------------------
--- Default constructor backtrace ---
$ /usr/local/bin/gdb -q --args python2-dbg -c 'import gc;
gc.set_debug(gc.DEBUG_UNCOLLECTABLE); import mylib; env =
mylib.Environment(); print "done"'
...
done
Debug memory block at address p=0x9a89a0: API 'm'
88 bytes originally requested
The 7 pad bytes at p-7 are FORBIDDENBYTE, as expected.
The 8 pad bytes at tail=0x9a89f8 are not all FORBIDDENBYTE (0xfb):
at tail+0: 0xff *** OUCH
at tail+1: 0xff *** OUCH
at tail+2: 0xff *** OUCH
at tail+3: 0xff *** OUCH
at tail+4: 0x00 *** OUCH
at tail+5: 0x00 *** OUCH
at tail+6: 0x00 *** OUCH
at tail+7: 0x00 *** OUCH
The block was made by call #0 to debug malloc/realloc.
Data at p: 70 92 33 f0 ff 7f 00 00 ... 00 00 00 00 00 00 00 00
Fatal Python error: bad trailing pad byte
Program received signal SIGABRT, Aborted.
0x00007ffff74582c5 in raise () from /usr/lib/libc.so.6
(gdb) bt
#0 0x00007ffff74582c5 in raise () from /usr/lib/libc.so.6
#1 0x00007ffff7459748 in abort () from /usr/lib/libc.so.6
#2 0x00007ffff7b0df60 in Py_FatalError (msg=msg@entry=0x7ffff7b45371 "bad
trailing pad byte") at Python/pythonrun.c:1677
#3 0x00007ffff7a7f4de in _PyObject_DebugCheckAddressApi
(api=api@entry=109 'm', p=p@entry=0x9a89a0) at Objects/obmalloc.c:1591
#4 0x00007ffff7a7f50d in _PyObject_DebugFreeApi (api=api@entry=109 'm',
p=0x9a89a0) at Objects/obmalloc.c:1478
#5 0x00007ffff7a7f55c in _PyMem_DebugFree (p=<optimized out>) at
Objects/obmalloc.c:1405
#6 0x00007ffff7a7e6ed in PyMem_Free (p=<optimized out>) at
Objects/object.c:2336
#7 0x00007ffff010b365 in boost::python::instance_holder::deallocate
(self_=self_@entry=0x9a74a0, storage=<optimized out>) at
libs/python/src/object/class.cpp:760
#8 0x00007ffff010b3b1 in boost::python::objects::instance_dealloc
(inst=0x9a74a0) at libs/python/src/object/class.cpp:336
#9 0x00007ffff7a9a642 in subtype_dealloc (self=self@entry=0x9a74a0) at
Objects/typeobject.c:1014
#10 0x00007ffff7a7b732 in _Py_Dealloc (op=0x9a74a0) at
Objects/object.c:2243
#11 0x00007ffff7a7366f in insertdict (mp=mp@entry=0x684760,
key=key@entry=0x7a6fb8, hash=-3051142320843333194,
value=value@entry=0x7ffff7da4fa0 <_Py_NoneStruct>) at
Objects/dictobject.c:530
#12 0x00007ffff7a76675 in PyDict_SetItem (op=op@entry=0x684760,
key=0x7a6fb8, value=0x7ffff7da4fa0 <_Py_NoneStruct>) at
Objects/dictobject.c:775
#13 0x00007ffff7a7a8fa in _PyModule_Clear (m=<optimized out>) at
Objects/moduleobject.c:138
#14 0x00007ffff7b01111 in PyImport_Cleanup () at Python/import.c:445
#15 0x00007ffff7b0f2bc in Py_Finalize () at Python/pythonrun.c:454
#16 0x00007ffff7b2125c in Py_Main (argc=<optimized out>, argv=<optimized
out>) at Modules/main.c:664
#17 0x0000000000400805 in main (argc=<optimized out>, argv=<optimized
out>) at ./Modules/python.c:23
------------------------
--- Interpretation 2 ---
The program runs to completion and seg-faults when trying to delete the
'env' instance from the global namespace. I haven't been able to figure
out what attribute of the Environment class is corrupted, but something
surely is.
I was thinking I should keep around the 'const char* const*' variable for
the lifetime of the Environment instance, destroying it in the destructor,
but am not really sure how to allocate such a variable, be it with one or
more calls to 'new' or 'malloc'...
Any help here much appreciated! If there's anything else that'd be useful
to know, please do ask!
Cheers,
Alex
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig
