I have gotten (3) to work, but I am wondering about two things. Since I
couldn't get the constructor to work, I went for a modifier that returns the
object. That allows me to do the following (where NewObject is a Python object
that derives from a C++ object):
class Factory:
def create( self ):
return NewObject().share()
The share function looks like this:
PyObject *share() {
PyObject *owner = boost::python::detail::wrapper_base_::get_owner( *this );
Py_INCREF(owner);
Py_INCREF(owner);
// Fix the C++ reference count
return owner;
}
So, two questions:
* Is this a legal/recommended way to do a modifier?
* If I only do a single INCREF, then the object is destroyed while
executing the NewObject().share() call. Why do I need two INCREFs?
Anyway, the unittests are passing, I now have intrusive pointers with shared
ownership between C++ and Python. Being forced to call the share method is a
little annoying, but it works. Thanks for making it possible.
-- Per.
From: cplusplus-sig-bounces+pknudsgaard=rgbnetworks....@python.org
[mailto:cplusplus-sig-bounces+pknudsgaard=rgbnetworks....@python.org] On Behalf
Of Per Knudsgaard
Sent: Friday, November 11, 2011 2:41 PM
To: Development of Python/C++ integration
Subject: Re: [C++-sig] [Boost.Python] How to adjust Python reference counts for
hybrid objects?
After playing around with this some more, I seem to be right in thinking
that this is a problem with Python reference count. I have tried 3 different
approaches to solving the problem:
1. My first thought was to get a hold of a PyObject *self pointer in the
constructor and incrementing its reference count forcing the Python object to
persist despite going out of scope in Python. I found two ways to do that,
described in [1] and [2]. They seem to fail for reasons described in the
answers to [2].
2. My second thought was to increment the reference count the moment the
object enters into C++. This stopped the crash, but if I instrument the
__del__ method in the Python object, it gets called before I increment the
reference count. I think that is a strong indication that this method left me
with memory corruption so I am hesitant to follow this path.
3. The third choice is that I implement a C++ method that marks the
object as "shared" and increments its reference count. I have to make an
explicit call to that method in the Python constructor and I check if the
object has been marked shared before passing it into C++. I have added the
appropriate smart pointer logic to maintain the extra reference count as long
as there are C++ pointers around.
The third option seems to work. Constructors and destructors are called in
reasonable ways indicating that objects are neither get deleted prematurely nor
do they stick around for too long. I just don't like it because I would prefer
the integration to be transparent, the person implementing the derived class in
python should not need to know that (s)he is deriving it from a C++ class.
Which returns me to the question of how one is supposed to handle this case?
My current solution is neither crashing nor leaking memory, but it isn't very
elegant. It kind of stands out against the elegance of Boost and Boost.Python.
Thanks,
-- Per.
[1]
http://wiki.python.org/moin/boost.python/HowTo#ownership_of_C.2B-.2B-_object_extended_in_Python
[2] http://mail.python.org/pipermail/cplusplus-sig/2007-March/011790.html
From:
cplusplus-sig-bounces+pknudsgaard=rgbnetworks....@python.org<mailto:cplusplus-sig-bounces+pknudsgaard=rgbnetworks....@python.org>
[mailto:cplusplus-sig-bounces+pknudsgaard=rgbnetworks....@python.org] On
Behalf Of Per Knudsgaard
Sent: Thursday, November 10, 2011 6:53 PM
To: cplusplus-sig@python.org<mailto:cplusplus-sig@python.org>
Subject: [C++-sig] [Boost.Python] How to adjust Python reference counts for
hybrid objects?
Hi,
I have run into a small problem with my understanding of Python reference
counts with Python/C++ hybrid objects. I have an abstract interface called A
and an associated factory interface called AFactory. I am looking to implement
various A+AFactory combinations in both C++ and Python. I am using boosts
intrusive smart pointers to make life a little more interesting.
I have attached two files that illustrate my problem (they are a little
long, sorry about that but I couldn't come up with a better example). On
linux, compile the cpp file with:
$ g++ foo.cpp -g -fPIC -shared -I/usr/include/python2.7 -lpython2.7
-lboost_python -o module.so
Run the py file with a single argument ('crash' to produce a segfault,
anything else to have it work).
What am I doing? The abstract factory interface looks roughly like this
(with most of the pointer stuff removed):
class AFactory {
public:
virtual A::Ptr newA() = 0;
};
A single function that returns an intrusive pointer to the A interface. I
have an AFactoryWrap class that will forward the call to a python
implementation. I derive it in Python as AFactoryDerived (where ADerived is
the python implementation of the A interface):
tmp = ADerived()
class AFactoryDerived( module.AFactory ):
def __init__( self ):
module.AFactory.__init__( self )
def newA( self ):
if( sys.argv[1] == 'crash' ):
return ADerived() # segfaults
else:
return tmp # works
I then trigger the whole thing from a simple C++ function (that I also call
from python, just to add to the fun):
void f2( const AFactory::Ptr &factory ) {
A::Ptr a = factory->newA();
a->f();
}
Get a new instance of A from the factory. Call a virtual function,
implemented in Python, on it that prints "In f" on stdout.
It works if the instance of ADerived is persistent (global variable) and
segfaults if it is temporary (local to the newA call). That makes me suspect
that the problem is with Python reference counts and that I somehow need to
maintain a reference from the c++ side so it doesn't get garbage collected.
The boost::python::handle<> class would seem like the way to go, except I can't
seem to find any decent examples of how to use it.
Am I on the right track? I would assume that the handle<> should be added
to the Wrapper constructors, and the ptrRelease function enhanced to count C++
and Python references, but the syntax of creating a handle and obtaining python
reference counts is escaping me.
Thanks,
-- Per.
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig
