[C++-sig] Nested Boost::Python dictionaries
I'm trying to create a boost::python::dict that stores another
boost::python::dict.
int main()
{
Py_Initialize();
boost::python::dict parent;
try{
parent["child_dict"] =
boost::make_shared(boost::python::dict());
}
catch(...){
PyErr_Print();
}
return 0;
}
TypeError: No to_python (by-value) converter found for C++ type: class
boost::shared_ptr
I was under the impression that boost::shared_ptr had special
treatment in boost::python(that is the library already knew what to do
with it). I'm not quite sure if I'm supposed to write this to_python
converter or if I'm supposed to achieve this in a different manner. If
I do write the to_python converter what's the method of keeping track
of both reference counts?
___
Cplusplus-sig mailing list
[email protected]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] Nested Boost::Python dictionaries
On 05/12/2012 12:22 PM, DarkAnt wrote:
I'm trying to create a boost::python::dict that stores another
boost::python::dict.
int main()
{
Py_Initialize();
boost::python::dict parent;
try{
parent["child_dict"] =
boost::make_shared(boost::python::dict());
}
catch(...){
PyErr_Print();
}
return 0;
}
TypeError: No to_python (by-value) converter found for C++ type: class
boost::shared_ptr
I was under the impression that boost::shared_ptr had special
treatment in boost::python(that is the library already knew what to do
with it). I'm not quite sure if I'm supposed to write this to_python
converter or if I'm supposed to achieve this in a different manner. If
I do write the to_python converter what's the method of keeping track
of both reference counts?
There's no need to use shared_ptr on Python objects; those will be
tracked using Python's own reference counting system.
Just doing
parent["child_dict"] = boost::python::dict();
will do what you want.
Note that boost::python::dict (as well as boost::python::object, etc.)
is actually a smart pointer itself, holding a PyObject* and
incrementing/decrementing the reference count in the C++ copy
constructor, assignment operator, and destructor.
HTH!
Jim
___
Cplusplus-sig mailing list
[email protected]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] Nested Boost::Python dictionaries
Thanks Jim!
On Sat, May 12, 2012 at 4:03 PM, Jim Bosch wrote:
> On 05/12/2012 12:22 PM, DarkAnt wrote:
>>
>> I'm trying to create a boost::python::dict that stores another
>> boost::python::dict.
>>
>> int main()
>> {
>> Py_Initialize();
>> boost::python::dict parent;
>> try{
>> parent["child_dict"] =
>> boost::make_shared(boost::python::dict());
>> }
>> catch(...){
>> PyErr_Print();
>> }
>> return 0;
>> }
>>
>> TypeError: No to_python (by-value) converter found for C++ type: class
>> boost::shared_ptr
>>
>> I was under the impression that boost::shared_ptr had special
>> treatment in boost::python(that is the library already knew what to do
>> with it). I'm not quite sure if I'm supposed to write this to_python
>> converter or if I'm supposed to achieve this in a different manner. If
>> I do write the to_python converter what's the method of keeping track
>> of both reference counts?
>
>
> There's no need to use shared_ptr on Python objects; those will be tracked
> using Python's own reference counting system.
>
> Just doing
>
> parent["child_dict"] = boost::python::dict();
>
> will do what you want.
>
> Note that boost::python::dict (as well as boost::python::object, etc.) is
> actually a smart pointer itself, holding a PyObject* and
> incrementing/decrementing the reference count in the C++ copy constructor,
> assignment operator, and destructor.
>
> HTH!
>
> Jim
> ___
> Cplusplus-sig mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/cplusplus-sig
___
Cplusplus-sig mailing list
[email protected]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] Extensive memory errors upon C++ destruction and data transfer from numpy
Well, I'm afraid you're jumping into a number of thorny bushes at once here, trying to make C++ code talk to NumPy through Boost.Python while learning C++. So I'll start with a few principles: 1) Never call a destructor yourself, in C++ or in Python. That should always happen automatically when an object goes out of scope unless you're doing really advanced things (and you'll know when that is when you learn about those advanced things). Doing this is probably the source of essentially all your problems. However, "del obj" isn't really a direct call to a destructor in Python; it just removes one reference to a variable, which might call the destructor if that's the last reference Python knows about - so you can do that if you like, but it's usually unnecessary because the same thing happens automatically when a variable goes out of scope or is reassigned. 2) shared_ptr is great, but I'm worried you might be using it more than you need to, especially enable_shared_from_this. Sometimes enable_shared_from_this is necessary, but often it's a sign you could have designed something better. Because Python does its own reference counting, you'll have an easier time figuring out what's going on with your memory if you only have one kind of reference counter floating around your code. 3) You seem to be using pointers as arguments for C++ functions more often that I'd consider necessary. And pointer arguments to numeric types will not work well with Python, because numbers are immutable in Python. For instance, all those ints in register_new_gene should probably be passed by value if you want them to work sensibly with Python. 4) numeric_array is a little old and limited, but those limitations may have saved you from creating even bigger memory headaches for you, because as far as I can tell there's no way to create NumPy arrays to C++ memory from it, or extract C++ arrays from a numeric::array object. That means you have to copy everything, which is inefficient, but it also means you don't have to worry about aliasing and ownership of array memory, which I think probably the right choice for right now. Finally, I'd really recommend making this work in plain C++, even in a limited fashion, before throwing it into Python. The Python/C++ barrier is tricky even with the best tools, and as a C++ beginner you'll have a much easier time tracking down memory issues using plain C++ and a debugger and/or a tool like valgrind. I haven't looked closely at your code yet, but I'm hoping the above will give you enough to work on that it will be substantially different before I need to anyhow :) HTH! Jim On 05/12/2012 04:10 PM, Christopher Cowing-Zitron wrote: Hello, I am writing a computational package integrating Python and C++ using Boost.Python. The interface and the initial data collection are in Python, but the workhorse data structures and computation are in C++. I am having difficulties with memory errors at two points: at the transfer of data between numpy and C++, and upon the destruction of wrapped C++ objects. I suspect the issues are related, and are not the result of limitations of Python, C++, or Boost.Python, but rather are the result of my lack of familiarity with C++. I apologize for the long email, but my lack of experience with C++ makes it hard to describe my problems succinctly, especially because I have no idea where the problems are occurring, nor do I know the right vocabulary. I've pasted below the C++ source for the relevant class, as well as the Boost.Python module wrapper code. This is all for academic research, and under the legal conditions of my grants it is necessarily open-source, but because I intend to publish this package in a research journal, I'd appreciate it if you didn't share the code beyond this mailing list; I'll create pypi and sourceforge entries for the source once it's ready for release. Most importantly, feel free to talk trash about and point out any errors in my code: I just started with C++ three weeks ago, and criticism is very helpful and much appreciated! :-) My package is for the analysis of large biological data sets, and a new C++ object must be created for each gene analyzed, which for the human genome amounts to about 30,000 times for the analysis of a single data set. A single gene object stores many variables, some of which are scalar and could be built into the class as static variables, but most variables are array data which greatly vary in size (and possibly dimension) depending upon the biological structure of the individual gene as well as the nature of the experiment being analyze, which cannot be known in advance. As such, all the array variables are dynamically allocated. I have used Boost.Python shared pointers and shared arrays for all dynamically allocated variables. In an attempt at uniformity, in this version of the code, all gene class variables are dynamically allocated. I have separated t
Re: [C++-sig] How can I make myself usefull?
On 05/08/2012 02:14 AM, Matthew Scouten wrote: Hello, I have recently been laid off from a job where a made a lot of good use of Boost.Python. I would like to give something back (while incidentally keeping my skills from getting rusty). I have already been following the boost-python tag on stackoverflow and answering a lot of question there (http://stackoverflow.com/users/8508/matthew-scouten). I haven't hacked on the internals of BP much, but now that I have time. While I was there, I had some code (now lost to me) that made BP easier to use. Here is some of that I had: * A deepcopyable suite, so that any c++ class with an appropriate copy ctor could be quickly given a __deepcopy__, a __copy__ and a copying __init__, with a single line. * A similar compare suite, so that classes with == and< could be given a full set of comparison operators, with a single line * 2 function templates, SafePointer2Object and SafeObject2Pointer which dealt with conversions between bp::objects that might be None and pointers to c++ classes that might be NULL * Simple RAII objects that dealt with acquiring and freeing the GIL around callbacks on different threads. A similar one for freeing the GIL around a code block. An idea that maybe this could be a call policy. * No_compare_indexing_suite is vector_indexing_suite for classes without == I would like to recreate some of these up for inclusion in BP, if you are interested. Or if there is other work that needs doing All of these sound very useful, and I'd love to see any of them in Boost.Python (well, to be honest, I don't use threading much, so I don't have much need for the GIL tools, but I know others on this list seem to be GIL-wrangling all the time). Another thing to consider is C++11 support, and particularly support for the C++11 standard library. I don't know think any of the original library builders or maintainers has any plans to devote any of their own time to add support for e.g. std::tuple or even std::shared_ptr and unique_ptr, but that's going to be increasingly important as C++11 compiler support gets better. Being able to wrap C++ functions that take rvalue reference arguments would be another important C++11 feature to support, but possibly a lot harder. Thanks! Jim ___ Cplusplus-sig mailing list [email protected] http://mail.python.org/mailman/listinfo/cplusplus-sig
