Re: [C++-sig] pybind11 -- alternative to Boost.Python

2015-10-19 Thread Axel Huebl 
Wow the docs and examples look great!
Thank you for the tremendous amount of work you put in!

I am eager to test this with CUDA7+/C++11 programs.

Best,
Axel

On October 18, 2015 3:56:42 PM EEST, Wenzel Jakob  wrote:
>Hello all,
>
>after being a long-time Boost.Python user, I’ve been working on an
>alternative that makes more effective use of recent C++11-capable
>compilers. The overall syntax and ideology are very similar to
>Boost.Python, but the implementation only requires a few header files
>with a a vastly smaller amount of code thanks to C++11 lambda
>functions, tuples and variadic templates. There is also dedicated
>support for Python’s buffer protocol and NumPy arrays, which is useful
>for scientific computing applications.
>
>So far it’s only used by a few projects, but I think it could be useful
>to this audience.
>
>Code: https://github.com/wjakob/pybind11
>
>Documentation: http://pybind11.readthedocs.org/en/latest/
>
>
>Best,
>Wenzel
>
>
>
>___
>Cplusplus-sig mailing list
>[email protected]
>https://mail.python.org/mailman/listinfo/cplusplus-sig

___
Cplusplus-sig mailing list
[email protected]
https://mail.python.org/mailman/listinfo/cplusplus-sig

Re: [C++-sig] Returning C++ buffer to Python 2.7

2015-10-19 Thread Trigve Siver via Cplusplus-sig 
>
> From: Tony Cappellini 
>To: [email protected] 
>Sent: Thursday, October 15, 2015 10:27 PM
>Subject: [C++-sig] Returning C++ buffer to Python 2.7
> 
>
>
>
>
>I've allocated memory in C++ and need to expose a function to Python
>so that Python can read this memory as a sequence.
>
>
>
>I've found this example
>
>
>http://pastebin.com/YGi61R4H
>
>
>
>from this URL
>http://stackoverflow.com/questions/16232520/how-to-expose-raw-byte-buffers-with-boostpython
>
>
>
>which **looks** like it will do what I need, from the description of the code 
>and the request from the original author.
>
>
>However, it looks incomplete to me. There is no code which defines, or 
>explains what
>handle<> is/does, in this line of code

>

See 
http://www.boost.org/doc/libs/1_55_0/libs/python/doc/tutorial/doc/html/python/object.html#python.creating_python_objectfor
 explanation of handle<>
___
Cplusplus-sig mailing list
[email protected]
https://mail.python.org/mailman/listinfo/cplusplus-sig

Re: [C++-sig] pybind11 -- alternative to Boost.Python

2015-10-19 Thread Trigve Siver via Cplusplus-sig 
>
> From: Wenzel Jakob 
>To: [email protected] 
>Sent: Sunday, October 18, 2015 2:56 PM
>Subject: [C++-sig] pybind11 -- alternative to Boost.Python
> 
>
>
>Hello all,
>
>
>after being a long-time Boost.Python user, I’ve been working on an alternative 
>that makes more effective use of recent C++11-capable compilers. The overall 
>syntax and ideology are very similar to Boost.Python, but the implementation 
>only requires a few header files with a a vastly smaller amount of code thanks 
>to C++11 lambda functions, tuples and variadic templates. There is also 
>dedicated support for Python’s buffer protocol and NumPy arrays, which is 
>useful for scientific computing applications.
>
>
>So far it’s only used by a few projects, but I think it could be useful to 
>this audience.
>
>
>Code: https://github.com/wjakob/pybind11
>Documentation: http://pybind11.readthedocs.org/en/latest/


It looks good.
Have you tried contacting the actual boost.python mantainer and maybe propose 
merge with the boost.python or make a boost.python3 from it? It would be shame 
not incorporate the useful stuff in boost.python.

>
>Best,
>Wenzel
>___
>Cplusplus-sig mailing list
>[email protected]
>https://mail.python.org/mailman/listinfo/cplusplus-sig
>

>



--
Trigve
___
Cplusplus-sig mailing list
[email protected]
https://mail.python.org/mailman/listinfo/cplusplus-sig

Re: [C++-sig] pybind11 -- alternative to Boost.Python

2015-10-19 Thread Wenzel Jakob 
I would be open to it but have my doubts about the feasibility of a merge. 
Consider the difference in code size alone: Boost.Python (without dependencies 
like MPL etc.) uses 26K lines of code, compared to about 2K for pybind11 (3K 
with all extensions). Apart from that, the libraries take very different 
internal design decisions, which would likely break existing software that 
ventures beyond the basic .def() syntax. 

Cheers,
Wenzel


> On Oct 19, 2015, at 11:24 AM, Trigve Siver via Cplusplus-sig 
>  wrote:
> 
>> 
>> From: Wenzel Jakob 
>> To: [email protected] 
>> Sent: Sunday, October 18, 2015 2:56 PM
>> Subject: [C++-sig] pybind11 -- alternative to Boost.Python
>> 
> It looks good.
> Have you tried contacting the actual boost.python mantainer and maybe propose 
> merge with the boost.python or make a boost.python3 from it? It would be 
> shame not incorporate the useful stuff in boost.python.
> 

___
Cplusplus-sig mailing list
[email protected]
https://mail.python.org/mailman/listinfo/cplusplus-sig

Re: [C++-sig] pybind11 -- alternative to Boost.Python

2015-10-19 Thread Stefan Seefeld 
On 19.10.2015 06:24, Wenzel Jakob wrote:
> I would be open to it but have my doubts about the feasibility of a
> merge. Consider the difference in code size alone: Boost.Python
> (without dependencies like MPL etc.) uses 26K lines of code, compared
> to about 2K for pybind11 (3K with all extensions). Apart from that,
> the libraries take very different internal design decisions, which
> would likely break existing software that ventures beyond the basic
> .def() syntax.

Hi Wenzel,

Indeed, I would be very interested in a detailed comparison. Modernizing
Boost.Python by using C++11 features (for example) and stripping out
obsolete compiler support is one way to move forward. That in itself
will help a lot, and may even allow Boost.Python to strip off
dependencies to other Boost libraries.

I'd be curious to learn about those design decisions that you are
alluding to that lead to incompatibilities. Such a document may
ultimately also be important for potential users when they consider the
alternatives.

Regards,
Stefan


-- 

  ...ich hab' noch einen Koffer in Berlin...

___
Cplusplus-sig mailing list
[email protected]
https://mail.python.org/mailman/listinfo/cplusplus-sig

Re: [C++-sig] pybind11 -- alternative to Boost.Python

2015-10-19 Thread Jim Bosch 
At first glance, this looks great.  I've been tinkering with something like
this for a while now, but it's never amounted to anything more than a C++11
learning project, and this looks quite solid in comparison.

It may be a long time before I get a chance to evaluate pybind11 for use in
my own projects, but as a prelude to that I was wondering if you could say
anything about support for custom converters and cross-module type
conversion, which I didn't see mentioned in the docs (though I just skimmed
them).  Are you using the same sort of global registry Boost.Python used?
If so, I'm curious how that works with a header-only library.

Jim




On Sun, Oct 18, 2015 at 9:56 PM, Wenzel Jakob  wrote:

> Hello all,
>
> after being a long-time Boost.Python user, I’ve been working on an
> alternative that makes more effective use of recent C++11-capable
> compilers. The overall syntax and ideology are very similar to
> Boost.Python, but the implementation only requires a few header files with
> a a vastly smaller amount of code thanks to C++11 lambda functions, tuples
> and variadic templates. There is also dedicated support for Python’s buffer
> protocol and NumPy arrays, which is useful for scientific computing
> applications.
>
> So far it’s only used by a few projects, but I think it could be useful to
> this audience.
>
> Code: https://github.com/wjakob/pybind11
> Documentation: http://pybind11.readthedocs.org/en/latest/
>
> Best,
> Wenzel
>
> ___
> Cplusplus-sig mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/cplusplus-sig
>
___
Cplusplus-sig mailing list
[email protected]
https://mail.python.org/mailman/listinfo/cplusplus-sig

Re: [C++-sig] pybind11 -- alternative to Boost.Python

2015-10-19 Thread Wenzel Jakob 
Hi,

it would take a long time to discuss all differences, but I can give some 
examples. There are basically three ways of interfacing with Python objects in 
pybind11.

1. using wrapper classes like pybind11::object (analogous to 
boost::python::object)
2. by creating bindings that map a C++ type to Python — this is done using 
pybind11::class_ (analogous to boost::python::class_)
3. by declaring a partial template overload that does transparent conversions 
between different types.

Boost.Python’s approach for communicating type information (item 2. in the 
above list) between modules entails linking against a shared library with a few 
containers storing the relevant data. In comparison, pybind11 installs a  
__pybind11__ capsule object in the global scope for this purpose, which avoids 
the library dependency. Any extra binding library that is loaded just registers 
its types there.

In terms of the underlying implementation, 1. and 2. are pretty basic, and 3. 
is where a lot of the interesting things happen. This is basically a big list 
of partial template overloads of a class named type_caster which try to match 
various common types recursively. I’ll show just one example of how C++11 can 
considerably simplify implementation details here.

For instance, consider the converter which enables transparent conversions 
between std::tuple<…> and Python’s ‘tuple’ class. Among other things, pybind11 
uses this to convert function arguments to Python objects. The top-level 
signature matches an arbitrary tuple (that could even be nested, or other kinds 
of type concoctions … :)) I’ll expand the snippet literal programming-style, 
adding code to the <…> part.

template  class type_caster> {
typedef std::tuple type;
enum { size = sizeof...(Tuple) };

<…>
};

The first thing we’ll do is to declare sub-converters to deal with the 
individual tuple element types. The decay template simplifies the base type as 
much as possible by stripping type modifiers like pointers, references, const, 
etc. (those are handled separately)

<…> +=
std::tuple::type>...> value;

The following function takes a tuple from Python and converts it into the 
corresponding C++ object, returning false if the conversion wasn’t possible. It 
expects a special type index_sequence<0,1,2,3,…., N-1> as an argument, where N 
is the length of the tuple. This is a pretty common workaround to enable 
something resembling a loop over variadic template arguments rather than 
writing a messy recursive function.

<…> +=
protected:
template  bool load(PyObject *src, 
index_sequence) {
if (!PyTuple_Check(src))
return false;
if (PyTuple_Size(src) != size)
return false;
std::array results {{
(PyTuple_GET_ITEM(src, Indices) != nullptr ? 
std::get(value).load(PyTuple_GET_ITEM(src, Indices)) : false)...
}};
for (bool r : results)
if (!r)
return false;
return true;
}

The following function function calls the above protected function with the 
needed index_sequence

<…> +=
public:
bool load(PyObject *src) {
return load(src, typename 
make_index_sequence::type());
}

which is constructed using a much shorter recursive implementation that runs at 
compile time:

template struct index_sequence  { };
template struct make_index_sequence : 
make_index_sequence  { };
template struct make_index_sequence <0, S...> { typedef 
index_sequence type; };

Here is another very short example that I like: this converts a Python function 
into a std::function<> using a stateful lambda closure that invokes the 
function object’s call() function.
With this partial template overload, we can easily call functions that take 
std::function<>s as argument using Python functions. Something similar is also 
possible for the reverse direction.


template  struct 
type_caster> {
typedef std::function type;
public:

bool load(PyObject *src_) {
if (!PyFunction_Check(src_))
return false;
object src(src_, true);
value = [src](Args... args) -> Return {
object retval(handle(src).call(std::move(args)...));
return retval.template cast();
};
return true;
}


   <…>
protected:
   type value;
}.

The codebase contains many other examples. For instance, the optional 
auto-vectorization support over NumPy array arguments is something that would 
have been very painful to do with C++03.

Best,
Wenzel

> On Oct 19, 2015, at 2:15 PM, Jim Bosch  wrote:
> 
> At first glance, this looks great.  I've been tinkering with something like 
> this for a while now, but it's never amounted to anything more than a C++11 
> learning project, and this looks quite solid in comparison.
> 
> It may be a long time before I get a chance to evaluate pybind11 for use in 
> my own projects, but as a prelude to that I was wondering if you could say 
> anything about support for custom conv

Re: [C++-sig] Returning C++ buffer to Python 2.7

2015-10-19 Thread Tony Cappellini 
>
> See
>
> http://www.boost.org/doc/libs/1_55_0/libs/python/doc/tutorial/doc/html/python/object.html#python.creating_python_objectfor
> explanation of handle<>
>

Thank You!
___
Cplusplus-sig mailing list
[email protected]
https://mail.python.org/mailman/listinfo/cplusplus-sig

Re: [C++-sig] pybind11 -- alternative to Boost.Python

2015-10-19 Thread Francesco Biscani 
This looks really neat!

Do you have any measure on the memory/cpu performance wrt Boost.Python when
compiling large bindings? I would expect that variadic templates and all
the other C++11 goodies are more efficient than the preprocessor magic used
in Boost Python, from this point of view.

Cheers,

  Francesco.

On 18 October 2015 at 14:56, Wenzel Jakob  wrote:

> Hello all,
>
> after being a long-time Boost.Python user, I’ve been working on an
> alternative that makes more effective use of recent C++11-capable
> compilers. The overall syntax and ideology are very similar to
> Boost.Python, but the implementation only requires a few header files with
> a a vastly smaller amount of code thanks to C++11 lambda functions, tuples
> and variadic templates. There is also dedicated support for Python’s buffer
> protocol and NumPy arrays, which is useful for scientific computing
> applications.
>
> So far it’s only used by a few projects, but I think it could be useful to
> this audience.
>
> Code: https://github.com/wjakob/pybind11
> Documentation: http://pybind11.readthedocs.org/en/latest/
>
> Best,
> Wenzel
>
> ___
> Cplusplus-sig mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/cplusplus-sig
>
___
Cplusplus-sig mailing list
[email protected]
https://mail.python.org/mailman/listinfo/cplusplus-sig

Re: [C++-sig] pybind11 -- alternative to Boost.Python

2015-10-19 Thread Václav Šmilauer 



Code: https://github.com/wjakob/pybind11
Documentation: http://pybind11.readthedocs.org/en/latest/

Hi Wenzel, it looks excelent.

There are a few points in boost::python which have been unresolved for a 
long time: straightforward support for aligned types (Eigen comes to 
mind) and some subtle bugs in shared_ptr/weak_ptr (e.g. not holding GIL 
when deleting some python objects and crashing) which are not being 
fixed upstream. Could you comment on those?


I would suggest that the documents include some migration hints from 
boost::python, if I if ever attempt it (something like: use 
reference_internal instead of with_custodian_and_ward, this is how you 
write an equivalent of raw_function/raw_constructor).


Cheers, Václav


___
Cplusplus-sig mailing list
[email protected]
https://mail.python.org/mailman/listinfo/cplusplus-sig

Re: [C++-sig] pybind11 -- alternative to Boost.Python

2015-10-19 Thread Wenzel Jakob 
Hi,

I became curious about this myself and ran a simple benchmark for automatically 
generated binding code of increasing size.

The compilation times for Boost.Python and pybind11 turn out to be fairly 
similar. However, there is a significant difference in terms of the size of the 
compilation result, which is almost twice as large for Boost.Python.

See the details here: http://pybind11.readthedocs.org/en/latest/benchmark.html 


Best,
Wenzel

> On Oct 19, 2015, at 9:58 PM, Francesco Biscani  wrote:
> 
> This looks really neat!
> 
> Do you have any measure on the memory/cpu performance wrt Boost.Python when 
> compiling large bindings? I would expect that variadic templates and all the 
> other C++11 goodies are more efficient than the preprocessor magic used in 
> Boost Python, from this point of view.
> 
> Cheers,
> 
>   Francesco.
> 
> On 18 October 2015 at 14:56, Wenzel Jakob  > wrote:
> Hello all,
> 
> after being a long-time Boost.Python user, I’ve been working on an 
> alternative that makes more effective use of recent C++11-capable compilers. 
> The overall syntax and ideology are very similar to Boost.Python, but the 
> implementation only requires a few header files with a a vastly smaller 
> amount of code thanks to C++11 lambda functions, tuples and variadic 
> templates. There is also dedicated support for Python’s buffer protocol and 
> NumPy arrays, which is useful for scientific computing applications.
> 
> So far it’s only used by a few projects, but I think it could be useful to 
> this audience.
> 
> Code: https://github.com/wjakob/pybind11 
> Documentation: http://pybind11.readthedocs.org/en/latest/ 
> 
> 
> Best,
> Wenzel
> 
> ___
> Cplusplus-sig mailing list
> [email protected] 
> https://mail.python.org/mailman/listinfo/cplusplus-sig 
> 
> 
> ___
> Cplusplus-sig mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/cplusplus-sig

___
Cplusplus-sig mailing list
[email protected]
https://mail.python.org/mailman/listinfo/cplusplus-sig