Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-12-07 Thread Ian Thomas
Yes, an excellent summary and neatly bringing us back to the crux of the matter. For completeness I should say that I wouldn't use SWIG. I used it about 5 years ago to wrap some C++ for Python and other languages. Initially it was very useful, but eventually the default mapping between C++ and

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-12-06 Thread Michael Droettboom
I think this has been a very helpful and useful discussion. I'm going to attempt to summarize this discussion and propose some ways forward here. The impetus for this discussion is that PyCXX seems to be not adequately maintained. It is difficult to build matplotlib with vanilla PyCXX in

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-12-06 Thread Michael Droettboom
I think this has been a very helpful and useful discussion. I'm going to attempt to summarize this discussion and propose some ways forward here. The impetus for this discussion is that PyCXX seems to be not adequately maintained. It is difficult to build matplotlib with vanilla PyCXX in

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-12-06 Thread Nicolas Rougier
Just for completeness, there is also ctypes. I wrapped the freetype library (http://code.google.com/p/freetype-py/) using it and it is quite easy (and boring). But this only works for C (not C++). Nicolas On Dec 6, 2012, at 18:06 , Michael Droettboom wrote: I think this has been a very

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-12-06 Thread Eric Firing
On 2012/12/06 7:16 AM, Michael Droettboom wrote: I think this has been a very helpful and useful discussion. I'm going to attempt to summarize this discussion and propose some ways forward here. The impetus for this discussion is that PyCXX seems to be not adequately maintained. It is

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-12-04 Thread Michael Droettboom
On 12/03/2012 07:00 PM, Chris Barker - NOAA Federal wrote: On Mon, Dec 3, 2012 at 12:24 PM, Chris Barker - NOAA Federal chris.bar...@noaa.gov wrote: but some of that complexity could be reduced by using Numpy arrays in place It would at least make this a more fair comparison to have the

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-12-04 Thread Michael Droettboom
On 12/03/2012 07:16 PM, Nathaniel Smith wrote: On Mon, Dec 3, 2012 at 11:50 PM, Chris Barker - NOAA Federal chris.bar...@noaa.gov wrote: On Mon, Dec 3, 2012 at 2:21 PM, Nathaniel Smith n...@pobox.com wrote: For the file handle, I would just write cdef FILE *fp = fdopen(file_obj.fileno(),

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-12-04 Thread Michael Droettboom
On 12/03/2012 08:01 PM, Chris Barker - NOAA Federal wrote: On Mon, Dec 3, 2012 at 4:16 PM, Nathaniel Smith n...@pobox.com wrote: Yeah, this is a general problem with the Python file API, trying to hook it up to stdio is not at all an easy thing. A better version of this code would skip that

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-12-04 Thread Michael Droettboom
Also -- this feedback is really helpful when writing some comments in the wrappers as to why certain things are the way they are... I'll make sure to include rationales for raw file fast path and the need to open the files on the Python side. Mike On 12/04/2012 08:45 AM, Michael Droettboom

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-12-04 Thread Damon McDougall
On Mon, Dec 3, 2012 at 12:12 PM, Chris Barker - NOAA Federal chris.bar...@noaa.gov wrote: generated code is ugly and hard to maintain, it is not designed to be human-readable, and we wouldn't get the advantages of bug-fixes further development in Cython. As far as I'm concerned, this is an

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-12-04 Thread Ryan May
On Tue, Dec 4, 2012 at 4:07 PM, Damon McDougall damon.mcdoug...@gmail.comwrote: On Mon, Dec 3, 2012 at 12:12 PM, Chris Barker - NOAA Federal chris.bar...@noaa.gov wrote: generated code is ugly and hard to maintain, it is not designed to be human-readable, and we wouldn't get the advantages

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-12-04 Thread Eric Firing
On 2012/12/04 12:07 PM, Damon McDougall wrote: On Mon, Dec 3, 2012 at 12:12 PM, Chris Barker - NOAA Federal chris.bar...@noaa.gov wrote: generated code is ugly and hard to maintain, it is not designed to be human-readable, and we wouldn't get the advantages of bug-fixes further development in

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-12-03 Thread Ian Thomas
I vote for using the raw Python/C API. I've written a couple of PyCXX extensions and whilst it is mostly convenient, PyCXX doesn't support the use of numpy arrays so for them you have to use the Python/C API. This means dealing with the reference counting yourself for numpy arrays; extending

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-12-03 Thread Michael Droettboom
On 12/03/2012 04:07 AM, Ian Thomas wrote: I vote for using the raw Python/C API. I've written a couple of PyCXX extensions and whilst it is mostly convenient, PyCXX doesn't support the use of numpy arrays so for them you have to use the Python/C API. This means dealing with the reference

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-12-03 Thread Chris Barker - NOAA Federal
On Sat, Dec 1, 2012 at 6:44 AM, Michiel de Hoon Since the Python/C glue code is modified only very rarely, there may not be a need for regenerating the Python/C glue code by developers or users from a Cython source code. True. In addition, it is much easier to maintain the Python/C glue

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-12-03 Thread Michael Droettboom
On 12/03/2012 01:12 PM, Chris Barker - NOAA Federal wrote: This argues against making the Cython source code a part of the matplotlib codebase. huh? are you suggesting that we use Cython to generate the glue, then hand-maintain that glue? I think that is a really, rally bad idea --

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-12-03 Thread Chris Barker - NOAA Federal
On Mon, Dec 3, 2012 at 11:59 AM, Michael Droettboom md...@stsci.edu wrote: but some of that complexity could be reduced by using Numpy arrays in place of the image buffer types that each of them contain OR Cython arrays and/or memoryviews -- this is indeed a real strength of Cython.

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-12-03 Thread Nathaniel Smith
On Mon, Dec 3, 2012 at 8:24 PM, Chris Barker - NOAA Federal chris.bar...@noaa.gov wrote: On Mon, Dec 3, 2012 at 11:59 AM, Michael Droettboom md...@stsci.edu wrote: so there are types in libpng, for example, that we don't actually know the size of. They are different on different platforms.

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-12-03 Thread Chris Barker - NOAA Federal
On Mon, Dec 3, 2012 at 2:21 PM, Nathaniel Smith n...@pobox.com wrote: For the file handle, I would just write cdef FILE *fp = fdopen(file_obj.fileno(), w) and be done with it. This will work with any version of Python etc. yeah, that makes sense -- though what if you want to be able to

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-12-03 Thread Chris Barker - NOAA Federal
On Mon, Dec 3, 2012 at 12:24 PM, Chris Barker - NOAA Federal chris.bar...@noaa.gov wrote: but some of that complexity could be reduced by using Numpy arrays in place It would at least make this a more fair comparison to have the Cython code as Cythonic as possible. However, I couldn't find

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-12-03 Thread Chris Barker - NOAA Federal
On Mon, Dec 3, 2012 at 4:16 PM, Nathaniel Smith n...@pobox.com wrote: Yeah, this is a general problem with the Python file API, trying to hook it up to stdio is not at all an easy thing. A better version of this code would skip that altogether like: cdef void write_to_pyfile(png_structp s,

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-12-03 Thread Eric Firing
On 2012/12/03 4:54 AM, Michael Droettboom wrote: I think Cython is well suited to writing new algorithmic code to speed up hot spots in Python code. I don't think it's as well suited as glue between C and Python -- that was not a main goal of the original Pyrex project, IIRC. It feels kind

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-12-01 Thread Nelle Varoquaux
One package (Pysam) that I use a lot relies on Cython, and requires users to install Cython before they can install Pysam itself. With Cython, is that always the case? Will all users need to install Cython? Or is it sufficient if only matplotlib developers install Cython? You can set

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-12-01 Thread Julian Taylor
On 12/01/2012 02:32 AM, Benjamin Root wrote: Since when has numpy used Cython? I specifically remember a rather involved discussion thread on numpy-discussion about the pros-and-cons of including cython. Now, SciPy on the other hand, does utilize Cython in some spots IIRC, but does it in

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-12-01 Thread Thomas Kluyver
Drat, re-sending on the list. On 1 December 2012 16:40, Thomas Kluyver tho...@kluyver.me.uk wrote: On 1 December 2012 14:44, Michiel de Hoon mjldeh...@yahoo.com wrote: At the same time, to minimize errors, we could use Cython to create the initial Python/C glue code, and then add the

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-12-01 Thread Michael Droettboom
For the PNG extension specifically, it was creating callbacks that can be called from C and the setjmp magic that libpng requires. I think it's possible to do it, but I was surprised at how non-obvious those pieces of Cython were. I was really hoping by creating this experiment that a Cython

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-12-01 Thread Michael Droettboom
For point of comparison, my branch now has a Cython and C++ version of the same thing. Here's the Cython version: https://github.com/mdboom/matplotlib/blob/no_cxx/src/_png.pyx Here's the C++ version: https://github.com/mdboom/matplotlib/blob/no_cxx/src/_png_wrap.cpp Some interesting things

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-11-30 Thread Jason Grout
On 11/29/12 10:59 AM, Michael Droettboom wrote: I've not had much luck with Cython for this kind of thing in the past, but I know it is popular. I'm curious about what problems you've run into and how long it was. In the past, Cython hasn't supported C++ very well, but the situation has

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-11-30 Thread Chris Barker - NOAA Federal
On Fri, Nov 30, 2012 at 6:06 AM, Michael Droettboom md...@stsci.edu wrote: If you read between the lines of what I was saying, that is basically where I fall as well. There seems to be a lot of desire to use Cython to make the code more accessible, I'll add a beat to that drum -- I'm a big

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-11-30 Thread Michiel de Hoon
. --- On Fri, 11/30/12, Chris Barker - NOAA Federal chris.bar...@noaa.gov wrote: From: Chris Barker - NOAA Federal chris.bar...@noaa.gov Subject: Re: [matplotlib-devel] Experiments in removing/replacing PyCXX To: Michael Droettboom md...@stsci.edu Cc: Michiel de Hoon mjldeh...@yahoo.com, matplotlib

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-11-30 Thread Nathaniel Smith
On Fri, Nov 30, 2012 at 11:40 PM, Michiel de Hoon mjldeh...@yahoo.com wrote: One package (Pysam) that I use a lot relies on Cython, and requires users to install Cython before they can install Pysam itself. With Cython, is that always the case? Will all users need to install Cython? Or is it

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-11-30 Thread Benjamin Root
On Fri, Nov 30, 2012 at 6:44 PM, Nathaniel Smith n...@pobox.com wrote: On Fri, Nov 30, 2012 at 11:40 PM, Michiel de Hoon mjldeh...@yahoo.com wrote: One package (Pysam) that I use a lot relies on Cython, and requires users to install Cython before they can install Pysam itself. With Cython,

[matplotlib-devel] Experiments in removing/replacing PyCXX

2012-11-29 Thread Michael Droettboom
Given the slow pace of development on PyCXX, I know it has been the desire of some here to remove our dependency on it. I thought a helpful starting point to evaluate the alternatives would be to restructure one of our extensions to not use PyCXX anymore. I've taken the PNG extension, which

Re: [matplotlib-devel] Experiments in removing/replacing PyCXX

2012-11-29 Thread Michiel de Hoon
in removing/replacing PyCXX To: matplotlib-devel@lists.sourceforge.net matplotlib-devel@lists.sourceforge.net Date: Thursday, November 29, 2012, 11:59 AM Given the slow pace of development on PyCXX, I know it has been the desire of some here to remove our dependency on it. I thought