[matplotlib-devel] OpenGL backend with Galry
Hi all, I am developing a high-performance interactive visualization package in Python based on PyOpenGL (http://rossant.github.com/galry/). It is primarily meant to be used as a framework for developing complex interactive GUIs (in QT) that deal with very large amounts of data (tens of millions of points). But it may also be used, like matplotlib, as a high-level interactive library to plot and visualize data. The low-level interface is mostly done at this point (the code is still in an experimental stage though), and I'm now focusing on my current research project which is to write a scientific GUI based on this interface. However, I think people (including myself!) may be interested in a matplotlib-like high-level interface. I was first thinking about writing such an interface from scratch, by implementing a very small fraction of the matplotlib interface (basic commands like figure(), plot(), subplot(), show(), etc.). One could then quickly visualize huge datasets with the same commands than matplotlib. Another solution would be to write a matplotlib backend based on this library. I am not familar enough with the internals of matplotlib to know how complicated it could be. I may do it myself, but it would probably take a long time since it is currently not my highest priority. I would be glad if someone experienced in writing backends was interested in working on it. Actually I could do everything that is specific to my library, which already provides commands to plot points, lines, textures, etc. The canvas is based on QT and may be similar to what is already implemented in the QT backend. Of course, it would already be great if only the most basic plotting features were available in the backend. A first step could be for example to have a simplistic example "plot(x, sin(x))" working (with interactive navigation). I am looking forward to your feedback. Best, Cyrille Rossant -- Monitor your physical, virtual and cloud infrastructure from a single web console. Get in-depth insight into apps, servers, databases, vmware, SAP, cloud infrastructure, etc. Download 30-day Free Trial. Pricing starts from $795 for 25 servers or applications! http://p.sf.net/sfu/zoho_dev2dev_nov___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] Interpolation in a triangular mesh (tri.Triangulation)
Hello Ian, Thank you for your second proposition ; I find it very interesting in fact. (But beware that, as I do not feel able to do this on my own, your code exemple / guidance would be needed for sure... ) I will have a look at http://matplotlib.org/devel/index.html I think your idea of having a separate TriFinder class is quite good. My search algo. is not optimised, only avoiding a O(N) for interpolations along a path or line. Some other parts of the code (especially the loop over the (x,y) points in __call__() ) may also be performance-critical, at least for the 'cubic' interpolator. But I dont have a clear picture on how to embed C++ code in python so I would need your example to figure out what is possible. Regards Geoffroy. 2012/11/14 Ian Thomas > Hi Geoffroy, > > I have had some time to look at your TriLinearInterpolator in some detail > (the other two files only briefly). I would indeed like to add something > like this to matplotlib - the mesh refinement looks very nice and the > interpolators would be useful to many people. > > As you suspected, the code does need significant changes before we can > include it. Some are merely cosmetic, as all code must adhere to PEP8 and > the matplotlib coding guidelines, but there are also some functional and > performance improvements. For example, your wavefront method for finding > the triangle containing a certain point must be able to deal with masked > triangulations and indeed triangulations that are discontinuous, for > example two islands in a masked-out ocean, which is unusual but must be > supported. In terms of performance, there is much explicit looping within > numpy arrays that could be improved using other numpy array commands, and > would also reduce the length of the source code. There is an argument for > some of the performance-critical code to be in C/C++. > > I think the code used to determine which triangle contains a certain point > should be factored out into its own TriFinder class, so that (1) it does > not need to be replicated in the two interpolator classes, and (2) > different algorithms can be easily swapped if necessary. I have a C++ > TriFinder class that I could modify to work within matplotlib, and it is > O(log N) so should be faster than your version for typical use cases. > > I expect that this is probably more work than you anticipated when you > asked if the code needed any improvement! I propose the following: if you > are happy to give matplotlib your source code as it stands and for us to > include it under our BSD-style license, then I will take on the > responsibility of getting it into a form that will be accepted by the other > developers. I will acknowledge your contribution in both the source code > and on the web site, something like "based on code contributed by Geoffroy > Billotey". > > Alternatively, if you would like to use this as an excuse to learn how to > contribute to matplotlib more actively but don't want to take on > everything, then we could divide up the work so that first I write my C++ > log(N) TriFinder class and the linear interpolator that uses it, and then > you could modify the cubic interpolator following the format of the linear > interpolator and using my guidance as and when you need it. > > Let me know your preference, > Ian > > P.S. Never apologise for not being a computer scientist! Many of our > developers, myself included, are proper scientists or engineers!!! > > > > On 29 October 2012 09:37, Ian Thomas wrote: > >> Hi Geoffroy >> >> This will certainly be very useful. I need to spend some time looking at >> it and seeing how it would best fit within the matplotlib framework, >> particularly as only a few days ago I committed to writing a triangular >> grid interpolator for quad grids and it would be sensible to group these >> interpolators together in some way. >> >> I'll get back to you when I've had time to look at it. >> >> Thanks for your efforts! >> Ian >> >> >> >> On 28 October 2012 20:17, GBillotey wrote: >> >>>Hi! >>> >>> >>> I had recently to develop interpolators for a function defined at the >>> nodes >>> of a user-specified triangular mesh. >>> (Beside interpolation, it can help producing higher-quality tricontour >>> plots, using interpolation on a refined mesh and matplotlib tricontour >>> function.) >>> >>> Being a regular user of matplotlib, I would be happy if it can be useful >>> to >>> others... >>> The code is hosted here: >>> https://github.com/GBillotey/trimesh-interpolator.git >>> >>> >>> Please let me know if it this dev. can be useful and if the code needs >>> some >>> cleaning (I am not a computer scientist, only a mechanical engineer) >>> >>> >>> Cheers, >>> Geoffroy. >>> >> > -- Monitor your physical, virtual and cloud infrastructure from a single web console. Get in-depth insight into apps, servers, databases, vmware, SAP, cloud infrastructure, etc.
Re: [matplotlib-devel] tick_params direction 'inout'
On Wed, Nov 14, 2012 at 7:37 PM, Mike Kaufman wrote: > Hi all, > > I don't have time to make a patch at the moment, but I thought I'd point > it out for anyone to give it a go... > > tick_params(axis='both', **kwargs) > Change the appearance of ticks and tick labels. > > Keyword arguments: > > ... > > *direction* : ['in' | 'out'] > Puts ticks inside or outside the axes. > > I just found that *direction* accepts 'inout' as well, which > does indeed place the tick on both sides of the spine. So the > documentation should be updated to reflect this. Thanks for the report, Mike, here's a PR for the patch: https://github.com/matplotlib/matplotlib/pull/1503 > If it were me, I'd allow 'both' to work as well. > I'm amenable to that, just not sure if that counts as a new feature and should go into master, or a bugfix and go into v1.2.x. I can add this functionality to #1503 if that makes sense to go to v1.2.x -- Paul Ivanov 314 address only used for lists, off-list direct email at: http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7 -- Monitor your physical, virtual and cloud infrastructure from a single web console. Get in-depth insight into apps, servers, databases, vmware, SAP, cloud infrastructure, etc. Download 30-day Free Trial. Pricing starts from $795 for 25 servers or applications! http://p.sf.net/sfu/zoho_dev2dev_nov___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] OpenGL backend with Galry
On Thu, Nov 15, 2012 at 2:24 PM, Cyrille Rossant wrote: > Hi all, > > I am developing a high-performance interactive visualization package in > Python based on PyOpenGL (http://rossant.github.com/galry/). It is > primarily meant to be used as a framework for developing complex > interactive GUIs (in QT) that deal with very large amounts of data (tens of > millions of points). But it may also be used, like matplotlib, as a > high-level interactive library to plot and visualize data. > > The low-level interface is mostly done at this point (the code is still in > an experimental stage though), and I'm now focusing on my current research > project which is to write a scientific GUI based on this interface. > However, I think people (including myself!) may be interested in a > matplotlib-like high-level interface. I was first thinking about writing > such an interface from scratch, by implementing a very small fraction of > the matplotlib interface (basic commands like figure(), plot(), subplot(), > show(), etc.). One could then quickly visualize huge datasets with the same > commands than matplotlib. > > Another solution would be to write a matplotlib backend based on this > library. I am not familar enough with the internals of matplotlib to know > how complicated it could be. I may do it myself, but it would probably take > a long time since it is currently not my highest priority. I would be glad > if someone experienced in writing backends was interested in working on it. > Actually I could do everything that is specific to my library, which > already provides commands to plot points, lines, textures, etc. The canvas > is based on QT and may be similar to what is already implemented in the QT > backend. > > Of course, it would already be great if only the most basic plotting > features were available in the backend. A first step could be for example > to have a simplistic example "plot(x, sin(x))" working (with interactive > navigation). > > I am looking forward to your feedback. > > Best, > Cyrille Rossant > > Great to hear another person interested in bringing opengl to matplotlib! Another project you might be interested in collaborating with is Glumpy: http://code.google.com/p/glumpy/ >From my limited knowledge of OpenGL, what my vision is that any of the existing backends have support for an OpenGL object, so we just need to be able to instantiate the opengl object in any figure object, and know how to send it the appropriate commands and data. So, it is not exactly a backend, more of a "middling". Anyway, I think the dev at Glumpy would be happy to have help, and probably have much more developed ideas on how to integrate with matplotlib. Cheers! Ben Root -- Monitor your physical, virtual and cloud infrastructure from a single web console. Get in-depth insight into apps, servers, databases, vmware, SAP, cloud infrastructure, etc. Download 30-day Free Trial. Pricing starts from $795 for 25 servers or applications! http://p.sf.net/sfu/zoho_dev2dev_nov___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] Interpolation in a triangular mesh (tri.Triangulation)
On Wed, Nov 14, 2012 at 1:50 AM, Ian Thomas wrote: > I think the code used to determine which triangle contains a certain point > should be factored out into its own TriFinder class, +1 -- this is a generally useful feature. In fact, it would be nice if a lot of this were in a pacakge that deals with triangular meshes, apart from MPL altogether (a scikit maybe?) > I have a C++ TriFinder class > that I could modify to work within matplotlib, and it is O(log N) so should > be faster than your version for typical use cases. What algorithm does this use? Is the code open source and/or availabel for other projects? I'm working on a package for working with unstructured grids in general, and also have a use for "what triangle is this point in" code for other purposes -- and I havne't found a fast, robust code for this yet. >> particularly as only a few days ago I committed to writing a triangular grid >> interpolator for quad grids what is a triangular interpolator for quad grids? sounds useful, too. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov -- Monitor your physical, virtual and cloud infrastructure from a single web console. Get in-depth insight into apps, servers, databases, vmware, SAP, cloud infrastructure, etc. Download 30-day Free Trial. Pricing starts from $795 for 25 servers or applications! http://p.sf.net/sfu/zoho_dev2dev_nov ___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] OpenGL backend with Galry
Yep, I'm still developing some OpenGL technics to provide both nice and fast rendering and I hope to be able to help the writing of a GL backend for matplotlib next summer (provided we get a GSoC student for the project). So far, my main concern is that for efficient rendering using OpenGL, you need to consider that drawing means to create objects on the graphic card (line, curve, image, points, etc) that can be later manipulated (scaling/rotating/coloring/properties change, etc.). From what I remember in my early attempts at writing an OpenGL backend, I did not find the proper way to enforce such framework. Said differently, the backend is supposed to implement drawing operations while I would need to know if the drawing operations actually relates to something that is already on the graphic card or not. I'm not sure I'm very clear but I can develop the point if necessary. Having read the post by Michael (http://mdboom.github.com/blog/2012/08/06/matplotlib-client-side/) on client-side rendering, I think the proposed three-way split might be a solution but I do not know how advanced are the ideas. To date, I've been working on different things: Text/font : http://code.google.com/p/freetype-gl/ (c code) Stroke/dash/paths: http://code.google.com/p/gl-agg/ (python) Images:http://code.google.com/p/glumpy/ (python) If you want to get a feel of how nice and fast rendering could be, have a look at 'demo-lines.py' from the gl-agg repository (and play with mouse). From these experiments, I think it is possible to achieve AGG quality using OpenGL. What is really exciting is the perspective of having a opengl/webgl backend that could be used with ipython (there has been a recent post on ipython list that show such integration for a molecule viewer). Anyway, you're more than welcome to contribute to glumpy, but in the long run, I hope it will disappear in favor of a matplolib GL backend. Nicolas On Nov 15, 2012, at 22:03 , Benjamin Root wrote: > > > On Thu, Nov 15, 2012 at 2:24 PM, Cyrille Rossant > wrote: > Hi all, > > I am developing a high-performance interactive visualization package in > Python based on PyOpenGL (http://rossant.github.com/galry/). It is primarily > meant to be used as a framework for developing complex interactive GUIs (in > QT) that deal with very large amounts of data (tens of millions of points). > But it may also be used, like matplotlib, as a high-level interactive library > to plot and visualize data. > > The low-level interface is mostly done at this point (the code is still in an > experimental stage though), and I'm now focusing on my current research > project which is to write a scientific GUI based on this interface. However, > I think people (including myself!) may be interested in a matplotlib-like > high-level interface. I was first thinking about writing such an interface > from scratch, by implementing a very small fraction of the matplotlib > interface (basic commands like figure(), plot(), subplot(), show(), etc.). > One could then quickly visualize huge datasets with the same commands than > matplotlib. > > Another solution would be to write a matplotlib backend based on this > library. I am not familar enough with the internals of matplotlib to know how > complicated it could be. I may do it myself, but it would probably take a > long time since it is currently not my highest priority. I would be glad if > someone experienced in writing backends was interested in working on it. > Actually I could do everything that is specific to my library, which already > provides commands to plot points, lines, textures, etc. The canvas is based > on QT and may be similar to what is already implemented in the QT backend. > > Of course, it would already be great if only the most basic plotting features > were available in the backend. A first step could be for example to have a > simplistic example "plot(x, sin(x))" working (with interactive navigation). > > I am looking forward to your feedback. > > Best, > Cyrille Rossant > > > Great to hear another person interested in bringing opengl to matplotlib! > Another project you might be interested in collaborating with is Glumpy: > http://code.google.com/p/glumpy/ > > From my limited knowledge of OpenGL, what my vision is that any of the > existing backends have support for an OpenGL object, so we just need to be > able to instantiate the opengl object in any figure object, and know how to > send it the appropriate commands and data. So, it is not exactly a backend, > more of a "middling". Anyway, I think the dev at Glumpy would be happy to > have help, and probably have much more developed ideas on how to integrate > with matplotlib. > > Cheers! > Ben Root > > -- > Monitor your physical, virtual and cloud infrastructure from a single > web console. Get in-depth insi
Re: [matplotlib-devel] OpenGL backend with Galry
OK so it seems that integrating any efficient OpenGL rendering code in matplotlib as a backend is much more complicated than what I thought. > I'm guessing with galry, you push the user-coordinates to the graphics > card, then as the user is interacting, you're changing the transforms > and re-rendering, but don't need to push the vertex data itself over > and over again, and with with shaders, you could do arbitrary > transforms. hence high performance. You're absolutely right, that's the way high performance is achieved in Galry. In fact, the low-level interface provides a way to write custom shaders to implement really anything (interactive transformations or any rendering effect). Whereas it is possible to push data on the GPU at any time, the most efficient way of rendering stuff in Galry is to push everything at the beginning, and then only update uniform shader variables to implement transforms and dynamic effects. That does not seem very compatible with the way backends appear to work then. I fear that in order to have an efficient GL backend, either this backend system would need to be updated so that it can be directly connected to transformation events, or one would need to get around the backend system. Anyway, it looks like the difficulty would come more from the matplotlib backend system than the OpenGL part (by the way, your rendering demos are really cool Nicolas!). It would be great if a GSoC student could work on this, and I would be happy to help if necessary. In the meantime, I might write sometime an extremely basic high-level matplotlib-like interface for Galry, with support in particular for scatter plots, continuous-time signals, textures, maybe other plot types if anyone asks. It may be useful until a fully working GL backend becomes available. Cyrille 2012/11/15 Nicolas Rougier > > > Yep, I'm still developing some OpenGL technics to provide both nice and > fast rendering and I hope to be able to help the writing of a GL backend > for matplotlib next summer (provided we get a GSoC student for the project). > > So far, my main concern is that for efficient rendering using OpenGL, you > need to consider that drawing means to create objects on the graphic card > (line, curve, image, points, etc) that can be later manipulated > (scaling/rotating/coloring/properties change, etc.). From what I remember > in my early attempts at writing an OpenGL backend, I did not find the > proper way to enforce such framework. Said differently, the backend is > supposed to implement drawing operations while I would need to know if the > drawing operations actually relates to something that is already on the > graphic card or not. I'm not sure I'm very clear but I can develop the > point if necessary. Having read the post by Michael ( > http://mdboom.github.com/blog/2012/08/06/matplotlib-client-side/) on > client-side rendering, I think the proposed three-way split might be a > solution but I do not know how advanced are the ideas. > > To date, I've been working on different things: > > Text/font : http://code.google.com/p/freetype-gl/ (c code) > Stroke/dash/paths: http://code.google.com/p/gl-agg/ (python) > Images:http://code.google.com/p/glumpy/ (python) > > If you want to get a feel of how nice and fast rendering could be, have a > look at 'demo-lines.py' from the gl-agg repository (and play with mouse). > From these experiments, I think it is possible to achieve AGG quality using > OpenGL. What is really exciting is the perspective of having a opengl/webgl > backend that could be used with ipython (there has been a recent post on > ipython list that show such integration for a molecule viewer). > > > Anyway, you're more than welcome to contribute to glumpy, but in the long > run, I hope it will disappear in favor of a matplolib GL backend. > > > > Nicolas > > > > > > > On Nov 15, 2012, at 22:03 , Benjamin Root wrote: > > > > > > > On Thu, Nov 15, 2012 at 2:24 PM, Cyrille Rossant < > cyrille.ross...@gmail.com> wrote: > > Hi all, > > > > I am developing a high-performance interactive visualization package in > Python based on PyOpenGL (http://rossant.github.com/galry/). It is > primarily meant to be used as a framework for developing complex > interactive GUIs (in QT) that deal with very large amounts of data (tens of > millions of points). But it may also be used, like matplotlib, as a > high-level interactive library to plot and visualize data. > > > > The low-level interface is mostly done at this point (the code is still > in an experimental stage though), and I'm now focusing on my current > research project which is to write a scientific GUI based on this > interface. However, I think people (including myself!) may be interested in > a matplotlib-like high-level interface. I was first thinking about writing > such an interface from scratch, by implementing a very small fraction of > the matplotlib interface (basic commands like figure(), plot(), subplot(), > show(), etc
Re: [matplotlib-devel] Interpolation in a triangular mesh (tri.Triangulation)
On Thu, Nov 15, 2012 at 3:25 PM, Chris Barker wrote: > On Wed, Nov 14, 2012 at 1:50 AM, Ian Thomas wrote: > >> I think the code used to determine which triangle contains a certain point >> should be factored out into its own TriFinder class, > > +1 -- this is a generally useful feature. In fact, it would be nice if > a lot of this were in a pacakge that deals with triangular meshes, > apart from MPL altogether (a scikit maybe?) > >> I have a C++ TriFinder class >> that I could modify to work within matplotlib, and it is O(log N) so should >> be faster than your version for typical use cases. > > What algorithm does this use? Is the code open source and/or availabel > for other projects? I'm pretty sure there is an O(log n) algorithm in the Numerical Recipes book. It requires you to construct the triangulation in a specific way (this allows one to set up a tree data structure of triangles nicely). There may be others that I am not aware of though. > > I'm working on a package for working with unstructured grids in > general, and also have a use for "what triangle is this point in" code > for other purposes -- and I havne't found a fast, robust code for this > yet. > >>> particularly as only a few days ago I committed to writing a triangular grid >>> interpolator for quad grids > > what is a triangular interpolator for quad grids? sounds useful, too. > > -Chris > > -- > > Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR&R(206) 526-6959 voice > 7600 Sand Point Way NE (206) 526-6329 fax > Seattle, WA 98115 (206) 526-6317 main reception > > chris.bar...@noaa.gov > > -- > Monitor your physical, virtual and cloud infrastructure from a single > web console. Get in-depth insight into apps, servers, databases, vmware, > SAP, cloud infrastructure, etc. Download 30-day Free Trial. > Pricing starts from $795 for 25 servers or applications! > http://p.sf.net/sfu/zoho_dev2dev_nov > ___ > Matplotlib-devel mailing list > Matplotlib-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel -- Damon McDougall http://www.damon-is-a-geek.com Institute for Computational Engineering Sciences 201 E. 24th St. Stop C0200 The University of Texas at Austin Austin, TX 78712-1229 -- Monitor your physical, virtual and cloud infrastructure from a single web console. Get in-depth insight into apps, servers, databases, vmware, SAP, cloud infrastructure, etc. Download 30-day Free Trial. Pricing starts from $795 for 25 servers or applications! http://p.sf.net/sfu/zoho_dev2dev_nov ___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
