Re: [matplotlib-devel] contourf masking bug
Eric,
Thanks for the scripts -- they make it quite clear what the problem is.
As you know, cntr.c is a fairly opaque chunk of code, and I'm not having
much luck so far tracking down why the branch-cutting between the outer
and inner polygons is not working.
In the meantime, I have some more general questions.
It looks like that the backend API isn't really set up to do compound
paths. The PolygonCollection class appears to be just a list of simple
polygons, rather than something that could be used for compound paths.
(Correct me if I'm wrong.) Is the only reason for wanting branch cuts
in contours because matplotlib doesn't support compound paths? Is there
a reason matplotlib doesn't support compound paths? I suspect you guys
have been down that path [haha] already. I would anticipate problems
if different backends have different winding rules, but perhaps there
are some ways to around that? This could be useful in general in
matplotlib...
Another option is to remove the mask from the contours in a
post-processing step (i.e. don't worry about trying to do it as part of
the main two passes in cntr.c). The result may be a bit of a
Frankenstein's monster, admittedly, but it is an option of last resort
if we choose to take it. cntr.c is creating the inner and outer
polygons correctly, just not connecting them with a slit, right?
In the meantime, I'll continue to look for the root cause.
Cheers,
Mike
Eric Firing wrote:
Mike,
The attached file masked_interior.py illustrates masking failure in a
very simple case; you can see masking working in the plot on the left,
where a contour intersects the masked region, but when that contour
level is removed the masked region is getting filled in.
The file contourf_demo.py is slightly modified from the one in the mpl
examples directory, and shows a failure of masking in a more
complicated setting. The masked region at the lower-left corner is
correct, but the masked region in the middle of the plot is getting
filled with gray instead of being left blank.
In cntr.c there is a function, print_Csite, that may be helpful for
debugging the simplest case, where the array size is not too large.
Note that the code path for filled contours is quite different, and
more complicated, than for line contours--and in fact, even neglecting
branch cuts, the two code paths don't always yield the same contours.
cntr.c is somewhat unusual among contour algorithms in that it works
with rectangles without subdividing them into triangles.
Eric
#!/usr/bin/env python
'''
This is a very simple illustration of what is causing
the problem with masked interior regions for filled contours.
The exterior contour and the interior contour are being
separated into two polygons instead of being joined by the
branch cut.
'''
import sys
from pylab import *
import matplotlib.numerix.npyma as ma
rc('figure', dpi=120)
x = y = arange(5)
X, Y = meshgrid(x, y)
Z = Y
ax = (0,4,0,4)
badmask = zeros(shape(X))
badmask[2, 2] = 1
Z = ma.array(Z, mask=badmask)
print Z
subplot(2,1,1)
CS = contourf(X, Y, Z, (-0.1, 0.8, 2.0, 6.0),
colors=('r', 'g', 'b'))
CS.collections[0].set_edgecolor('c')
CS.collections[0].set_linewidth(6)
CS.collections[1].set_edgecolor('y')
CS.collections[1].set_linewidth(2)
CS.collections[2].set_edgecolor('m')
CS.collections[2].set_linewidth(4)
title('Masked correctly')
subplot(2,1,2)
CS = contourf(X, Y, Z, (-0.1, 0.8, 6.0),
colors=('r', 'b'))
CS.collections[0].set_edgecolor('c')
CS.collections[0].set_linewidth(6)
CS.collections[1].set_edgecolor('y')
CS.collections[1].set_linewidth(2)
title('Wrong: masked region filled')
print len(CS.collections), len(CS.collections)
colls = CS.collections
# We want to look at the actual polygons individually.
for i, coll in enumerate(colls):
print level , i
for j, T in enumerate(coll._verts):
print polygon , j
print T
# The second collection has two polygons, but should only have
# one; the two should be joined by a branch cut.
show()
#!/usr/bin/env python
from pylab import *
import matplotlib.numerix.npyma as ma
origin = 'lower'
#origin = 'upper'
test_masking = True # There is a bug in filled contour masking.
if test_masking:
# Use a coarse grid so only a few masked points are needed.
delta = 0.5
else:
delta = 0.025
x = y = arange(-3.0, 3.01, delta)
X, Y = meshgrid(x, y)
Z1 = bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0)
Z2 = bivariate_normal(X, Y, 1.5, 0.5, 1, 1)
Z = 10 * (Z1 - Z2)
# interior badmask doesn't work yet for filled contours
if test_masking:
badmask = zeros(shape(Z))
badmask[5,5] = 1
badmask[5,6] = 1
Z[5,5] = 0
Z[5,6] = 0
badmask[0,0] = 1
Re: [matplotlib-devel] contourf masking bug
On 7/12/07, Michael Droettboom [EMAIL PROTECTED] wrote: It looks like that the backend API isn't really set up to do compound paths. The PolygonCollection class appears to be just a list of simple polygons, rather than something that could be used for compound paths. (Correct me if I'm wrong.) Is the only reason for wanting branch cuts in contours because matplotlib doesn't support compound paths? Is there a reason matplotlib doesn't support compound paths? I suspect you guys have been down that path [haha] already. I would anticipate problems if different backends have different winding rules, but perhaps there are some ways to around that? This could be useful in general in matplotlib... Actually we haven't. In fact, paths have been a fairly recent addition to mpl. The drawing model is based on GTK, which did not have paths at the time. The renderer API is basically the GTK API, and all the other backends simply implement the GTK drawing model. In the early days, there was only a GTK backend So one reason why the backends are a bit of a kludge is because we have tried to throw some extra stuff into the GTK drawing model as an afterthough. We could redo all the collection stuff w/ compounds paths. JDH - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] contourf masking bug
John Hunter wrote: [...] So one reason why the backends are a bit of a kludge is because we have tried to throw some extra stuff into the GTK drawing model as an afterthough. We could redo all the collection stuff w/ compounds paths. John, Do all the backend devices or libraries we *need* now support compound paths (1) at all, and (2) in a consistent enough way to solve the contouring problem as well as to make something like polygon collections more efficient? I guess this is really part of a larger question about the degree to which the various graphics drawing models have converged with a rich intersection of capabilities, including arbitrary text rotation, precise text positioning, arcs, bezier curves, etc. Eric - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] contourf masking bug
On Thu, 12 Jul 2007 09:29:06 -1000, Eric Firing wrote: So one reason why the backends are a bit of a kludge is because we have tried to throw some extra stuff into the GTK drawing model as an afterthough. We could redo all the collection stuff w/ compounds paths. What's the GTK drawing model here that is problematic? Is that pre-cairo stuff? And can using cairo directly help at all? -Carl pgpWrQYtE8QUH.pgp Description: PGP signature - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] contourf masking bug
On 7/12/07, Eric Firing [EMAIL PROTECTED] wrote: John Hunter wrote: Do all the backend devices or libraries we *need* now support compound paths (1) at all, and (2) in a consistent enough way to solve the contouring problem as well as to make something like polygon collections more efficient? I guess this is really part of a larger question about the degree to which the various graphics drawing models have converged with a rich intersection of capabilities, including arbitrary text rotation, precise text positioning, arcs, bezier curves, etc. Yes, I think they do, though we will encounter some differences between spline handling that might make some of the path stuff tricky. We need agg, cairo, svg, ps and pdf, all of which use vector graphics. We would need to sacrifice native GDK and WX, which would inconvenience some but these users could certainly get by with GTK* or WX*. One reason a few still use native GTK is because it is faster, but this would likely disappear with some intelligent design. One might envision just three or four fundamental objects which the backends need to understand. Path CompoundPath Image Text and maybe use a stateful graphics context (linewidth, facecolor, etc...) numpify everything, traitify everything and rebuild the transformation architecture from scratch. Borrowing some language from ipython, this would truly be a chainsaw branch. I'm halfway inclined to take a crack at it.. JDH - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] contourf masking bug
On 7/12/07, Carl Worth [EMAIL PROTECTED] wrote: What's the GTK drawing model here that is problematic? Is that pre-cairo stuff? And can using cairo directly help at all? Yes, this is pre cairo and unfortunately, no I don't think cairo can help. When I wrote MPL, it was GTK only and I used GDK for rendering. The backend canvas was a gtk.DrawingArea calls and the renderer was a gdk.Drawable which defined methods like draw_rectangle and draw_polygon http://www.pygtk.org/pygtk2reference/class-gdkdrawable.html As I added other backends, backend_ps first, I abstracted the gtk.DrawingArea, gdk.Drawable and gdk.GC to the FigureCanvas, Renderer and GraphicsContext respectively which are are core backend base classes, and then for each backend did a concrete implemention of the gdk.Drawable API, etc.This was back in 2001. If I were smarter or had done more research at the time, I would have chosen a different drawing mode, eg DisplayPDF or something like that, but I guess I was too busy coding (reminds me of a joke we used to tell in the lab Why spend and hour in the library when you can get the same result in the lab in a year?). So the fact that GTK now uses Cairo doesn't help us too much, since all of our backends implement the gdk.Drawable API more or less. What we are discussing here is a clean room reimplementation of our backend API and drawing model. JDH - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] contourf masking bug
Michael Droettboom wrote: cntr.c is creating the inner and outer polygons correctly, just not connecting them with a slit, right? To answer my own question, unfortunately, this isn't always the case. There is also a bug where if a masked region is inside of the *inner* polygon of a contour, the inner contour is incorrect. So, other advantages aside, adding support for compound polygons won't magically fix any of the contour problems. Cheers, Mike - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] contourf masking bug
John Hunter wrote: On 7/12/07, Eric Firing [EMAIL PROTECTED] wrote: John Hunter wrote: Do all the backend devices or libraries we *need* now support compound paths (1) at all, and (2) in a consistent enough way to solve the contouring problem as well as to make something like polygon collections more efficient? I guess this is really part of a larger question about the degree to which the various graphics drawing models have converged with a rich intersection of capabilities, including arbitrary text rotation, precise text positioning, arcs, bezier curves, etc. Yes, I think they do, though we will encounter some differences between spline handling that might make some of the path stuff tricky. We need agg, cairo, svg, ps and pdf, all of which use vector graphics. We would need to sacrifice native GDK and WX, which would From a purely technical standpoint--getting the desired output formats--perhaps cairo could substitute for pdf and svg, and also for ps if only cairo would make eps files. So, even if eventually you want to restore native pdf, svg, and ps backends, it could be done at a later stage than the initial chainsaw sculpture. inconvenience some but these users could certainly get by with GTK* or WX*. One reason a few still use native GTK is because it is faster, but this would likely disappear with some intelligent design. One might envision just three or four fundamental objects which the backends need to understand. Path CompoundPath Image Text and maybe use a stateful graphics context (linewidth, facecolor, etc...) numpify everything, traitify everything and rebuild the transformation architecture from scratch. Borrowing some language from ipython, this would truly be a chainsaw branch. I'm halfway inclined to take a crack at it.. Great! I wonder whether your company could be convinced to give you some slack for that... It might be worth a bit of thought and planning to see how parts of the work could be divided up so that you would not have to do everything yourself. To minimize the interval during which a separate branch would be needed, maybe some initial traitification should be done without a fork. Eric JDH - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] contourf masking bug
On Thursday 12 July 2007 05:11:31 pm Carl Worth wrote: On Thu, 12 Jul 2007 10:31:19 -1000, Eric Firing wrote: if only cairo would make eps files Isn't EPS a trivially change compared to PS? Something like a modified header and the addition of bounding-box information? The following is listed in the Encapsulated Postscript File Format Specification, http://partners.adobe.com/public/developer/en/ps/5002.EPSF_Spec.pdf: There are some PostScript language operators plus statusdict and userdict operators that are intended for system-level jobs or page descriptions that are not appropriate in an EPS file. In addition to all operators in statusdict and the operators in userdict for establishing an imageable area, the following operators must not be used in an EPS file: banddevice exitserver initmatrix setshared clear framedevice quit startjob cleardictstack grestoreall renderbands copypageinitclip setglobal erasepage initgraphics setpagedevice If used properly, the following operators are allowed in an EPS file. However, use of any of these must comply with the rules in Appendix I of the PostScript Language Reference Manual, Second Edition. Improper use can cause unpredictable results. nulldevice sethalftone setscreen undefinefont setgstate setmatrixsettransfer - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] contourf masking bug
Carl Worth wrote: On Thu, 12 Jul 2007 10:31:19 -1000, Eric Firing wrote: if only cairo would make eps files Isn't EPS a trivially change compared to PS? Something like a modified header and the addition of bounding-box information? Carl, My knowledge of graphics topics is sketchy, but yes, in many cases the difference between PS and EPS is just a few lines. This is true for Matlab files--ps and eps are identical except for something like 3 lines. This works *provided* the ps file doesn't have anything in it that is not allowed in eps. I don't recall the constraints, apart from the file being single-page, but I know there are some. I suspect this is not a problem for cairo output, but I haven't tried to check. It sounds like something that should be really easy to change in cairo. Yes, so much so that I don't understand why it is not already done. (I know you are a cairo dev--is there not a postscript specialist in the cairo dev community?) Maybe you could take a cairo-generated .ps file and manually munge it into the .eps that you'd like? If you did that, I'd be glad to write the necessary cairo code to get the same result, (and come up with whatever tiny new API is necessary to instruct cairo to do that). If no one else steps forward, I can try to look at it fairly soon--but I am really not the right person to do it. As for the rest of what you say. From my point of view, yes, using cairo and its multi-backend capabilities seems to make a fair amount more sense than inventing a new system with multiple backends. (Not a criticism against the original MPL stuff---cairo 1.0 has only existed since August 2005). But yeah, I also understand that there are licensing concerns. Anything else? Are there cairo performance concerns? If so, I'd love to hear about them so we can fix them. Well, I think that for screen display and for png generation, Agg is significantly faster. Backend_driver.py gives Backend agg took 0.76 minutes to complete Backend cairo.png took 0.95 minutes to complete Backend template took 0.46 minutes to complete I haven't tried to test it, but I suspect that for some types of plots the optimizations in Agg and the agg backend will make a much bigger difference than the averages above would indicate. Eric And even if you don't use cairo directly, I will offer the suggestion of its drawing model as being very good, (using source and mask pattern objects and just 5 different drawing operations: stroke, fill, paint, show_text/glyphs, and mask). Whatever you decide to go with, have fun! -Carl - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] contourf masking bug
Carl Worth wrote: [...] But yeah, I also understand that there are licensing concerns. Anything else? Are there cairo performance concerns? If so, I'd love to hear about them so we can fix them. Backend cairo.ps took 0.87 minutes to complete Backend ps took 0.66 minutes to complete So the mpl Cairo ps output is also relatively slow. I don't know whether this is inherent in Cairo, or whether it could be optimized away in the mpl Cairo backend. And, it may be caused by a few outliers among the examples; I have not tried to compare times for individual plots. Eric - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] contourf masking bug
Darren Dale wrote: On Thursday 12 July 2007 05:11:31 pm Carl Worth wrote: On Thu, 12 Jul 2007 10:31:19 -1000, Eric Firing wrote: if only cairo would make eps files Isn't EPS a trivially change compared to PS? Something like a modified header and the addition of bounding-box information? The following is listed in the Encapsulated Postscript File Format Specification, http://partners.adobe.com/public/developer/en/ps/5002.EPSF_Spec.pdf: There are some PostScript language operators plus statusdict and userdict operators that are intended for system-level jobs or page descriptions that are not appropriate in an EPS file. In addition to all operators in statusdict and the operators in userdict for establishing an imageable area, the following operators must not be used in an EPS file: banddevice exitserver initmatrix setshared clear framedevice quit startjob cleardictstack grestoreall renderbands copypageinitclip setglobal erasepage initgraphics setpagedevice Aha! Cairo ps output is full of initclip commands, so it can't be converted to eps without substantial reworking. This is a big limitation. To put a figure in a LaTeX document, for example, eps usually is what you need. So until Cairo solves this problem and provides eps, it is locked out of a large domain of graphics use. Eric - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] contourf masking bug
On Thursday 12 July 2007 5:57:27 pm Eric Firing wrote: Carl Worth wrote: [...] But yeah, I also understand that there are licensing concerns. Anything else? Are there cairo performance concerns? If so, I'd love to hear about them so we can fix them. Backend cairo.ps took 0.87 minutes to complete Backend ps took 0.66 minutes to complete So the mpl Cairo ps output is also relatively slow. I don't know whether this is inherent in Cairo, or whether it could be optimized away in the mpl Cairo backend. And, it may be caused by a few outliers among the examples; I have not tried to compare times for individual plots. Probably mpl's cairo backend can be optimized. The PS backend is heavily optimized, making use of the newer API to draw lines and markers (RendererPS.draw_lines and draw_markers). - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] contourf masking bug
On 7/12/07, Carl Worth [EMAIL PROTECTED] wrote: Anyway, I'll try to get setup to profile the cairo run above and see if any obvious things stick out that could be easily fixed. Most likely, the biggest win by far will be to implement draw_markers. You will see the start of an attmept in draw_markers_OLD. Not having this function means symbol plots must make poosibly thousands of separate calls to draw_polygon and friends, each with their own set of gc calls. backend_ps is probably the best place to look for inspiration. If you search the devel archives for draw_markers, you'll find lots of discussion JDH - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
[matplotlib-devel] Bus error using interactive plotting commands
I recently updated matplotlib, and I am now getting a bus error when plotting (any command) after I have already closed a matplotlib window. I can open as many windows as I want, but the first plot command executed after closing any window results in a bus error. Relavent things: OS: Mac OS X Backend: TkAgg (same bus error with qt4, though) very recent svn versions of numpy/mpl/ipython. I hadn't updated in a while (weeks) prior to the last update, so I am not exactly sure when the bug appeared. -Rob Rob Hetland, Associate Professor Dept. of Oceanography, Texas AM University http://pong.tamu.edu/~rob phone: 979-458-0096, fax: 979-845-6331 - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
