Re: [matplotlib-devel] 2954 removal of log kwarg from bar?
Hey Eric, I'm CC-ing the devel list because some of this has potentially far reaching consequences. > "Eric" == Eric Firing <[EMAIL PROTECTED]> writes: Eric> In connection with the colorbar sizing problem I have been Eric> thinking about adding a function list to axes, with the Eric> functions to be executed early in the draw() method, to Eric> handle this sort of thing. Using this mechanism to handle I think this is a very nice idea -- I saw a talk at pycon one year about the utility of adding hooks, eg before_somefunc_hook and after_somefunc_hook that are customizable *for every function in your API*. The questions are 1) what is the right data structure -- a single callable, a sequence, or a dictionary registry with connect/disconnect semantics (eg the backend event callback handler) 2) what is the signature (see below) 3) which artists should support it Most elegant, but most expensive, is something like a ordered dictionary which is supported for every artist. By renaming all the artist draw methods _draw, we can do this quite simply in backend_bases.Renderer with def draw(self, renderer): for func in self.before_draw_hook.ordered_values(): func(self, renderer) self._draw(renderer) for func in self.after_draw_hook.ordered_values(): func(self, renderer) I think this could be really useful and would help handle problems that crop up all the time -- how to handle physical layout problems where we need the renderer to get size information like you are experiencing with colorbar. This problem also crops up in a lot of text layout problems. Unfortunately, draw time layout creates other problems (sometimes users want this info at the API level), and order effects becomes very important and potentially complicated. But that said, given the current architecture, something like this would be a definite improvement. I'm inclined to use an ordered dictionary because one wants the connect/disconnect semantics that keys make easy, and because ordering matters. There is a python cookbook recipe for an ordered dictionary, and we could easily roll our own. Because the calls are potentially expensive, we might want an optimization like class MyOrderedDictionary: def __init__(blah): self.isempty = True so we can do def draw(renderer): if not self.before_draw_hook.isempty: for func in self.before_draw_hook.ordered_values(): func(self, renderer) self._draw(renderer) if not self.after_draw_hook.isempty: for func in self.after_draw_hook.ordered_values(): func(self, renderer) Since 99% of the time I guess these will be empty and then we are just adding a few attribute lookups and a boolean check for each draw. JDH - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Matplotlib-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
[matplotlib-devel] coding guide
Eric Firing had the excellent idea of making a CODING_GUIDE which
summarizes the conventions used in matplotlib development, and I've
added this to svn. Feel free to make changes, additions and
comments. I think we could add a lot here, including an overview of
the API. Here is the document I just committed::
Devs, feel free to edit this document. This is meant to be a guide to
developers on the mpl coding practices and standards
== Committing Changes ==
When committing changes to matplotlib, there are a few things to bear
in mind.
* if your changes are nontrivial, please make an entry in the
CHANGELOG
* if you change the API, please document it in API_CHANGES, and
consider posing to mpl-devel
* Are your changes python2.3 compatible? We are still trying to
support 2.3, so avoid 2.4 only features like decorators until we
remove 2.3 support
* Are your changes Numeric, numarray and numpy compatible? Try
running simple_plot.py or image_demo.py with --Numeric, --numarray
and --numpy (Note, someone should add examples to
backend_driver.py which explicitly require numpy, numarray and
Numeric so we can automatically catch these)
* Can you pass examples/backend_driver.py? This is our poor man's
unit test.
* If you have altered extension code, do you pass
unit/memleak_hawaii.py?
== Naming conventions ==
functions and class methods : lower or lower_underscore_separated
attributes and variables : lower or lowerUpper
classes : Upper or MixedCase
Personally, I prefer the shortest names that are still readable.
== kwargs processing ==
Matplotlib makes extensive use of **kwargs for pass through
customizations from one function to another, eg the pylab plot ->
Axes.plot pass through. As a general rule, the use of **kwargs should
be reserved for pass-through keyword arguments, eg
def somefunc(x, k1='something', **kwargs):
# do some thing with x, k1
return some_other_func(..., **kwargs)
If I intend for all the keyword args to be used in somefunc alone, I
just use the key/value keyword args in the function definition rather
than the **kwargs idiom. In some cases I want to consume some keys
and pass through the others, in which case I pop the ones I want to
use locally and pass on the rest, eg I pop scalex and scaley in
Axes.plot and assume the rest are Line2D keyword arguments. Whenever
you mutate a kwargs dictionary (eg by popping it), you must first copy
it since the user may be explitly passing in a dictionary which is
used across many function calls. As an example of a copy, pop,
passthrough usage, see Axes.plot:
def plot(self, *args, **kwargs):
kwargs = kwargs.copy()
scalex = popd(kwargs, 'scalex', True)
scaley = popd(kwargs, 'scaley', True)
if not self._hold: self.cla()
lines = []
for line in self._get_lines(*args, **kwargs):
self.add_line(line)
lines.append(line)
popd is a matplotlib.cbook function to pop an item from a dictionary
with a default value if the item doesn't exist
Note there is a use case when kwargs are meant to be used locally in
the function (not passed on), but you still need the **kwargs idiom.
That is when you want to use *args to allow variable numbers of
non-keyword args. In this case, python will not allow you to use
named keyword args after the *args usage, so you will be forced to use
**kwargs. An example is matplotlib.contour.ContourLabeler.clabel
def clabel(self, *args, **kwargs):
fontsize = kwargs.get('fontsize', None)
inline = kwargs.get('inline', 1)
self.fmt = kwargs.get('fmt', '%1.3f')
colors = kwargs.get('colors', None)
if len(args) == 0:
levels = self.levels
indices = range(len(self.levels))
elif len(args) == 1:
...etc...
== class documentation ==
matplotlib uses artist instrospection of docstrings to support
properties. All properties that you want to support through setp and
getp should have a set_property and get_property method in the Artist
class. Yes this is not ideal given python properties or enthought
traits, but it is a historical legacy for now. The setter methods use
the docstring with the ACCEPTS token to indicate the type of argument
the method accepts. Eg in matplotlib.lines.Line2D
def set_linestyle(self, linestyle):
"""
Set the linestyle of the line
ACCEPTS: [ '-' | '--' | '-.' | ':' | 'steps' | 'None' | ' ' | '' ]
"""
Since matplotlib uses a lot of pass through kwargs, eg in every
function that creates a line (plot, semilogx, semilogy, etc...), it
can be difficult for the new user to know which kwargs are supported.
I have developed a docstring interpolation scheme to support
documentation of every function that takes a **kwargs. The
requirements are:
1) single point of configuration so changes to the properties don't
Re: [matplotlib-devel] 2954 removal of log kwarg from bar?
John Hunter wrote: > Hey Eric, I'm CC-ing the devel list because some of this has > potentially far reaching consequences. > >> "Eric" == Eric Firing <[EMAIL PROTECTED]> writes: > > Eric> In connection with the colorbar sizing problem I have been > Eric> thinking about adding a function list to axes, with the > Eric> functions to be executed early in the draw() method, to > Eric> handle this sort of thing. Using this mechanism to handle > > I think this is a very nice idea -- I saw a talk at pycon one year about > the utility of adding hooks, eg before_somefunc_hook and after_somefunc_hook > that are customizable *for every function in your API*. The questions are > > 1) what is the right data structure -- a single callable, a > sequence, or a dictionary registry with connect/disconnect > semantics (eg the backend event callback handler) > > 2) what is the signature (see below) > > 3) which artists should support it > > Most elegant, but most expensive, is something like a ordered > dictionary which is supported for every artist. By renaming all the > artist draw methods _draw, we can do this quite simply in > backend_bases.Renderer with > > def draw(self, renderer): > for func in self.before_draw_hook.ordered_values(): > func(self, renderer) > > self._draw(renderer) > > for func in self.after_draw_hook.ordered_values(): > func(self, renderer) > Nice, simple, elegant, but it doesn't quite match what I had in mind for axes. I was thinking that the place to put the hook call is inside the draw() method, so it can take advantage of the intial work done there: getting a cached renderer, checking for visibility. Maybe this doesn't matter. Also, looking at backend_bases, I can't figure out where this goes. There is a RendererBase class, but it doesn't have a draw method. And there is a FigureCanvasBase, but its draw method is overridden by each backend. What am I missing? > > I think this could be really useful and would help handle problems > that crop up all the time -- how to handle physical layout problems > where we need the renderer to get size information like you are > experiencing with colorbar. This problem also crops up in a lot of > text layout problems. Unfortunately, draw time layout creates other > problems (sometimes users want this info at the API level), and order > effects becomes very important and potentially complicated. But that > said, given the current architecture, something like this would be a > definite improvement. > > I'm inclined to use an ordered dictionary because one wants the > connect/disconnect semantics that keys make easy, and because ordering > matters. There is a python cookbook recipe for an ordered dictionary, > and we could easily roll our own. Because the calls are potentially > expensive, we might want an optimization like > I agree, an ordered dictionary is ideal if one needs the option of multiple functions being executed. I would probably have it inherit from list and tack on the dictionary; I think this might make for the fastest cycling through the list, and that is the place where speed is needed. > class MyOrderedDictionary: > def __init__(blah): > self.isempty = True > > > so we can do > > def draw(renderer): > if not self.before_draw_hook.isempty: > for func in self.before_draw_hook.ordered_values(): > func(self, renderer) > > self._draw(renderer) > > if not self.after_draw_hook.isempty: > for func in self.after_draw_hook.ordered_values(): > func(self, renderer) > > Since 99% of the time I guess these will be empty and then we are just > adding a few attribute lookups and a boolean check for each draw. Detail: Maybe it would be simpler and faster to initialize like this: self.before_draw_hook=None and have def register_before_draw(self, name, func): if self.before_draw_hook is None: self.before_draw_hook = OrderedDictionary() self.before_draw_hook.append(name, func) It saves one attribute lookup at draw time. Larger picture: Looking at the axes draw method, I am wondering whether a more general application of an OrderedDictionary (or ListDict, or whatever) wouldn't be a better approach. Why should there be before_draw_hook, draw, and after_draw_hook, with a big list generation and sorting operation at draw time? Could everything be done by having artists register all draw-time functions in a single ListDict, with sufficient ordering information to keep everything straight? That's pretty vague, and there is a lot I don't understand--but it just seems like there is an opportunity here for simplifying the overall structure with improved flexibility and no loss of speed. (Again, some kind of result-caching and need-for-recomputation-signalling would be needed.) Eric
Re: [matplotlib-devel] coding guide
> * Are your changes python2.3 compatible? We are still trying to > support 2.3, so avoid 2.4 only features like decorators until we > remove 2.3 support Good, I thought we were still restricted to 2.2! Does this mean boilerplate.py should be updated? (Or maybe it isn't worth fiddling with it.) # wrap the plot commands defined in axes. The code generated by this # file is pasted into pylab.py. We did try to do this the smart way, # with callable functions and new.function, but could never get the # docstrings right for python2.2. See http://groups-beta.google.com/group/comp.lang.python/browse_thread/thread/dcd63ec13096a0f6/17739e70ac6f710c?lnk=gst&q=dcd63ec13096a0f6&rnum=1#17739e70ac6f710c Eric - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Matplotlib-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] coding guide
> "Eric" == Eric Firing <[EMAIL PROTECTED]> writes: >> * Are your changes python2.3 compatible? We are still trying >> to support 2.3, so avoid 2.4 only features like decorators >> until we remove 2.3 support Eric> Good, I thought we were still restricted to 2.2! I was wondering how long it would take for someone to notice that I think we could lose python2.2 support if there is a good reason. Is anyone still using it? I'll also post to the user's list. Perhaps in the next release we should issue deprecation warnings for 2.2 and encourage people to post to the list if they have an objection. Typically with these things you don't see complaints until you take something away. We haven't been compiling 2.2 for windows for sometime now. JDH - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Matplotlib-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] 2954 removal of log kwarg from bar?
> "Eric" == Eric Firing <[EMAIL PROTECTED]> writes:
Eric> Nice, simple, elegant, but it doesn't quite match what I had
Eric> in mind for axes. I was thinking that the place to put the
Eric> hook call is inside the draw() method, so it can take
Eric> advantage of the intial work done there: getting a cached
Eric> renderer, checking for visibility. Maybe this doesn't
Eric> matter.
Well for these two pieces, perhaps the method should look like
def draw(self, renderer=None, **kwargs):
if renderer is None:
renderer = self._cachedRenderer
if renderer is None:
raise RuntimeError('No renderer defined')
if not self.get_visible(): return
for func in self.before_draw_hook.ordered_values():
func(self, renderer)
self._draw(renderer, **kwargs)
for func in self.after_draw_hook.ordered_values():
func(self, renderer)
and perhaps the renderer caching should be done at the figure level
if renderer is None:
renderer = self.figure._cachedRenderer
Eric> Also, looking at backend_bases, I can't figure out where
Eric> this goes. There is a RendererBase class, but it doesn't
Eric> have a draw method. And there is a FigureCanvasBase, but
Eric> its draw method is overridden by each backend. What am I
Eric> missing?
I meant artist.Artist.draw -- sorry for the bad info.
Eric> and have def register_before_draw(self, name, func): if
Eric> self.before_draw_hook is None: self.before_draw_hook =
Eric> OrderedDictionary() self.before_draw_hook.append(name, func)
We also need a connectionid for connecting and disconnecting or
something like it.
Eric> Looking at the axes draw method, I am wondering whether a
Eric> more general application of an OrderedDictionary (or
Eric> ListDict, or whatever) wouldn't be a better approach. Why
Eric> should there be before_draw_hook, draw, and after_draw_hook,
Eric> with a big list generation and sorting operation at draw
Eric> time? Could everything be done by having artists register
Eric> all draw-time functions in a single ListDict, with
Eric> sufficient ordering information to keep everything straight?
Eric> That's pretty vague, and there is a lot I don't
Eric> understand--but it just seems like there is an opportunity
Eric> here for simplifying the overall structure with improved
Eric> flexibility and no loss of speed. (Again, some kind of
Eric> result-caching and need-for-recomputation-signalling would
Eric> be needed.)
This is interesting -- the devil will be in the details. Eg when one
artist changes it's zorder, the whole connection hierarchy will need
to be redone, which is currently accomplished by doing the sort w/
every draw. Your way would be faster since we would only need to do
the ordering when the zscore changes, or when new artists are added,
and not with every draw. But it might become difficult to manage
resorting if users have added their own hooks. And then it wouldn't
really be a before draw hook but a draw hook
Actually, if we go this route, we may need to have one registry for
user hooks, and another registry from internal calls. So any artist
could support drawing other artists, eg Lines and Labels register with
Ticks register which registers with an Axis which registers with an
Axes which registers with a Figure. Of course at this point we are
talking about a major architectural change and so there would need to
be some clear utility, and I'm not sure what it is. The current
design assumes an artist knows how to draw it's child artist, the one
you are proposing it seems assumes the child artist knows how to
insert itself into the parent.
JDH
-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Matplotlib-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] coding guide
> "John" == John Hunter <[EMAIL PROTECTED]> writes: John> I think we could lose python2.2 support if there is a good John> reason. Is anyone still using it? I'll also post to the John> user's list. Perhaps in the next release we should issue John> deprecation warnings for 2.2 and encourage people to post to John> the list if they have an objection. Typically with these John> things you don't see complaints until you take something John> away. Well, I was just about to post to the user list to see if anyone objected to remove 2.2 support, and wondered, do we really support 2.2 currently? I just tried to compile under 2.2 and got peds-pc311:~/mpl> sudo python2.2 setup.py install Traceback (most recent call last): File "setup.py", line 63, in ? from setupext import build_agg, build_gtkagg, build_tkagg, build_wxagg,\ File "setupext.py", line 120, in ? win32_compiler = get_win32_compiler() File "setupext.py", line 117, in get_win32_compiler if 'mingw32' in v: TypeError: 'in ' requires character as left operand And this has probably been in there for a while. So we haven't supported 2.2 for sometime and noone is complaining so let's make 2.3 the official target rather than fix this. At least 2.3 compiles and runs. I'll update the docs and website when I get a minute. JDH - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Matplotlib-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] coding guide
John Hunter wrote: [...] I am in the process of changing your shiny new kwdocd machinery slightly. I added artist.kwdoc so that > artist.kwdocd['Line2D'] = > '\n'.join(artist.ArtistInspector(Line2D).pprint_setters(leadingspace=12)) > becomes artist.kwdocd['Line2D'] = artist.kwdoc(Line2D) > Then in any function accepting Line2D passthrough kwargs, eg > matplotlib.axes.Axes.plot > > def plot(self, *args, **kwargs): > """ > Some stuff omitted > > The kwargs are Line2D properties: > %(Line2D)s and the line above is changed to be indented the same as the rest of the string (improves readability to my eye) > > kwargs scalex and scaley, if defined, are passed on > to autoscale_view to determine whether the x and y axes are > autoscaled; default True. See Axes.autoscale_view for more > information > """ > pass because this > plot.__doc__ = plot.__doc__ % artist.kwdocd becomes plot.__doc__ = dedent(plot.__doc__) % artist.kwdocd where dedent is now in cbook.py I can continue making the necessary changes if that is OK with you, but I don't want our versions to get tangled up if you are still working on this aspect, or if you don't like the strategy modification outlined above. I committed dedent but not the other changes. Eric - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Matplotlib-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] coding guide
> "Eric" == Eric Firing <[EMAIL PROTECTED]> writes: Eric> I can continue making the necessary changes if that is OK Eric> with you, but I don't want our versions to get tangled up if Eric> you are still working on this aspect, or if you don't like Eric> the strategy modification outlined above. I committed Eric> dedent but not the other changes. This looks very good and a definite improvement. You might consider extending this a bit to have a custom class that handles all the details. The general sketch is class Kwdocd(dict): def inspect(self, artist): # add artist to kwdocd, using __name__ or __class__ to get the key def expand_doc(self, method): # interpolate kwdocd into method.__doc__ altering # method.__doc__ in place kwdocd = Kwdocd() and then we have class Line2D: pass kwdocd.inspect(Line2D) class Axes: def plot(self, *args, **kwargs): """ %(Line2D)s """ pass kwdocd.expand_doc(plot) This is untested but it is consistent with the work you've done so far to simplify the API at the level of the derived Artist classes by hiding the gory details. JDH - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Matplotlib-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] coding guide
John Hunter wrote: >> "Eric" == Eric Firing <[EMAIL PROTECTED]> writes: > > Eric> I can continue making the necessary changes if that is OK > Eric> with you, but I don't want our versions to get tangled up if > Eric> you are still working on this aspect, or if you don't like > Eric> the strategy modification outlined above. I committed > Eric> dedent but not the other changes. > > This looks very good and a definite improvement. You might consider > extending this a bit to have a custom class that handles all the > details. The general sketch is Very nice. I thought about making a standalone expand_doc() function, but your incorporation of it in the class is much nicer. I think I will put this on the stack and not do it immediately. I already made all the changes I outlined (couldn't stop once I got going) and can't quite face another pass through the files right now. But in the process of making that pass, I ran into a small worm can: in the collections the kwargs and the setters don't match, so kwdocd['PatchCollection'], for example, is not quite right. The collection setter names are singular (set_color) to mirror the corresponding non-collection objects, but the kwargs are plural (colors). Although I see the (grammatical) logic of the plural form I like the singular form because I see little gain and considerable loss in having different names for the same functionality in LineCollection as compared to Line2D, for example. We could get around the problem by allowing the singular forms of the kwargs as well and deprecating the plural forms. Another small glitch: it looks like you used the inspector to assemble the kwargs list for the Axes docstring, but some of these, like figure and position, duplicate required arguments so they don't really makes sense as kwargs--even though they would work. Did you edit out other such examples? Is that the reason the Axes.__init__ kwargs list is not being generated automatically? Eric > > class Kwdocd(dict): > > def inspect(self, artist): > # add artist to kwdocd, using __name__ or __class__ to get the key > > def expand_doc(self, method): > # interpolate kwdocd into method.__doc__ altering > # method.__doc__ in place > > kwdocd = Kwdocd() > > and then we have > > class Line2D: > pass > > kwdocd.inspect(Line2D) > > class Axes: > def plot(self, *args, **kwargs): > """ > %(Line2D)s > """ > pass > kwdocd.expand_doc(plot) > > > This is untested but it is consistent with the work you've done so far > to simplify the API at the level of the derived Artist classes by > hiding the gory details. > > JDH - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Matplotlib-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
