Re: [matplotlib-devel] Plot or Not: voting to create better matplotlibrc

[Michael Droettboom]

On 07/20/2013 09:07 PM, Eric Firing wrote:
> On 2013/07/20 2:38 PM, David P. Sanders wrote:
>> And this is my problem with 'rc':  it brings to mind an arcane config
>> file hidden away somewhere that has a terrible syntax and must not be
>> touched.
>>
>> As Chris and Adrian have emphasized, the point is that we *should* be
>> tweaking away at the parameters all the time.
>> I propose to rename as  mpl_params=rcParams
>>
> Yes, mpl_params is more descriptive and easy to remember.  rcParams is
> ugly, obscure, and archaic.  It will have to remain available for a long
> time, but I don't object to changing standard usage to a nicer name.
> There might even be a better name than "mpl_params"--"settings" comes to
> mind, or maybe "style".

I agree those are better.  This is *such* a core method, though, that 
the old name can probably never go away -- there are tons of references 
to it in the documentation that would need to be updated, not to  
mention all of the non-documentation information (mailing list, 
stackoverflow) that Google finds.  I worry that adding an alias will 
actually contribute to confusion, not eliminate it.  I'd prefer to make 
it more obvious what it is and does in the documentation rather than 
change its name.

>
> As for parameter tweaking: the defaults shipped with mpl should be
> changed only rarely, but we should make it as easy as possible for users
> to customize plots, including coming up with good combinations of style
> parameters.  In some cases it makes sense to do this via a matplotlibrc
> file, in other cases it is better to do the parameter setting explicitly
> at the top of a script.
>
> Regarding defaults, note that I said "rarely", not "never".
>
> The whole rc system could use a good review--maybe resulting in lots of
> changes, maybe not--so I welcome the attention you are directing to it.
>
My biggest pet peeve is the way that some parameters take effect at 
"construction time" and thus their changes have no effect on existing 
plots.  Some take effect at draw time, and are thus more dynamic.  We 
should strive for the latter whenever possible.  Some things are just 
required to be the former, so these should be documented clearly as 
exceptions.

As for a low-hanging fruit project, the formatting of 
matplotlibrc.template is a mess.  It would be great if someone could go 
through it and make the line lengths and tabbing consistent etc.

Mike

--
See everything from the browser to the database with AppDynamics
Get end-to-end visibility with application monitoring from AppDynamics
Isolate bottlenecks and diagnose root cause in seconds.
Start your free trial of AppDynamics Pro today!
http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
___
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] How to use STIX fonts in matplotlib plots?

[Michael Droettboom]

The STIX fonts are included with matplotlib, as the licensing permits 
this.  We actually ship TTF versions of the fonts, converted from the 
original OTF files, since the built-in font rendering (i.e. when not 
using XeTeX etc.) does not support OTF.  See MEP14 for a discussion of 
some of the gory details, if you're interested.


On 07/21/2013 04:08 AM, David P. Sanders wrote:




On Sat, Jul 20, 2013 at 2:03 PM, Benjamin Root > wrote:


David,

IIRC, we were just starting to investigate how to produce retina
graphics. Perhaps you might be able to help Mike D and Michael de
Hoon with there efforts because very few of us have retina displays.


Sure, I'm very happy to help.
First let me return to the fonts issue.
I had been misunderstanding the rcParams (this seems to be a recurring 
problem at the moment ;) - some new documentation is definitely 
required; I will try to get round to add it to my matplotlib.settings 
notebook).


The fuzziness I referred to was indeed a retina issue, stemming from 
the fact *that the default output format is still PNG*. It seems to me 
that these days the default output should be SVG, which immediately 
resolves all retina issues!! (And a lot of other issues, it seems to me.)


The current status of retina support is actually reasonable. There are 
two options:


%load_ext retina
%config InlineBackend.figure_format = 'retina'

In the absence of tab completion for %load_ext and %config, and not 
understanding the code, I am not sure if these are synonyms or not. 
But the effect is to have PNGs produced with twice the vertical and 
horizontal resolution. (The problem comes if, for example, these are 
included in output sent to nbviewer, in which case they appear twice 
as large.)


So it appears the fuzziness is specific to the IPython notebook.  I 
think at the Scipy sprints we determined that using the MacOSX backend 
directly that there were no issues with the retina display. Let's maybe 
file an issue with the IPython folks about this.  Since there is already 
a retina display plugin in IPython, perhaps a dicussion should be 
started about autodetecting the retina case and switching it on in that 
case.  (I have no idea if that's technically feasible -- I don't know 
how Safari etc. implement the retina support).


BTW: This is IPython's PR where this was added: 
https://github.com/ipython/ipython/pull/3381




One STIX font question remains: How can I get the text of the tick 
labels and other things to also be in STIX?

settings['font.family'] = 'stix'
does not work, apparently.


STIX is designed to blend seamlessly with Times (New Roman), so you can 
set the default family to that.  It might be worth discussing whether 
setting `math.fontset` to `stix` should do that by default. (It's tricky 
because it creates a dependency between different rcParams), but in the 
meantime, that is the best workaround.




And could the default font finally be changed to something else? What 
are the licensing requirements for the font? Is it distributed with 
matplotlib, or how does it work?


The default font is Bitstream Vera Sans.  At the time it was chosen, it 
was the only really obvious option for an open source font. There are 
better options now -- it would need to be something not only freely 
redistibutable, but also open source (to meet Debian and RedHat 
requirements).  I don't know if we need to change the default, so much 
as make it easier to change (a la Tony's PR #2236).  If there is a way 
to distribute fonts along with styles, that would be killer.  I also 
hope to support WebFonts (which would be "installation free") as part of 
my ongoing work on MEP14, so that will get better, too.


Mike



David.




--
See everything from the browser to the database with AppDynamics
Get end-to-end visibility with application monitoring from AppDynamics
Isolate bottlenecks and diagnose root cause in seconds.
Start your free trial of AppDynamics Pro today!
http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk


___
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel


--
See everything from the browser to the database with AppDynamics
Get end-to-end visibility with application monitoring from AppDynamics
Isolate bottlenecks and diagnose root cause in seconds.
Start your free trial of AppDynamics Pro today!
http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk___
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] Plot or Not: voting to create better matplotlibrc

[Nathaniel Smith]

On Sun, Jul 21, 2013 at 2:48 AM, Chris Beaumont  wrote:
> I don't fully agree with Eric that changing the defaults should be treated
> as an API break -- yes, it may irritate a minority of users, but their code
> will still run. I'd flip around your argument for the role of rcParams and
> customization: the majority user is probably someone who doesn't know much
> about rcParams, or all the ways plots can be customized. *That* is the use
> case to optimize, not the "legacy" users invested in the current style.

As one small data point here, from a really excellent paper reviewing
the trade-offs in colormap design and deriving the "cool-warm"
colormap as a good default:

"This diverging color map interpolation has also been added to
ParaView, a free, open-source end-user scientific visualization
application, and was first featured in the 3.4 release in October
2008. Although ParaView does let users change the color map and there
is no way to track who does so, in our experience few users actually
do this. In the nearly 3000 messages on the ParaView users’ mailing
list from October 2008 to July 2009, there was no mention of the
change of color map from rainbow to cool-warm diverging. Users seem to
have accepted the change with little notice despite most users’
affinity for rainbow color maps."

- http://www.cs.unm.edu/~kmorel/documents/ColorMaps/ColorMapsExpanded.pdf

It absolutely should be easy to go back to the way things were, for
reproduction of old carefully hand-tweaked plots etc. But, there are
cases where our defaults are demonstrably terrible; and, these
defaults are demonstrably the direct cause of people producing
unnecessarily inferior plots; and, it's been many years now that this
situation has continued out and nothing has happened because we don't
want to be hasty. At some point we need to bite the bullet and make
these things better.

-n

--
See everything from the browser to the database with AppDynamics
Get end-to-end visibility with application monitoring from AppDynamics
Isolate bottlenecks and diagnose root cause in seconds.
Start your free trial of AppDynamics Pro today!
http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
___
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] Plot or Not: voting to create better matplotlibrc

[Chris Beaumont]

I had the same question about opt-out vs opt-in. Personally, I vote for
opt-out. I would like to see each release of MPL have an associated style
(which may be the same as the last release, but maybe not). With Tony's
style PR, users that need constant styles would either put
`style.use('1.3')` in their script, or `style: 1.3` in an rcParams file.
This would then freeze the style FOR-EV-ER (sandlot voice). However, the
default would be `style: latest`, so that the default user benefits from
the community's effort into making the best possible set of defaults. Is
that sufficiently pain-free for people who want future proofing? Or do you
think styles must be opt-in (which, effectively, means that defaults cannot
change).

I'm not sure if I understand what you're getting at re:
approximately_emulate. I'm in favor of two modes for style loading -- some
kind of lazy version that only touches the options explicitly addressed in
the file, and an explicit version that defaults all other options to
something. Thus, explicit loading would guarantee that a style never
changes.

chris
--
See everything from the browser to the database with AppDynamics
Get end-to-end visibility with application monitoring from AppDynamics
Isolate bottlenecks and diagnose root cause in seconds.
Start your free trial of AppDynamics Pro today!
http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk___
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] Plot or Not: voting to create better matplotlibrc

[Michael Droettboom]

On 07/22/2013 11:34 AM, Chris Beaumont wrote:
I had the same question about opt-out vs opt-in. Personally, I vote 
for opt-out. I would like to see each release of MPL have an 
associated style (which may be the same as the last release, but maybe 
not). With Tony's style PR, users that need constant styles would 
either put `style.use('1.3')` in their script, or `style: 1.3` in an 
rcParams file. This would then freeze the style FOR-EV-ER (sandlot 
voice). However, the default would be `style: latest`, so that the 
default user benefits from the community's effort into making the best 
possible set of defaults. Is that sufficiently pain-free for people 
who want future proofing? Or do you think styles must be opt-in 
(which, effectively, means that defaults cannot change).


In my experience, the vast majority of users don't read changelog notes, 
so we can't expect people to opt out of something that will change 
and/or break their existing plots.  This is particularly a problem where 
matplotlib is upgraded by a sysadmin or distribution (about 58% percent 
of users, by our survey), because it doesn't even get upgraded on the 
user's timetable, necessarily.  So I think any changes to the defaults 
have to be opt-in.  However, we do have a policy of breaking things 
after a release cycle of deprecation warnings.  So we can change the 
defaults in 1.5 after a round of warnings about them in 1.4 (if an 
explicit style is not set).




I'm not sure if I understand what you're getting at re: 
approximately_emulate.


I took Nathaniel's suggestion to just mean that default style should be 
selectable by a version number, so the user doesn't have to keep track 
of the mapping between a style set and a version of matplotlib.


I'm in favor of two modes for style loading -- some kind of lazy 
version that only touches the options explicitly addressed in the 
file, and an explicit version that defaults all other options to 
something. Thus, explicit loading would guarantee that a style never 
changes.


Yes -- I agree there, too.  "set_style" vs. "update_style" perhaps? 
("update" used for its similarity to dict.update?)


Mike
--
See everything from the browser to the database with AppDynamics
Get end-to-end visibility with application monitoring from AppDynamics
Isolate bottlenecks and diagnose root cause in seconds.
Start your free trial of AppDynamics Pro today!
http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk___
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] Plot or Not: voting to create better matplotlibrc

[Nathaniel Smith]

On Mon, Jul 22, 2013 at 2:55 PM, Michael Droettboom  wrote:
> This is why I suggested that the best way forward is to implement some sort
> of easy styling functionality (like what Tony Yu has submitted in #2236,
> though I haven't had a chance to look at it yet), and make it explicit for
> the user to select a new style.  This could be either by adding a new line
> to the top of the plotting script, or adding a single switch in the global
> matplotlibrc file.  But there needs to be an explicit turning on of this.

Are you saying that the defaults can't change (i.e., any changes must
be opt-in), or just that there needs to be some transition period
before the defaults can change and a simple way to opt out afterwards?
I agree with the latter :-).

I have mixed feelings about a full "styling system" though, since, how
many coherent "styles" for plots are there? If we provide a menu of
plot styles right in the main documentation then it seems like it
would just end up being an excuse for people to procrastinate playing
with the different settings, and increase the number of manuscripts I
read that have baroque and poorly chosen colormaps, plots that use the
"ggplot as drawn by xkcd" style, etc. And what value would it really
add? IMO we have a responsibility to nudge users towards making good
plots, and that means having good defaults, and perhaps also means
encouraging people to use them instead of just picking things that
optimize some vague aesthetic judgement at 2am before the submission
deadline...

How about
  mpl.approximately_emulate()
which sets the defaults to whatever version of matplotlib is named?
That could be used a-priori by people who want to future-proof their
scripts, is very easy to add after the fact if you upgrade matplotlib
and discover some plot of yours has been broken, and also encompasses
the "future" functionality (you could ask your current matplotlib to
emulate the next version, if it knows how).

The advantage of a limited API that just takes a version number is not
just that it's simpler on the backend (no need for a system to name
and discover styles, etc. etc.), but it can also easily encapsulate
knowledge like "the defaults were the same from 0.99 through 1.2, so
if anything in that range is requested use *this* file, but then in
1.3...". This means that if a user knows that their plot worked on 1.1
but broke on 1.4, they don't have to care -- they can just say
  mpl.approximately_emulate("1.1")
instead of having to somehow figure out that the right call is:
  mpl.style("0.99-through-1.2")

-n

--
See everything from the browser to the database with AppDynamics
Get end-to-end visibility with application monitoring from AppDynamics
Isolate bottlenecks and diagnose root cause in seconds.
Start your free trial of AppDynamics Pro today!
http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
___
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] How to use STIX fonts in matplotlib plots?

[Michael Droettboom]

On 07/20/2013 10:59 AM, David P. Sanders wrote:



By the way, the following is a useful idiom to search for relevant 
parameters in the rcParams:


[k for k in mpl.rcParams.keys() if 'font' in k]

I think it would be useful to document this -- where would be a good 
place?


You might want to look at rcParams.find_all().

Mike


--
See everything from the browser to the database with AppDynamics
Get end-to-end visibility with application monitoring from AppDynamics
Isolate bottlenecks and diagnose root cause in seconds.
Start your free trial of AppDynamics Pro today!
http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk___
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] MEP14: Improve text handling

[Jouni K . Seppänen]

Michael Droettboom  writes:

> I've drafted a MEP with a plan to improve some of the text and font
> handling in matplotlib.
>
> I'd love any and all feedback.
>
> https://github.com/matplotlib/matplotlib/wiki/Mep14

I'm a bit late to the party, but here are a few thoughts:

What I see as the biggest problem in the current font-selection system
is its opaqueness. You can attempt to specify a style you'd like, but
it's up to the backend to find the relevant font. The naive user has no
way of knowing which font actually got selected, and no way of knowing
how to modify the parameters to get what they want (except if they
stumble upon the way to specify the full path to a font file). Each
backend can override the font-selection code, so e.g. the ps backend
has an option to use only "AFM fonts", meaning the core fonts built into
PostScripts viewers.

The subsetting system proposed in MEP14 (reading the font via FreeType,
then rendering or outputting the outlines into the result) would make
backends consistent with each other, as long as the same text engine is
used. Then at least the OO API could have font selection as an explicit
step, i.e. instead of 

  ax.text(x, y, s, family='serif', style='oblique')

you could write

  font = text_engine.find_font(family='serif', style='oblique')
  ax.text(x, y, s, font=font)

and also query the `font` object for what actual font is being used.
(Or would it look more like ax.text(x, y, text_engine.layout(s, font))?)

If we want to continue support for backend idiosyncracies like
ps.use_afm, I suppose those would need to be parameters to the text
engine.

  

The approach of subsetting fonts by writing a new Type-3 font in the
PostScript or PDF file would allow supporting any fonts that FreeType
can read, but this would lose hinting information in TTF and Type-1
fonts. I think we should at least leave open the possibility to embed
the original font (or a directly-derived subset).

  

The code that parses DVI files from TeX outputs not only glyphs but also
boxes, which are black rectangles used to implement things like the
underscore character and the varying-length part of the square-root
sign. To support this, I guess TextSpan.get_chars should be able to
return not only TextChar instances but also boxes.

-- 
Jouni K. Seppänen
http://www.iki.fi/jks


--
See everything from the browser to the database with AppDynamics
Get end-to-end visibility with application monitoring from AppDynamics
Isolate bottlenecks and diagnose root cause in seconds.
Start your free trial of AppDynamics Pro today!
http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
___
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] How to use STIX fonts in matplotlib plots?

[Michael Droettboom]

On 07/21/2013 04:12 AM, David P. Sanders wrote:


Breaking news from the MathJax site:

The *SVG output processor* is new in MathJax version 2.0, and it uses 
Scalable Vector Graphics to render the mathematics on the page.


Not everything that views SVG is a web browser with Javascript support, 
so doing so would break using the SVG files in Inkscape and Adobe 
Illustrator, for example.  I think that's kind of a non-starter, 
unfortunately.




Mike: Could we use this to finally render all text in STIX *without* 
using an external TeX installation? This would be fantastic!


You already can render all text in STIX without an external TeX 
installation.  That's the purpose of the mathtext support in 
matplotlib.  I agree it has the one wart that the default font also 
needs to be set when using stix for the math, but beyond that, it does 
already work today.


Cheers,
Mike



David


--
See everything from the browser to the database with AppDynamics
Get end-to-end visibility with application monitoring from AppDynamics
Isolate bottlenecks and diagnose root cause in seconds.
Start your free trial of AppDynamics Pro today!
http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk


___
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel


--
See everything from the browser to the database with AppDynamics
Get end-to-end visibility with application monitoring from AppDynamics
Isolate bottlenecks and diagnose root cause in seconds.
Start your free trial of AppDynamics Pro today!
http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk___
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] MEP14: Improve text handling

[Michael Droettboom]

On 07/22/2013 01:05 PM, Jouni K. Seppänen wrote:
> The following message is a courtesy copy of an article
> that has been posted to gmane.comp.python.matplotlib.devel as well.
>
> Michael Droettboom  writes:
>
>> I've drafted a MEP with a plan to improve some of the text and font
>> handling in matplotlib.
>>
>> I'd love any and all feedback.
>>
>> https://github.com/matplotlib/matplotlib/wiki/Mep14
> I'm a bit late to the party

Not too late -- the implementation has barely begun.

> , but here are a few thoughts:
>
> What I see as the biggest problem in the current font-selection system
> is its opaqueness. You can attempt to specify a style you'd like, but
> it's up to the backend to find the relevant font. The naive user has no
> way of knowing which font actually got selected, and no way of knowing
> how to modify the parameters to get what they want (except if they
> stumble upon the way to specify the full path to a font file). Each
> backend can override the font-selection code, so e.g. the ps backend
> has an option to use only "AFM fonts", meaning the core fonts built into
> PostScripts viewers.

Good point.  I should add that in the MEP as an explicit example of 
another case where the font selection needs to be special-cased.

>
> The subsetting system proposed in MEP14 (reading the font via FreeType,
> then rendering or outputting the outlines into the result) would make
> backends consistent with each other, as long as the same text engine is
> used. Then at least the OO API could have font selection as an explicit
> step, i.e. instead of
>
>ax.text(x, y, s, family='serif', style='oblique')
>
> you could write
>
>font = text_engine.find_font(family='serif', style='oblique')
>ax.text(x, y, s, font=font)
>
> and also query the `font` object for what actual font is being used.
> (Or would it look more like ax.text(x, y, text_engine.layout(s, font))?)
>
> If we want to continue support for backend idiosyncracies like
> ps.use_afm, I suppose those would need to be parameters to the text
> engine.

My thinking was that there would just be engine-specific attributes on 
the font selector, e.g.:

font = FontProperties(family="serif", ps_family="Helvetica")

and then you can pass this into the regular text engine or the PS 
AFM-specific one and they would both know what to do.

But maybe that needs some more thinking.  What I want to avoid is the 
current situation where things change radically when switching text 
engines because they *need* to handle fonts so differently. I'd rather 
make that more explicit -- because I don't think there's any way to make 
font selection work the same way across all of them.  I think that's the 
assumption in the current design and it's not great.  It works fine if 
you say "give me a sans serif font, I don't care which", but beyond 
that, the user needs domain-specific knowldege.  Honestly, this is a 
part of the MEP that I think needs work -- I basically threw up my hands 
as a solution to the problem. Maybe there is a better way.

>
>
>
> The approach of subsetting fonts by writing a new Type-3 font in the
> PostScript or PDF file would allow supporting any fonts that FreeType
> can read, but this would lose hinting information in TTF and Type-1
> fonts. I think we should at least leave open the possibility to embed
> the original font (or a directly-derived subset).

Yes.  But that's not a change from current behavior.  We've had 
subsetting fonts as Type 3 fonts for *years* and no one has complained 
about the lack of hinting.  And we do provide an option to embed the 
entire original font if necessary (and no reason to remove that).  
What's new in the MEP is that the subsetting would be based on the 
freetype API (and thus be able to read virtually any font as input), 
rather than ttconv (which can only read well-behaved TrueType fonts with 
Macintosh metadata).  This will allow us to support Microsoft-specific 
TTF fonts (the ones that ship with Windows 7 and 8), OpenType fonts and 
Web fonts.

>
>
>
> The code that parses DVI files from TeX outputs not only glyphs but also
> boxes, which are black rectangles used to implement things like the
> underscore character and the varying-length part of the square-root
> sign. To support this, I guess TextSpan.get_chars should be able to
> return not only TextChar instances but also boxes.
>
Good point.  We'll need that for the built-in mathtext renderer as well, 
for the same reason.

Mike

--
See everything from the browser to the database with AppDynamics
Get end-to-end visibility with application monitoring from AppDynamics
Isolate bottlenecks and diagnose root cause in seconds.
Start your free trial of AppDynamics Pro today!
http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
___
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforg

Re: [matplotlib-devel] Plot or Not: voting to create better matplotlibrc

[Michael Droettboom]

On 07/22/2013 10:42 AM, Nathaniel Smith wrote:
> On Mon, Jul 22, 2013 at 2:55 PM, Michael Droettboom  wrote:
>> This is why I suggested that the best way forward is to implement some sort
>> of easy styling functionality (like what Tony Yu has submitted in #2236,
>> though I haven't had a chance to look at it yet), and make it explicit for
>> the user to select a new style.  This could be either by adding a new line
>> to the top of the plotting script, or adding a single switch in the global
>> matplotlibrc file.  But there needs to be an explicit turning on of this.
> Are you saying that the defaults can't change (i.e., any changes must
> be opt-in), or just that there needs to be some transition period
> before the defaults can change and a simple way to opt out afterwards?
> I agree with the latter :-).

I'm saying that there needs to be a transition period, and the start of 
that transition would be to require an explicit opt-in of the new defaults.

>
> I have mixed feelings about a full "styling system" though, since, how
> many coherent "styles" for plots are there? If we provide a menu of
> plot styles right in the main documentation then it seems like it
> would just end up being an excuse for people to procrastinate playing
> with the different settings, and increase the number of manuscripts I
> read that have baroque and poorly chosen colormaps, plots that use the
> "ggplot as drawn by xkcd" style, etc. And what value would it really
> add? IMO we have a responsibility to nudge users towards making good
> plots, and that means having good defaults, and perhaps also means
> encouraging people to use them instead of just picking things that
> optimize some vague aesthetic judgement at 2am before the submission
> deadline...
>
> How about
>mpl.approximately_emulate()
> which sets the defaults to whatever version of matplotlib is named?
> That could be used a-priori by people who want to future-proof their
> scripts, is very easy to add after the fact if you upgrade matplotlib
> and discover some plot of yours has been broken, and also encompasses
> the "future" functionality (you could ask your current matplotlib to
> emulate the next version, if it knows how).
>
> The advantage of a limited API that just takes a version number is not
> just that it's simpler on the backend (no need for a system to name
> and discover styles, etc. etc.), but it can also easily encapsulate
> knowledge like "the defaults were the same from 0.99 through 1.2, so
> if anything in that range is requested use *this* file, but then in
> 1.3...". This means that if a user knows that their plot worked on 1.1
> but broke on 1.4, they don't have to care -- they can just say
>mpl.approximately_emulate("1.1")
> instead of having to somehow figure out that the right call is:
>mpl.style("0.99-through-1.2")
>
I like this version idea.  (Not sure about the name 
"approximately_emulate", but that's a detail...)  Other tools (sphinx, 
for example) have a way of declaring what version was used when 
something was created.  If the user says:

matplotlib.styling_required('1.2')

then, if we're on 1.3, we do our best to load the styling defaults from 
1.2, and display a warning to the effect "this plot was written for 
matplotlib 1.2, we're running 1.3, so we're entering 1.2 compatibility 
mode, but some things still may be broken.  See the changes 
documentation".  (Again, we can work on the exact wording later).

I think this is a nice approach.  I still think the ability to load 
arbitrary styles from files, online etc. is required, though. There's 
really two issues to resolve here: one is to make sharing of styles 
easier (to have an institutional or publication style, for example), the 
other is to transitition to better defaults with as little breakage and 
pain as possible.  I think we need to do both.

Mike

--
See everything from the browser to the database with AppDynamics
Get end-to-end visibility with application monitoring from AppDynamics
Isolate bottlenecks and diagnose root cause in seconds.
Start your free trial of AppDynamics Pro today!
http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
___
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] MEP19: Continuous Integration

[Jouni K . Seppänen]

Michael Droettboom  writes:

> I've started a MEP related to improving our continuous integration
> system for matplotlib.
>
> https://github.com/matplotlib/matplotlib/wiki/Mep19
>
> Rather than deal to much with implementation at this point, I thought
> it best to start by outlining our requirements.  At this point, let's
> just get everything we'd like in, and we can worry about prioritizing
> things later.

Testing all pull requests means that sandboxing must be taken
seriously. Imagine a pull request that sends spam via email or web
forms, or reads the buildslave password and embeds it in the output. 
I suppose Travis must handle this somehow, but if we're going to roll 
our own, this may need serious thinking.

One thing I would like is to have results from all test cases in a
format that allows them to be compared across the git history and the
build environments, to discover things like "the text tests are failing
with FreeType version X on Python version Y". There's an XUnit XML
plugin for nose, and at least Jenkins has a reporting plugin that can
read that format.

> I would particularly like feedback from others who have set up similar
> things.  I have some experience with ShiningPanda (a service based on
> Jenkins), and Travis.  We used buildbot in the past, but I have little
> direct experience with it.  Are there other obvious candidates or
> approaches?

I've used buildbot at work, but with a much smaller range of build
environments. It takes some work to configure but at least the
configuration file is Python, and build steps can run pretty much any
code. The waterfall display you get with the default settings isn't very
much, but e.g. the Chromium project has a useful-looking setup:

https://chromium-build.appspot.com/p/chromium/console

Other options include at least CircleCI (a paid service), but I have no
experience with it.

-- 
Jouni K. Seppänen
http://www.iki.fi/jks


--
See everything from the browser to the database with AppDynamics
Get end-to-end visibility with application monitoring from AppDynamics
Isolate bottlenecks and diagnose root cause in seconds.
Start your free trial of AppDynamics Pro today!
http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
___
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] MEP19: Continuous Integration

[Michael Droettboom]

On 07/22/2013 03:04 PM, Jouni K. Seppänen wrote:
> The following message is a courtesy copy of an article
> that has been posted to gmane.comp.python.matplotlib.devel as well.
>
> Michael Droettboom  writes:
>
>> I've started a MEP related to improving our continuous integration
>> system for matplotlib.
>>
>> https://github.com/matplotlib/matplotlib/wiki/Mep19
>>
>> Rather than deal to much with implementation at this point, I thought
>> it best to start by outlining our requirements.  At this point, let's
>> just get everything we'd like in, and we can worry about prioritizing
>> things later.
> Testing all pull requests means that sandboxing must be taken
> seriously. Imagine a pull request that sends spam via email or web
> forms, or reads the buildslave password and embeds it in the output.
> I suppose Travis must handle this somehow, but if we're going to roll
> our own, this may need serious thinking.

This should made explicit in the MEP, but I really hope not to our own.  
I'm a developer, not a sysadmin -- I don't have the skills or time to do 
this stuff effectively.

To address your question: both Travis and ShiningPanda (and I suspect 
other hosted testing services) fire up temporary virtual machines for 
each test run.  By design, this virtual machine has no sensitive data on 
it, and thus none to steal in this way. ShiningPanda lets the virtual 
machine be customized upfront, and then cloned and thrown away on each 
test run, and is therefore a little more powerful IMHO.

>
> One thing I would like is to have results from all test cases in a
> format that allows them to be compared across the git history and the
> build environments, to discover things like "the text tests are failing
> with FreeType version X on Python version Y". There's an XUnit XML
> plugin for nose, and at least Jenkins has a reporting plugin that can
> read that format.

Indeed.  Testing on a larger matrix of dependencies is something I'd 
like to do, and the results should be managable by human beings ;)

>
>> I would particularly like feedback from others who have set up similar
>> things.  I have some experience with ShiningPanda (a service based on
>> Jenkins), and Travis.  We used buildbot in the past, but I have little
>> direct experience with it.  Are there other obvious candidates or
>> approaches?
> I've used buildbot at work, but with a much smaller range of build
> environments. It takes some work to configure but at least the
> configuration file is Python, and build steps can run pretty much any
> code. The waterfall display you get with the default settings isn't very
> much, but e.g. the Chromium project has a useful-looking setup:
>
> https://chromium-build.appspot.com/p/chromium/console
>
> Other options include at least CircleCI (a paid service), but I have no
> experience with it.
>
I will add CircleCI to the "to consider" list.

Mike

--
See everything from the browser to the database with AppDynamics
Get end-to-end visibility with application monitoring from AppDynamics
Isolate bottlenecks and diagnose root cause in seconds.
Start your free trial of AppDynamics Pro today!
http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
___
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] 1.3.0rc5 tagged

[Russell E. Owen]

In article <51e9681b.3010...@stsci.edu>,
 Michael Droettboom  
 wrote:

> At long last, I have a 1.3.0rc5 tagged.  I really hope to the software 
> deities that this is the last rc before final.
> 
> If you wouldn't mind creating and posting the binaries, I'll make an 
> announcment on matplotlib-users, give this a week and then put final out.

I have uploaded a binary for MacOS X 10.6 and later.


There were a few oddities this time around:
- matplotlib now requires pyparsing. I don't remember that being a 
requirement before -- even for previous 1.3 candidates.

- I had a lot of trouble with matplotlib complaining that dateutil was 
not present, even though it was in my site-packages. So I tried to 
reinstall it using pip install -U dateutil. Unfortunately pip has never 
heard of "dateutil". After some searching I realized the package name is 
actually "python-dateutil" (and not dateutils, which is a different 
package, alas). Even then, I had to install/upgrade it twice -- for some 
reason matplotlib couldn't find it at first. Very puzzling. I have no 
idea what was going on, but also didn't want to spend a lot of time 
trying to debug it.

- I get a few unit tests failures, including a slew of 
DeprecationWarnings about Operator '<<' that I don't remember seeing 
before. I have appended the test log.

- I first tried building on 10.8 and running on 10.6 (since that's much 
simpler for me). Unfortunately it doesn't work; on 10.6 it acts as if 
the unit tests themselves aren't part of the package. I have no idea 
why. I appended a log snippet showing the basic message, but I haven't 
looked into it further. This sounds worth spending some time tracking 
down.

-- Russell


Log of unit tests, with a few \n inserted for readability.
This is for a package built on 10.6 and running on 10.8.

python -c "import matplotlib as m ; m.test(verbosity=1)"
.
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-pack
ages/matplotlib/mathtext.py:2182: DeprecationWarning: Operator '<<' is 
deprecated, use '<<=' instead
  float_literal << Regex(r"[-+]?([0-9]+\.?[0-9]*|\.[0-9]+)")
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-pack
ages/matplotlib/mathtext.py:2183: DeprecationWarning: Operator '<<' is 
deprecated, use '<<=' instead
  int_literal   << Regex("[-+]?[0-9]+")
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-pack
ages/matplotlib/mathtext.py:2185: DeprecationWarning: Operator '<<' is 
deprecated, use '<<=' instead
  lbrace<< Literal('{').suppress()
...(and 40-odd more examples of the same that I omitted)...
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-pack
ages/matplotlib/mathtext.py:2319: DeprecationWarning: Operator '<<' is 
deprecated, use '<<=' instead
  main  << (non_math + ZeroOrMore(math_string + non_math)) + 
StringEnd()
.
.
..
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-pack
ages/matplotlib/font_manager.py:1236: UserWarning: findfont: Font family 
['sans-serif'] not found. Falling back to Helvetica
  (prop.get_family(), self.defaultFamily[fontext]))
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-pack
ages/matplotlib/font_manager.py:1246: UserWarning: findfont: Could not 
match 
:family=Helvetica:style=normal:variant=normal:weight=normal:stretch=norma
l:size=medium. Returning 
/Applications/TUI.app/Contents/Resources/mpl-data/fonts/afm/putri8a.afm
  UserWarning)
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-pack
ages/matplotlib/font_manager.py:1246: UserWarning: findfont: Could not 
match 
:family=Helvetica:style=normal:variant=normal:weight=normal:stretch=norma
l:size=24.0. Returning 
/Applications/TUI.app/Contents/Resources/mpl-data/fonts/afm/putri8a.afm
  UserWarning)
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-pack
ages/matplotlib/font_manager.py:1246: UserWarning: findfont: Could not 
match 
:family=Helvetica:style=normal:variant=normal:weight=normal:stretch=norma
l:size=large. Returning 
/Applications/TUI.app/Contents/Resources/mpl-data/fonts/afm/putri8a.afm
  UserWarning)
F...kpathsea: Invalid fontname `Bitstream Vera Serif', contains ' '
Ekpathsea: Invalid fontname `Bitstream Vera Serif', contains ' '
EK.K.
.
.
.
.
.

Re: [matplotlib-devel] 1.3.0rc5 tagged

[Michael Droettboom]

On 07/22/2013 05:59 PM, Russell E. Owen wrote:
> In article <51e9681b.3010...@stsci.edu>,
>   Michael Droettboom 
>   wrote:
>
>> At long last, I have a 1.3.0rc5 tagged.  I really hope to the software
>> deities that this is the last rc before final.
>>
>> If you wouldn't mind creating and posting the binaries, I'll make an
>> announcment on matplotlib-users, give this a week and then put final out.
> I have uploaded a binary for MacOS X 10.6 and later.
>
>
> There were a few oddities this time around:
> - matplotlib now requires pyparsing. I don't remember that being a
> requirement before -- even for previous 1.3 candidates.

It's been a requirement for time immemorial, but only in 1.3.0 (all of 
the release candidates) has it become an external dependency. What error 
occurred that suggests something changed?

>
> - I had a lot of trouble with matplotlib complaining that dateutil was
> not present, even though it was in my site-packages. So I tried to
> reinstall it using pip install -U dateutil. Unfortunately pip has never
> heard of "dateutil". After some searching I realized the package name is
> actually "python-dateutil" (and not dateutils, which is a different
> package, alas). Even then, I had to install/upgrade it twice -- for some
> reason matplotlib couldn't find it at first. Very puzzling. I have no
> idea what was going on, but also didn't want to spend a lot of time
> trying to debug it.

Does `python setup.py install` (of matplotlib) not install it 
automatically?  We are bearing all the pain of setuptools in order for 
that to happen, so if it's not, that's a real problem.

>
> - I get a few unit tests failures, including a slew of
> DeprecationWarnings about Operator '<<' that I don't remember seeing
> before. I have appended the test log.

That's probably related to pyparsing 2.0.1 (released just this week).  
I'd like to fix those warnings, but then we'd have to *require* 
pyparsing 2.0.1 (no earlier Python 2.x release of pyparsing includes an 
alternative syntax).  I think pyparsing moved too quickly on this one, 
but I'm not sure what we can do about it. It does make me long for the 
days when we included our dependencies so we have some control over this 
stuff.

>
> - I first tried building on 10.8 and running on 10.6 (since that's much
> simpler for me). Unfortunately it doesn't work; on 10.6 it acts as if
> the unit tests themselves aren't part of the package. I have no idea
> why. I appended a log snippet showing the basic message, but I haven't
> looked into it further. This sounds worth spending some time tracking
> down.

That's a puzzler.  I've seen that crop up on Travis erratically as well, 
but not consistently.  It's not clear what's going on here -- will have 
to think on it.

Thanks for all of your work on this!

Mike

>
> -- Russell
>
>
> Log of unit tests, with a few \n inserted for readability.
> This is for a package built on 10.6 and running on 10.8.
>
> python -c "import matplotlib as m ; m.test(verbosity=1)"
> .
> /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-pack
> ages/matplotlib/mathtext.py:2182: DeprecationWarning: Operator '<<' is
> deprecated, use '<<=' instead
>float_literal << Regex(r"[-+]?([0-9]+\.?[0-9]*|\.[0-9]+)")
> /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-pack
> ages/matplotlib/mathtext.py:2183: DeprecationWarning: Operator '<<' is
> deprecated, use '<<=' instead
>int_literal   << Regex("[-+]?[0-9]+")
> /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-pack
> ages/matplotlib/mathtext.py:2185: DeprecationWarning: Operator '<<' is
> deprecated, use '<<=' instead
>lbrace<< Literal('{').suppress()
> ...(and 40-odd more examples of the same that I omitted)...
> /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-pack
> ages/matplotlib/mathtext.py:2319: DeprecationWarning: Operator '<<' is
> deprecated, use '<<=' instead
>main  << (non_math + ZeroOrMore(math_string + non_math)) +
> StringEnd()
> .
> .
> ..
> /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-pack
> ages/matplotlib/font_manager.py:1236: UserWarning: findfont: Font family
> ['sans-serif'] not found. Falling back to Helvetica
>(prop.get_family(), self.defaultFamily[fontext]))
> /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-pack
> ages/matplotlib/font_manager.py:1246: UserWarning: findfont: Could not
> match
> :family=Helvetica:style=normal:variant=normal:weight=normal:stretch=norma
> l:size=medium. Returning
> /Applications/TUI.app/Contents/Resources/mpl-data/fonts/afm/putri8a.afm
>UserWarning)
> /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-pack
> ages/matplotlib/font_manager.py:1246: Us