Re: [matplotlib-devel] Nose tests: Font mismatch

2009-09-08 Thread Andrew Straw
John Hunter wrote: > On Tue, Sep 8, 2009 at 12:34 PM, Andrew Straw wrote: > >> Michael Droettboom wrote: >> >>> More information after another build iteration. >>> >>> The two tests that failed after updating to the unhinted images were >>> subtests of tests that were failing earlier. If a

Re: [matplotlib-devel] Nose tests: Font mismatch

2009-09-08 Thread John Hunter
On Tue, Sep 8, 2009 at 12:34 PM, Andrew Straw wrote: > Michael Droettboom wrote: >> More information after another build iteration. >> >> The two tests that failed after updating to the unhinted images were >> subtests of tests that were failing earlier.  If a single test >> function outputs multip

Re: [matplotlib-devel] Nose tests: Font mismatch

2009-09-08 Thread Andrew Straw
John Hunter wrote: > I wrote a script at scipy when Andrew and I worked on this to > recursively move known good actuals into the baselines directory, with > some yes/no prompting, but it looks like it did not survive the test > code migration, so we may want to develop something to replace it. Yes

Re: [matplotlib-devel] Nose tests: Font mismatch

2009-09-08 Thread Andrew Straw
Michael Droettboom wrote: > More information after another build iteration. > > The two tests that failed after updating to the unhinted images were > subtests of tests that were failing earlier. If a single test > function outputs multiple images, image comparison stops after the > first mismatch

Re: [matplotlib-devel] Nose tests: Font mismatch

2009-09-08 Thread Michael Droettboom
More information after another build iteration. The two tests that failed after updating to the unhinted images were subtests of tests that were failing earlier. If a single test function outputs multiple images, image comparison stops after the first mismatched image. So there's nothing pecu

Re: [matplotlib-devel] Nose tests: Font mismatch

2009-09-08 Thread Andrew Straw
Michael Droettboom wrote: > Interesting result. I pulled all of the new "actual" files from the 21 > failing tests on the buildbots to my local machine and all of those > tests now pass for me. Good. Interestingly, there are still two tests > failing on my machine which did not fail on the bu

Re: [matplotlib-devel] Nose tests: Font mismatch

2009-09-08 Thread John Hunter
On Tue, Sep 8, 2009 at 11:47 AM, Michael Droettboom wrote: > Interesting result.  I pulled all of the new "actual" files from the 21 > failing tests on the buildbots to my local machine and all of those tests > now pass for me.  Good.  Interestingly, there are still two tests failing on > my machin

Re: [matplotlib-devel] Nose tests: Font mismatch

2009-09-08 Thread Michael Droettboom
Interesting result. I pulled all of the new "actual" files from the 21 failing tests on the buildbots to my local machine and all of those tests now pass for me. Good. Interestingly, there are still two tests failing on my machine which did not fail on the buildbots, so I can't grab the buil

Re: [matplotlib-devel] Nose tests: Font mismatch

2009-09-08 Thread Andrew Straw
Michael Droettboom wrote: > Doing so, my results are even *less* in agreement with the baseline, but > the real question is whether my results are in agreement with those on > the buildbot machines with this change to forcibly turn hinting off. I > should no pretty quickly when the buildbots st

Re: [matplotlib-devel] Nose tests: Font mismatch

2009-09-08 Thread Andrew Straw
John Hunter wrote: > Perhaps with hinting turned off this won't be necessary. Ie, maybe we > can get more agreement across a wide range of freetype versions w/o > hinting. Are you planning on committing the unhinted baselines? I have a presentation to give tomorrow, so I'd just as soon let you an

Re: [matplotlib-devel] Nose tests: Font mismatch

2009-09-08 Thread Darren Dale
On Tue, Sep 8, 2009 at 11:46 AM, Andrew Straw wrote: > Michael Droettboom wrote: >> On 09/08/2009 10:24 AM, John Hunter wrote: >> >>> On Tue, Sep 8, 2009 at 8:54 AM, Michael Droettboom  wrote: >>> >>> I've been only skimming the surface of the discussion about the new test framework up un

Re: [matplotlib-devel] Nose tests: Font mismatch

2009-09-08 Thread John Hunter
On Tue, Sep 8, 2009 at 10:46 AM, Andrew Straw wrote: > While the default test usage should probably set as much as possible to > ensure things are identical, we also want to be able to test other code > paths, so I think I'll add some kind of kwarg to matplotlib.test() to handle > non-testing-defa

Re: [matplotlib-devel] Nose tests: Font mismatch

2009-09-08 Thread Andrew Straw
Michael Droettboom wrote: > On 09/08/2009 10:24 AM, John Hunter wrote: > >> On Tue, Sep 8, 2009 at 8:54 AM, Michael Droettboom wrote: >> >> >>> I've been only skimming the surface of the discussion about the new test >>> framework up until now. >>> >>> Just got around to trying it, and

Re: [matplotlib-devel] Nose tests: Font mismatch

2009-09-08 Thread Michael Droettboom
On 09/08/2009 10:24 AM, John Hunter wrote: > On Tue, Sep 8, 2009 at 8:54 AM, Michael Droettboom wrote: > >> I've been only skimming the surface of the discussion about the new test >> framework up until now. >> >> Just got around to trying it, and every comparison failed because it was >> sele

Re: [matplotlib-devel] Nose tests: Font mismatch

2009-09-08 Thread John Hunter
On Tue, Sep 8, 2009 at 8:54 AM, Michael Droettboom wrote: > I've been only skimming the surface of the discussion about the new test > framework up until now. > > Just got around to trying it, and every comparison failed because it was > selecting a different font than that used in the baseline ima