On Sun, Sep 6, 2009 at 12:58 AM, Andrew Straw<straw...@astraw.com> wrote:
> Today I committed to svn a simplified testing infrastructure, which I've
> committed to matplotlib/testing/*. A few sample tests are in
> matplotlib/tests/*. I also wrote some docs, which are now in the
> developer coding guide. See that (
> http://matplotlib.svn.sourceforge.net/viewvc/matplotlib/trunk/matplotlib/doc/devel/coding_guide.rst?revision=7654&view=markup
> , starting at line 675) for information about how to write tests.
>
> Now, I have a question. As currently written, the baseline (a.k.a.
> expected) images are stored in the sample_data directory and
> matplotlib.cbook.get_sample_data() downloads the images. However, I
> suspect that the Mac Sage buildslave doesn't like to download stuff
> while not in an interactive login. (Remember the initial problems
> running tests on that machine?) That's probably a good indication that
> we probably don't want to require network access to run the tests. So,
> the next question is whether we want to install baseline images with
> standard MPL installs so that any user can run the full test suite? That
> would be my preference, as it would be the simplest and most robust to
> implement, but it comes at the cost of using additional disk space.
> Otherwise, I'm open to suggestions.
>
> (John, to confirm my suspicions about the network access issue, could
> you ssh into the Sage Mac and run test/_buildbot_mac_sage.sh by hand to
> see if that eliminates the bus error we're getting when run from the
> buildbot?)
I was able to run the buildbot mac script when logged into sage with::
Test plotting empty axes with dates along one axis. ... KNOWNFAIL:
Fails due to SF bug 2850075
Test Some formatter and ticker issues. ... ok
Test the 'is_string_like cookbook' function. ... ok
Test DateFormatter ... ok
Test RRuleLocator ... ok
Basic Annotations ... ok
Polar Plot Annotations ... ok
Polar Coordinate Annotations ... ok
Test the fill method with unitized-data. ... ok
Test constant xy data. ... ok
Test numpy shaped data. ... ok
Test single-point date plots. ... ok
Test single-point plots. ... ok
Test polar plots with unitized data. ... ok
Test polar plots where data crosses 0 degrees. ... ok
Test the axhspan method with Epochs. ... ok
Test the axvspan method with Epochs. ... ok
very simple example test ... ok
very simple example test that should fail ... KNOWNFAIL: Test known to fail
matplotlib.tests.test_transforms.test_Affine2D_from_values ... ok
matplotlib.tests.test_spines.test_spines_axes_positions ... ok
----------------------------------------------------------------------
Ran 21 tests in 8.342s
OK (KNOWNFAIL=2)
I am not sure I want to distribute the baseline images with the main
mpl distribution, but I am open to considering it. As the number of
tests and baseline images grows, which hopefully will happen soon,
this could potentially become large -- the reason I added
get_sampledata in the first place was to get the distribution size
down. We could add support to get_sampledata to use an environment
variable for the cache directory. Then I could do an svn checkout of
the sample_data tree and svn up this dir in my buildbot script. If I
point the sample data environment var to this directory, it would have
the latest data for the buildbot and would not need to make an http
request (it is odd though that svn checkouts on the sage buildbot work
fine even when not interactively logged in but http requests
apparently do not).
If you think the sample_data w/ support for local svn checkouts is the
way to go for the baseline data and images, let me know. I would like
to utilize a subdir, eg, sample_data/baseline, if we go this route, to
keep the top-level directory a bit cleaner for user data. We could
also release a tarball of the sample_data/baseline directory with each
release, so people who want to untar, set the environment var and test
could do so.
I am not sure this is the right approach by any means, just putting it
up for consideration. One disadvantage of the sample_data approach is
that it would probably work well with HEAD but not with releases,
because as the baseline images changes, it becomes difficult to test
existing releases against it, which may be assuming a prior baseline.
This is why I mentioned releasing the baseline images too, but it does
raise the barrier for doing tests.
JDH
------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
