Michael Droettboom <md...@stsci.edu> writes: > I've started a MEP related to improving our continuous integration > system for matplotlib. > > https://github.com/matplotlib/matplotlib/wiki/Mep19 > > Rather than deal to much with implementation at this point, I thought > it best to start by outlining our requirements. At this point, let's > just get everything we'd like in, and we can worry about prioritizing > things later.
Testing all pull requests means that sandboxing must be taken seriously. Imagine a pull request that sends spam via email or web forms, or reads the buildslave password and embeds it in the output. I suppose Travis must handle this somehow, but if we're going to roll our own, this may need serious thinking. One thing I would like is to have results from all test cases in a format that allows them to be compared across the git history and the build environments, to discover things like "the text tests are failing with FreeType version X on Python version Y". There's an XUnit XML plugin for nose, and at least Jenkins has a reporting plugin that can read that format. > I would particularly like feedback from others who have set up similar > things. I have some experience with ShiningPanda (a service based on > Jenkins), and Travis. We used buildbot in the past, but I have little > direct experience with it. Are there other obvious candidates or > approaches? I've used buildbot at work, but with a much smaller range of build environments. It takes some work to configure but at least the configuration file is Python, and build steps can run pretty much any code. The waterfall display you get with the default settings isn't very much, but e.g. the Chromium project has a useful-looking setup: https://chromium-build.appspot.com/p/chromium/console Other options include at least CircleCI (a paid service), but I have no experience with it. -- Jouni K. Seppänen http://www.iki.fi/jks ------------------------------------------------------------------------------ See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk _______________________________________________ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
