On 01/30/2017 05:39 PM, Daniel Ruggeri wrote:
I'm tremendously inspired by this work. What are your thoughts on the idea of having a series of docker container builds that compile and run the test suite on various distributions? I'll volunteer to give this a whack since it's something that's been in the back of my mind for a long while...
I think that would be awesome. The cheaper we can make new test distributions, the easier we can test all sorts of different configurations (which, given how many knobs and buttons we expose, is important).
I don't know how much of Infra's current Puppet/Buildbot framework is Docker-friendly, but if there's currently no cheap virtualization solution there for build slaves, then anything we added would potentially be useful for other ASF projects as well. Definitely something to start a conversation over.
(Side thought: since raw speed is typically one of the top priorities for a CI test platform, we'd have to carefully consider which items we tested by spinning up containers and which we ran directly on a physical machine. Though I don't know how fast Docker has gotten with all of the fancy virtualization improvements.)
I think with the work you've done and plan to do, a step like above to increase our ability to test against many distributions all at once (and cheaply) and also making the test framework more approachable, we could seriously increase our confidence when pulling the trigger on a release or accepting a backport.
+1. It'll take some doing (mostly focused on the coverage of the test suite itself), but we can get there.
I'm also a big fan of backports requiring tests, but am honestly intimidated by the testing framework...
What would make it less intimidating for you? (I agree with you, but I'm hoping to get your take without biasing it with my already-strong opinions. :D)
--Jacob
