> -----Original Message----- > From: crowbar-bounces On Behalf Of Adam Spiers > Sent: Monday, March 11, 2013 2:24 PM > To: crowbar > Subject: [Crowbar] testing and pull requests > > "What testing do we run before submitting?" is a key question; thanks for > bringing it up. Unfortunately the answer is currently "only whatever we run > manually", since Travis only runs on a merged repository containing master > from each barclamp repository, due to barclamps currently being unable to run > their unit tests standalone. > (There are Trello cards to address this, but it's unlikely to get fixed soon.)
The general pattern we've been following is to have barclamps extend framework provided classes, and that code currently resides in crowbar/crowbar_framework... so to get to a point where we can run unit tests for standalone barclamps will require copying in the framework.... (chicken, meet egg...). > > That means we all need to be as conscientous as possible by running tests > before every pull request, otherwise master is likely to break often. I just > spoke > with James and agree with his proposal to prototype barclamps as gems which > can run their unit tests standalone (I get the impression engines should help > with this but I am not an expert in the area). That would allow Travis to > run on > pull requests to individual barclamp repos. A possible short term approach that would give us testing prior to pull requests being merged would be to reflect Pull requests from the different repo, into the travis repo. How is this different from what's happening now? Now, the update-git script checks for changed files between the travis repo (as checked-out locally on the server running the script) and the main crowbar repo. If it finds differences, it pushes an update to the travis-ci-crowbar repo. This process will only catch all the merged pull requests. The approach I proposing works differently: On a periodic basis, pull the crowbar repos (crowbar and each barclamp) for pending pull requests. For each open pull request, create a pull request against the travis-ci-crowbar repo. This in turn will trigger travis to test the pull request, before it is actually merged. There are a few complications that need to be handled (but I believe a solution exists for both): a) Bundles - often there are sets of pull requests against different barclamps (or framework + barclamp) that will only pass tests if they're evaluated as a 'bundle' (i.e. both sets of changes are required). b) Treating the separate barclamp repos and the framework as one repo from travis's perspective. There is some git magic that can let us setup a repo for travis-ci to monitor, that appears ""flat"" while still maintaining the correct relationship to the real repos (so we can identify what pull-request on what barclamp actually failed). [3] I was playing with some of these over the weekend, and am likely to continue playing with this for a bit till it's somewhat at least wobbly [4] and [5] for instructions but shows promise. That is, unless someone sees a some horrible flaw in the above. [1] https://github.com/crowbar/crowbar/blob/master/travis-ci/update-git.sh [2] https://github.com/crowbar/travis-ci-crowbar [3] http://git-scm.com/book/ch6-7.html (following links are totally optional, but feel free to youtube search "wobble dance") [4] http://www.youtube.com/watch?v=qd6UI6wEIsU [5] http://www.youtube.com/watch?v=8vTIY0xHBUg > > However unfortunately we can't prioritise that over the current dev work, so > for the next few weeks at very least I expect we will just all have to be > self- > disciplined and not submit pulls for untested code ;-) > _______________________________________________ Crowbar mailing list [email protected] https://lists.us.dell.com/mailman/listinfo/crowbar For more information: http://crowbar.github.com/
