Hi, I agree with Remi on the hurdles he mentioned. It is difficult to integrate 3rd party CI, If someone wants to implement their own CI, the link below gives one way to do it. https://cwiki.apache.org/confluence/display/CLOUDSTACK/Test+bed+orchestrator+and+test+runner+to+enable+continuos+integration
Thanks, Bharat. On 28-Jan-2016, at 2:52 AM, Wido den Hollander <w...@widodh.nl<mailto:w...@widodh.nl>> wrote: On 01/27/2016 10:02 PM, Remi Bergsma wrote: Hi all, We should keep the simple approach that was used until now: one LGTM based on code review and one LGTM based on integration tests (that’s not the same as 2xLGTM). If we care about master stability, every change has to be tested for regression. Period. Things may look OK, but still break something else in an unexpected way. IMHO, giving up on it would be a shame. In a perfect world, CI would automate this for us and run it against every PR. Current impediments I see for proper CI on every PR: - there is no community hardware to run integration tests on (there was supposed to be something) - no support from Apache Infra: need Github write access (also to hook 3rd party CI). It’s nowhere close. - the community has very few people that are able to run the integration tests on their own laptop/hardware and post the results on PRs We tried to address points 1 and 2 and I propose to give up on them. If all devs would stop developing for a couple of days and make sure they have a box that can run KVM, then clone the code we use [1] it should be pretty straight forward to run the integration tests. We choose to make it all virtual and used nested-virtualisation, but that is optional. A simple Intel Nuc or similar will do. Yes, that's great code. I still have to master it, but the README should get us there. I still need to create one which also spins up a Ceph cluster. Probably a good thing to do now ;) Wido People can then test the PRs that they find interesting and post results, after which it can be merged. Distribute it. Share the load. This is what Wilder, Miguel, Boris, Daan an I have been doing for months. Simply running the tests. Hundreds of times. That’s why we can run a 100% Mission Critical Cloud close to master branch at Schuberg Philis. Regards, Remi [1] https://github.com/schubergphilis/MCT-shared/ On 27/01/16 21:33, "Sebastien Goasguen" <run...@gmail.com<mailto:run...@gmail.com>> wrote: On Jan 27, 2016, at 9:25 PM, Wido den Hollander <w...@widodh.nl<mailto:w...@widodh.nl>> wrote: On 01/27/2016 09:18 PM, Sebastien Goasguen wrote: Folks, How about we freeze our repo entirely until we get proper CI in place. Seems to me all the hard work from Remi and co could be lost if we start committing again. Now Travis is not running, Jenkins fails all the time and nobody cares… So how about we figure out CI now ? and not do anything else. I think forces have to be combined to make this work. Questions which come to mind: Who runs Jenkins? Do we need a additional slave? I haven't figured out the Integration tests completely personally. In an ideal case, PR should trigger tests totally distributed on everyone’s own hardware. Then tests would report back on the PR. Only when all are green can we merge. there is an issue with creating triggers in github on our own, but I think that’s what we should aspire to. for instance, how can pcextreme automate its testing and report back on each PR ? Wido -Sebastien
