yes, we would be sharing it with the community and get this running in the ACS infra. Currently it can create a cloudstack test bed, runs tests and email the results.
Here are some details on how this works and what is needed to set this up. * we use jenkins, cobbler, puppet and marvin to create cloudstack setup. * jenkins triggers the test runs, collects the test results and mails them. * cobbler is use to image the hosts and create Management server. * The management server is a VM and each time a test run is triggered we pull the latest code, build (dev setup) the MS and run it. * Need IPMI enabled servers to uses and Hosts in cloudstack setup. Cobbler installs the required OS on these hosts. * We use a XenServer to create management server VMs. The resources required to set this up. * We need two servers to host the VMs used in CI, one XenServer to host the Cloustack management servers and at least 3 IPMI enabled servers per cloudstack setup to run the BVTs. * some set of IPs (public and private IPs) and vlans. Once we have the resources in ACS infra we can start setting this up. But some work needs to be done to integrate this with the github to test and post the results in the PRs instead of mailing them. I think the best way to share it will be by implementing this in the ACS infra. Once we do this every one can pitch in, replicate and further contribute to this. Meanwhile i will commit the scripts to set this up and keep this going. Thanks, Bharat. On 28-Jan-2016, at 7:37 PM, Erik Weber <terbol...@gmail.com<mailto:terbol...@gmail.com>> wrote: Why not share it as is, then the community could help improving this, rather than this being a single company effort? -- Erik On Thu, Jan 28, 2016 at 1:49 PM, Bharat Kumar <bharat.ku...@citrix.com<mailto:bharat.ku...@citrix.com>> wrote: Hi All, I agree that we need to have a CI to deal with the large volume of PRs. The current travis CI is not good enough as it runs only simulator tests. We identified this issue and came up with a effective CI for automating test runs for a each PR. This is already functional, with few github integration aspects pending. We are internally stabilizing it before sharing it. We have been in touch with David Nalley ( CC’ed ) in making this operational for entire community using ACS infra. For your reference, here is the FS I have shared with the community earlier and also in this thread before, your feedback is welcome. ( https://cwiki.apache.org/confluence/display/CLOUDSTACK/Test+bed+orchestrator+and+test+runner+to+enable+continuos+integration ). Thanks, Bharat. On 28-Jan-2016, at 4:26 PM, Rohit Yadav <rohit.ya...@shapeblue.com<mailto: rohit.ya...@shapeblue.com>> wrote: All, I’m sorry to get to have the PRs merged without adhering to the strict testing requirements. While I think PRs were alright and it did not break anything, the way it was merged made people uncomfortable that there is some sort of haste in doing this fast which there is none. I’ll not repeat this and hope you understand that I never had any hidden agenda but to simply help people with some PRs. Regards. On 28-Jan-2016, at 11:36 AM, Sebastien Goasguen <run...@gmail.com<mailto: run...@gmail.com>> wrote: Hi Folks, My proposal to freeze until we get CI was indeed due to seeing Rohit’s commit but was by no means a personal attack or judgment. We have lots of PR pending (as mentioned before by Remi) and we need people to help review and test. So thanks to Rohit. My only concerns were two fold: 1- We need to keep to adhere to our release principles: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Release+principles+for+Apache+CloudStack+4.6+and+up Hence I replied to some PR asking if they needed to be merged directly in master or not and wondered about the release branches. With so many releases in flight it is not yet clear to me where we start to apply a PR ? 2- We need to keep testing and post results of tests. Currently it is manual and but there has been a strong guarantee in the last releases that the PR where not going to break things. While I agree that some PR are small and *should* not break things, history has shown that even small unrelated things *somehow* can affect the behavior of cloudstack. So I proposed a freeze because: - Remi stepped down as RM and we don’t have an official RM yet. - The code has reached a solid state and we don’t want to do anything that changes that - We have a proposal for LTS on the floor - We still don’t have CI. So my standpoint is that we focused in the last 6 months on getting our release principles right (pending LTS principles), code has stabilized and we can release. Awesome. Now is probably a good time to concentrate our limited resources on figuring out automated CI. - For instance as far as I know Travis is bonkers…(reports green but does not do anything) - And with citrix stepping out, we need to take control of the jenkins slaves (some of which are on AWS and still paid by Citrix…) My email while triggered by seeing Rohit’s commits, was not a judgement or critic of his actions, so let’s not get into a personal argument here. -Sebastien On Jan 28, 2016, at 11:00 AM, Rohit Yadav <rohit.ya...@shapeblue.com> wrote: So, since some have directly (over IM etc) or indirectly have thrown allegations on me since I merged most of the PRs. Here is a list of those 12 PRs and answers on why they were merged on case-by-case basis. Please keep any further replies technical and to the specific PR, please point out and revert if needed: 1. https://github.com/apache/cloudstack/pull/1288 Enough LGTMs, JS related change and fix tested with UI screenshot from Remi. I personally looked at the diff and therefore then merged. 2. https://github.com/apache/cloudstack/pull/1274/files Enough LGTMs, a simple NPE fix one-liner. I personally thought we can cheat here and given Travis/Jenkins passed I merged it. 3. https://github.com/apache/cloudstack/pull/1261/files Enough LGTMs, the diff only removed unused variable leading to change in the constructor definition. Explicit integration tests are not necessary as it’d simply dead-code removal and as the simulator smoke tests passed with Travis/Jenkins passed so I merged it. 4. https://github.com/apache/cloudstack/pull/1048 Enough LGTMs. This change is related to a marvin test itself where it adds 2 new test methods — so no need to run regression integration test. The integration test result of the marvin test was shared in the comment. PR merged on this basis. 5. https://github.com/apache/cloudstack/pull/1044 Enough LGTMs and regression tests results (shared as attachments by Daan, in case someone missed), so merged. 6. https://github.com/apache/cloudstack/pull/969 Enough LGTMs and regression tests results (shared as attachments by Daan, in case someone missed), so merged. 7. https://github.com/apache/cloudstack/pull/855 Enough LGTMs and regression tests results by Remi, so merged. 8. https://github.com/apache/cloudstack/pull/831 Enough LGTMs and only text changes in API doc-string so merged given Travis/Jenkins passed. 9. https://github.com/apache/cloudstack/pull/830/files Enough LGTMs and NPE fixes, so no explicit integration tests required given Travis/Jenkins passed. 10. https://github.com/apache/cloudstack/pull/1256/files Enough LGTMs and simple Java OOP fix with Travis/Jenkins passed so merged this. I’m aware of this codebase. 11. https://github.com/apache/cloudstack/pull/1240 Enough LGTMs, the changes would require manual tests wrt usage server etc as well as confirmed in comments. I had seen the regression test result (of the new/modified marvin test wrt of the feature) so merged. The regression test suite does not include this among other tests. 12. https://github.com/apache/cloudstack/pull/1289 Enough LGTMs, this was a findbugs related fix. Travis/Jenkins passed on it and the findbugs mvn job result was shared to confirm that the fix works now. This was not merged by me. Regards. Rohit Yadav Software Architect , ShapeBlue d: | s: +44 203 603 0540 | m: +91 8826230892 e: rohit.ya...@shapeblue.com | t: | w: www.shapeblue.com a: 53 Chandos Place, Covent Garden London WC2N 4HS UK Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark. This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Find out more about ShapeBlue and our range of CloudStack related services: IaaS Cloud Design & Build | CSForge – rapid IaaS deployment framework CloudStack Consulting | CloudStack Software Engineering CloudStack Infrastructure Support | CloudStack Bootcamp Training Courses Regards. Find out more about ShapeBlue and our range of CloudStack related services: IaaS Cloud Design & Build< http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> | CloudStack Software Engineering< http://shapeblue.com/cloudstack-software-engineering/> CloudStack Infrastructure Support< http://shapeblue.com/cloudstack-infrastructure-support/> | CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
