Sure Rohit, sometimes it makes no sense to run integration tests. That’s why for example I build an environment and check the UI when someone changes a javascript file.
Just saying it “looks” good is valuable from a code review point of view, but not from a verification point of view. I say we always need both. As a general rule, every PR should be “tested” so one way or the other you need to show it works. Can be integration tests, can be UI screenshot, can be successful package build result or whatever. Currently, there is not much real “testing” going on and people expect this to magically happen somehow by someone else. Currently I feel quite alone in performing that "magic". I can’t and won’t keep that up. Regards, Remi From: Rohit Yadav <rohit.ya...@shapeblue.com<mailto:rohit.ya...@shapeblue.com>> Reply-To: "dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>" <dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>> Date: Thursday 28 January 2016 09:50 To: "dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>" <dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>> Subject: Re: [PROPOSAL] Freeze everything until we get CI [ShapeBlue]<http://www.shapeblue.com> Rohit Yadav Software Architect , ShapeBlue d: | s: +44 203 603 0540<tel:|%20s:%20+44%20203%20603%200540> | m: +91 8826230892<tel:+91%208826230892> e: rohit.ya...@shapeblue.com | t: <mailto:rohit.ya...@shapeblue.com%20|%20t:> | w: www.shapeblue.com<http://www.shapeblue.com> a: 53 Chandos Place, Covent Garden London WC2N 4HS UK [cid:image24a672.png@c43ecb78.4d9cef18] Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark. This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. On 27-Jan-2016, at 10:02 PM, Remi Bergsma <rberg...@schubergphilis.com<mailto:rberg...@schubergphilis.com>> wrote: We should keep the simple approach that was used until now: one LGTM based on code review and one LGTM based on integration tests (that’s not the same as 2xLGTM). In general, a good idea though integration tests won’t apply everywhere to help us maintain quality. We can cheat by avoiding integration tests but do other kinds of tests (manually for now) based on that Travis/Jenkins strictly pass and diffs are entirely: - Text based in API doc/params for example, README files, comments, etc. - Non-java code, not even systemvm related scripts where they may require alternative tests; such as changes in systemvm-template building scripts, marvin, docker etc. - Javascript file changes, as that would require some manual testing and screenshots and won’t be caught by integration tests Regards. Find out more about ShapeBlue and our range of CloudStack related services: IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> | CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/> CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/> | CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
