1. If tests are failing and it is already a known product issue, we should find a way to show it as other away from "failed". This can add some enhancements to test cases, map bugs logged to failures and only new failures away from those. If it is a known issue, the failure should report it as known issue rather than failure. I believe we can add these facilitation to marvin.
2. Also, if we are to integrate test cases from some test case management solution to test cases under repo, there should be mapping faciliation provided for each test case in tms to test caseid or name in tests and reverse faciliation to update results for those test cases. This can be more of automated approach as an integration to test management solution i believe. Santhosh ________________________________________ From: Ashutosh Kelkar [nore...@reviews.apache.org] on behalf of Ashutosh Kelkar [ashut...@clogeny.com] Sent: Tuesday, November 19, 2013 5:52 AM To: Girish Shilamkar; Srikanteswararao Talluri Cc: Ashutosh Kelkar; cloudstack; Prasanna Santhanam Subject: Re: Review Request 15646: CLOUDSTACK-5197: portable ip - Skipping the tests which are failing due to product issue ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15646/ ----------------------------------------------------------- (Updated Nov. 19, 2013, 10:52 a.m.) Review request for cloudstack, Girish Shilamkar and SrikanteswaraRao Talluri. Changes ------- Removing skip tag from test cases which are failing due to product issue. Bugs: CLOUDSTACK-5197 https://issues.apache.org/jira/browse/CLOUDSTACK-5197 Repository: cloudstack-git Description ------- 2 test cases are failing while SSHing to the VM through portable ip. Logged a product defect for this - https://issues.apache.org/jira/browse/CLOUDSTACK-5196. Skipping the test cases until the product defect is resolved. Diffs (updated) ----- test/integration/component/test_portable_ip.py d397ec4 Diff: https://reviews.apache.org/r/15646/diff/ Testing ------- Thanks, Ashutosh Kelkar
