We (Abhi, Boris and me) opened couple of marvin test bugs to automate the gaps [1], we will start to work on them. If any one wishes to contribute please feel free to pick them up.
[1] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Virtual+Router%3A+Smoke+and+Integration+tests+gap+analysis On 26/08/16, 9:35 PM, "Murali Reddy" <muralimmre...@gmail.com> wrote: >Raja, > >Thanks for sharing insightful automation analysis. We are trying to figure out >how to convert our efforts in to actionable items. As community we can work on >them to get good automated test suite for VR. > > > >Will be sharing details early next week. > >Thanks. > >On 19/08/16, 2:25 PM, "Raja Pullela" <raja.pull...@accelerite.com> wrote: > >>Hi Murali, >> >>Great initiative… and VR is one area that could help greatly if we have more >>automation. we have done high-level analysis on VR functionality/automation >>recently and have posted our findings to the wiki under “high-level >>Automation analysis” section. Please take a look, >> >>[1] >>https://cwiki.apache.org/confluence/display/CLOUDSTACK/Virtual+Router%3A+Smoke+and+Integration+tests+gap+analysis >> >>Best, >>Raja >>Senior Manager, Product Development >>Accelerite, www.accelerite.com, @accelerite >>2055, Laurelwood Road, Santa Clara, CA 95054, USA >>Phone: 1-408-216-7010 >> >>On 8/19/16, 12:33 PM, "Murali Reddy" <muralimmre...@gmail.com> wrote: >> >>All, >> >>We (at ShapeBlue) did a gap analysis to figure if current set of smoke and >>component tests sufficiently test the VR functionality for the regressions. I >>have posted the analysis at [1]. I went through the test suites, and listed >>down all the tests that touch virtual router functionality. There is listing >>of general observations on grey areas. >> >>One particular area where there were no tests, was related multiple public >>IP’s from different public IP ranges associated with a network. From 4.6, all >>the way to master only IP’s from one public IP range (eth2 on the VR) is >>working, any network services on the public IP’s on eth3, eth4 etc on VR are >>not functional. This is a common use case and is broken for last few >>releases. Bug in this area are reported [2] and PR is yet to be merged [3]. I >>will be work on the patch to get this fixed in LTS. I have also added Marvin >>tests [4] cover multiple public IP scenarios. >> >>Given the flexibility and rich set of network functionality is CloudStack, we >>could catch regression only if we have good test suite. If there are any >>other areas related to virtual router functionality that you see there are >>significant gaps, please chime in share your thoughts or add the the wiki. >> >>[1] >>https://cwiki.apache.org/confluence/display/CLOUDSTACK/Virtual+Router%3A+Smoke+and+Integration+tests+gap+analysis >>[2] https://issues.apache.org/jira/browse/CLOUDSTACK-9339 >>[3] https://github.com/apache/cloudstack/pull/1519 >>[4] >>https://github.com/murali-reddy/cloudstack/commit/0b6fbc29fcadb39b08d0050ca473680a614dfab4 >> >> >> >> >> >> >> >>DISCLAIMER >>========== >>This e-mail may contain privileged and confidential information which is the >>property of Accelerite, a Persistent Systems business. It is intended only >>for the use of the individual or entity to which it is addressed. If you are >>not the intended recipient, you are not authorized to read, retain, copy, >>print, distribute or use this message. If you have received this >>communication in error, please notify the sender and delete all copies of >>this message. Accelerite, a Persistent Systems business does not accept any >>liability for virus infected mails. >
