I agree, we need to work on automating testing for most of scenarios. My idea was to create a checklist and then think about automation. My few thoughts while compiling the list.
1. We need to create more API samples to demonstrate API capabilities and those samples can help us automate testing. (current samples need updates) 2. Creating API documentation for every capability can help us to improve API and to add required integration tests. I found swagger [1] to be very use for REST API documentation. We can find similar options for Thrift and Java API. Thanks Raminder 1. http://swagger.wordnik.com/ On Nov 22, 2013, at 9:18 AM, Marlon Pierce <[email protected]> wrote: > This is why I set a bounty on integration tests. > > > Marlon > > On 11/21/13 4:20 PM, Suresh Marru wrote: >> One more reason for automated testing: >> * Humans are lazy by nature :) >> >> + 1 for everything. >> >> Suresh >> >> On Nov 21, 2013, at 2:12 PM, Amila Jayasekara <[email protected]> >> wrote: >> >>> Hi Raman, >>> >>> Thank you for initiating this [1]. This list [1] is great. >>> >>> I have few comments related to verifying samples. >>> >>> It will be really great if we can automate most of these testing. Ideally >>> we should only do UI testing manually (Maybe we can automate that too). >>> Further we need to automate real scenarios (like running job against >>> trestles/stampede) >>> >>> I see automation is necessary due to following reasons; >>> >>> 1. Humans are not perfect >>> 2. We have lot of things in our minds so we are always eager to finish one >>> thing and go to next. Therefore high chance we miss something. Further we >>> dont have a way to assure that release manager did all the verifications. >>> 3. We dont need to wait till release time to figure out something is broken. >>> 4. We are a small team is small. Therefore we need to automate as much as >>> possible. Infact that is the way most of the apache projects do. >>> >>> [1] >>> https://cwiki.apache.org/confluence/display/AIRAVATA/Airavata+Release+Testing >>> >>> Thanks >>> Amila >
