Hi All, Here is the latest update on test automation of API manager integration tests. Last week i was mainly focused on API store and publisher operations(operations exposed to out side by hostobject classes). For that we need to add sample jaggery application and invoke all possible operations for API store and publisher host objects. Then we can get single response for all operations available and fetch the results according to method order. After that we need to validate response against method. With these tests line coverage reached* 55%* and method and class coverage reached to *70%* and *73%*. I will add some more tests to cover other remaining edge cases. Still there are some issues and we need to address them as well.
01. Integrating with BAM and BPS to cover stats and workflow scenarios(we have considerable amount of code to cover usage metering related stuff). 02. Need to add some more negative test cases to cover exception classes and exception code blocks. 03. Need to add test case to cover external API store scenario. WDYT? Thanks, sanjeewa. On Mon, Jan 27, 2014 at 8:48 PM, Sanjeewa Malalgoda <[email protected]>wrote: > Hi All, > After adding tests to cover comment, rating, token generation and host > object functionalities i was able to reach 37% line coverage and 52% method > coverage. At this point we have covered most of the common API management > related use cases. Now we need to pay attention to workflow integration, > usage publishing, token partition and some other edge cases. Also we might > need to add test cases to cover negative test cases to cover exception > classes as well. I found that auto generated thrift code is having large > number of lines and some of them never get executed. Shall we filter out > them? Any suggestions or ideas? > > OVERALL COVERAGE SUMMARY name line, %block, % method, % class, % > all classes 37% (5976.3/16191) 35% (24800/70841)52% (969/1847) > 67% (130/194)OVERALL STATS SUMMARY total packages: 33 > total executable files: 154 total classes:194 total methods:1847total > executable lines: > 16191 > > > Thanks, > Sanjeewa. > > > On Tue, Jan 21, 2014 at 4:53 PM, Sanjeewa Malalgoda <[email protected]>wrote: > >> Hi All, >> Here is a brief update about API Manager integration test improvement. >> Before adding test cases code coverage was about 2% (default test set). >> Once we added nuwanW implemented you tube sample it was about 12%. After >> that we added 4 existing JMeter scripts it was about 24% (and i need to >> fix some remaining issues in other 2 scripts). So at this moment code >> coverage is about 24%. As we do not have usage related tests i removed >> usage packages from instrumentation file. We might need to add tests to >> cover API Manager and BAM integration to cover usage related code. Here are >> some of the areas we might need to focus on (based on emma report). >> >> >> - Tests to cover Key management web service API as we use thrift by >> default. >> - Also found that exception classes are playing major role as we do >> not test exceptions in our tests. For this we can skip exception classes >> of >> add tests to cover exceptions. >> - We need to implement throttling scenario tests to cover throttling >> code base. >> - For the moment we can safely ignore uri-template scenarios as we >> have comprehensive unit tests for that. But we might need tests for url >> mapping. >> - We need to cover JWT generation and signing scenario (include >> claims retrieving) as we have significant amount of code written for that. >> - We need to add tests to cover tenant scenarios. >> - We might need to add token parser test cases. >> - Also we should add some test cases to cover all operations exposed >> by host object classes(we can achieve this through more scenarios). >> - Some tests to cover workflow scenarios(user creation, application >> creation and subscription). >> - Application level throttling tests and we found that significant >> amount of throttling code will not execute. >> - Add some test to cover comments, rating related operations. >> >> >> Please let know your ideas and feedback. We have planned to add test >> cases to cover missed areas. >> >> Thanks, >> sanjeewa. >> >> -- >> >> *Sanjeewa Malalgoda* >> Senior Software Engineer >> WSO2 Inc. >> Mobile : +94713068779 >> >> <http://sanjeewamalalgoda.blogspot.com/>blog >> :http://sanjeewamalalgoda.blogspot.com/<http://sanjeewamalalgoda.blogspot.com/> >> >> >> > > > -- > > *Sanjeewa Malalgoda* > Senior Software Engineer > WSO2 Inc. > Mobile : +94713068779 > > <http://sanjeewamalalgoda.blogspot.com/>blog > :http://sanjeewamalalgoda.blogspot.com/<http://sanjeewamalalgoda.blogspot.com/> > > > -- *Sanjeewa Malalgoda* Senior Software Engineer WSO2 Inc. Mobile : +94713068779 <http://sanjeewamalalgoda.blogspot.com/>blog :http://sanjeewamalalgoda.blogspot.com/<http://sanjeewamalalgoda.blogspot.com/>
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
