Hi, Matthew Thank you very much for your reply. > I'm glad that others have an interest in this topic. I've started an etherpad > for that discussion here: > > https://etherpad.openstack.org/p/icehouse-summit-qa-coverage-tooling > > Right now it's a very rough outline, without much on it. I'm planning to add > more later. But, feel free to add any discussion points or information that > you > think needs to be a part of the session.
Thanks. I'll add discussion points and information. By the way, I think another way to increase test coverage. That is Test Driven Development by using Tempest. https://etherpad.openstack.org/p/icehouse-summit-qa-tdd-by-tempest I'd like to propose this topic to Icehouse design summit.(not yet) Because we implement a Tempest test code after implementing an execution code such as Nova, Cinder.. and so on now. But I think this flow is one of the missing Tempest test reasons. I think we can increase the test coverage of Tempest if we write a Tempest code first. Best Regards, -- Masayuki Igawa On 2013/10/15 23:24:56 +0900, Matthew Treinish wrote: > On Tue, Oct 15, 2013 at 06:25:28AM +0000, Masayuki Igawa wrote: > > Hi, > > > > First, thank you to an anonymous for updating this list! > > -> GET /{project_id}/servers/:server_id/diagnostics > > > > And, I have updated: Nova API List for Missing Tempest Tests. > > > > https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc > > > > The summary of this list: > > different count from > > Tested or not # of APIs ratio the last time > > ------------------------------------------------------- > > Tested API 124 49.6% +2 > > Not Tested API 66 26.4% -2 > > Not Need to Test(*1) 60 24.0% 0 > > ------------------------------------------------------- > > Total(*2): 250 100.0% 0 > > (*1) Because they are deprecated APIs such as nova-network and volume. > > (*2) not included v3 APIs > > > > The tempest version is: > > commit f55f4e54ceab7c6a4d330f92c8059e46233e3560 > > Merge: 86ab238 062e30a > > Author: Jenkins <[email protected]> > > Date: Mon Oct 14 15:55:59 2013 +0000 > > > > By the way, I saw a design summit proposal related to this topic(*3). I > > think > > this information should be generated automatically. So I'd like to talk > > about > > this topic at the summit session. > > (*3) Coverage analysis tooling: http://summit.openstack.org/cfp/details/171 > > I'm glad that others have an interest in this topic. I've started an etherpad > for that discussion here: > > https://etherpad.openstack.org/p/icehouse-summit-qa-coverage-tooling > > Right now it's a very rough outline, without much on it. I'm planning to add > more later. But, feel free to add any discussion points or information that > you > think needs to be a part of the session. > > -Matt Treinish > > > > > This information would be useful for creating Tempest tests. > > Any comments/questions/suggestions are welcome. > > > > Best Regards, > > -- Masayuki Igawa > > > > > > > Hi, > > > > > > # I'm sorry for this resending because my last mail has unnecessary > > > messages. > > > > > > > > > I have updated: Nova API List for Missing Tempest Tests. > > > > > > https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc > > > > > > The summary of this list: > > > different count from > > > Tested or not # of APIs ratio the last time > > > ------------------------------------------------------- > > > Tested API 122 48.8% +5 > > > Not Tested API 68 27.2% -5 > > > Not Need to Test(*1) 60 24.0% 0 > > > ------------------------------------------------------- > > > Total(*2): 250 100.0% 0 > > > > > > (*1) Because they are deprecated APIs such as nova-network and volume. > > > (*2) not included v3 APIs > > > > > > I hope this information would be helpful for creating Tempest tests. > > > Any comments and questions are welcome. > > > > > > Best Regards, > > > -- Masayuki Igawa > > > > > > > > > > Hi, Tempest developers > > > > > > > > I have made: > > > > Nova API List for Missing Tempest Tests. > > > > > > > > https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc > > > > > > > > This list shows what we should test. That is: > > > > * Nova has 250 APIs(not include v3 APIs). > > > > * 117 APIs are executed(maybe tested). > > > > * 73 APIs are not executed. > > > > * 60 APIs are not executed. But they maybe not need to test. > > > > -> Because they are deprecated APIs such as nova-network and > > > > volume. > > > > > > > > So I think we need more tempest test cases. > > > > If this idea is acceptable, can you put your name to 'assignee' at your > > > > favorites, > > > > and implement tempest tests. > > > > > > > > Any comments are welcome. > > > > > > > > Additional information: > > > > I made this API list with modification of nova's code that based on > > > > https://review.openstack.org/#/c/25882/ (Abandoned). > > > > > > > > Best Regards, > > > > -- Masayuki Igawa > > > > > > > > > > _______________________________________________ > OpenStack-dev mailing list > [email protected] > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >
smime.p7s
Description: S/MIME cryptographic signature
_______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
