Hi, Matthew
Thank you very much for your reply.

> I'm glad that others have an interest in this topic. I've started an etherpad
> for that discussion here:
> 
> https://etherpad.openstack.org/p/icehouse-summit-qa-coverage-tooling
> 
> Right now it's a very rough outline, without much on it. I'm planning to add
> more later. But, feel free to add any discussion points or information that 
> you
> think needs to be a part of the session.

Thanks. I'll add discussion points and information.

By the way, I think another way to increase test coverage. That is 
 Test Driven Development by using Tempest.
 https://etherpad.openstack.org/p/icehouse-summit-qa-tdd-by-tempest

I'd like to propose this topic to Icehouse design summit.(not yet)
Because we implement a Tempest test code after implementing an execution code 
such as
Nova, Cinder.. and so on now. But I think this flow is one of the missing 
Tempest test reasons.
I think we can increase the test coverage of Tempest if we write a Tempest code 
first.

Best Regards,
-- Masayuki Igawa


On 2013/10/15 23:24:56 +0900, Matthew Treinish wrote:

> On Tue, Oct 15, 2013 at 06:25:28AM +0000, Masayuki Igawa wrote:
> > Hi, 
> > 
> > First, thank you to an anonymous for updating this list!
> >     -> GET /{project_id}/servers/:server_id/diagnostics
> > 
> > And, I have updated: Nova API List for Missing Tempest Tests.
> >   
> > https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc
> > 
> > The summary of this list:
> >                                     different count from
> > Tested or not     # of APIs ratio   the last time
> > -------------------------------------------------------
> > Tested API          124      49.6%          +2
> > Not Tested API               66      26.4%          -2
> > Not Need to Test(*1)         60      24.0%           0
> > -------------------------------------------------------
> > Total(*2):          250     100.0%           0
> > (*1) Because they are deprecated APIs such as nova-network and volume.
> > (*2) not included v3 APIs
> > 
> > The tempest version is:
> >  commit f55f4e54ceab7c6a4d330f92c8059e46233e3560
> >  Merge: 86ab238 062e30a
> >  Author: Jenkins <jenk...@review.openstack.org>
> >  Date:   Mon Oct 14 15:55:59 2013 +0000
> > 
> > By the way, I saw a design summit proposal related to this topic(*3). I 
> > think
> > this information should be generated automatically. So I'd like to talk 
> > about
> > this topic at the summit session.
> > (*3) Coverage analysis tooling: http://summit.openstack.org/cfp/details/171
> 
> I'm glad that others have an interest in this topic. I've started an etherpad
> for that discussion here:
> 
> https://etherpad.openstack.org/p/icehouse-summit-qa-coverage-tooling
> 
> Right now it's a very rough outline, without much on it. I'm planning to add
> more later. But, feel free to add any discussion points or information that 
> you
> think needs to be a part of the session.
> 
> -Matt Treinish
> 
> > 
> > This information would be useful for creating Tempest tests.
> > Any comments/questions/suggestions are welcome.
> > 
> > Best Regards,
> > -- Masayuki Igawa
> > 
> > 
> > > Hi,
> > > 
> > > # I'm sorry for this resending because my last mail has unnecessary 
> > > messages.
> > > 
> > > 
> > > I have updated: Nova API List for Missing Tempest Tests.
> > >  
> > > https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc
> > > 
> > > The summary of this list:
> > >                                   different count from
> > > Tested or not    # of APIs        ratio   the last time
> > > -------------------------------------------------------
> > > Tested API                122      48.8%          +5
> > > Not Tested API             68      27.2%          -5
> > > Not Need to Test(*1)       60      24.0%           0
> > > -------------------------------------------------------
> > > Total(*2):                250     100.0%           0
> > > 
> > > (*1) Because they are deprecated APIs such as nova-network and volume.
> > > (*2) not included v3 APIs
> > > 
> > > I hope this information would be helpful for creating Tempest tests.
> > > Any comments and questions are welcome.
> > > 
> > > Best Regards,
> > > -- Masayuki Igawa
> > > 
> > > 
> > > > Hi, Tempest developers
> > > > 
> > > > I have made:
> > > >  Nova API List for Missing Tempest Tests.
> > > >  
> > > > https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc
> > > > 
> > > > This list shows what we should test. That is:
> > > >  * Nova has 250 APIs(not include v3 APIs).
> > > >  * 117 APIs are executed(maybe tested).
> > > >  * 73 APIs are not executed.
> > > >  * 60 APIs are not executed. But they maybe not need to test.
> > > >      -> Because they are deprecated APIs such as nova-network and 
> > > > volume.
> > > > 
> > > > So I think we need more tempest test cases.
> > > > If this idea is acceptable, can you put your name to 'assignee' at your 
> > > > favorites,
> > > > and implement tempest tests.
> > > > 
> > > > Any comments are welcome.
> > > > 
> > > > Additional information:
> > > >  I made this API list with modification of nova's code that based on 
> > > >  https://review.openstack.org/#/c/25882/ (Abandoned).
> > > > 
> > > > Best Regards,
> > > > -- Masayuki Igawa
> > > > 
> > > > 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

Attachment: smime.p7s
Description: S/MIME cryptographic signature

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Reply via email to