Re: Good practice for disabling/ignoring failing test cases

[Raymond Feng]

Hi,

It seems that you need to improve the script to filter out the @Ignore 
annotations in the java comments :-).

For example,
   // @Ignore
   // @Ignore("TUSCANY-2349")

Thanks,
Raymond

--------------------------------------------------
From: "Kevin Williams" <[EMAIL PROTECTED]>
Sent: Wednesday, June 11, 2008 10:26 PM
To: <tuscany-dev@ws.apache.org>
Subject: Re: Good practice for disabling/ignoring failing test cases

Since the vtest script assumes a test comment structure around
specification line numbers, I created a new script specific to this
purpose and added it to /java/etc.  It scans for files named
*testCase.java and collects methods annotated with @Ignore.

I could use some help getting this automated but, in the meantime,
I'll run this scan weekly and post results to the dev-list

Here is output from tonight's scan:
---------------------------------------------------------

Content generated by from C:/tuscanysvn/java/sca
Test case files scanned 2008-06-11 (svn:Revision: 663374)
* Total files processed = 591
* Total ingnored test cases = 26

The following test cases have ignored test methods ...

org.apache.tuscany.sca.itest.databindings.jaxb.DatabindingTestCase
  testWSMap() , no associated jira
  testWSHashMap() , no associated jira

org.apache.tuscany.sca.itest.databindings.jaxb.PrimitivesDatabindingTestCase
  testSCANegateByteArray() , T-2351
  testWSNegateByteArray() , T-2349

org.apache.tuscany.sca.itest.references.ManualWiredReferenceTestCase
  testD2Reference() , no associated jira

client.CatalogEJBClientTestCase
  testCatalogEJB() , no associated jira

org.apache.tuscany.sca.vtest.assembly.ctypefile.CompomnentTypeFileTestCase
  typeFile3() , T-2383

org.apache.tuscany.sca.vtest.javaapi.annotations.conversationattributes.ConversationAttributesAnnotationTestCase
  singlePrincipal() , no associated jira

org.apache.tuscany.sca.vtest.javaapi.annotations.property.PropertyAnnotationTestCase
  atProperty7() , T-2289

org.apache.tuscany.sca.vtest.javaapi.annotations.reference.ReferenceAnnotationTestCase
  bogusComponentName() , no associated jira

org.apache.tuscany.sca.vtest.javaapi.annotations.scope.ScopeAnnotationTestCase
  atScope1() , T-2274
  atScope7() , no associated jira

org.apache.tuscany.sca.vtest.javaapi.apis.componentcontext.ComponentContextTestCase
  testGetRequestContext() , no associated jira
  testServiceLookup() , no associated jira

org.apache.tuscany.sca.vtest.javaapi.apis.exception.ExceptionTestCase
  testServiceUnavailableException() , no associated jira

org.apache.tuscany.sca.vtest.javaapi.apis.requestcontext.RequestContextTestCase
  testGetSecuritySubject() , no associated jira

org.apache.tuscany.sca.vtest.javaapi.conversation.callback.CallbackTestCase
  statefulMixedCallback() , T-2291
  statelessCallback3() , T-2306
  customCallback() , T-2312

org.apache.tuscany.sca.vtest.javaapi.conversation.lifetime.LifetimeTestCase
  lifetime2() , T-2243
  lifetime11() , T-2283

org.apache.tuscany.sca.vtest.wsbinding.nowsdl.multisoapbindings.GeneratedWSDLTestCase
  testMultiSoapBindings4() , no associated jira

org.apache.tuscany.sca.vtest.wsbinding.nowsdl.soapversion.GeneratedWSDLTestCase
  testSoapVersion4() , no associated jira
  testSoapVersion5() , no associated jira

org.apache.tuscany.sca.vtest.wsbinding.WsdlEndpointTestCase
  testWsdlEndpoint() , no associated jira

org.apache.tuscany.sca.vtest.wsbinding.WsdlServiceTestCase
  testWsdlService() , T-2298



On Wed, Jun 11, 2008 at 6:42 PM, Jean-Sebastien Delfino
<[EMAIL PROTECTED]> wrote:
Simon Laws wrote:

On Wed, Jun 11, 2008 at 6:43 PM, Vamsavardhana Reddy 
<[EMAIL PROTECTED]>
wrote:

Thanks Raymond for pointing this out.  We never used this strategy in
Geronimo (infact, I have never put any JIRA number in any of the source
files other that Release Notes may be).  I have come across this in
Tuscany
source code and have been following it consistently from then on. 
Should
give credit to who ever started this in Tuscany :)

++Vamsi

On Wed, Jun 11, 2008 at 11:02 PM, Raymond Feng <[EMAIL PROTECTED]>
wrote:

Hi,

I see Vamsi uses the following strategy to disable failing unit test

cases.

@Test
@Ignore("TUSCANY-xxxx") // Ignore the test case due to JIRA 
TUSCANY-xxxx
public void testMySrtuff() {
}

I think it's a very good practice we should follow to maintain a clean
build while keep track of the ignored failing test cases with JIRAs.

Thanks,
Raymond


+1 good idea

Simon


+1

--
Jean-Sebastien