Re: Good practice for disabling/ignoring failing test cases

2008-06-12 Thread Raymond Feng 

Hi,

It seems that you need to improve the script to filter out the @Ignore 
annotations in the java comments :-).


For example,
   // @Ignore
   // @Ignore(TUSCANY-2349)

Thanks,
Raymond

--
From: Kevin Williams [EMAIL PROTECTED]
Sent: Wednesday, June 11, 2008 10:26 PM
To: tuscany-dev@ws.apache.org
Subject: Re: Good practice for disabling/ignoring failing test cases


Since the vtest script assumes a test comment structure around
specification line numbers, I created a new script specific to this
purpose and added it to /java/etc.  It scans for files named
*testCase.java and collects methods annotated with @Ignore.

I could use some help getting this automated but, in the meantime,
I'll run this scan weekly and post results to the dev-list

Here is output from tonight's scan:
-

Content generated by from C:/tuscanysvn/java/sca
Test case files scanned 2008-06-11 (svn:Revision: 663374)
* Total files processed = 591
* Total ingnored test cases = 26

The following test cases have ignored test methods ...

org.apache.tuscany.sca.itest.databindings.jaxb.DatabindingTestCase
  testWSMap() , no associated jira
  testWSHashMap() , no associated jira

org.apache.tuscany.sca.itest.databindings.jaxb.PrimitivesDatabindingTestCase
  testSCANegateByteArray() , T-2351
  testWSNegateByteArray() , T-2349

org.apache.tuscany.sca.itest.references.ManualWiredReferenceTestCase
  testD2Reference() , no associated jira

client.CatalogEJBClientTestCase
  testCatalogEJB() , no associated jira

org.apache.tuscany.sca.vtest.assembly.ctypefile.CompomnentTypeFileTestCase
  typeFile3() , T-2383

org.apache.tuscany.sca.vtest.javaapi.annotations.conversationattributes.ConversationAttributesAnnotationTestCase
  singlePrincipal() , no associated jira

org.apache.tuscany.sca.vtest.javaapi.annotations.property.PropertyAnnotationTestCase
  atProperty7() , T-2289

org.apache.tuscany.sca.vtest.javaapi.annotations.reference.ReferenceAnnotationTestCase
  bogusComponentName() , no associated jira

org.apache.tuscany.sca.vtest.javaapi.annotations.scope.ScopeAnnotationTestCase
  atScope1() , T-2274
  atScope7() , no associated jira

org.apache.tuscany.sca.vtest.javaapi.apis.componentcontext.ComponentContextTestCase
  testGetRequestContext() , no associated jira
  testServiceLookup() , no associated jira

org.apache.tuscany.sca.vtest.javaapi.apis.exception.ExceptionTestCase
  testServiceUnavailableException() , no associated jira

org.apache.tuscany.sca.vtest.javaapi.apis.requestcontext.RequestContextTestCase
  testGetSecuritySubject() , no associated jira

org.apache.tuscany.sca.vtest.javaapi.conversation.callback.CallbackTestCase
  statefulMixedCallback() , T-2291
  statelessCallback3() , T-2306
  customCallback() , T-2312

org.apache.tuscany.sca.vtest.javaapi.conversation.lifetime.LifetimeTestCase
  lifetime2() , T-2243
  lifetime11() , T-2283

org.apache.tuscany.sca.vtest.wsbinding.nowsdl.multisoapbindings.GeneratedWSDLTestCase
  testMultiSoapBindings4() , no associated jira

org.apache.tuscany.sca.vtest.wsbinding.nowsdl.soapversion.GeneratedWSDLTestCase
  testSoapVersion4() , no associated jira
  testSoapVersion5() , no associated jira

org.apache.tuscany.sca.vtest.wsbinding.WsdlEndpointTestCase
  testWsdlEndpoint() , no associated jira

org.apache.tuscany.sca.vtest.wsbinding.WsdlServiceTestCase
  testWsdlService() , T-2298



On Wed, Jun 11, 2008 at 6:42 PM, Jean-Sebastien Delfino
[EMAIL PROTECTED] wrote:

Simon Laws wrote:


On Wed, Jun 11, 2008 at 6:43 PM, Vamsavardhana Reddy 
[EMAIL PROTECTED]

wrote:


Thanks Raymond for pointing this out.  We never used this strategy in
Geronimo (infact, I have never put any JIRA number in any of the source
files other that Release Notes may be).  I have come across this in
Tuscany
source code and have been following it consistently from then on. 
Should

give credit to who ever started this in Tuscany :)

++Vamsi

On Wed, Jun 11, 2008 at 11:02 PM, Raymond Feng [EMAIL PROTECTED]
wrote:


Hi,

I see Vamsi uses the following strategy to disable failing unit test


cases.


@Test
@Ignore(TUSCANY-) // Ignore the test case due to JIRA 
TUSCANY-

public void testMySrtuff() {
}

I think it's a very good practice we should follow to maintain a clean
build while keep track of the ignored failing test cases with JIRAs.

Thanks,
Raymond



+1 good idea

Simon



+1

--
Jean-Sebastien

Re: Good practice for disabling/ignoring failing test cases

2008-06-12 Thread Vamsavardhana Reddy 
Shouldn't be a big problem.  We can get rid of those commented ones from
code as soon as we find them.

++Vamsi

On Thu, Jun 12, 2008 at 11:35 AM, Raymond Feng [EMAIL PROTECTED] wrote:

 Hi,

 It seems that you need to improve the script to filter out the @Ignore
 annotations in the java comments :-).

 For example,
   // @Ignore
   // @Ignore(TUSCANY-2349)

 Thanks,
 Raymond

 --
 From: Kevin Williams [EMAIL PROTECTED]
 Sent: Wednesday, June 11, 2008 10:26 PM
 To: tuscany-dev@ws.apache.org
 Subject: Re: Good practice for disabling/ignoring failing test cases

  Since the vtest script assumes a test comment structure around
 specification line numbers, I created a new script specific to this
 purpose and added it to /java/etc.  It scans for files named
 *testCase.java and collects methods annotated with @Ignore.

 I could use some help getting this automated but, in the meantime,
 I'll run this scan weekly and post results to the dev-list

 Here is output from tonight's scan:
 -

 Content generated by from C:/tuscanysvn/java/sca
 Test case files scanned 2008-06-11 (svn:Revision: 663374)
 * Total files processed = 591
 * Total ingnored test cases = 26

 The following test cases have ignored test methods ...

 org.apache.tuscany.sca.itest.databindings.jaxb.DatabindingTestCase
  testWSMap() , no associated jira
  testWSHashMap() , no associated jira


 org.apache.tuscany.sca.itest.databindings.jaxb.PrimitivesDatabindingTestCase
  testSCANegateByteArray() , T-2351
  testWSNegateByteArray() , T-2349

 org.apache.tuscany.sca.itest.references.ManualWiredReferenceTestCase
  testD2Reference() , no associated jira

 client.CatalogEJBClientTestCase
  testCatalogEJB() , no associated jira

 org.apache.tuscany.sca.vtest.assembly.ctypefile.CompomnentTypeFileTestCase
  typeFile3() , T-2383


 org.apache.tuscany.sca.vtest.javaapi.annotations.conversationattributes.ConversationAttributesAnnotationTestCase
  singlePrincipal() , no associated jira


 org.apache.tuscany.sca.vtest.javaapi.annotations.property.PropertyAnnotationTestCase
  atProperty7() , T-2289


 org.apache.tuscany.sca.vtest.javaapi.annotations.reference.ReferenceAnnotationTestCase
  bogusComponentName() , no associated jira


 org.apache.tuscany.sca.vtest.javaapi.annotations.scope.ScopeAnnotationTestCase
  atScope1() , T-2274
  atScope7() , no associated jira


 org.apache.tuscany.sca.vtest.javaapi.apis.componentcontext.ComponentContextTestCase
  testGetRequestContext() , no associated jira
  testServiceLookup() , no associated jira

 org.apache.tuscany.sca.vtest.javaapi.apis.exception.ExceptionTestCase
  testServiceUnavailableException() , no associated jira


 org.apache.tuscany.sca.vtest.javaapi.apis.requestcontext.RequestContextTestCase
  testGetSecuritySubject() , no associated jira


 org.apache.tuscany.sca.vtest.javaapi.conversation.callback.CallbackTestCase
  statefulMixedCallback() , T-2291
  statelessCallback3() , T-2306
  customCallback() , T-2312


 org.apache.tuscany.sca.vtest.javaapi.conversation.lifetime.LifetimeTestCase
  lifetime2() , T-2243
  lifetime11() , T-2283


 org.apache.tuscany.sca.vtest.wsbinding.nowsdl.multisoapbindings.GeneratedWSDLTestCase
  testMultiSoapBindings4() , no associated jira


 org.apache.tuscany.sca.vtest.wsbinding.nowsdl.soapversion.GeneratedWSDLTestCase
  testSoapVersion4() , no associated jira
  testSoapVersion5() , no associated jira

 org.apache.tuscany.sca.vtest.wsbinding.WsdlEndpointTestCase
  testWsdlEndpoint() , no associated jira

 org.apache.tuscany.sca.vtest.wsbinding.WsdlServiceTestCase
  testWsdlService() , T-2298



 On Wed, Jun 11, 2008 at 6:42 PM, Jean-Sebastien Delfino
 [EMAIL PROTECTED] wrote:

 Simon Laws wrote:


 On Wed, Jun 11, 2008 at 6:43 PM, Vamsavardhana Reddy 
 [EMAIL PROTECTED]
 wrote:

  Thanks Raymond for pointing this out.  We never used this strategy in
 Geronimo (infact, I have never put any JIRA number in any of the source
 files other that Release Notes may be).  I have come across this in
 Tuscany
 source code and have been following it consistently from then on.
 Should
 give credit to who ever started this in Tuscany :)

 ++Vamsi

 On Wed, Jun 11, 2008 at 11:02 PM, Raymond Feng [EMAIL PROTECTED]
 wrote:

  Hi,

 I see Vamsi uses the following strategy to disable failing unit test


 cases.


 @Test
 @Ignore(TUSCANY-) // Ignore the test case due to JIRA
 TUSCANY-
 public void testMySrtuff() {
 }

 I think it's a very good practice we should follow to maintain a clean
 build while keep track of the ignored failing test cases with JIRAs.

 Thanks,
 Raymond


 +1 good idea

 Simon


 +1

 --
 Jean-Sebastien

Re: Good practice for disabling/ignoring failing test cases

2008-06-12 Thread Dan Becker 

On Wed, Jun 11, 2008 at 11:02 PM, Raymond Feng [EMAIL PROTECTED]
wrote:

I see Vamsi uses the following strategy to disable failing unit test

cases.

@Test
@Ignore(TUSCANY-) // Ignore the test case due to JIRA TUSCANY-
public void testMySrtuff() {
}


+1 Very good practice.

It documents the issue, it centralizes accountability, it is easy to 
scan for issues, ...

--
Thanks, Dan Becker

Re: Good practice for disabling/ignoring failing test cases

2008-06-12 Thread Mike Edwards 

Raymond Feng wrote:

Hi,

I see Vamsi uses the following strategy to disable failing unit test cases.

@Test
@Ignore(TUSCANY-) // Ignore the test case due to JIRA TUSCANY-
public void testMySrtuff() {
}

I think it's a very good practice we should follow to maintain a clean 
build while keep track of the ignored failing test cases with JIRAs.


Thanks,
Raymond


+1 from me


Yours,  Mike.

Re: Good practice for disabling/ignoring failing test cases

2008-06-11 Thread Vamsavardhana Reddy 
Thanks Raymond for pointing this out.  We never used this strategy in
Geronimo (infact, I have never put any JIRA number in any of the source
files other that Release Notes may be).  I have come across this in Tuscany
source code and have been following it consistently from then on.  Should
give credit to who ever started this in Tuscany :)

++Vamsi

On Wed, Jun 11, 2008 at 11:02 PM, Raymond Feng [EMAIL PROTECTED] wrote:

 Hi,

 I see Vamsi uses the following strategy to disable failing unit test cases.

 @Test
 @Ignore(TUSCANY-) // Ignore the test case due to JIRA TUSCANY-
 public void testMySrtuff() {
 }

 I think it's a very good practice we should follow to maintain a clean
 build while keep track of the ignored failing test cases with JIRAs.

 Thanks,
 Raymond

Re: Good practice for disabling/ignoring failing test cases

2008-06-11 Thread Simon Laws 
On Wed, Jun 11, 2008 at 6:43 PM, Vamsavardhana Reddy [EMAIL PROTECTED]
wrote:

 Thanks Raymond for pointing this out.  We never used this strategy in
 Geronimo (infact, I have never put any JIRA number in any of the source
 files other that Release Notes may be).  I have come across this in Tuscany
 source code and have been following it consistently from then on.  Should
 give credit to who ever started this in Tuscany :)

 ++Vamsi

 On Wed, Jun 11, 2008 at 11:02 PM, Raymond Feng [EMAIL PROTECTED]
 wrote:

  Hi,
 
  I see Vamsi uses the following strategy to disable failing unit test
 cases.
 
  @Test
  @Ignore(TUSCANY-) // Ignore the test case due to JIRA TUSCANY-
  public void testMySrtuff() {
  }
 
  I think it's a very good practice we should follow to maintain a clean
  build while keep track of the ignored failing test cases with JIRAs.
 
  Thanks,
  Raymond
 


+1 good idea

Simon

Re: Good practice for disabling/ignoring failing test cases

2008-06-11 Thread Kevin Williams 
The vtest contributors get credit for this pattern! :-)  We also have
a Ruby script that scans the test source and, among other things,
lists ignored tests and the associated blocking Jiras.


On Wed, Jun 11, 2008 at 11:49 AM, Simon Laws [EMAIL PROTECTED] wrote:
 On Wed, Jun 11, 2008 at 6:43 PM, Vamsavardhana Reddy [EMAIL PROTECTED]
 wrote:

 Thanks Raymond for pointing this out.  We never used this strategy in
 Geronimo (infact, I have never put any JIRA number in any of the source
 files other that Release Notes may be).  I have come across this in Tuscany
 source code and have been following it consistently from then on.  Should
 give credit to who ever started this in Tuscany :)

 ++Vamsi

 On Wed, Jun 11, 2008 at 11:02 PM, Raymond Feng [EMAIL PROTECTED]
 wrote:

  Hi,
 
  I see Vamsi uses the following strategy to disable failing unit test
 cases.
 
  @Test
  @Ignore(TUSCANY-) // Ignore the test case due to JIRA TUSCANY-
  public void testMySrtuff() {
  }
 
  I think it's a very good practice we should follow to maintain a clean
  build while keep track of the ignored failing test cases with JIRAs.
 
  Thanks,
  Raymond
 


 +1 good idea

 Simon

Re: Good practice for disabling/ignoring failing test cases

2008-06-11 Thread Luciano Resende 
+1 for using this pattern to disable/ignore tests that are failing.

How about also running the script in an automated fashion and sending
an e-mail to the list with the results ? What would be a good interval
for running the script ?

On Wed, Jun 11, 2008 at 1:03 PM, Kevin Williams [EMAIL PROTECTED] wrote:
 The vtest contributors get credit for this pattern! :-)  We also have
 a Ruby script that scans the test source and, among other things,
 lists ignored tests and the associated blocking Jiras.


 On Wed, Jun 11, 2008 at 11:49 AM, Simon Laws [EMAIL PROTECTED] wrote:
 On Wed, Jun 11, 2008 at 6:43 PM, Vamsavardhana Reddy [EMAIL PROTECTED]
 wrote:

 Thanks Raymond for pointing this out.  We never used this strategy in
 Geronimo (infact, I have never put any JIRA number in any of the source
 files other that Release Notes may be).  I have come across this in Tuscany
 source code and have been following it consistently from then on.  Should
 give credit to who ever started this in Tuscany :)

 ++Vamsi

 On Wed, Jun 11, 2008 at 11:02 PM, Raymond Feng [EMAIL PROTECTED]
 wrote:

  Hi,
 
  I see Vamsi uses the following strategy to disable failing unit test
 cases.
 
  @Test
  @Ignore(TUSCANY-) // Ignore the test case due to JIRA TUSCANY-
  public void testMySrtuff() {
  }
 
  I think it's a very good practice we should follow to maintain a clean
  build while keep track of the ignored failing test cases with JIRAs.
 
  Thanks,
  Raymond
 


 +1 good idea

 Simon





-- 
Luciano Resende
Apache Tuscany Committer
http://people.apache.org/~lresende
http://lresende.blogspot.com/

Re: Good practice for disabling/ignoring failing test cases

2008-06-11 Thread Raymond Feng 
I think weekly report makes sense. When we try to cut a release, we would 
like to see the reports too.


Thanks,
Raymond
--
From: Luciano Resende [EMAIL PROTECTED]
Sent: Wednesday, June 11, 2008 2:49 PM
To: tuscany-dev@ws.apache.org
Subject: Re: Good practice for disabling/ignoring failing test cases


+1 for using this pattern to disable/ignore tests that are failing.

How about also running the script in an automated fashion and sending
an e-mail to the list with the results ? What would be a good interval
for running the script ?

On Wed, Jun 11, 2008 at 1:03 PM, Kevin Williams [EMAIL PROTECTED] 
wrote:

The vtest contributors get credit for this pattern! :-)  We also have
a Ruby script that scans the test source and, among other things,
lists ignored tests and the associated blocking Jiras.


On Wed, Jun 11, 2008 at 11:49 AM, Simon Laws [EMAIL PROTECTED] 
wrote:
On Wed, Jun 11, 2008 at 6:43 PM, Vamsavardhana Reddy 
[EMAIL PROTECTED]

wrote:


Thanks Raymond for pointing this out.  We never used this strategy in
Geronimo (infact, I have never put any JIRA number in any of the source
files other that Release Notes may be).  I have come across this in 
Tuscany
source code and have been following it consistently from then on. 
Should

give credit to who ever started this in Tuscany :)

++Vamsi

On Wed, Jun 11, 2008 at 11:02 PM, Raymond Feng [EMAIL PROTECTED]
wrote:

 Hi,

 I see Vamsi uses the following strategy to disable failing unit test
cases.

 @Test
 @Ignore(TUSCANY-) // Ignore the test case due to JIRA 
 TUSCANY-

 public void testMySrtuff() {
 }

 I think it's a very good practice we should follow to maintain a 
 clean

 build while keep track of the ignored failing test cases with JIRAs.

 Thanks,
 Raymond




+1 good idea

Simon







--
Luciano Resende
Apache Tuscany Committer
http://people.apache.org/~lresende
http://lresende.blogspot.com/

Re: Good practice for disabling/ignoring failing test cases

2008-06-11 Thread Jean-Sebastien Delfino 

Simon Laws wrote:

On Wed, Jun 11, 2008 at 6:43 PM, Vamsavardhana Reddy [EMAIL PROTECTED]
wrote:


Thanks Raymond for pointing this out.  We never used this strategy in
Geronimo (infact, I have never put any JIRA number in any of the source
files other that Release Notes may be).  I have come across this in Tuscany
source code and have been following it consistently from then on.  Should
give credit to who ever started this in Tuscany :)

++Vamsi

On Wed, Jun 11, 2008 at 11:02 PM, Raymond Feng [EMAIL PROTECTED]
wrote:


Hi,

I see Vamsi uses the following strategy to disable failing unit test

cases.

@Test
@Ignore(TUSCANY-) // Ignore the test case due to JIRA TUSCANY-
public void testMySrtuff() {
}

I think it's a very good practice we should follow to maintain a clean
build while keep track of the ignored failing test cases with JIRAs.

Thanks,
Raymond



+1 good idea

Simon



+1

--
Jean-Sebastien

Re: Good practice for disabling/ignoring failing test cases

2008-06-11 Thread Kevin Williams 
Since the vtest script assumes a test comment structure around
specification line numbers, I created a new script specific to this
purpose and added it to /java/etc.  It scans for files named
*testCase.java and collects methods annotated with @Ignore.

I could use some help getting this automated but, in the meantime,
I'll run this scan weekly and post results to the dev-list

Here is output from tonight's scan:
-

Content generated by from C:/tuscanysvn/java/sca
Test case files scanned 2008-06-11 (svn:Revision: 663374)
* Total files processed = 591
* Total ingnored test cases = 26

The following test cases have ignored test methods ...

org.apache.tuscany.sca.itest.databindings.jaxb.DatabindingTestCase
   testWSMap() , no associated jira
   testWSHashMap() , no associated jira

org.apache.tuscany.sca.itest.databindings.jaxb.PrimitivesDatabindingTestCase
   testSCANegateByteArray() , T-2351
   testWSNegateByteArray() , T-2349

org.apache.tuscany.sca.itest.references.ManualWiredReferenceTestCase
   testD2Reference() , no associated jira

client.CatalogEJBClientTestCase
   testCatalogEJB() , no associated jira

org.apache.tuscany.sca.vtest.assembly.ctypefile.CompomnentTypeFileTestCase
   typeFile3() , T-2383

org.apache.tuscany.sca.vtest.javaapi.annotations.conversationattributes.ConversationAttributesAnnotationTestCase
   singlePrincipal() , no associated jira

org.apache.tuscany.sca.vtest.javaapi.annotations.property.PropertyAnnotationTestCase
   atProperty7() , T-2289

org.apache.tuscany.sca.vtest.javaapi.annotations.reference.ReferenceAnnotationTestCase
   bogusComponentName() , no associated jira

org.apache.tuscany.sca.vtest.javaapi.annotations.scope.ScopeAnnotationTestCase
   atScope1() , T-2274
   atScope7() , no associated jira

org.apache.tuscany.sca.vtest.javaapi.apis.componentcontext.ComponentContextTestCase
   testGetRequestContext() , no associated jira
   testServiceLookup() , no associated jira

org.apache.tuscany.sca.vtest.javaapi.apis.exception.ExceptionTestCase
   testServiceUnavailableException() , no associated jira

org.apache.tuscany.sca.vtest.javaapi.apis.requestcontext.RequestContextTestCase
   testGetSecuritySubject() , no associated jira

org.apache.tuscany.sca.vtest.javaapi.conversation.callback.CallbackTestCase
   statefulMixedCallback() , T-2291
   statelessCallback3() , T-2306
   customCallback() , T-2312

org.apache.tuscany.sca.vtest.javaapi.conversation.lifetime.LifetimeTestCase
   lifetime2() , T-2243
   lifetime11() , T-2283

org.apache.tuscany.sca.vtest.wsbinding.nowsdl.multisoapbindings.GeneratedWSDLTestCase
   testMultiSoapBindings4() , no associated jira

org.apache.tuscany.sca.vtest.wsbinding.nowsdl.soapversion.GeneratedWSDLTestCase
   testSoapVersion4() , no associated jira
   testSoapVersion5() , no associated jira

org.apache.tuscany.sca.vtest.wsbinding.WsdlEndpointTestCase
   testWsdlEndpoint() , no associated jira

org.apache.tuscany.sca.vtest.wsbinding.WsdlServiceTestCase
   testWsdlService() , T-2298



On Wed, Jun 11, 2008 at 6:42 PM, Jean-Sebastien Delfino
[EMAIL PROTECTED] wrote:
 Simon Laws wrote:

 On Wed, Jun 11, 2008 at 6:43 PM, Vamsavardhana Reddy [EMAIL PROTECTED]
 wrote:

 Thanks Raymond for pointing this out.  We never used this strategy in
 Geronimo (infact, I have never put any JIRA number in any of the source
 files other that Release Notes may be).  I have come across this in
 Tuscany
 source code and have been following it consistently from then on.  Should
 give credit to who ever started this in Tuscany :)

 ++Vamsi

 On Wed, Jun 11, 2008 at 11:02 PM, Raymond Feng [EMAIL PROTECTED]
 wrote:

 Hi,

 I see Vamsi uses the following strategy to disable failing unit test

 cases.

 @Test
 @Ignore(TUSCANY-) // Ignore the test case due to JIRA TUSCANY-
 public void testMySrtuff() {
 }

 I think it's a very good practice we should follow to maintain a clean
 build while keep track of the ignored failing test cases with JIRAs.

 Thanks,
 Raymond


 +1 good idea

 Simon


 +1

 --
 Jean-Sebastien