Re: Proposed Release Criteria for Beehive 1.0

[Richard Feit]

Sounds great, Steve -- thanks.  I'll update the bug with info about the 
JSF RI.
Rich

Steven Tocco wrote:
Rich,
I'll amend the release criteria.
I added a JIRA issue for the struts1.1 tests.  These are planned for
integration into the test dist as part of 1.0.
http://issues.apache.org/jira/browse/BEEHIVE-545
I also added JIRA issue for the JSF tests to be added as well (post V1)
to the test dist.  For now they will be run from within the source tree
before we affirm a vote on v1.
http://issues.apache.org/jira/browse/BEEHIVE-546
I would suggest any known limitations or issues with shipping the JSF RI
or MyFaces be added to the bug for future reference.
Thanks
Steve
-----Original Message-----
From: Richard Feit 
Sent: Thursday, April 21, 2005 12:09 PM
To: Beehive Developers
Subject: Re: Proposed Release Criteria for Beehive 1.0

Sure, that sounds fine to me.  It would just be really sad to break our 
JSF integration right before we ship.  :S

Rich
Steven Tocco wrote:
 

Rich, 

I think for 1.0 we should only take this as part of the source tree
criteria (not part of the dist).  The release criteria have the source
issues and distribution issues separate so this should only apply to
   

the
 

first.
I'm willing to add it to the wiki.
Any other feedback/concerns before going forward?
Thanks
Steve
-----Original Message-----
From: Richard Feit 
Sent: Thursday, April 21, 2005 11:32 AM
To: Beehive Developers
Subject: Re: Proposed Release Criteria for Beehive 1.0

I see.  If necessary, we could compromise and say that the JSF DRTs
   

have
 

to run in the dev tree (although ideally we'd know that they run
   

against
 

the distribution).
Rich
Julie Zhuo wrote:

   

Well, currently the test distribution does not include the jsf tests.
The jsf test is separate from the normal drt coreWeb. So, there will
     

be
 

some work to do there if we want to run it against the distribution.
I believe everything needed for the bvt.struts11 is already in the
     

test
 

distribution. We just need to add that target to run the test and
perhaps a bit work on how to keep and organize the test result files.
Julie
-----Original Message-----
From: Richard Feit 
Sent: Thursday, April 21, 2005 11:15 AM
To: Beehive Developers
Subject: Re: Proposed Release Criteria for Beehive 1.0

Well, each of these is currently run individually, e.g., 'ant 
bvt.struts11' in in netui/test/webapps/drt.  I *don't* think they
  

     

should
   

run as part of the DRTs normally.  As to the JSF RI, we don't
  

     

distribute
   

it.  To run the drt.jsf-ri test, you need to set the JSF_HOME 
environment variable to a directory that contains the RI.  MyFaces is 
included in the tree, and that's OK because it's another Apache
  

     

project.
   

Rich
Steven Tocco wrote:

  

     

Rich,
I think I'm ok with this, but have a few questions.
Would all of these run by default as part of the DRTs?
Would they run both in source tree as well as from distribution/test
distribution?
I seem to recall there were some problems with the jsf runtime and
       

our
 

ability to distribute it.
Thanks
Steve
-----Original Message-----
From: Richard Feit 
Sent: Thursday, April 21, 2005 10:52 AM
To: Beehive Developers
Subject: Re: Proposed Release Criteria for Beehive 1.0

Would anyone object to me adding the following tests (100% pass rate)
 

    

       

to
  

     

the release criteria?
*bvt.struts11* in netui/test/webapps/drt (tests of NetUI running 
against Struts 1.1)
*drt.myfaces* in netui/test/webapps/jsf (tests of NetUI/JSF 
integration using MyFaces)
*drt.jsf-ri* in netui/test/webapps/jsf (tests of NetUI/JSF 
integration using the JSF Reference Implementation)

Rich
Steven Tocco wrote:

 

    

       

These have also been posted to the wiki here:
http://wiki.apache.org/beehive/V1ReleaseCriteria
Thanks
Steve
-----Original Message-----
From: Steven Tocco 
Sent: Wednesday, April 06, 2005 2:43 PM
To: Beehive Developers
Subject: Proposed Release Criteria for Beehive 1.0

Beehive Developers,
Now that we have gotten our second Beta approved, I thought it an
appropriate time to present what I feel are the release criteria for
Beehive 1.0.  

I would expect the process around these to go as follows.  

1) Present criteria (this email) and post the same on the wiki.
2) Give roughly 48 hrs for disagreement (removals or additions).
3) Then solicit owners of the various objectives.
So here is the proposed list of release criteria for v1 Beehive. In
   

      

         

the
  

     

remainder of this document, the Subversion revision number that
(eventually) reflects Beehive 1.0 will be simply referred to as the
release candidate.  

The release criteria generally fall into one of the following areas:
Source Tree, Distribution Bits, Release Planning, and Documentation.
      

         


   

SOURCE TREE
The following indicate activities that must occur in the Beehive
Subversion Source Tree before we are ready to ship Beehive 1.0.
#1.0    
Objective: All DRTs/Checkin tests pass at 100% when executed against
   

      

         

the
 

    

       

release candidate revision number in the source tree.
Background: All DRTs must pass at 100% for NetUI, JWS and Controls.
Gating: Yes
#1.1
Objective: All BVT/detailed tests pass at 100% when executed against
   

      

         

the
 

    

       

release candidate revision number in the source tree.	
Background: Any BVTs/detailed tests not passing should have JIRA
      

         

issue
   

filed and be commented out for the release candidate.  If at a later
date a fix is to be added to 1.0 (something like 1.0.1) the test can
   

      

         

be
  

     

re-enabled in the 1.0 branch or in the next major release (1.1 or
      

         

2.0)
   

Gating: Yes
#1.2
Objective: Any tests commented out due to failures should have JIRA
issues filed (defects or enhancements) to be fixed in subsequent
releases. Comments within test configuration files should indicate
      

         

the
   

JIRA issue tracking the problem.
Background: N/A
Gating: Yes
#1.3
Objective: The website content must be generated in support of the
release candidate.
Background: N/A
Gating: Yes
#1.4
Objective: The javadoc target for the release candidate must be able
   

      

         

to
  

     

pass successfully.
Background: We must be able to successfully build the javadoc.
Gating: Yes
DISTRIBUTION BITS
The following indicate activities that must occur in support of the
Beehive Distribution.
#2.0
Objective: The Beehive Distribution must be able to be created from
   

      

         

the
  

     

release candidate.
Background: We must be able to build the distribution from the
      

         

release
   

candidate.
Gating: Yes
#2.1
Objective: The Beehive Test Distribution must be able to be created
   

      

         

from
 

    

       

the release candidate.
Background: We must be able to build the test distribution from the
release candidate. This is really needed to verify bits in the
distribution.
Gating: Yes
#2.2
Objective: All tests (DRTs/checkin/BVTs/detailed) in support of
   

      

         

Beehive
  

     

must be able to be executed from the test distribution against the
distribution derived from the release candidate.
Background: Thus test code must fully be enabled to run in this
   

      

         

manner.
  

     

Gating: Yes
#2.3
Objective: All DRTs/Checkin tests must pass at 100% when executed
      

         

from
   

the test distribution against the distribution for the release
   

      

         

candidate
 

    

       

for Windows 2003 Server.
Background: We need the same pass rate to verify it is a good
distribution.
Gating: Yes
#2.4
Objective: All DRTs/Checkin tests must pass at 100% when executed
      

         

from
   

the test distribution against the distribution for the release
   

      

         

candidate
 

    

       

for Redhat Linux AS 3.0.
Background: We need the same pass rate to verify it is a good
distribution.
Gating: Yes
#2.5
Objective: All BVT/Detailed tests must pass at 100% when executed
      

         

from
   

the test distribution against the distribution for the release
   

      

         

candidate
 

    

       

for Window 2003 Server.
Background: We need the same pass rate to verify it is a good
distribution.
Gating: Yes
#2.6
Objective: All BVT/Detailed tests must pass at 100% when executed
      

         

from
   

the test distribution against the distribution for the release
   

      

         

candidate
 

    

       

for Redhat Linux AS 3.0.
Background: We need the same pass rate to verify it is a good
distribution.
Gating: Yes
#2.7
Objective: The distribution created from the release candidate must
      

         

be
   

published for download and signed.
Background: N/A
Gating: Yes
#2.8
Objective: The JSR 181 Technology Compatibility Kit (TCK) must pass
against the distribution derived from the release candidate. 
Background: N/A
Gating: Yes

RELEASE PLANNING
The following indicate activities that must occur in support of the
Beehive Distribution for Release Planning.
#3.0
Objective: The release candidate needs to be proposed for vote
         

within
 

Apache.  This needs to be a minimum of three days prior to release
      

         

and
   

needs to include all documentation, tests.
Background: We should not go to vote until all docs are ready
   

      

         

including
  

     

samples, user guides, java doc, etc.
Gating: Yes
#3.1
Objective: The vote submitted for the release candidate must pass.
Background: N/A
Gating: Yes
#3.2
Objective: A list of known issues must be produced and published in
support of the release candidate.  This should capture all known
   

      

         

defects
 

    

       

(from JIRA) the user may encounter.  This need not include
   

      

         

enhancements
  

     

and documentation bugs.
Background: Could be published as part of the dist itself or perhaps
   

      

         

on
  

     

the Beehive website.  If part of the dist, must be made before the
   

      

         

vote.
 

    

       

Gating: No
#3.3
Objective: For the release candidate, no open bugs will exist.  All
items will be fixed or assigned to the new release.
Background: Currently all things for the future are assigned to Fix
   

      

         

For
  

     

version equal to "TBD".
Gating: No
DOCUMENTATION
#4.0
Objective: All samples must be completed for the release candidate.
   

      

         

The
  

     

samples list includes: 
1) Petstore 
2) External config feature samples 
3) DB control 
4) Service control 
5) JSF sample
Background: List may grow.
Gating: Yes

#4.1
Objective: All samples must be part of the distribution for the
   

      

         

release
  

     

candidate.  
Background: These need to be proofed and executed.
Gating: Yes

#4.2
Objective: All Java Doc for "Public Use" APIs must be authored as
      

         

part
   

of the release candidate.
Background: This is mainly a driver to indicate if there is any API
   

      

         

that
 

    

       

we intend users to implement/understand, Javadoc must be provided.
These need to be authored/proofed by developers.
Gating: Yes
#4.3
Objective: All tutorials on the website must be able to be
   

      

         

successfully
  

     

executed against the distribution created from the release
         

candidate.
 

Background: N/A
Gating: Yes 

#4.4
Objective: The Beehive website must be updated to support the
         

release
 

candidate, including samples, download information, etc.
Background:
Gating: Yes
Thanks
Steve