>>> On 8/16/2008 at 11:26 AM, in message <[EMAIL PROTECTED]>, Carlo
Marcelo Arenas Belon <[EMAIL PROTECTED]> wrote:
> On Fri, Aug 15, 2008 at 11:48:50AM -0600, Brad Nicholes wrote:
>>
>> But there have been some significant patches added to 3.1.1 including a
>> fix for a segfault in gmetad.
> 
> and a fix for tcpconn failures and errors as well as spelled in the release
> notes from the STATUS file.
> 
>> Due mainly to the segfault patch, I am proposing that we tag and roll a
>> testing tarball of 3.1.1 within the next week
> 
> agree, but even if we agreed before to the process of releasing requiring
> a tag and a stable package, I suspect (specially based on the recent history
> with 3.1.0) that the process will be instead :
> 
> * release a snapshot package for testing (3.1.0.1696?) and wait for feedback
> * tag 3.1.1 (hopefully after no changes were required from the testing cycle
>   and except for the required changes in configure to make it a stable
>   package) and then wait for the 2 weeks.
> 

I'm not sure why an extra snapshot is necessary.  The above two steps are the 
same thing just with different revision numbers.  Basically the way it works, 
as outlined on the wiki (http://ganglia.wiki.sourceforge.net/ganglia_works), a 
tag is created in SVN that carries the proposed release version number.  The 
purpose for this is so that every release (whether official or unofficial 
testing release) can be tracked and reproducible.   Then a tarball is rolled 
from the tag and posted in the ganglia testing area for download.  An 
announcement is made to the ganglia community for testing/feedback and a time 
period is specified.  If problems are found with the tarball that require a 
patch, the tarball and version number are scrapped.  A patch to fix the problem 
is produced and proposed for backport under the RTC rules just like any other 
patch to the stable branch.  A new tag is created that carries a new version 
number and the testing process starts all over again.  The whole pro
 cess is very simple:

1. tag and roll a potential release with the proposed release version number
2. test the tarball
3. if a problem is found, increment the proposed version number and go to step 1
4. if no problems are found, the proposed version becomes the official version
5. release the official version

If two weeks is insufficient time for testing, the let's propose a testing 
period that works better.  But so far there is no evidence that a two week 
testing period is insufficient.

>> with a goal of shipping 3.1.1 two weeks after the tag.
> 
> I am not sure about the two weeks here either, as the last release cycle had
> shown can't be that effective in finding bugs, probably because of the
> complexity of setting up a ganglia test environment or to the fact that no
> much testing is being done anyway, because we haven't provided for clear
> guidelines on what needs to be tested.
> 
> and which should include at least :
> 
> * which platform are you testing it on (OS name and version), which
>   architecture (CPU type and bitness), do you have binary packages provided
>   to use, do they work?
> * does the ganglia package build from source in your platform? does also
>   gmetad? does it provide for loadable modules and static builds or only 
> static
>   builds? does it build python module support? any warnings reported when
>   building it?
> * after building it, does gmond -m provide a list of available modules?
> * if python module support was enabled, can a simple python module be loaded
>   and used? does tcpconn.py work stable?
> * if upgrading from an older version of ganglia (not including 3.1.0) does 
> the
>   configuration converter (for 2.5) or manual update (for 3.0) produce an
>   equivalent working configuration?
> * if able to update (or setup for testing) two or more gmond are they able 
> to
>   work together and report valid metrics for a cluster?, do they work OK
>   using multicast, unicast or both.
> * can your new 3.1.1 gmetad be configured to pull data from your existing
>   environment without problems or misreporting metrics when compared with 
> your
>   current gmetad?
> * can you use 3.1.1 in a hierarchical dispatcher configuration with your
>   current gmetad?, if able to upgrade (or setup for testing) two 3.1.1 
> gmetad
>   can they be configured in a hierarchical dispatcher configuration?
> * do gmond or gmetad work reliably (not crashing or needing a restart) 
> during
>   the test window, is the memory utilization stable and CPU load under the
>   expected values when compared with your current production environment?
> * is there anything else you had found during your test cycle that you would
>   like to be addressed?, is it something that worked before and doesn't now 
> (a
>   regression)?

I like this set of testing criteria.  This list should be included in the 
testing release announcement to the list and we should probably make similar 
testing criteria lists for all future releases that point out exactly what we 
believe should be the focus of the testing for any given release.

Brad


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Ganglia-developers mailing list
Ganglia-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ganglia-developers

Reply via email to