On Mon, May 22, 2017 at 5:23 AM, Amar Tumballi <[email protected]> wrote:
> All, > > NOTE: Currently sent to only maintainers list, planning to send it to > 'gluster-devel@' by Wednesday (24th) if there are no serious comments. > > Below is mainly a proposal, and I would like to hear people's thought on > these. > > > - Over the years, we have added many test cases to our regression test > suite, and currently the testing time stands at ~5hrs for 1 patch. But the > current way of '*.t' based regression can't be called as a 'complete' test > for the filesystem. > > > - Correct me if I am wrong, but even when we have to make a release, > other than the same set of tests, we don't have much to validate the build. > > > Now considering above points and taking the proposal of 'Good Build' from > Nigel > <http://lists.gluster.org/pipermail/gluster-devel/2017-March/052245.html> > [1], I am thinking of making below changes to how we look at testing and > stability. > > 'What to test on nightly build': > > - Build verification > - Run all the regression as it runs now. > - Run CentOS regression > - Run NetBSD regression > - Run coverity > - Run gcov/lcov (for coverage) > - Run more tests with currently optional options made as default (like > brick-multiplexing etc). > - Open up the infra to community contribution, so anyone can write > test cases to make sure GlusterFS passes their usecases, everynight. > - Should be possible to run a python script, ruby script, or a bash > script, need not be in a 'prove' like setup. > - <Add more here> > > 'master' branch: > > - make the overall regression lightweight. > - Run what netbsd tests run now (ie, basic and features in tests). > - Don't run netbsd builds, instead add a compilation test on centos > 32bit machine to keep reminding ourself how many warnings we get. > - Make sure 'master' branch is well tested in 'Nightly'. > - Let the approach of maintainers and over all project is to promote > new changes, instead of being very sceptical about new patches, ideas. > - Provide option to run the whole nightly build suit with a given > patchset to maintainers, so when in doubt, they can ask for the build to > complete before merging. Mostly applies to new feature or some changes > which change the way things behave fundamentally. > > 'release-x.y' branch: > > - During the release-plan come out with target number of 'coverity' > issues, and line coverage % to meet. Also consider number of 'glusto-tests' > to pass. > - Agree to branch out early (at least 45days, compared to current > 30days), so we can iron-out the issues caused by the making the 'master' > branch process lean. > > > - Change the voting logic, add more tests now (Example: fall back to > current regression suit). > - On the first build, run agreed performance tests and compare the > numbers with previous versions. > - Run NetBSD regression now. > - Btw, noticed the latest NetBSD package is for 3.8.9 (done in Jan). > - Work with Emmanuel <[email protected]> for better options here. > - Run nightly on the branch. > - Block the release till we meet the initially agreed metrics during > the release-plan. (like coverity/glusto-tests, line coverage etc). > - On the final build, run agree performance tests and publish the > numbers. Make sure it is not regressed. > > > We all agreeing to this is critical, as I saw that (refer mail on 90days > old patches) > <http://lists.gluster.org/pipermail/gluster-devel/2017-May/052844.html>[2], > there were more than 600 patches which were old, and many of these are good > patches which would make the project better. > > Please take time to review the points here and comment if any. Planning to > raise a ticket to Infra team to validate these changes by June 1st. Lets > move towards these changes by June 15th if there are not any serious > concerns. > > > +1 to bring about more agility in our processes. -Vijay
_______________________________________________ maintainers mailing list [email protected] http://lists.gluster.org/mailman/listinfo/maintainers
