Re: [openstack-dev] [Fuel] Feature delivery rules and automated tests
Hi all, I agree with Alexandra about importance of the feature info on early implementation stage, And seems devops need to take part in review of specs too, clarify scenarios here and other data, that they need. Отправлено с iPhone > 26 дек. 2014, в 19:36, Aleksandra Fedorova > написал(а): > > Actually with JJB in place we create jobs slower than without it. > Because we cannot just click through Jenkins web ui but have to > properly describe and test the job and make it pass the review > process. JJB helps to manage jobs but doesn't simplify job creation. > > But let me point out that Jenkins job configuration is the least > important part of the task. The most troublesome part is to define the > exact scenario and implement environment for the job. Take, for > example, glustrefs cluster we needed to setup for plugins tests. > > Thus, if we want CI job to be ready at SCF we need to get this > information in advance. Which means feature itself should have stable > interface and working test implementation several days before the > freeze. > > >> On Wed, Dec 24, 2014 at 2:26 AM, Igor Shishkin >> wrote: >> I believe yes. >> >> With jenkins job builder we could create jobs faster, QA can be involved in >> that or even create jobs on their own. >> >> I think we have to try it during next release cycle, currently I can’t see >> blockers/problems here. >> -- >> Igor Shishkin >> DevOps >> >> >> >>> On 24 Dec 2014, at 2:20 am GMT+3, Mike Scherbakov >>> wrote: >>> >>> Igor, >>> would that be possible? >>> >>> On Mon, Dec 22, 2014 at 7:49 PM, Anastasia Urlapova >>> wrote: >>> Mike, Dmitry, team, >>> let me add 5 cents - tests per feature have to run on CI before SCF, it is >>> mean that jobs configuration also should be implemented. >>> >>> On Wed, Dec 17, 2014 at 7:33 AM, Mike Scherbakov >>> wrote: >>> I fully support the idea. >>> >>> Feature Lead has to know, that his feature is under threat if it's not yet >>> covered by system tests (unit/integration tests are not enough!!!), and >>> should proactive work with QA engineers to get tests implemented and >>> passing before SCF. >>> >>> On Fri, Dec 12, 2014 at 5:55 PM, Dmitry Pyzhov wrote: >>> Guys, >>> >>> we've done a good job in 6.0. Most of the features were merged before >>> feature freeze. Our QA were involved in testing even earlier. It was much >>> better than before. >>> >>> We had a discussion with Anastasia. There were several bug reports for >>> features yesterday, far beyond HCF. So we still have a long way to be >>> perfect. We should add one rule: we need to have automated tests before HCF. >>> >>> Actually, we should have results of these tests just after FF. It is quite >>> challengeable because we have a short development cycle. So my proposal is >>> to require full deployment and run of automated tests for each feature >>> before soft code freeze. And it needs to be tracked in checklists and on >>> feature syncups. >>> >>> Your opinion? >>> >>> ___ >>> OpenStack-dev mailing list >>> OpenStack-dev@lists.openstack.org >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> >>> >>> -- >>> Mike Scherbakov >>> #mihgen >>> >>> >>> ___ >>> OpenStack-dev mailing list >>> OpenStack-dev@lists.openstack.org >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> >>> >>> ___ >>> OpenStack-dev mailing list >>> OpenStack-dev@lists.openstack.org >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> >>> >>> -- >>> Mike Scherbakov >>> #mihgen >> >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > -- > Aleksandra Fedorova > Fuel Devops Engineer > bookwar > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] Feature delivery rules and automated tests
Actually with JJB in place we create jobs slower than without it. Because we cannot just click through Jenkins web ui but have to properly describe and test the job and make it pass the review process. JJB helps to manage jobs but doesn't simplify job creation. But let me point out that Jenkins job configuration is the least important part of the task. The most troublesome part is to define the exact scenario and implement environment for the job. Take, for example, glustrefs cluster we needed to setup for plugins tests. Thus, if we want CI job to be ready at SCF we need to get this information in advance. Which means feature itself should have stable interface and working test implementation several days before the freeze. On Wed, Dec 24, 2014 at 2:26 AM, Igor Shishkin wrote: > I believe yes. > > With jenkins job builder we could create jobs faster, QA can be involved in > that or even create jobs on their own. > > I think we have to try it during next release cycle, currently I can’t see > blockers/problems here. > -- > Igor Shishkin > DevOps > > > >> On 24 Dec 2014, at 2:20 am GMT+3, Mike Scherbakov >> wrote: >> >> Igor, >> would that be possible? >> >> On Mon, Dec 22, 2014 at 7:49 PM, Anastasia Urlapova >> wrote: >> Mike, Dmitry, team, >> let me add 5 cents - tests per feature have to run on CI before SCF, it is >> mean that jobs configuration also should be implemented. >> >> On Wed, Dec 17, 2014 at 7:33 AM, Mike Scherbakov >> wrote: >> I fully support the idea. >> >> Feature Lead has to know, that his feature is under threat if it's not yet >> covered by system tests (unit/integration tests are not enough!!!), and >> should proactive work with QA engineers to get tests implemented and passing >> before SCF. >> >> On Fri, Dec 12, 2014 at 5:55 PM, Dmitry Pyzhov wrote: >> Guys, >> >> we've done a good job in 6.0. Most of the features were merged before >> feature freeze. Our QA were involved in testing even earlier. It was much >> better than before. >> >> We had a discussion with Anastasia. There were several bug reports for >> features yesterday, far beyond HCF. So we still have a long way to be >> perfect. We should add one rule: we need to have automated tests before HCF. >> >> Actually, we should have results of these tests just after FF. It is quite >> challengeable because we have a short development cycle. So my proposal is >> to require full deployment and run of automated tests for each feature >> before soft code freeze. And it needs to be tracked in checklists and on >> feature syncups. >> >> Your opinion? >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> >> -- >> Mike Scherbakov >> #mihgen >> >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> >> -- >> Mike Scherbakov >> #mihgen >> > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Aleksandra Fedorova Fuel Devops Engineer bookwar ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] Feature delivery rules and automated tests
I believe yes. With jenkins job builder we could create jobs faster, QA can be involved in that or even create jobs on their own. I think we have to try it during next release cycle, currently I can’t see blockers/problems here. -- Igor Shishkin DevOps > On 24 Dec 2014, at 2:20 am GMT+3, Mike Scherbakov > wrote: > > Igor, > would that be possible? > > On Mon, Dec 22, 2014 at 7:49 PM, Anastasia Urlapova > wrote: > Mike, Dmitry, team, > let me add 5 cents - tests per feature have to run on CI before SCF, it is > mean that jobs configuration also should be implemented. > > On Wed, Dec 17, 2014 at 7:33 AM, Mike Scherbakov > wrote: > I fully support the idea. > > Feature Lead has to know, that his feature is under threat if it's not yet > covered by system tests (unit/integration tests are not enough!!!), and > should proactive work with QA engineers to get tests implemented and passing > before SCF. > > On Fri, Dec 12, 2014 at 5:55 PM, Dmitry Pyzhov wrote: > Guys, > > we've done a good job in 6.0. Most of the features were merged before feature > freeze. Our QA were involved in testing even earlier. It was much better than > before. > > We had a discussion with Anastasia. There were several bug reports for > features yesterday, far beyond HCF. So we still have a long way to be > perfect. We should add one rule: we need to have automated tests before HCF. > > Actually, we should have results of these tests just after FF. It is quite > challengeable because we have a short development cycle. So my proposal is to > require full deployment and run of automated tests for each feature before > soft code freeze. And it needs to be tracked in checklists and on feature > syncups. > > Your opinion? > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > -- > Mike Scherbakov > #mihgen > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > -- > Mike Scherbakov > #mihgen > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] Feature delivery rules and automated tests
Igor, would that be possible? On Mon, Dec 22, 2014 at 7:49 PM, Anastasia Urlapova wrote: > > Mike, Dmitry, team, > let me add 5 cents - tests per feature have to run on CI before SCF, it is > mean that jobs configuration also should be implemented. > > On Wed, Dec 17, 2014 at 7:33 AM, Mike Scherbakov > wrote: > >> I fully support the idea. >> >> Feature Lead has to know, that his feature is under threat if it's not >> yet covered by system tests (unit/integration tests are not enough!!!), and >> should proactive work with QA engineers to get tests implemented and >> passing before SCF. >> >> On Fri, Dec 12, 2014 at 5:55 PM, Dmitry Pyzhov >> wrote: >> >>> Guys, >>> >>> we've done a good job in 6.0. Most of the features were merged before >>> feature freeze. Our QA were involved in testing even earlier. It was much >>> better than before. >>> >>> We had a discussion with Anastasia. There were several bug reports for >>> features yesterday, far beyond HCF. So we still have a long way to be >>> perfect. We should add one rule: we need to have automated tests before HCF. >>> >>> Actually, we should have results of these tests just after FF. It is >>> quite challengeable because we have a short development cycle. So my >>> proposal is to require full deployment and run of automated tests for each >>> feature before soft code freeze. And it needs to be tracked in checklists >>> and on feature syncups. >>> >>> Your opinion? >>> >>> ___ >>> OpenStack-dev mailing list >>> OpenStack-dev@lists.openstack.org >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> >> >> -- >> Mike Scherbakov >> #mihgen >> >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -- Mike Scherbakov #mihgen ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] Feature delivery rules and automated tests
Mike, Dmitry, team, let me add 5 cents - tests per feature have to run on CI before SCF, it is mean that jobs configuration also should be implemented. On Wed, Dec 17, 2014 at 7:33 AM, Mike Scherbakov wrote: > I fully support the idea. > > Feature Lead has to know, that his feature is under threat if it's not yet > covered by system tests (unit/integration tests are not enough!!!), and > should proactive work with QA engineers to get tests implemented and > passing before SCF. > > On Fri, Dec 12, 2014 at 5:55 PM, Dmitry Pyzhov > wrote: > >> Guys, >> >> we've done a good job in 6.0. Most of the features were merged before >> feature freeze. Our QA were involved in testing even earlier. It was much >> better than before. >> >> We had a discussion with Anastasia. There were several bug reports for >> features yesterday, far beyond HCF. So we still have a long way to be >> perfect. We should add one rule: we need to have automated tests before HCF. >> >> Actually, we should have results of these tests just after FF. It is >> quite challengeable because we have a short development cycle. So my >> proposal is to require full deployment and run of automated tests for each >> feature before soft code freeze. And it needs to be tracked in checklists >> and on feature syncups. >> >> Your opinion? >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > > -- > Mike Scherbakov > #mihgen > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] Feature delivery rules and automated tests
I fully support the idea. Feature Lead has to know, that his feature is under threat if it's not yet covered by system tests (unit/integration tests are not enough!!!), and should proactive work with QA engineers to get tests implemented and passing before SCF. On Fri, Dec 12, 2014 at 5:55 PM, Dmitry Pyzhov wrote: > > Guys, > > we've done a good job in 6.0. Most of the features were merged before > feature freeze. Our QA were involved in testing even earlier. It was much > better than before. > > We had a discussion with Anastasia. There were several bug reports for > features yesterday, far beyond HCF. So we still have a long way to be > perfect. We should add one rule: we need to have automated tests before HCF. > > Actually, we should have results of these tests just after FF. It is quite > challengeable because we have a short development cycle. So my proposal is > to require full deployment and run of automated tests for each feature > before soft code freeze. And it needs to be tracked in checklists and on > feature syncups. > > Your opinion? > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -- Mike Scherbakov #mihgen ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Fuel] Feature delivery rules and automated tests
Guys, we've done a good job in 6.0. Most of the features were merged before feature freeze. Our QA were involved in testing even earlier. It was much better than before. We had a discussion with Anastasia. There were several bug reports for features yesterday, far beyond HCF. So we still have a long way to be perfect. We should add one rule: we need to have automated tests before HCF. Actually, we should have results of these tests just after FF. It is quite challengeable because we have a short development cycle. So my proposal is to require full deployment and run of automated tests for each feature before soft code freeze. And it needs to be tracked in checklists and on feature syncups. Your opinion? ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
