Hi Fatih & Trevor, What we need is a new variable(like build_id) which can identify a combination of (Deployment + Functest + Yardstick) CI jobs. For example, if in one curtain scenario, we will run deployment job, Functest job and Yardstick job, and then their id will be the same. Since I am not familiar with the Jenkins, can you give us some suggestions? Or can you help us to add a new variable which can satisfy our requirements? Thanks in advance~
BRs, Jack Chan -----邮件原件----- 发件人: opnfv-tech-discuss-boun...@lists.opnfv.org [mailto:opnfv-tech-discuss-boun...@lists.opnfv.org] 代表 Trevor Bramwell 发送时间: 2018年1月30日 7:21 收件人: Fatih Degirmenci 抄送: Serena Feng; TECH-DISCUSS OPNFV 主题: Re: [opnfv-tech-discuss] [test-wg]Requirements for test resources collection Hi, Previous versions of Jenkins set BUILD_ID to the timestamp of the build but now BUILD_ID = BUILD_NUMBER. There are two plugins that appear to bring back 'BUILD_TIMESTAMP'[1][2]. It also wouldn't be too hard to generate this as part of a build step and we could keep the format TIMESTAMP_FORMAT as a global variable. Regards, Trevor Bramwell [1] https://plugins.jenkins.io/build-timestamp [2] https://plugins.jenkins.io/zentimestamp On Tue, Jan 30, 2018 at 12:04:10AM +0100, Fatih Degirmenci wrote: > Hi, > > BUILD_ID should not be used for this type of purposes since it gets > reset when certain things happen such as > > - recreation of jobs > - migration to a new jenkins instance without preserving the build > history > > If any of these things happen, there might be results with the same id > in db over time. It is better to find something else that never gets > reset such as timestamp. > > /Fatih > > On 29 Jan 2018, at 07:39, SerenaFeng(zte) > <serena.feng.711+...@gmail.com> > wrote: > > Hi Jack > > Eventually, we hope all the installers push their deployment results > to TestAPI, in the current stage, we just leverage Daisy to have a try > and find a common framework to achieve that, we will promote it to > other installers soon after it works. > > In terms of Jenkins id, almost all the installers leverage muti-job to > manage deployment and drive test projects. there is an entry job to > control all the jobs, such as > daisy-os-nosdn-nofeature-ha-baremetal-daily-master > <https://build.opnfv.org/ci/job/daisy-os-nosdn-nofeature-ha-baremetal- > daily-master/>, maybe we can leverage the combination of the Jenkins > job and build_id > daisy-os-nosdn-nofeature-ha-baremetal-daily-master_185 > <https://build.opnfv.org/ci/job/daisy-os-nosdn-nofeature-ha-baremetal- > daily-master/185/>(or jest the Jenkins job) to link to the Jenkins. As > for naming, instead of 'Jenkins' I would recommend using 'Job_name', > because, in zuulv3, Jenkins will be discarded, but the job idea will > be left(maybe in some other way) > > For the status show, if only show the last and latest *one* iteration, > simple pass/fail is fine, or else multi-results will be included, I > don't recommend that. Personally, my suggestion is either integrate > last one iteration into last x iterations, all use statistics to show > the results, or separate into two pages: the current status(simple > pass/fail) and the history statistic status(8/9/10). > > For Functest health check, Sure, Functest reports per execution result > per test case separately, we can get them from TestAPI, there's no > problem with that. But for health check actually includes 3 test > cases(connection_check/api_check/snaps_health_check), we need to cope > with that carefully. > > > BRs > Serena > > On Sat, Jan 27, 2018 at 11:42 PM Julien <julien...@gmail.com> wrote: > > > Hi Jack, > > > > I have finished the macro, but it failed for a Jenkins plugin is missing. > > I cc the issue email to you, which will be discussed in the infra meeting. > > > > BR/Julien > > > > David McBride <dmcbr...@linuxfoundation.org>于2018年1月27日周六 上午12:13写道： > > > >> +Julien > >> > >> Jack, > >> > >> I like the table format and the filtering options that you proposed. > >> Also +1 to Serena's suggestion to break out the health check data. > >> > >> I think that Julien is working on a Jenkins macro to enable the > >> test API to retrieve the "jenkins id". Julien - please confirm. > >> > >> David > >> > >> On Tue, Jan 23, 2018 at 10:53 PM, chenjiankun > >> <chenjiank...@huawei.com> > >> wrote: > >> > >>> Hi Serena, > >>> > >>> > >>> > >>> Thanks for your information updating. > >>> > >>> I am so glad to see TestAPI is ready to collecting deployment > >>> results, and it would be much more convenient. > >>> > >>> One question: does all installer will push result to DB, or just > >>> Daisy for now? > >>> > >>> > >>> > >>> For the Jenkins id, it means a combination of deployment + > >>> functest + yardstick job. I am not sure if there are a > >>> ‘jenkins_id’ for now. This field is for identify the combination jobs. > >>> > >>> For the second comment, maybe there are some misunderstanding, as > >>> David’s requirements, we will show data last x iterations(not > >>> days), so I think there will be no such problems. > >>> > >>> For the healthcheck column, I totally agree with you. But maybe it > >>> will depend on functest to upload test result independently. > >>> > >>> > >>> > >>> What’s your opinions? @David, @Serena > >>> > >>> > >>> > >>> BRs, > >>> > >>> Jack Chan > >>> > >>> > >>> > >>> *发件人:* SerenaFeng(zte) [mailto:serena.feng.711+...@gmail.com] > >>> *发送时间:* 2018年1月22日 18:08 > >>> *收件人:* chenjiankun > >>> *抄送:* David McBride; Serena Feng; TECH-DISCUSS OPNFV > >>> *主题:* Re: [opnfv-tech-discuss] [test-wg]Requirements for test > >>> resources collection > >>> > >>> > >>> > >>> Hi David & Jack, > >>> > >>> > >>> > >>> Thanks for mentioning the task, an interface for collecting > >>> deployment results has been ready in TestAPI[1], > >>> > >>> the Macro for pushing deployment result is also ready by > >>> Jelien[2], I believe it will facilitate all the installers' work, > >>> > >>> currently, Julien is working on pushing daisy results to TestAPI > >>> leveraging that Macro, I think it will be finished soon. > >>> > >>> > >>> > >>> And regarding how to show all the information in a table, I > >>> suggest we can take a look at Jack's proposal first. > >>> > >>> > >>> > >>> @Jack, a few comments: > >>> > >>> 1. what's the opinion of jenkins id? > >>> > >>> 2. for a scenario-installer combination, some will not run once in > >>> a day(trigger multiple times or run in multiple pods), > >>> > >>> in this case, a simple pass/fail will be too vague, and to > >>> facilitate the support of data iteration, I would suggest > >>> > >>> leveraging 8/9/10(8 passed, 9 triggered, 10total), delete the > >>> final statistic line > >>> > >>> 3. how about adding a healthcheck column(functest-healthcheck test > >>> cases), to see if the installer meets the milestone 3.0 > >>> > >>> > >>> > >>> > >>> > >>> [1]: https://gerrit.opnfv.org/gerrit/#/c/49895/ > >>> > >>> [2]: https://gerrit.opnfv.org/gerrit/#/c/48515/ > >>> > >>> > >>> > >>> BRs > >>> > >>> Serena > >>> > >>> > >>> > >>> On Mon, Jan 22, 2018 at 5:02 PM chenjiankun > >>> <chenjiank...@huawei.com> > >>> wrote: > >>> > >>> Thanks, David. > >>> > >>> > >>> > >>> According your descriptions, I have created a demo table as > >>> below(wish I do not misunderstanding your meaning): > >>> > >>> > >>> > >>> *scenario * > >>> > >>> *date * > >>> > >>> *Jenkins * > >>> > >>> *Version * > >>> > >>> *Installer * > >>> > >>> *Deployment * > >>> > >>> *Functest * > >>> > >>> *yardstick * > >>> > >>> os-nosdn-nofeature-ha > >>> > >>> 2018-01-21 > >>> > >>> Jenkins id > >>> > >>> euphrates > >>> > >>> compass > >>> > >>> pass > >>> > >>> pass > >>> > >>> pass > >>> > >>> 2018-01-21 > >>> > >>> Jenkins id > >>> > >>> euphrates > >>> > >>> compass > >>> > >>> fail > >>> > >>> not trigger > >>> > >>> not trigger > >>> > >>> statistic > >>> > >>> 8/9/10 > >>> > >>> (pass:8,triggered:9, total:10) > >>> > >>> 6/7/8 > >>> > >>> 6/7/8 > >>> > >>> > >>> > >>> > >>> > >>> This last line in table body is the statistics information, and > >>> lines above are the detailed information(and it can be folded). > >>> > >>> The score 8/9/10 is pass/triggered/total. Total means should run, > >>> triggered means actually run. > >>> > >>> Also we can add three filters: > >>> > >>> > >>> > >>> If you select installer as compass, then will show all data > >>> related to compass. > >>> > >>> Iterations means last x data points to be displayed. > >>> > >>> > >>> > >>> Does this table satisfied your requirements? > >>> > >>> > >>> > >>> BRs, > >>> > >>> Jack Chan > >>> > >>> *发件人:* David McBride [mailto:dmcbr...@linuxfoundation.org] > >>> *发送时间:* 2018年1月20日 3:07 > >>> *收件人:* chenjiankun > >>> *抄送:* TECH-DISCUSS OPNFV; tro...@redhat.com; Brattain, Ross B; > >>> Rao, Sridhar; OLLIVIER Cédric IMT/OLN; mark.bei...@dell.com; > >>> Yuyang (Gabriel); ALFRED C 'MORTON ' (acmor...@att.com); > >>> emma.l.fo...@intel.com; Liyin (Ace); Wangwulin (Linda); > >>> georg.k...@ericsson.com; Serena Feng; Julien > >>> *主题:* Re: [opnfv-tech-discuss][test-wg]Requirements for test > >>> resources collection > >>> > >>> > >>> > >>> +Serena, Julien > >>> > >>> > >>> > >>> Thanks, Jack. > >>> > >>> 1. Data reported per scenario (i.e., jenkins job, deployment, > >>> functest, yardstick, etc. displayed together for each scenario) > >>> instead of > >>> separate test silos. > >>> 2. Include deployment results > >>> 3. Include all Jenkins job results (failure to start, failure to > >>> complete, etc.) > >>> 4. Clear date/time stamps for every data point > >>> 5. Display the data above for the last x data points (e.g., 4, 5, 10 > >>> ?) > >>> 6. Use an easy-to-understand, unified scoring method for all test > >>> frameworks. > >>> > >>> As I mentioned, yesterday, Julien and Serena have been working on > >>> this, as well. Julien has developed a macro > >>> <https://gerrit.opnfv.org/gerrit/#/c/48515/> to enable > >>> consolidation of all results per scenario. He is intending to use > >>> the Daisy installer as a platform to verify the macro, which then can be > >>> adapted to other installers. > >>> > >>> > >>> > >>> In addition, Serena has agreed to help manage an intern who can > >>> assist with the project. I have an action to create an intern > >>> proposal for that purpose. > >>> > >>> > >>> > >>> David > >>> > >>> > >>> > >>> On Fri, Jan 19, 2018 at 1:23 AM, chenjiankun > >>> <chenjiank...@huawei.com> > >>> wrote: > >>> > >>> Hi, > >>> > >>> > >>> > >>> As we discussed last test working group weekly meeting, we want to > >>> do test resources aggregation. > >>> > >>> We plan to offer a new friendly web portal which contain all > >>> existing test resource and more functions. > >>> > >>> > >>> > >>> I have a broad classification as bellow: > >>> > >>> 1. Data analysis > >>> > >>> a) Reporting(existing, For release) > >>> > >>> b) Bitergia(existing) > >>> > >>> c) Grafana(existing, For detailed test results) > >>> > >>> d) ……(maybe we can develop more tools to show our detailed test > >>> results) > >>> > >>> 2. Test working group information(What information you want to > >>> see from test working group? Test working group event? Event of > >>> each > >>> project?) > >>> > >>> 3. Tools of each project(Need each project member to complete) > >>> > >>> 4. ……(waiting for you to improve) > >>> > >>> > >>> > >>> > >>> > >>> This email is aim at collecting requirements for test resources, > >>> so if you have any idea about classification, existing tools(such > >>> as reporting), new functions you want, please do not hesitate to comment > >>> here. > >>> > >>> As Gabriel said, he will create a new wiki page for test resources > >>> collection, so you can also comment there. > >>> > >>> > >>> > >>> @David, @Tim, can you repeat your advice about reporting here? I > >>> will try my best to implement it. > >>> > >>> @All, all requirements, advice, comment are welcome~J > >>> > >>> > >>> > >>> BRs, > >>> > >>> Jack Chan > >>> > >>> > >>> > >>> > >>> > >>> -- > >>> > >>> *David McBride* > >>> > >>> Release Manager, OPNFV > >>> > >>> Mobile: +1.805.276.8018 > >>> > >>> Email/Google Talk: dmcbr...@linuxfoundation.org > >>> > >>> Skype: davidjmcbride1 > >>> > >>> IRC: dmcbride > >>> > >>> _______________________________________________ > >>> opnfv-tech-discuss mailing list > >>> opnfv-tech-discuss@lists.opnfv.org > >>> https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss > >>> > >>> > >> > >> > >> -- > >> *David McBride* > >> Release Manager, OPNFV > >> Mobile: +1.805.276.8018 > >> Email/Google Talk: dmcbr...@linuxfoundation.org > >> Skype: davidjmcbride1 > >> IRC: dmcbride > >> > > _______________________________________________ > opnfv-tech-discuss mailing list > opnfv-tech-discuss@lists.opnfv.org > https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss > _______________________________________________ > opnfv-tech-discuss mailing list > opnfv-tech-discuss@lists.opnfv.org > https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss _______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
