Hi all， In our case, a link back to jobs on Jenkins is necessary for the latest iteration(s, for a short period), but for the very historical records, I don't think it is needed, after all, in very few conditions, the user needs to check the historical log, what they care more is the success/fail/duration/faiure-reason...etc information, which are all already included in the database. And we have already saved start_date/stop_date to distinguish each test results.
Considering the cost and effort of saving the logs somewhere and permanently, VS, our requirement, I think we can still leverage build_id for the link work, there may be some wrong leading due to the two aforementioned reasons, but we can take it as the tolerable mistake. And personally comments: - recreation of jobs very little possibility to do it - migration to a new jenkins instance without preserving the build history it is quite impossible to do like that in an official usage What do you think? @Jack @David @Fatih @Trevor BUILD_TIMESTAMP is something we can leverage, currently, the start_date/stop_date is generated based on the local time, the time is quite inconsistent due to time zone and/or ntp server issues, @Trevor and @Fatih, do you think it can solve our mis-date issues by using BUILD_TIMESTAMP? BRs Serena On Tue, Jan 30, 2018 at 4:19 PM Fatih Degirmenci < fatih.degirme...@ericsson.com> wrote: > Hi Jack, > > As Serena noted, all installers except Apex use same job structure; 1 > parent/upstream job starting and controlling the execution of the > child/downstream jobs that do deployment, functest, and yardstick for a > single scenario. > A variable can be generated by parent job and passed to child jobs so you > can have one variable to identify a full run in the end. > > This could be BUILD_TIMESTAMP as Trevor talked about below or something > else as long as it is guaranteed to be unique within that installer or > scenario. > What is important here is how you intend to push data to database. If you > will track results on installer level than there is very small possibility > of having same timestamp for 2 separate runs if they start exactly at the > same time on millisecond. > In this case, UUID can be generated and used instead. > > Another thing I missed mentioning is that you intend to link back to jobs > on Jenkins if I understood it correctly. We have build retention policy on > Jenkins so after certain period or no of builds (X days or Y no of builds), > they get deleted and you will have links pointing to non-existing logs. > If you need to keep the logs then my suggestion would be to dump logs to > somewhere that can also use the variable generated by the parent job so you > have references to the things correctly. > > /Fatih > > On 2018-01-30, 08:22, "opnfv-tech-discuss-boun...@lists.opnfv.org on > behalf of chenjiankun" <opnfv-tech-discuss-boun...@lists.opnfv.org on > behalf of chenjiank...@huawei.com> wrote: > > Hi Fatih & Trevor, > > What we need is a new variable(like build_id) which can identify a > combination of (Deployment + Functest + Yardstick) CI jobs. For example, if > in one curtain scenario, we will run deployment job, Functest job and > Yardstick job, and then their id will be the same. > Since I am not familiar with the Jenkins, can you give us some > suggestions? > Or can you help us to add a new variable which can satisfy our > requirements? > Thanks in advance~ > > BRs, > Jack Chan > > -----邮件原件----- > 发件人: opnfv-tech-discuss-boun...@lists.opnfv.org [mailto: > opnfv-tech-discuss-boun...@lists.opnfv.org] 代表 Trevor Bramwell > 发送时间: 2018年1月30日 7:21 > 收件人: Fatih Degirmenci > 抄送: Serena Feng; TECH-DISCUSS OPNFV > 主题: Re: [opnfv-tech-discuss] [test-wg]Requirements for test resources > collection > > Hi, > > Previous versions of Jenkins set BUILD_ID to the timestamp of the > build but now BUILD_ID = BUILD_NUMBER. > > There are two plugins that appear to bring back > 'BUILD_TIMESTAMP'[1][2]. > > It also wouldn't be too hard to generate this as part of a build step > and we could keep the format TIMESTAMP_FORMAT as a global variable. > > Regards, > Trevor Bramwell > > [1] https://plugins.jenkins.io/build-timestamp > [2] https://plugins.jenkins.io/zentimestamp > > On Tue, Jan 30, 2018 at 12:04:10AM +0100, Fatih Degirmenci wrote: > > Hi, > > > > BUILD_ID should not be used for this type of purposes since it gets > > reset when certain things happen such as > > > > - recreation of jobs > > - migration to a new jenkins instance without preserving the build > > history > > > > If any of these things happen, there might be results with the same > id > > in db over time. It is better to find something else that never gets > > reset such as timestamp. > > > > /Fatih > > > > On 29 Jan 2018, at 07:39, SerenaFeng(zte) > > <serena.feng.711+...@gmail.com> > > wrote: > > > > Hi Jack > > > > Eventually, we hope all the installers push their deployment results > > to TestAPI, in the current stage, we just leverage Daisy to have a > try > > and find a common framework to achieve that, we will promote it to > > other installers soon after it works. > > > > In terms of Jenkins id, almost all the installers leverage muti-job > to > > manage deployment and drive test projects. there is an entry job to > > control all the jobs, such as > > daisy-os-nosdn-nofeature-ha-baremetal-daily-master > > < > https://build.opnfv.org/ci/job/daisy-os-nosdn-nofeature-ha-baremetal- > > daily-master/>, maybe we can leverage the combination of the Jenkins > > job and build_id > > daisy-os-nosdn-nofeature-ha-baremetal-daily-master_185 > > < > https://build.opnfv.org/ci/job/daisy-os-nosdn-nofeature-ha-baremetal- > > daily-master/185/>(or jest the Jenkins job) to link to the Jenkins. > As > > for naming, instead of 'Jenkins' I would recommend using 'Job_name', > > because, in zuulv3, Jenkins will be discarded, but the job idea will > > be left(maybe in some other way) > > > > For the status show, if only show the last and latest *one* > iteration, > > simple pass/fail is fine, or else multi-results will be included, I > > don't recommend that. Personally, my suggestion is either integrate > > last one iteration into last x iterations, all use statistics to show > > the results, or separate into two pages: the current status(simple > > pass/fail) and the history statistic status(8/9/10). > > > > For Functest health check, Sure, Functest reports per execution > result > > per test case separately, we can get them from TestAPI, there's no > > problem with that. But for health check actually includes 3 test > > cases(connection_check/api_check/snaps_health_check), we need to cope > > with that carefully. > > > > > > BRs > > Serena > > > > On Sat, Jan 27, 2018 at 11:42 PM Julien <julien...@gmail.com> wrote: > > > > > Hi Jack, > > > > > > I have finished the macro, but it failed for a Jenkins plugin is > missing. > > > I cc the issue email to you, which will be discussed in the infra > meeting. > > > > > > BR/Julien > > > > > > David McBride <dmcbr...@linuxfoundation.org>于2018年1月27日周六 > 上午12:13写道： > > > > > >> +Julien > > >> > > >> Jack, > > >> > > >> I like the table format and the filtering options that you > proposed. > > >> Also +1 to Serena's suggestion to break out the health check data. > > >> > > >> I think that Julien is working on a Jenkins macro to enable the > > >> test API to retrieve the "jenkins id". Julien - please confirm. > > >> > > >> David > > >> > > >> On Tue, Jan 23, 2018 at 10:53 PM, chenjiankun > > >> <chenjiank...@huawei.com> > > >> wrote: > > >> > > >>> Hi Serena, > > >>> > > >>> > > >>> > > >>> Thanks for your information updating. > > >>> > > >>> I am so glad to see TestAPI is ready to collecting deployment > > >>> results, and it would be much more convenient. > > >>> > > >>> One question: does all installer will push result to DB, or just > > >>> Daisy for now? > > >>> > > >>> > > >>> > > >>> For the Jenkins id, it means a combination of deployment + > > >>> functest + yardstick job. I am not sure if there are a > > >>> ‘jenkins_id’ for now. This field is for identify the combination > jobs. > > >>> > > >>> For the second comment, maybe there are some misunderstanding, as > > >>> David’s requirements, we will show data last x iterations(not > > >>> days), so I think there will be no such problems. > > >>> > > >>> For the healthcheck column, I totally agree with you. But maybe > it > > >>> will depend on functest to upload test result independently. > > >>> > > >>> > > >>> > > >>> What’s your opinions? @David, @Serena > > >>> > > >>> > > >>> > > >>> BRs, > > >>> > > >>> Jack Chan > > >>> > > >>> > > >>> > > >>> *发件人:* SerenaFeng(zte) [mailto:serena.feng.711+...@gmail.com] > > >>> *发送时间:* 2018年1月22日 18:08 > > >>> *收件人:* chenjiankun > > >>> *抄送:* David McBride; Serena Feng; TECH-DISCUSS OPNFV > > >>> *主题:* Re: [opnfv-tech-discuss] [test-wg]Requirements for test > > >>> resources collection > > >>> > > >>> > > >>> > > >>> Hi David & Jack, > > >>> > > >>> > > >>> > > >>> Thanks for mentioning the task, an interface for collecting > > >>> deployment results has been ready in TestAPI[1], > > >>> > > >>> the Macro for pushing deployment result is also ready by > > >>> Jelien[2], I believe it will facilitate all the installers' work, > > >>> > > >>> currently, Julien is working on pushing daisy results to TestAPI > > >>> leveraging that Macro, I think it will be finished soon. > > >>> > > >>> > > >>> > > >>> And regarding how to show all the information in a table, I > > >>> suggest we can take a look at Jack's proposal first. > > >>> > > >>> > > >>> > > >>> @Jack, a few comments: > > >>> > > >>> 1. what's the opinion of jenkins id? > > >>> > > >>> 2. for a scenario-installer combination, some will not run once > in > > >>> a day(trigger multiple times or run in multiple pods), > > >>> > > >>> in this case, a simple pass/fail will be too vague, and to > > >>> facilitate the support of data iteration, I would suggest > > >>> > > >>> leveraging 8/9/10(8 passed, 9 triggered, 10total), delete the > > >>> final statistic line > > >>> > > >>> 3. how about adding a healthcheck column(functest-healthcheck > test > > >>> cases), to see if the installer meets the milestone 3.0 > > >>> > > >>> > > >>> > > >>> > > >>> > > >>> [1]: https://gerrit.opnfv.org/gerrit/#/c/49895/ > > >>> > > >>> [2]: https://gerrit.opnfv.org/gerrit/#/c/48515/ > > >>> > > >>> > > >>> > > >>> BRs > > >>> > > >>> Serena > > >>> > > >>> > > >>> > > >>> On Mon, Jan 22, 2018 at 5:02 PM chenjiankun > > >>> <chenjiank...@huawei.com> > > >>> wrote: > > >>> > > >>> Thanks, David. > > >>> > > >>> > > >>> > > >>> According your descriptions, I have created a demo table as > > >>> below(wish I do not misunderstanding your meaning): > > >>> > > >>> > > >>> > > >>> *scenario * > > >>> > > >>> *date * > > >>> > > >>> *Jenkins * > > >>> > > >>> *Version * > > >>> > > >>> *Installer * > > >>> > > >>> *Deployment * > > >>> > > >>> *Functest * > > >>> > > >>> *yardstick * > > >>> > > >>> os-nosdn-nofeature-ha > > >>> > > >>> 2018-01-21 > > >>> > > >>> Jenkins id > > >>> > > >>> euphrates > > >>> > > >>> compass > > >>> > > >>> pass > > >>> > > >>> pass > > >>> > > >>> pass > > >>> > > >>> 2018-01-21 > > >>> > > >>> Jenkins id > > >>> > > >>> euphrates > > >>> > > >>> compass > > >>> > > >>> fail > > >>> > > >>> not trigger > > >>> > > >>> not trigger > > >>> > > >>> statistic > > >>> > > >>> 8/9/10 > > >>> > > >>> (pass:8,triggered:9, total:10) > > >>> > > >>> 6/7/8 > > >>> > > >>> 6/7/8 > > >>> > > >>> > > >>> > > >>> > > >>> > > >>> This last line in table body is the statistics information, and > > >>> lines above are the detailed information(and it can be folded). > > >>> > > >>> The score 8/9/10 is pass/triggered/total. Total means should run, > > >>> triggered means actually run. > > >>> > > >>> Also we can add three filters: > > >>> > > >>> > > >>> > > >>> If you select installer as compass, then will show all data > > >>> related to compass. > > >>> > > >>> Iterations means last x data points to be displayed. > > >>> > > >>> > > >>> > > >>> Does this table satisfied your requirements? > > >>> > > >>> > > >>> > > >>> BRs, > > >>> > > >>> Jack Chan > > >>> > > >>> *发件人:* David McBride [mailto:dmcbr...@linuxfoundation.org] > > >>> *发送时间:* 2018年1月20日 3:07 > > >>> *收件人:* chenjiankun > > >>> *抄送:* TECH-DISCUSS OPNFV; tro...@redhat.com; Brattain, Ross B; > > >>> Rao, Sridhar; OLLIVIER Cédric IMT/OLN; mark.bei...@dell.com; > > >>> Yuyang (Gabriel); ALFRED C 'MORTON ' (acmor...@att.com); > > >>> emma.l.fo...@intel.com; Liyin (Ace); Wangwulin (Linda); > > >>> georg.k...@ericsson.com; Serena Feng; Julien > > >>> *主题:* Re: [opnfv-tech-discuss][test-wg]Requirements for test > > >>> resources collection > > >>> > > >>> > > >>> > > >>> +Serena, Julien > > >>> > > >>> > > >>> > > >>> Thanks, Jack. > > >>> > > >>> 1. Data reported per scenario (i.e., jenkins job, deployment, > > >>> functest, yardstick, etc. displayed together for each > scenario) instead of > > >>> separate test silos. > > >>> 2. Include deployment results > > >>> 3. Include all Jenkins job results (failure to start, failure > to > > >>> complete, etc.) > > >>> 4. Clear date/time stamps for every data point > > >>> 5. Display the data above for the last x data points (e.g., > 4, 5, 10 > > >>> ?) > > >>> 6. Use an easy-to-understand, unified scoring method for all > test > > >>> frameworks. > > >>> > > >>> As I mentioned, yesterday, Julien and Serena have been working on > > >>> this, as well. Julien has developed a macro > > >>> <https://gerrit.opnfv.org/gerrit/#/c/48515/> to enable > > >>> consolidation of all results per scenario. He is intending to use > > >>> the Daisy installer as a platform to verify the macro, which > then can be adapted to other installers. > > >>> > > >>> > > >>> > > >>> In addition, Serena has agreed to help manage an intern who can > > >>> assist with the project. I have an action to create an intern > > >>> proposal for that purpose. > > >>> > > >>> > > >>> > > >>> David > > >>> > > >>> > > >>> > > >>> On Fri, Jan 19, 2018 at 1:23 AM, chenjiankun > > >>> <chenjiank...@huawei.com> > > >>> wrote: > > >>> > > >>> Hi, > > >>> > > >>> > > >>> > > >>> As we discussed last test working group weekly meeting, we want > to > > >>> do test resources aggregation. > > >>> > > >>> We plan to offer a new friendly web portal which contain all > > >>> existing test resource and more functions. > > >>> > > >>> > > >>> > > >>> I have a broad classification as bellow: > > >>> > > >>> 1. Data analysis > > >>> > > >>> a) Reporting(existing, For release) > > >>> > > >>> b) Bitergia(existing) > > >>> > > >>> c) Grafana(existing, For detailed test results) > > >>> > > >>> d) ……(maybe we can develop more tools to show our > detailed test > > >>> results) > > >>> > > >>> 2. Test working group information(What information you > want to > > >>> see from test working group? Test working group event? Event of > > >>> each > > >>> project?) > > >>> > > >>> 3. Tools of each project(Need each project member to > complete) > > >>> > > >>> 4. ……(waiting for you to improve) > > >>> > > >>> > > >>> > > >>> > > >>> > > >>> This email is aim at collecting requirements for test resources, > > >>> so if you have any idea about classification, existing tools(such > > >>> as reporting), new functions you want, please do not hesitate to > comment here. > > >>> > > >>> As Gabriel said, he will create a new wiki page for test > resources > > >>> collection, so you can also comment there. > > >>> > > >>> > > >>> > > >>> @David, @Tim, can you repeat your advice about reporting here? I > > >>> will try my best to implement it. > > >>> > > >>> @All, all requirements, advice, comment are welcome~J > > >>> > > >>> > > >>> > > >>> BRs, > > >>> > > >>> Jack Chan > > >>> > > >>> > > >>> > > >>> > > >>> > > >>> -- > > >>> > > >>> *David McBride* > > >>> > > >>> Release Manager, OPNFV > > >>> > > >>> Mobile: +1.805.276.8018 <(805)%20276-8018> > > >>> > > >>> Email/Google Talk: dmcbr...@linuxfoundation.org > > >>> > > >>> Skype: davidjmcbride1 > > >>> > > >>> IRC: dmcbride > > >>> > > >>> _______________________________________________ > > >>> opnfv-tech-discuss mailing list > > >>> opnfv-tech-discuss@lists.opnfv.org > > >>> https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss > > >>> > > >>> > > >> > > >> > > >> -- > > >> *David McBride* > > >> Release Manager, OPNFV > > >> Mobile: +1.805.276.8018 <(805)%20276-8018> > > >> Email/Google Talk: dmcbr...@linuxfoundation.org > > >> Skype: davidjmcbride1 > > >> IRC: dmcbride > > >> > > > _______________________________________________ > > opnfv-tech-discuss mailing list > > opnfv-tech-discuss@lists.opnfv.org > > https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss > > > _______________________________________________ > > opnfv-tech-discuss mailing list > > opnfv-tech-discuss@lists.opnfv.org > > https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss > > _______________________________________________ > opnfv-tech-discuss mailing list > opnfv-tech-discuss@lists.opnfv.org > https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss > > > _______________________________________________ > opnfv-tech-discuss mailing list > opnfv-tech-discuss@lists.opnfv.org > https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss >
_______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
