Hi Serena, Thanks for your information updating. I am so glad to see TestAPI is ready to collecting deployment results, and it would be much more convenient. One question: does all installer will push result to DB, or just Daisy for now?
For the Jenkins id, it means a combination of deployment + functest + yardstick job. I am not sure if there are a ‘jenkins_id’ for now. This field is for identify the combination jobs. For the second comment, maybe there are some misunderstanding, as David’s requirements, we will show data last x iterations(not days), so I think there will be no such problems. For the healthcheck column, I totally agree with you. But maybe it will depend on functest to upload test result independently. What’s your opinions? @David, @Serena BRs, Jack Chan 发件人: SerenaFeng(zte) [mailto:serena.feng.711+...@gmail.com] 发送时间: 2018年1月22日 18:08 收件人: chenjiankun 抄送: David McBride; Serena Feng; TECH-DISCUSS OPNFV 主题: Re: [opnfv-tech-discuss] [test-wg]Requirements for test resources collection Hi David & Jack, Thanks for mentioning the task, an interface for collecting deployment results has been ready in TestAPI[1], the Macro for pushing deployment result is also ready by Jelien[2], I believe it will facilitate all the installers' work, currently, Julien is working on pushing daisy results to TestAPI leveraging that Macro, I think it will be finished soon. And regarding how to show all the information in a table, I suggest we can take a look at Jack's proposal first. @Jack, a few comments: 1. what's the opinion of jenkins id? 2. for a scenario-installer combination, some will not run once in a day(trigger multiple times or run in multiple pods), in this case, a simple pass/fail will be too vague, and to facilitate the support of data iteration, I would suggest leveraging 8/9/10(8 passed, 9 triggered, 10total), delete the final statistic line 3. how about adding a healthcheck column(functest-healthcheck test cases), to see if the installer meets the milestone 3.0 [1]: https://gerrit.opnfv.org/gerrit/#/c/49895/ [2]: https://gerrit.opnfv.org/gerrit/#/c/48515/ BRs Serena On Mon, Jan 22, 2018 at 5:02 PM chenjiankun <chenjiank...@huawei.com<mailto:chenjiank...@huawei.com>> wrote: Thanks, David. According your descriptions, I have created a demo table as below(wish I do not misunderstanding your meaning): scenario date Jenkins Version Installer Deployment Functest yardstick os-nosdn-nofeature-ha 2018-01-21 Jenkins id euphrates compass pass pass pass 2018-01-21 Jenkins id euphrates compass fail not trigger not trigger statistic 8/9/10 (pass:8,triggered:9, total:10) 6/7/8 6/7/8 This last line in table body is the statistics information, and lines above are the detailed information(and it can be folded). The score 8/9/10 is pass/triggered/total. Total means should run, triggered means actually run. Also we can add three filters: [cid:image003.png@01D393A2.AC65E3E0] If you select installer as compass, then will show all data related to compass. Iterations means last x data points to be displayed. Does this table satisfied your requirements? BRs, Jack Chan 发件人: David McBride [mailto:dmcbr...@linuxfoundation.org<mailto:dmcbr...@linuxfoundation.org>] 发送时间: 2018年1月20日 3:07 收件人: chenjiankun 抄送: TECH-DISCUSS OPNFV; tro...@redhat.com<mailto:tro...@redhat.com>; Brattain, Ross B; Rao, Sridhar; OLLIVIER Cédric IMT/OLN; mark.bei...@dell.com<mailto:mark.bei...@dell.com>; Yuyang (Gabriel); ALFRED C 'MORTON ' (acmor...@att.com<mailto:acmor...@att.com>); emma.l.fo...@intel.com<mailto:emma.l.fo...@intel.com>; Liyin (Ace); Wangwulin (Linda); georg.k...@ericsson.com<mailto:georg.k...@ericsson.com>; Serena Feng; Julien 主题: Re: [opnfv-tech-discuss][test-wg]Requirements for test resources collection +Serena, Julien Thanks, Jack. 1. Data reported per scenario (i.e., jenkins job, deployment, functest, yardstick, etc. displayed together for each scenario) instead of separate test silos. 2. Include deployment results 3. Include all Jenkins job results (failure to start, failure to complete, etc.) 4. Clear date/time stamps for every data point 5. Display the data above for the last x data points (e.g., 4, 5, 10 ?) 6. Use an easy-to-understand, unified scoring method for all test frameworks. As I mentioned, yesterday, Julien and Serena have been working on this, as well. Julien has developed a macro<https://gerrit.opnfv.org/gerrit/#/c/48515/> to enable consolidation of all results per scenario. He is intending to use the Daisy installer as a platform to verify the macro, which then can be adapted to other installers. In addition, Serena has agreed to help manage an intern who can assist with the project. I have an action to create an intern proposal for that purpose. David On Fri, Jan 19, 2018 at 1:23 AM, chenjiankun <chenjiank...@huawei.com<mailto:chenjiank...@huawei.com>> wrote: Hi, As we discussed last test working group weekly meeting, we want to do test resources aggregation. We plan to offer a new friendly web portal which contain all existing test resource and more functions. I have a broad classification as bellow: 1. Data analysis a) Reporting(existing, For release) b) Bitergia(existing) c) Grafana(existing, For detailed test results) d) ……(maybe we can develop more tools to show our detailed test results) 2. Test working group information(What information you want to see from test working group? Test working group event? Event of each project?) 3. Tools of each project(Need each project member to complete) 4. ……(waiting for you to improve) This email is aim at collecting requirements for test resources, so if you have any idea about classification, existing tools(such as reporting), new functions you want, please do not hesitate to comment here. As Gabriel said, he will create a new wiki page for test resources collection, so you can also comment there. @David, @Tim, can you repeat your advice about reporting here? I will try my best to implement it. @All, all requirements, advice, comment are welcome~☺ BRs, Jack Chan -- David McBride Release Manager, OPNFV Mobile: +1.805.276.8018<tel:%2B1.805.276.8018> Email/Google Talk: dmcbr...@linuxfoundation.org<mailto:dmcbr...@linuxfoundation.org> Skype: davidjmcbride1 IRC: dmcbride _______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org> https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
_______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
