+Julien Jack,
I like the table format and the filtering options that you proposed. Also +1 to Serena's suggestion to break out the health check data. I think that Julien is working on a Jenkins macro to enable the test API to retrieve the "jenkins id". Julien - please confirm. David On Tue, Jan 23, 2018 at 10:53 PM, chenjiankun <[email protected]> wrote: > Hi Serena, > > > > Thanks for your information updating. > > I am so glad to see TestAPI is ready to collecting deployment results, and > it would be much more convenient. > > One question: does all installer will push result to DB, or just Daisy for > now? > > > > For the Jenkins id, it means a combination of deployment + functest + > yardstick job. I am not sure if there are a ‘jenkins_id’ for now. This > field is for identify the combination jobs. > > For the second comment, maybe there are some misunderstanding, as David’s > requirements, we will show data last x iterations(not days), so I think > there will be no such problems. > > For the healthcheck column, I totally agree with you. But maybe it will > depend on functest to upload test result independently. > > > > What’s your opinions? @David, @Serena > > > > BRs, > > Jack Chan > > > > *发件人:* SerenaFeng(zte) [mailto:[email protected]] > *发送时间:* 2018年1月22日 18:08 > *收件人:* chenjiankun > *抄送:* David McBride; Serena Feng; TECH-DISCUSS OPNFV > *主题:* Re: [opnfv-tech-discuss] [test-wg]Requirements for test resources > collection > > > > Hi David & Jack, > > > > Thanks for mentioning the task, an interface for collecting deployment > results has been ready in TestAPI[1], > > the Macro for pushing deployment result is also ready by Jelien[2], I > believe it will facilitate all the installers' work, > > currently, Julien is working on pushing daisy results to TestAPI > leveraging that Macro, I think it will be finished soon. > > > > And regarding how to show all the information in a table, I suggest we can > take a look at Jack's proposal first. > > > > @Jack, a few comments: > > 1. what's the opinion of jenkins id? > > 2. for a scenario-installer combination, some will not run once in a > day(trigger multiple times or run in multiple pods), > > in this case, a simple pass/fail will be too vague, and to facilitate > the support of data iteration, I would suggest > > leveraging 8/9/10(8 passed, 9 triggered, 10total), delete the > final statistic line > > 3. how about adding a healthcheck column(functest-healthcheck test cases), > to see if the installer meets the milestone 3.0 > > > > > > [1]: https://gerrit.opnfv.org/gerrit/#/c/49895/ > > [2]: https://gerrit.opnfv.org/gerrit/#/c/48515/ > > > > BRs > > Serena > > > > On Mon, Jan 22, 2018 at 5:02 PM chenjiankun <[email protected]> > wrote: > > Thanks, David. > > > > According your descriptions, I have created a demo table as below(wish I > do not misunderstanding your meaning): > > > > *scenario * > > *date * > > *Jenkins * > > *Version * > > *Installer * > > *Deployment * > > *Functest * > > *yardstick * > > os-nosdn-nofeature-ha > > 2018-01-21 > > Jenkins id > > euphrates > > compass > > pass > > pass > > pass > > 2018-01-21 > > Jenkins id > > euphrates > > compass > > fail > > not trigger > > not trigger > > statistic > > 8/9/10 > > (pass:8,triggered:9, total:10) > > 6/7/8 > > 6/7/8 > > > > > > This last line in table body is the statistics information, and lines > above are the detailed information(and it can be folded). > > The score 8/9/10 is pass/triggered/total. Total means should run, > triggered means actually run. > > Also we can add three filters: > > > > If you select installer as compass, then will show all data related to > compass. > > Iterations means last x data points to be displayed. > > > > Does this table satisfied your requirements? > > > > BRs, > > Jack Chan > > *发件人:* David McBride [mailto:[email protected]] > *发送时间:* 2018年1月20日 3:07 > *收件人:* chenjiankun > *抄送:* TECH-DISCUSS OPNFV; [email protected]; Brattain, Ross B; Rao, > Sridhar; OLLIVIER Cédric IMT/OLN; [email protected]; Yuyang (Gabriel); > ALFRED C 'MORTON ' ([email protected]); [email protected]; Liyin > (Ace); Wangwulin (Linda); [email protected]; Serena Feng; Julien > *主题:* Re: [opnfv-tech-discuss][test-wg]Requirements for test resources > collection > > > > +Serena, Julien > > > > Thanks, Jack. > > 1. Data reported per scenario (i.e., jenkins job, deployment, > functest, yardstick, etc. displayed together for each scenario) instead of > separate test silos. > 2. Include deployment results > 3. Include all Jenkins job results (failure to start, failure to > complete, etc.) > 4. Clear date/time stamps for every data point > 5. Display the data above for the last x data points (e.g., 4, 5, 10 ?) > 6. Use an easy-to-understand, unified scoring method for all test > frameworks. > > As I mentioned, yesterday, Julien and Serena have been working on this, as > well. Julien has developed a macro > <https://gerrit.opnfv.org/gerrit/#/c/48515/> to enable consolidation of > all results per scenario. He is intending to use the Daisy installer as a > platform to verify the macro, which then can be adapted to other installers. > > > > In addition, Serena has agreed to help manage an intern who can assist > with the project. I have an action to create an intern proposal for that > purpose. > > > > David > > > > On Fri, Jan 19, 2018 at 1:23 AM, chenjiankun <[email protected]> > wrote: > > Hi, > > > > As we discussed last test working group weekly meeting, we want to do test > resources aggregation. > > We plan to offer a new friendly web portal which contain all existing test > resource and more functions. > > > > I have a broad classification as bellow: > > 1. Data analysis > > a) Reporting(existing, For release) > > b) Bitergia(existing) > > c) Grafana(existing, For detailed test results) > > d) ……(maybe we can develop more tools to show our detailed test > results) > > 2. Test working group information(What information you want to see > from test working group? Test working group event? Event of each project?) > > 3. Tools of each project(Need each project member to complete) > > 4. ……(waiting for you to improve) > > > > > > This email is aim at collecting requirements for test resources, so if you > have any idea about classification, existing tools(such as reporting), new > functions you want, please do not hesitate to comment here. > > As Gabriel said, he will create a new wiki page for test resources > collection, so you can also comment there. > > > > @David, @Tim, can you repeat your advice about reporting here? I will try > my best to implement it. > > @All, all requirements, advice, comment are welcome~J > > > > BRs, > > Jack Chan > > > > > > -- > > *David McBride* > > Release Manager, OPNFV > > Mobile: +1.805.276.8018 > > Email/Google Talk: [email protected] > > Skype: davidjmcbride1 > > IRC: dmcbride > > _______________________________________________ > opnfv-tech-discuss mailing list > [email protected] > https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss > > -- *David McBride* Release Manager, OPNFV Mobile: +1.805.276.8018 Email/Google Talk: [email protected] Skype: davidjmcbride1 IRC: dmcbride
_______________________________________________ opnfv-tech-discuss mailing list [email protected] https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
