Hello all, After your feedback I made this draft with QA Stats: http://dev.xwiki.org/xwiki/bin/view/Drafts/QA+Stats
I await your reply and feedback regarding the draft and stats. Thank you. On Thu, Oct 2, 2014 at 3:29 PM, [email protected] <[email protected]> wrote: > Hi Andrea, > > On 2 Oct 2014 at 10:01:17, Andreea Popescu ([email protected] > (mailto:[email protected])) wrote: > > > Hello everyone, > > > > > > If you’re wondering what kept the QA team busy during the last few days, > > What QA team, there’s no such team/role in the XWiki project ATM :) > > I think yo meant the XWiki SAS QA Team. Since I’m also from XWiki SAS I > can explain a bit more about this team: > - it’s globally in charge of ensuring the quality of the XWiki releases > from the point of view of XWiki SAS clients (for example some XWiki SAS > clients are using Oracle and this team takes special care to ensure there’s > no regression on Oracle, same for browser versions, and globally for > anything that could impact XWiki SAS clients) > - XWiki SAS is delegating Andrea full time (and Manuel 1/3rd of this time) > to help the xwiki.org project by doing manual testing of XWiki releases. > They are maintaining the http://test.xwiki.org wiki and filling the > manual testing part of the Release notes. > > > the answer is a lot of testing and a brainstorming effort in order to > > devise a series of statistics for the product, which would be featured > with > > the product version 6.2. > > cool > > > We believe that the statistics ought to be implemented for the following > > reasons: > > > > a) They would allow for better communication between us and the community > > You’re not community? :) Personally I’ve always considered you community > like anyone else contributing something to the xwiki project! > > > b) Each version would be more easily evaluated this way > > > > c) The product’s evolution would be more easily observable by tracking > the > > changes in these statistics from one version to another > > > > d) A more complete overview on the long-term evolution of the product > > would be possible, with an opportunity to analyze a series of items in > > detail > > Yes what’s important are not the raw values, but the trends between > versions. > > I started working on this here: > http://dev.xwiki.org/xwiki/bin/view/Community/ProjectHealth > > Thomas also recently started working on performance stats here: > http://test.xwiki.org/xwiki/bin/view/Performances/Jetty+HSQLDB+single+wiki > > See also this mail thread on the topic of having regular perf reports at > each release: > http://markmail.org/message/dprnorr37ox7tvfu > > IMO this needs to be included in the page you’re going to create for each > release. > > See below for more on that. > > > We consider some of the above-mentioned items to be quite important. Here > > is a list of the most important ones: > > > > - No. of downloads > > - No. of active installs > > - No. of tests executed / added > > Not very easy to compute but I know how to do it if you need help on that. > > > - Jira issues fixed - by resolution, by priority, by type > > - Jira issues opened - by priority, by type > > - Closed vs open tickets > > - Stats for important tags: e.g. ie10, mobile, flamingo > > - Stats for major features: e.g. flamingo, extension manager, solr > > > > > > - Top overall issues reporters > > - Top non-XWiki SAS issues reporters > > > > > > - Extensions quality: issues reported (top 10 extensions) > > - Extensions quality: issues closed (top 10 extensions) > > > > > > - l10n translations: existent, missing > > > > > > - Performance stats > > Yes! > > > Therefore, your opinion on the following issues would be of great use to > > us: > > > > a) The list offered a number of items that we deem important for our > > measurements and assessments. We would like to know whether you agree > with > > the list and we would like to have your opinion regarding other items > that > > we would consider inserting in the list? > > Just to stress it again, what’s important is to get the figure for the > past releases and compare so that we get trends over several releases and > see where we’re going. > > - Global TPC is also interesting to compute and see the evolution. Sorin > started doing this in the past (he did it once only unfortunately ;)). I > can also explain how to compute this. > > Now you don’t have to start with 100 metrics. Just 3-4 is enough. It’s > more important to send the reports regularly and have the trends for each > metrics than to have too many metrics. > > Also, what’s important is to analyze them. There’s no point in having the > metrics and not doing anything about them. So we’ll need to think about > that too but we can decide after. > > > b) Where would you like us to publish the statistics and the conclusions > > that would result from their analysis? We were thinking about the > following: > > > > > > - On a ‘Project Health’ page > > - On a page designated to each product version (each version would have > > its own page with statistics) > > - In the test.xwiki.org > > - Within a blogpost > > - A mix of the above > > Here’s my POV: > > 1) > > * I’d really like you to continue the page I started at > http://dev.xwiki.org/xwiki/bin/view/Community/ProjectHealth but to morph > it > * I see a ProjectHealth space on dev.xwiki.org > * On the home page of the ProjectHealth space, show all the trends across > the years or across the versions + have a Livetable listing specific > reports for each version analyzed > * The raw data should be stored in the pages or in xobjects (they needs to > be stored on xwiki.org that’s the important part) > > 2) Once a report page is ready for a version, ask for feedback about it on > the devs list > > 3) After the feedback has been incorporated or after a few days without > answer, add a blog post on xwiki.org pointing to the page > > 4) Tweet about it with the xwiki.org account > > > We await your reply and feedback regarding the viability and usefulness > of > > our proposal. > > Great, let’s do it and we’ll tune the details as we progress! > > Thanks > -Vincent > > > Thank you for your time > > > > -- > > > > Andreea Zenovia Popescu > -- Andreea Zenovia Popescu QA Engineer @XWiki SAS _______________________________________________ users mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/users
