Hi Andrea, On 2 Oct 2014 at 10:01:17, Andreea Popescu ([email protected](mailto:[email protected])) wrote:
> Hello everyone, > > > If you’re wondering what kept the QA team busy during the last few days, What QA team, there’s no such team/role in the XWiki project ATM :) I think yo meant the XWiki SAS QA Team. Since I’m also from XWiki SAS I can explain a bit more about this team: - it’s globally in charge of ensuring the quality of the XWiki releases from the point of view of XWiki SAS clients (for example some XWiki SAS clients are using Oracle and this team takes special care to ensure there’s no regression on Oracle, same for browser versions, and globally for anything that could impact XWiki SAS clients) - XWiki SAS is delegating Andrea full time (and Manuel 1/3rd of this time) to help the xwiki.org project by doing manual testing of XWiki releases. They are maintaining the http://test.xwiki.org wiki and filling the manual testing part of the Release notes. > the answer is a lot of testing and a brainstorming effort in order to > devise a series of statistics for the product, which would be featured with > the product version 6.2. cool > We believe that the statistics ought to be implemented for the following > reasons: > > a) They would allow for better communication between us and the community You’re not community? :) Personally I’ve always considered you community like anyone else contributing something to the xwiki project! > b) Each version would be more easily evaluated this way > > c) The product’s evolution would be more easily observable by tracking the > changes in these statistics from one version to another > > d) A more complete overview on the long-term evolution of the product > would be possible, with an opportunity to analyze a series of items in > detail Yes what’s important are not the raw values, but the trends between versions. I started working on this here: http://dev.xwiki.org/xwiki/bin/view/Community/ProjectHealth Thomas also recently started working on performance stats here: http://test.xwiki.org/xwiki/bin/view/Performances/Jetty+HSQLDB+single+wiki See also this mail thread on the topic of having regular perf reports at each release: http://markmail.org/message/dprnorr37ox7tvfu IMO this needs to be included in the page you’re going to create for each release. See below for more on that. > We consider some of the above-mentioned items to be quite important. Here > is a list of the most important ones: > > - No. of downloads > - No. of active installs > - No. of tests executed / added Not very easy to compute but I know how to do it if you need help on that. > - Jira issues fixed - by resolution, by priority, by type > - Jira issues opened - by priority, by type > - Closed vs open tickets > - Stats for important tags: e.g. ie10, mobile, flamingo > - Stats for major features: e.g. flamingo, extension manager, solr > > > - Top overall issues reporters > - Top non-XWiki SAS issues reporters > > > - Extensions quality: issues reported (top 10 extensions) > - Extensions quality: issues closed (top 10 extensions) > > > - l10n translations: existent, missing > > > - Performance stats Yes! > Therefore, your opinion on the following issues would be of great use to > us: > > a) The list offered a number of items that we deem important for our > measurements and assessments. We would like to know whether you agree with > the list and we would like to have your opinion regarding other items that > we would consider inserting in the list? Just to stress it again, what’s important is to get the figure for the past releases and compare so that we get trends over several releases and see where we’re going. - Global TPC is also interesting to compute and see the evolution. Sorin started doing this in the past (he did it once only unfortunately ;)). I can also explain how to compute this. Now you don’t have to start with 100 metrics. Just 3-4 is enough. It’s more important to send the reports regularly and have the trends for each metrics than to have too many metrics. Also, what’s important is to analyze them. There’s no point in having the metrics and not doing anything about them. So we’ll need to think about that too but we can decide after. > b) Where would you like us to publish the statistics and the conclusions > that would result from their analysis? We were thinking about the following: > > > - On a ‘Project Health’ page > - On a page designated to each product version (each version would have > its own page with statistics) > - In the test.xwiki.org > - Within a blogpost > - A mix of the above Here’s my POV: 1) * I’d really like you to continue the page I started at http://dev.xwiki.org/xwiki/bin/view/Community/ProjectHealth but to morph it * I see a ProjectHealth space on dev.xwiki.org * On the home page of the ProjectHealth space, show all the trends across the years or across the versions + have a Livetable listing specific reports for each version analyzed * The raw data should be stored in the pages or in xobjects (they needs to be stored on xwiki.org that’s the important part) 2) Once a report page is ready for a version, ask for feedback about it on the devs list 3) After the feedback has been incorporated or after a few days without answer, add a blog post on xwiki.org pointing to the page 4) Tweet about it with the xwiki.org account > We await your reply and feedback regarding the viability and usefulness of > our proposal. Great, let’s do it and we’ll tune the details as we progress! Thanks -Vincent > Thank you for your time > > -- > > Andreea Zenovia Popescu _______________________________________________ users mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/users
