[openstack-dev] [metrics] New version of the Activity Board
Hi everyone, I'd like to introduce the new activity board look and feel and other improvements in the metrics side. * What is it? Activity board is the place where you can find development metrics of the OpenStack Foundation projects. * Where to get it? === There is a live version at [1]. If you want to run a version by yourself, you should download in first place the OpenStack activity-board git repository at [2] and clone the daily updated dataset of JSON files at [3] under the ~/activity-board/browser/data/json dir. Then use your favourite web server. * What's changed? Main changes include: - Navigation is now focused on projects. [4] - Information is now divided into the several releases of the project (top right menu). - Organizations page site improved [5] - Personal page site improved [6] - Mailing lists hotspots [6]. Hot topics are calculated by the last 30 days, 365 days and the whole history. * Further work = - Add Juno release information - Allow to have projects navigation per release - Add Askbot data per release - Add IRC data per release - Improve navigation (feedback is more than welcome here). - Update documentation Cheers, Daniel. [1] http://activity.openstack.org/dash/browser/ [2] http://git.openstack.org/cgit/openstack-infra/activity-board/ [3] https://github.com/Bitergia/openstack-dashboard-json [4] Example navigating through the metrics of Nova: http://activity.openstack.org/dash/browser/project.html?project=nova [5] http://activity.openstack.org/dash/browser/company.html?company=OpenStack%20Foundation [6] http://activity.openstack.org/dash/browser/people.html?id=18 [7] http://activity.openstack.org/dash/browser/mls.html -- Daniel Izquierdo Cortazar, PhD Chief Data Officer - Software Analytics for your peace of mind www.bitergia.com @bitergia ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [metrics] Old reviews (2011) with strange uploaded dates in review.openstack.org
Hi there, I was checking some datasets [1] by the Activity Board [2] and realized that there are some inconsistencies in the old reviews, around July 2011, at the very beginning. An example of this [3] shows that the uploaded date is Dec 16, 2012, while the review was opened on the 25th of July, 2011. I know that this is not a big issue nowadays and even more for the day to day work of the developers, but for the Activity Board, this was producing some negative numbers in the review process, what seemed a bit strange. So, just to point the focus to those dates for those working on metrics :). And a question: was there a migration around the 2012-12-16 of the review system or some other noticeable event?. On such date there was around 1,200 submitted reviews, while around those days, in mean, there are some dozens of them. Cheers, Daniel. [1] http://activity.openstack.org/dash/browser/data/db/reviews.mysql.7z [2] http://activity.openstack.org/dash/browser/ [3] https://review.openstack.org/#/c/44/ -- Daniel Izquierdo Cortazar, PhD Chief Data Officer - Software Analytics for your peace of mind www.bitergia.com @bitergia ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [metrics] Old reviews (2011) with strange uploaded dates in review.openstack.org
On 12/09/14 15:38, Jeremy Stanley wrote: On 2014-09-12 11:54:19 +0200 (+0200), Daniel Izquierdo wrote: [...] And a question: was there a migration around the 2012-12-16 of the review system or some other noticeable event?. On such date there was around 1,200 submitted reviews, while around those days, in mean, there are some dozens of them. We discovered that Gerrit configures datestamps in most of its tables to reset on row updates (a particularly insane design choice for things like a created_on column). Before we realized this, intrusive maintenance activities--most notably project renames--were mass-resetting the creation dates of changes and comments to the date and time we ran the necessary update queries. Now we special-case those fields in our update queries to forcibly reset them to themselves so that they retain their original values, but at this point there's no easy way to go back and fix the ones we did before we noticed this unfortunate loss of date/time information. That makes totally sense, thanks a lot for the info!. Then we should try to avoid those reviews when calculating time to review and other time-based metrics. This maintenance notification looks relevant... http://lists.openstack.org/pipermail/openstack-dev/2012-December/003934.html Oops, thanks for the pointer. It's exactly that date (I didn't check the infra mailing list for that exactly date, my fault u_u). Thanks a lot! Regards, Daniel. -- Daniel Izquierdo Cortazar, PhD Chief Data Officer - Software Analytics for your peace of mind www.bitergia.com @bitergia ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [metrics] How to group activity in git/gerrit repositories
On 16/06/14 21:25, Ilya Shakhat wrote: Let me explain how Stackalytics grouping works. Thanks for the clarification here. Stefano, given that it's simple to group projects, a Most of groups are created from the official programs http://programs.yaml.yaml. Every program turns into item in the module list (colored in violet), for example 'Nova Compute' is a group containing 'nova', 'python-novaclient' and 'nova-specs'. Every type of repo (integrated, incubated and others) turns into the project type, for example 'integrated' type would contain all modules for a chosen release. Also Stackalytics has a few custom project types https://github.com/stackforge/stackalytics/blob/master/etc/default_data.json#L7833-L7879, for example 'infra' is every project under 'openstack-infra' git, or 'documentation' which is the group 'documentation' from programs.yaml. Custom module groups https://github.com/stackforge/stackalytics/blob/master/etc/default_data.json#L7749-L7778 are also possible, but actually used for stackforge projects only. Currently there's no group for python clients, but it would be very easy to add such group. Thanks, Ilya 2014-06-16 21:57 GMT+04:00 Stefano Maffulli stef...@openstack.org mailto:stef...@openstack.org: On Fri 13 Jun 2014 10:51:24 AM PDT, Stangel, Dan wrote: You can also refer to the example of Stackalytics, who have created their own hierarchy and groupings for metrics reporting: https://github.com/stackforge/stackalytics/blob/master/etc/default_data.json It's a very neat grouping. It seems to me that the clients are grouped with their parent git/gerrit repo (nova with python-novaclient, under 'Compute' program) and Nova is shown alone. I don't see the python clients individual repositories or grouped: is that correct? For the quarterly reports I will need granularity because I believe that clients have different dynamics than their parent project (and if that proves not to be the case, we can remove this complexity later and merge data). can you share a concrete example of how you group things? -- Ask and answer questions on https://ask.openstack.org ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Daniel Izquierdo Cortazar, PhD Chief Data Officer - Software Analytics for your peace of mind www.bitergia.com @bitergia ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [metrics] How to group activity in git/gerrit repositories
On 17/06/14 23:53, Daniel Izquierdo wrote: On 16/06/14 21:25, Ilya Shakhat wrote: Let me explain how Stackalytics grouping works. Thanks for the clarification here. Stefano, given that it's simple to group projects, a Sorry about this incomplete email u_u. I was trying to wrap up ideas but well... fat trigger finger Daniel. Most of groups are created from the official programs http://programs.yaml.yaml. Every program turns into item in the module list (colored in violet), for example 'Nova Compute' is a group containing 'nova', 'python-novaclient' and 'nova-specs'. Every type of repo (integrated, incubated and others) turns into the project type, for example 'integrated' type would contain all modules for a chosen release. Also Stackalytics has a few custom project types https://github.com/stackforge/stackalytics/blob/master/etc/default_data.json#L7833-L7879, for example 'infra' is every project under 'openstack-infra' git, or 'documentation' which is the group 'documentation' from programs.yaml. Custom module groups https://github.com/stackforge/stackalytics/blob/master/etc/default_data.json#L7749-L7778 are also possible, but actually used for stackforge projects only. Currently there's no group for python clients, but it would be very easy to add such group. Thanks, Ilya 2014-06-16 21:57 GMT+04:00 Stefano Maffulli stef...@openstack.org mailto:stef...@openstack.org: On Fri 13 Jun 2014 10:51:24 AM PDT, Stangel, Dan wrote: You can also refer to the example of Stackalytics, who have created their own hierarchy and groupings for metrics reporting: https://github.com/stackforge/stackalytics/blob/master/etc/default_data.json It's a very neat grouping. It seems to me that the clients are grouped with their parent git/gerrit repo (nova with python-novaclient, under 'Compute' program) and Nova is shown alone. I don't see the python clients individual repositories or grouped: is that correct? For the quarterly reports I will need granularity because I believe that clients have different dynamics than their parent project (and if that proves not to be the case, we can remove this complexity later and merge data). can you share a concrete example of how you group things? -- Ask and answer questions on https://ask.openstack.org ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Daniel Izquierdo Cortazar, PhD Chief Data Officer - Software Analytics for your peace of mind www.bitergia.com @bitergia ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Daniel Izquierdo Cortazar, PhD Chief Data Officer - Software Analytics for your peace of mind www.bitergia.com @bitergia ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Metrics] Working on affiliations details: Activity Board
Hi!, During the last days we've been improving the Activity Board [1] where you can find updated development activity and affiliation information. For this purpose, we have finally created (sorry for the delay u_u ) an affiliation file [2]. This is a csv file with the following information: name, affiliation, init date, end date and focused on the source code activity. Feedback is more than welcome. If inconsistencies, repeated identities or any other issue are found, please, feel free to directly change that using the usual Gerrit approach [3]. Other options are: open a bug report in the OpenStack Community project in Launchpad [4], reply this email here or personal reply (probably better option than adding too much noise to the dev list). This has been based on several data sources and manual work, so hopefully, there are a few minor issues! Regards, Daniel. ps: not sure if it's a good idea to resend this to the general list without the topic [metrics] [1] http://activity.openstack.org/dash/newbrowser/browser/ [2] http://activity.openstack.org/dash/browser/data/affs/openstack-community-affs.csv [3] http://git.openstack.org/cgit/openstack-infra/activity-board/ [4] https://launchpad.net/openstack-community -- Daniel Izquierdo Cortazar, PhD Chief Data Officer - Software Analytics for your peace of mind www.bitergia.com @bitergia ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Metrics] Working on affiliations details: Activity Board
On 10/15/2013 08:07 PM, Daniel Izquierdo wrote: Hi!, During the last days we've been improving the Activity Board [1] where you can find updated development activity and affiliation information. For this purpose, we have finally created (sorry for the delay u_u ) an affiliation file [2]. This is a csv file with the following information: name, affiliation, init date, end date and focused on the source code activity. oops, the csv file columns are indeed: name, affiliation, init date, end date, number of commits. I missed the number of commits column in the previous email. Sorry about the noise. Regards, Daniel. Feedback is more than welcome. If inconsistencies, repeated identities or any other issue are found, please, feel free to directly change that using the usual Gerrit approach [3]. Other options are: open a bug report in the OpenStack Community project in Launchpad [4], reply this email here or personal reply (probably better option than adding too much noise to the dev list). This has been based on several data sources and manual work, so hopefully, there are a few minor issues! Regards, Daniel. ps: not sure if it's a good idea to resend this to the general list without the topic [metrics] [1] http://activity.openstack.org/dash/newbrowser/browser/ [2] http://activity.openstack.org/dash/browser/data/affs/openstack-community-affs.csv [3] http://git.openstack.org/cgit/openstack-infra/activity-board/ [4] https://launchpad.net/openstack-community -- Daniel Izquierdo Cortazar, PhD Chief Data Officer - Software Analytics for your peace of mind www.bitergia.com @bitergia ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [metrics] New metrics navigator and data sources
Hi! We've been working in a new look and feel for the metrics dash, and that would be awesome if we get some feedback from you :). Please, take into account that this is still work in progress... so there are a lot of missing details. The current version of the dash can be found at [1], while the new and improved version can be found at [2]. With this step, we've tried to focus on the community side, having numbers about participants in mailing lists, developers or tickets activity. Then, in each of the data sources you'll find a summary (examples of git at [3]) and a list of repositories and companies participating in the development, discussions or tickets activity in each of the cases. As you can see, Code review and IRC do not properly work (work in progress) and we have added specific studies regarding the demographics of the OpenStack community at [4]. (still not really understandable, but they are basically typical pyramids of population). Finally, some examples of specific pages (it takes a bit to load... things to improve u_u): - Nova activity (Source Code - by repositories - nova.git) [5] - University of Melbourne members activity (Source Code - by companies - University of Melbourne) [6] In the following days I'll try to keep you informed about how to query the database to get this datasets or directly to play with the JSON files. In any case, please remember that you can easily create your own evolutionary charts using the actionable dashboard [7] or directly feeding your favorite viz tool using the JSON files [8]. Cheers, Daniel. [1] http://activity.openstack.org/dash/browser/ [2] http://activity.openstack.org/dash/newbrowser/browser/ [3] http://activity.openstack.org/dash/newbrowser/browser/scm.html [4] http://activity.openstack.org/dash/newbrowser/browser/demographics.html [5] http://activity.openstack.org/dash/newbrowser/browser/repository.html?repository=nova.git [6] http://activity.openstack.org/dash/newbrowser/browser/company.html?company=University%20of%20Melbourne [7] http://activity.openstack.org/dash/dashboard/ [8] http://activity.openstack.org/dash/browser/data/json/ -- Daniel Izquierdo Cortazar, PhD Chief Data Officer - Software Analytics for your peace of mind www.bitergia.com @bitergia ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
