Re: [gentoo-dev] [Gentoo Pheonix] Heartbeat team force
On 04/07/10 21:24, Ben de Groot wrote: with the herd embedded it's difficult though: the page http://www.gentoo.org/proj/en/base/embedded/index.xml lists members for it but the page source does not hold it, explicitly. where does it come from? It does, as can be seen in http://www.gentoo.org/proj/en/base/embedded/index.xml?passthru=1 you refer to the members of the embedded team. i was refering to the members of the embedded herd. the rendered page lists dagger, kumba, pebenito, solar to be working for that herd. the link you gave does not contain the word dagger. how does it get in there? sebastian
Re: [gentoo-dev] [Gentoo Pheonix] Heartbeat team force
On Wed, Apr 07, 2010 at 01:20:11PM +0200, Sebastian Pipping wrote: Current results === Bug load per developer -- http://dev.gentoo.org/~sping/bug-heartbeat/report--bug-count-by-person.html What's the actual math that you're using (it wasn't clear in your repo). Question to be answered === - herds.xml does not hold membership lists for all projects, not even herds. The load evaluator needs access to complete mappings to generate output close to reality. How can such a mapping be made without duplicating data? herds, project files, aliases: - 3 sets of non-overlapping data. - lets ignore the private aliases for now, very few of them should get package bugs. Report on unresolved bugs owned by hidden aliases: - 11 of the 93 aliases have bugs - 258 bugs in total - Those with 10 bugs: - infra, 78 - bugzilla, 47 - retirement, 28 - mirror-admin, 21 - forum-mods, 20 - devrel, 19 - recruiters, 19 - Pulling XML out of Bugzilla does no longer feel right with this amount of data: it's 15,000 open bugs that need to be refreshed periodically in chunks of 100 bugs (Bugzilla's limit) each. That's 150 request for a full sync. How can that be improved? Lets talk more about what queries you're using, and we can probably work something out. -- Robin Hugh Johnson Gentoo Linux: Developer, Trustee Infrastructure Lead E-Mail : robb...@gentoo.org GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85 pgp1RnvDCkrSG.pgp Description: PGP signature
Re: [gentoo-dev] [Gentoo Pheonix] Heartbeat team force
On 04/08/10 21:16, Robin H. Johnson wrote: On Wed, Apr 07, 2010 at 01:20:11PM +0200, Sebastian Pipping wrote: Current results === Bug load per developer -- http://dev.gentoo.org/~sping/bug-heartbeat/report--bug-count-by-person.html What's the actual math that you're using (it wasn't clear in your repo). let me try in python: load[dev] = bugs[dev] \ + reduce(sum, [bugs(h)/members(h) for h in herds[dev]]) it's personal bugs plus a fraction of all herds you're in. makes sense? - Pulling XML out of Bugzilla does no longer feel right with this amount of data: it's 15,000 open bugs that need to be refreshed periodically in chunks of 100 bugs (Bugzilla's limit) each. That's 150 request for a full sync. How can that be improved? Lets talk more about what queries you're using, and we can probably work something out. speaking of queries would limit me in what i may ask the data in the future. i have a full dump on open bugs more or less so i can ask them whatever i like. it's more flexible to me and makes much easier code than SQL stuff would. i guess that doesn't make it easier? sebastian
Re: [gentoo-dev] [Gentoo Pheonix] Heartbeat team force
On 04/04/10 00:24, Sebastian Pipping wrote: Concrete tasks == - [..] - work on association between bugs and packages (for all bugs, not just bugday ones) I've been playing with code answering the questions: - How do bugs and packages relate? - How is bugload distributed across developers? I came to a point where - I have something that starts to be useful - I am facing troubles that need further consideration first, see below. Current results === Bug load per developer -- http://dev.gentoo.org/~sping/bug-heartbeat/report--bug-count-by-person.html Bug load per herd - http://dev.gentoo.org/~sping/bug-heartbeat/report--bug-count-by-herd.html Bugs in a tree of category and package -- http://dev.gentoo.org/~sping/bug-heartbeat/report--bugs-by-package.html Bugs that may lack mention of package name -- http://dev.gentoo.org/~sping/bug-heartbeat/report--bugs-without-package.html Question to be answered === - herds.xml does not hold membership lists for all projects, not even herds. The load evaluator needs access to complete mappings to generate output close to reality. How can such a mapping be made without duplicating data? - Pulling XML out of Bugzilla does no longer feel right with this amount of data: it's 15,000 open bugs that need to be refreshed periodically in chunks of 100 bugs (Bugzilla's limit) each. That's 150 request for a full sync. How can that be improved? Generating reports yourself === In case you want to play with the code yourself: http://git.goodpoint.de/?p=gentoo-bug-heartbeat.git;a=summary Run the report generator as following: # bzip2 -d xmlbugs.xml.bz2 # python modules/heartbeat/reporter.py The .bz2 file needed is up at http://dev.gentoo.org/~sping/bug-heartbeat/, too. Sebastian
Re: [gentoo-dev] [Gentoo Pheonix] Heartbeat team force
On 7 April 2010 13:20, Sebastian Pipping sp...@gentoo.org wrote: - herds.xml does not hold membership lists for all projects, not even herds. The load evaluator needs access to complete mappings to generate output close to reality. How can such a mapping be made without duplicating data? herds.xml should be the definitive resource for herds, so any herds that are not on there should be added, and outdated information should be brought up to date. Other projects should have membership listed on their project pages. Again, where these are outdated, they should be brought up to date. Cheers, -- Ben de Groot Gentoo Qt project lead developer Gentoo Wiki project lead
Re: [gentoo-dev] [Gentoo Pheonix] Heartbeat team force
On 04/07/10 14:00, Ben de Groot wrote: On 7 April 2010 13:20, Sebastian Pipping sp...@gentoo.org wrote: - herds.xml does not hold membership lists for all projects, not even herds. The load evaluator needs access to complete mappings to generate output close to reality. How can such a mapping be made without duplicating data? herds.xml should be the definitive resource for herds, so any herds that are not on there should be added, and outdated information should be brought up to date. they are in there but keep their member list elsewhere. an example is herd bsd: herd namebsd/name emailb...@nospam/email descriptionSupport for *BSD system packages, and *BSD derived packages./description maintainingproject /proj/en/gentoo-alt/bsd/index.xml /maintainingproject /herd now that i have a closer look maybe the herds itself are manageable, besides the annoyance of being in another CVS tree: subproject ref=/proj/en/gentoo-alt/bsd/obsd/index.xml inheritmembers=yes / subproject ref=/proj/en/gentoo-alt/bsd/fbsd/index.xml inheritmembers=yes / subproject ref=/proj/en/gentoo-alt/bsd/nbsd/index.xml inheritmembers=yes / dev role=leadwelp/dev dev role=lead description=Deputy leaddrizzt/dev dev role=memberaballier/dev dev role=memberthe_paya/dev that looks usable. with the herd embedded it's difficult though: the page http://www.gentoo.org/proj/en/base/embedded/index.xml lists members for it but the page source does not hold it, explicitly. where does it come from? Other projects should have membership listed on their project pages. Again, where these are outdated, they should be brought up to date. seems like they use the same syntax, too. i start to like guidexml :-) some of the mail aliases i don't handle yet, could also be a problem (especially those i don't have permission to look at). sebastian
Re: [gentoo-dev] [Gentoo Pheonix] Heartbeat team force
On 7 April 2010 20:55, Sebastian Pipping sp...@gentoo.org wrote: dev role=leadwelp/dev dev role=lead description=Deputy leaddrizzt/dev dev role=memberaballier/dev dev role=memberthe_paya/dev that looks usable. Except that it's out of date as welp is retired. with the herd embedded it's difficult though: the page http://www.gentoo.org/proj/en/base/embedded/index.xml lists members for it but the page source does not hold it, explicitly. where does it come from? It does, as can be seen in http://www.gentoo.org/proj/en/base/embedded/index.xml?passthru=1 -- Ben de Groot Gentoo Qt project lead developer Gentoo Wiki project lead
Re: [gentoo-dev] [Gentoo Pheonix] Heartbeat team force
On Sat, Apr 3, 2010 at 5:54 PM, Sebastian Pipping sp...@gentoo.org wrote: I'm calling for participation in a new team working on things around reports, bug analysis, heartbeat tracking in Gentoo: Goals = - help track the heartbeat/breath of Gentoo (bugs fixed per day, bug distribution per herd, ..) - point to problems we may be overlooking in Gentoo (letting numbers speak to us and listening to them) - help to prioritize when distributing available resources (specific calls for help, people switching teams, ..) Data we will be working with - The Bug database (XML from Bugzilla, ..) - Package tree history (VCS logs, ..) - User setup details (gentoo-smolt, ..) - .. Concrete tasks == - bugday.g.o re-write (work in progress) - work on association between bugs and packages (for all bugs, not just bugday ones) - get real numbers on how much active manpower we have People == These poeple are in de facto: - deathwing00 - sping - you here? Sebastian, count me in. I believe this sort of stuff, or small parts of what you are mentioning, was done in the newsletter. I believe this is something real nice we should put somewhere public. If this is stuff which - you have time for - you are good at please join us! Sebastian Best regards, -- Jesus Rivero (Neurogeek) Gentoo Python Team
Re: [gentoo-dev] [Gentoo Pheonix] Heartbeat team force
On Sat, Apr 3, 2010 at 3:24 PM, Sebastian Pipping sp...@gentoo.org wrote: I'm calling for participation in a new team working on things around reports, bug analysis, heartbeat tracking in Gentoo: I will help with scripting; but probably not much else. -A Goals = - help track the heartbeat/breath of Gentoo (bugs fixed per day, bug distribution per herd, ..) - point to problems we may be overlooking in Gentoo (letting numbers speak to us and listening to them) - help to prioritize when distributing available resources (specific calls for help, people switching teams, ..) Data we will be working with - The Bug database (XML from Bugzilla, ..) - Package tree history (VCS logs, ..) - User setup details (gentoo-smolt, ..) - .. Concrete tasks == - bugday.g.o re-write (work in progress) - work on association between bugs and packages (for all bugs, not just bugday ones) - get real numbers on how much active manpower we have People == These poeple are in de facto: - deathwing00 - sping - you here? If this is stuff which - you have time for - you are good at please join us! Sebastian
Re: [gentoo-dev] [Gentoo Pheonix] Heartbeat team force
On 04/04/10 00:30, Jesus Rivero (Neurogeek) wrote: Sebastian, count me in. Cool! I believe this sort of stuff, or small parts of what you are mentioning, was done in the newsletter. I believe this is something real nice we should put somewhere public. Can you elaborate on that? I'm not sure what what you refer to. Sebastian
Re: [gentoo-dev] [Gentoo Pheonix] Heartbeat team force
On 04/04/10 00:42, Alec Warner wrote: I will help with scripting; but probably not much else. Just right, there will be lots of scripting :-) The current script dumping groud is hosted here: http://git.goodpoint.de/?p=gentoo-bug-heartbeat.git;a=summary Sebastian
