Public bug reported: This is kind of a halfway between Nova and Infra, but I'm adding it here for tracking purposes. Basically we have a tool for bug reporting in Nova that should be hosted in an Infra resource so that we're not dependent on whoever is running the bugs team to host it. Additionally other teams might find it useful.
To that end, I spoke with Infra about it and they expressed interest in having it integrated with their current tools if we wanted to host it. So this bug is basically figure out how to integrate the nova bugs dashboard with what's in Infra. Here's the discussion log (http://eavesdrop.openstack.org/meetings/infra/2016/infra.2016-12-13-19.03.log.txt): 19:40:41 <fungi> #topic possible hosting for a nova bugs dashboard (auggy) 19:40:47 <fungi> #link http://45.55.105.55:8082/bugs-dashboard.html current nova bugs dashboard 19:41:01 <auggy> yes! we chatted in #openstack-infra about this 19:41:16 <auggy> i just wanted to check on what you all need from me in order to evaluate this request 19:41:22 <fungi> #link http://lists.openstack.org/pipermail/openstack-dev/2016-December/108872.html Useful metrics? 19:41:28 <fungi> (related ml thread) 19:41:58 <auggy> basically we have this dashboard we use that markus wrote and hosted, and we just want it centralized so it's not depending on a single person 19:42:28 <auggy> so i thought i'd ask if it made sense to have you all host it, but if you have other suggestions then I'm open to that too 19:42:54 <fungi> #link https://github.com/markuszoeller/openstack/blob/master/scripts/launchpad/bugs_dashboard.py Creates a HTML file which can be used as a dashboard for cleanup tasks of the bug management. 19:42:57 <auggy> It's a python script that runs via cron on the hour that makes some read only queries to launchpand and then renders the results to html 19:43:05 <auggy> thx fungi ! 19:43:29 <auggy> it's also potentially usable by other openstack projects but i'm not sure how custom our bug queries are in that code 19:44:11 <fungi> we have this... 19:44:16 <JayF> auggy: just as a note, Ironic has http://ironic-divius.rhcloud.com -- I don't know where the source is that generates that though. It looks like yours covers our use case + maybe more. 19:44:18 <fungi> #link http://git.openstack.org/cgit/openstack-infra/puppet-bugdaystats/tree/manifests/site.pp 19:44:28 <fungi> could likely call it from an adjacent cron 19:44:53 <jeblair> i think the infrastructure is run by and for all the openstack projects, so these should absolutely be run centrally. any project should feel free to propose changes to infra systems to add things like this. :) 19:44:56 <clarkb> my preference would be to incorporate wahtever we do into bugday, we can evolve it if we need to, but seems like every project has their own one of these thigns and if we had to host a different one for each project well thats not really scalable 19:45:12 <jeblair> clarkb: collaboration, if possible would be nice 19:45:19 <clarkb> (neutron had similar things at one point that did get incorporated into bugday fwiw) 19:45:26 <jeblair> clarkb: this seems a little different than bugday though 19:45:38 <fungi> #link https://git.openstack.org/cgit/openstack-infra/bugdaystats 19:45:42 <jeblair> clarkb: are you thinking of the neutron reviewday thing? 19:45:51 <clarkb> jeblair: ya 19:46:07 <fungi> yeah, neutron's thing is a gerrit dashboard url generator run as part of reviewday 19:46:19 <clarkb> ya using bug info as an input iirc 19:46:45 <fungi> well, sure, reviewday itself is also doing lp queries to line reviews up with open bugs, blueprints, et cetera 19:46:57 <auggy> so if you want me to look at bugday or another tool you already have and see about integrating this into that then i can do that and then come back 19:48:10 <fungi> bugday has graphs of bug status changes over time but is currently short-duration and high-frequency updates 19:48:24 <persia> Is the scale issue about execution resources, or maintenance over time? 19:48:29 <clarkb> right, I guess my point is I think its ok to fix deficiencies in bugday so that it works for the larger set of use cases 19:48:37 <clarkb> like neutron has done 19:48:41 <clarkb> and potentially ironic and nova could do 19:48:58 <fungi> persia: more about target audience/use case i think 19:48:58 <jeblair> yeah, any integration with existing tools would be great as collectively we can spend time on making fewer tools better rather than all writing the same tool over 19:49:33 <jeblair> however, i'd rather see us running 5 redundant tools centrally than 5 redundant tools in random hosting places 19:49:46 <persia> So, one tool with per-project views? 19:50:00 <fungi> i think sdague also makes a good point that we have a lot of tooling built around graphite now, and bugday is relatively simple and could stand to be redone in something more inline with our present toolset too 19:50:16 <jeblair> yeah, bugday -> graphite would be pretty easy 19:51:05 <fungi> so maybe this script could seed a new bugstat project of some kind that can repace the bugday use cases as well with some graphs once someone wants to implement that part of it 19:51:59 <auggy> yeah i could work with markus to set it up in an openstack repo under infra or wherever you all think is best 19:52:05 <fungi> with graphite/grafana we can easily accommodate the old bugday use case (what did our squash impact look like in a 24 hour period) along with longer trending bug managers want to see 19:52:34 <persia> With nova workflow, but for selectable project? 19:52:52 <auggy> well that's the work that would have to be done on it 19:53:09 <fungi> it wouldn't be the first time nova's workflow for something got adopted by a ton of other projects ;) 19:53:30 <auggy> i haven't looked to see how specific the lp queries are but potentially one could specify them seperately to match their bugs 19:53:48 <fungi> so i'm not going to push for adding more to bugday unless this is the start of a bugday v2 reimplementation anyway 19:54:33 <auggy> eg, what queries do you want to go with which tabs kind of thing 19:54:33 <auggy> kk how about i work with markus to set this up as an openstack-infra repo project thingy? 19:54:33 <auggy> and then do some work to make it more project universal? 19:54:33 <fungi> but running this from a cron in the bugdaystats::site class and adding a tab on status.o.o for it or whatever seems fine to me 19:55:19 <fungi> auggy: i think that sounds like a fine plan 19:55:29 <fungi> anyone disagree? 19:56:02 <jeblair> sgtm 19:56:16 <pabelanger> ++ 19:56:23 <auggy> thanks :) 19:56:24 <fungi> #agreed The current Nova bug dashboard authors are welcome to import their work into a new Infra repo and then work on making it generalized for other projects' use. 19:56:56 <fungi> once that step is done, we can get more into the weeds on how to automate deployment and where to link/display it ** Affects: nova Importance: Undecided Assignee: Maciej Szankin (mszankin) Status: Confirmed ** Changed in: nova Status: New => Confirmed -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1665810 Title: Nova Bugs Dashboard should be hosted by Infra Status in OpenStack Compute (nova): Confirmed Bug description: This is kind of a halfway between Nova and Infra, but I'm adding it here for tracking purposes. Basically we have a tool for bug reporting in Nova that should be hosted in an Infra resource so that we're not dependent on whoever is running the bugs team to host it. Additionally other teams might find it useful. To that end, I spoke with Infra about it and they expressed interest in having it integrated with their current tools if we wanted to host it. So this bug is basically figure out how to integrate the nova bugs dashboard with what's in Infra. Here's the discussion log (http://eavesdrop.openstack.org/meetings/infra/2016/infra.2016-12-13-19.03.log.txt): 19:40:41 <fungi> #topic possible hosting for a nova bugs dashboard (auggy) 19:40:47 <fungi> #link http://45.55.105.55:8082/bugs-dashboard.html current nova bugs dashboard 19:41:01 <auggy> yes! we chatted in #openstack-infra about this 19:41:16 <auggy> i just wanted to check on what you all need from me in order to evaluate this request 19:41:22 <fungi> #link http://lists.openstack.org/pipermail/openstack-dev/2016-December/108872.html Useful metrics? 19:41:28 <fungi> (related ml thread) 19:41:58 <auggy> basically we have this dashboard we use that markus wrote and hosted, and we just want it centralized so it's not depending on a single person 19:42:28 <auggy> so i thought i'd ask if it made sense to have you all host it, but if you have other suggestions then I'm open to that too 19:42:54 <fungi> #link https://github.com/markuszoeller/openstack/blob/master/scripts/launchpad/bugs_dashboard.py Creates a HTML file which can be used as a dashboard for cleanup tasks of the bug management. 19:42:57 <auggy> It's a python script that runs via cron on the hour that makes some read only queries to launchpand and then renders the results to html 19:43:05 <auggy> thx fungi ! 19:43:29 <auggy> it's also potentially usable by other openstack projects but i'm not sure how custom our bug queries are in that code 19:44:11 <fungi> we have this... 19:44:16 <JayF> auggy: just as a note, Ironic has http://ironic-divius.rhcloud.com -- I don't know where the source is that generates that though. It looks like yours covers our use case + maybe more. 19:44:18 <fungi> #link http://git.openstack.org/cgit/openstack-infra/puppet-bugdaystats/tree/manifests/site.pp 19:44:28 <fungi> could likely call it from an adjacent cron 19:44:53 <jeblair> i think the infrastructure is run by and for all the openstack projects, so these should absolutely be run centrally. any project should feel free to propose changes to infra systems to add things like this. :) 19:44:56 <clarkb> my preference would be to incorporate wahtever we do into bugday, we can evolve it if we need to, but seems like every project has their own one of these thigns and if we had to host a different one for each project well thats not really scalable 19:45:12 <jeblair> clarkb: collaboration, if possible would be nice 19:45:19 <clarkb> (neutron had similar things at one point that did get incorporated into bugday fwiw) 19:45:26 <jeblair> clarkb: this seems a little different than bugday though 19:45:38 <fungi> #link https://git.openstack.org/cgit/openstack-infra/bugdaystats 19:45:42 <jeblair> clarkb: are you thinking of the neutron reviewday thing? 19:45:51 <clarkb> jeblair: ya 19:46:07 <fungi> yeah, neutron's thing is a gerrit dashboard url generator run as part of reviewday 19:46:19 <clarkb> ya using bug info as an input iirc 19:46:45 <fungi> well, sure, reviewday itself is also doing lp queries to line reviews up with open bugs, blueprints, et cetera 19:46:57 <auggy> so if you want me to look at bugday or another tool you already have and see about integrating this into that then i can do that and then come back 19:48:10 <fungi> bugday has graphs of bug status changes over time but is currently short-duration and high-frequency updates 19:48:24 <persia> Is the scale issue about execution resources, or maintenance over time? 19:48:29 <clarkb> right, I guess my point is I think its ok to fix deficiencies in bugday so that it works for the larger set of use cases 19:48:37 <clarkb> like neutron has done 19:48:41 <clarkb> and potentially ironic and nova could do 19:48:58 <fungi> persia: more about target audience/use case i think 19:48:58 <jeblair> yeah, any integration with existing tools would be great as collectively we can spend time on making fewer tools better rather than all writing the same tool over 19:49:33 <jeblair> however, i'd rather see us running 5 redundant tools centrally than 5 redundant tools in random hosting places 19:49:46 <persia> So, one tool with per-project views? 19:50:00 <fungi> i think sdague also makes a good point that we have a lot of tooling built around graphite now, and bugday is relatively simple and could stand to be redone in something more inline with our present toolset too 19:50:16 <jeblair> yeah, bugday -> graphite would be pretty easy 19:51:05 <fungi> so maybe this script could seed a new bugstat project of some kind that can repace the bugday use cases as well with some graphs once someone wants to implement that part of it 19:51:59 <auggy> yeah i could work with markus to set it up in an openstack repo under infra or wherever you all think is best 19:52:05 <fungi> with graphite/grafana we can easily accommodate the old bugday use case (what did our squash impact look like in a 24 hour period) along with longer trending bug managers want to see 19:52:34 <persia> With nova workflow, but for selectable project? 19:52:52 <auggy> well that's the work that would have to be done on it 19:53:09 <fungi> it wouldn't be the first time nova's workflow for something got adopted by a ton of other projects ;) 19:53:30 <auggy> i haven't looked to see how specific the lp queries are but potentially one could specify them seperately to match their bugs 19:53:48 <fungi> so i'm not going to push for adding more to bugday unless this is the start of a bugday v2 reimplementation anyway 19:54:33 <auggy> eg, what queries do you want to go with which tabs kind of thing 19:54:33 <auggy> kk how about i work with markus to set this up as an openstack-infra repo project thingy? 19:54:33 <auggy> and then do some work to make it more project universal? 19:54:33 <fungi> but running this from a cron in the bugdaystats::site class and adding a tab on status.o.o for it or whatever seems fine to me 19:55:19 <fungi> auggy: i think that sounds like a fine plan 19:55:29 <fungi> anyone disagree? 19:56:02 <jeblair> sgtm 19:56:16 <pabelanger> ++ 19:56:23 <auggy> thanks :) 19:56:24 <fungi> #agreed The current Nova bug dashboard authors are welcome to import their work into a new Infra repo and then work on making it generalized for other projects' use. 19:56:56 <fungi> once that step is done, we can get more into the weeds on how to automate deployment and where to link/display it To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1665810/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
