Re: [openstack-dev] [all] create periodic-ci-reports mailing-list
On Wed, Apr 13, 2016 at 05:12:08PM -0500, Dolph Mathews wrote: > On Wed, Apr 13, 2016 at 2:37 PM, Emilien Macchi wrote: > > > On Wed, Apr 13, 2016 at 12:13 PM, Matthew Treinish > > wrote: > > > On Wed, Apr 13, 2016 at 10:59:10AM -0400, Emilien Macchi wrote: > > >> Hi, > > >> > > >> Current OpenStack Infra Periodic jobs do not send e-mails (only > > >> periodic-stable do), so I propose to create periodic-ci-reports > > >> mailing list [1] and to use it when our periodic jobs fail [2]. > > >> If accepted, people who care about periodic jobs would like to > > >> subscribe to this new ML so they can read quick feedback from > > >> failures, thanks to e-mail filters. > > > > > > So a big motivation behind openstack-health was to make doing this not > > > necessary. In practice the ML posts never get any real attention from > > people > > > and things just sit. [3] So, instead of trying to create another ML here > > > I think it would be better to figure out why openstack-health isn't > > working > > > for doing this and figure out how to improve it. > > > > I like openstack-health, I use it mostly every day. > > Though I miss notifications, like we have with emails. > > Something we could investigate is RSS support in openstack-health. > > > > I guess everyone is different. Outside of automated systems, I don't > interface with RSS/atom anymore myself (~since Google Reader was shutdown). > > I've investigated every mailing-list based notification I've ever received, > however I don't feel compelled to respond to the mailing list thread (if > that is a metric anyone is looking at here). TBH, I don't think that's a metric that's relevant here. The argument that was being made earlier was that people want a ML to report results to because it acts as a notification system for periodic gate failures. The contention was that it enables people to respond quickly when things start to fail. But, what I was saying is that past experience has shown that in practice this is never the case. No one is actually on call for dealing with failures when they come in. So what ends up really happening is that failures just sit on the list and repeat every day. The ML is also pretty bad interface for dealing with this sort of thing over time. This problem was a big part of why openstack-health was developed. The RSS/atom idea is a compromise to provide an alternative for everyone who still says they want a notification on failures but would be more tightly integrated with the rest of the systems we're working on here. FWIW, I started doodling on doing this here: https://review.openstack.org/305496 it's still pretty far from complete, but it's a starting point. > > OpenStack Health is cool, but I certainly won't check it with any > regularity. This is actually the behavior I think we want to address. One of the goals for the project is to make it the go to spot for investigating anything related to the gate. Identifying the gaps that are preventing it from being a useful tool for you and other people is important for this goal. So I'm gonna put you on the spot, what do you think is missing from making this really useful for you today? -Matt Treinish > > >> > > >> The use-case is described in [2], please use Gerrit to give feedback. > > >> > > >> Thanks, > > >> > > >> [1] https://review.openstack.org/305326 > > >> [2] https://review.openstack.org/305278 > > > [3] > > http://lists.openstack.org/pipermail/openstack-dev/2016-February/086706.html signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] create periodic-ci-reports mailing-list
On 04/14/2016 03:22 AM, Jeremy Stanley wrote: Mentioned in IRC as well, but would an RSS/ATOM feed be a good compromise between active notification and focus on the dashboard as an entry point to researching job failures? For myself, simply ordering by date on the log page as per [1] would make it one step easier to write a local cron job to pick up the latest. -i [1] https://review.openstack.org/#/c/301989/ __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] create periodic-ci-reports mailing-list
On Wed, Apr 13, 2016 at 2:37 PM, Emilien Macchi wrote: > On Wed, Apr 13, 2016 at 12:13 PM, Matthew Treinish > wrote: > > On Wed, Apr 13, 2016 at 10:59:10AM -0400, Emilien Macchi wrote: > >> Hi, > >> > >> Current OpenStack Infra Periodic jobs do not send e-mails (only > >> periodic-stable do), so I propose to create periodic-ci-reports > >> mailing list [1] and to use it when our periodic jobs fail [2]. > >> If accepted, people who care about periodic jobs would like to > >> subscribe to this new ML so they can read quick feedback from > >> failures, thanks to e-mail filters. > > > > So a big motivation behind openstack-health was to make doing this not > > necessary. In practice the ML posts never get any real attention from > people > > and things just sit. [3] So, instead of trying to create another ML here > > I think it would be better to figure out why openstack-health isn't > working > > for doing this and figure out how to improve it. > > I like openstack-health, I use it mostly every day. > Though I miss notifications, like we have with emails. > Something we could investigate is RSS support in openstack-health. > I guess everyone is different. Outside of automated systems, I don't interface with RSS/atom anymore myself (~since Google Reader was shutdown). I've investigated every mailing-list based notification I've ever received, however I don't feel compelled to respond to the mailing list thread (if that is a metric anyone is looking at here). OpenStack Health is cool, but I certainly won't check it with any regularity. > > -Matt Treinish > > > >> > >> The use-case is described in [2], please use Gerrit to give feedback. > >> > >> Thanks, > >> > >> [1] https://review.openstack.org/305326 > >> [2] https://review.openstack.org/305278 > > [3] > http://lists.openstack.org/pipermail/openstack-dev/2016-February/086706.html > > > > > __ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > -- > Emilien Macchi > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] create periodic-ci-reports mailing-list
On Wed, Apr 13, 2016 at 12:13 PM, Matthew Treinish wrote: > On Wed, Apr 13, 2016 at 10:59:10AM -0400, Emilien Macchi wrote: >> Hi, >> >> Current OpenStack Infra Periodic jobs do not send e-mails (only >> periodic-stable do), so I propose to create periodic-ci-reports >> mailing list [1] and to use it when our periodic jobs fail [2]. >> If accepted, people who care about periodic jobs would like to >> subscribe to this new ML so they can read quick feedback from >> failures, thanks to e-mail filters. > > So a big motivation behind openstack-health was to make doing this not > necessary. In practice the ML posts never get any real attention from people > and things just sit. [3] So, instead of trying to create another ML here > I think it would be better to figure out why openstack-health isn't working > for doing this and figure out how to improve it. I like openstack-health, I use it mostly every day. Though I miss notifications, like we have with emails. Something we could investigate is RSS support in openstack-health. > -Matt Treinish > >> >> The use-case is described in [2], please use Gerrit to give feedback. >> >> Thanks, >> >> [1] https://review.openstack.org/305326 >> [2] https://review.openstack.org/305278 > [3] > http://lists.openstack.org/pipermail/openstack-dev/2016-February/086706.html > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Emilien Macchi __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] create periodic-ci-reports mailing-list
On Wed, Apr 13, 2016 at 05:22:28PM +, Jeremy Stanley wrote: > On 2016-04-13 12:58:47 -0400 (-0400), Matthew Treinish wrote: > > So, sure I understand the attraction of an active notification. > [...] > > Instead, of pretending the ML work for doing this I think it'll be > > better if we concentrate on making the dashboard for this better. > > Mentioned in IRC as well, but would an RSS/ATOM feed be a good > compromise between active notification and focus on the dashboard as > an entry point to researching job failures? That sounds like a really good idea, I like it. It enables individuals who say they want notifications for failures to subscribe to something and receive them, but in the normal case there is no extra noise being injected. We can add the rss links off of http://status.openstack.org/openstack-health to give the appearance of a uniformity. (regardless of how we actually implement the feeds) We can also have it generate links for jobs to openstack-health pages, etc. This definitely feels like a better approach to me. -Matt Treinish signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] create periodic-ci-reports mailing-list
On 2016-04-13 12:58:47 -0400 (-0400), Matthew Treinish wrote: > So, sure I understand the attraction of an active notification. [...] > Instead, of pretending the ML work for doing this I think it'll be > better if we concentrate on making the dashboard for this better. Mentioned in IRC as well, but would an RSS/ATOM feed be a good compromise between active notification and focus on the dashboard as an entry point to researching job failures? -- Jeremy Stanley __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] create periodic-ci-reports mailing-list
On Wed, Apr 13, 2016 at 06:22:50PM +0200, Ihar Hrachyshka wrote: > Matthew Treinish wrote: > > > On Wed, Apr 13, 2016 at 10:59:10AM -0400, Emilien Macchi wrote: > > > Hi, > > > > > > Current OpenStack Infra Periodic jobs do not send e-mails (only > > > periodic-stable do), so I propose to create periodic-ci-reports > > > mailing list [1] and to use it when our periodic jobs fail [2]. > > > If accepted, people who care about periodic jobs would like to > > > subscribe to this new ML so they can read quick feedback from > > > failures, thanks to e-mail filters. > > > > So a big motivation behind openstack-health was to make doing this not > > necessary. In practice the ML posts never get any real attention from > > people > > and things just sit. [3] So, instead of trying to create another ML here > > I think it would be better to figure out why openstack-health isn't working > > for doing this and figure out how to improve it. > > > > -Matt Treinish > > What I like in ML is that you get notified, instead of going to a website > each day to check job health, while it passes. In the end, you go there not > daily, but weekly or monthly, and so the failure response is not immediate. > So, sure I understand the attraction of an active notification. But, for that to work that assumes people actively looking at things and taking action as soon as they get an email. That very rarely happens, even for people who look at the failures regularly. Let's take a look at the stable maint list: http://lists.openstack.org/pipermail/openstack-stable-maint/2016-February/thread.html http://lists.openstack.org/pipermail/openstack-stable-maint/2016-March/thread.html That's 2 months, where is the immediate response on those? Only a handful only have a single isolated failure, and looking at them they are mostly transient failure (like a down mirror, git failure, upstream release, etc) which was likely blocking other development and was almost certainly fixed without looking at the ML results. The majority of the failure just sat for a long time. The issue with the ML is that it's not actually used as a notification but instead people only look at the details periodically. The ML provides a really bad interface for visualizing these results over time. That's why using openstack-health is a better solution for doing this. Now, I don't pretend that it's complete or perfect, it's still a relatively young project. But, the thing is everyone can help us work on fixing any issues or gaps with it: http://git.openstack.org/cgit/openstack/openstack-health/ Instead, of pretending the ML work for doing this I think it'll be better if we concentrate on making the dashboard for this better. -Matt Treinish signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] create periodic-ci-reports mailing-list
Matthew Treinish wrote: On Wed, Apr 13, 2016 at 10:59:10AM -0400, Emilien Macchi wrote: Hi, Current OpenStack Infra Periodic jobs do not send e-mails (only periodic-stable do), so I propose to create periodic-ci-reports mailing list [1] and to use it when our periodic jobs fail [2]. If accepted, people who care about periodic jobs would like to subscribe to this new ML so they can read quick feedback from failures, thanks to e-mail filters. So a big motivation behind openstack-health was to make doing this not necessary. In practice the ML posts never get any real attention from people and things just sit. [3] So, instead of trying to create another ML here I think it would be better to figure out why openstack-health isn't working for doing this and figure out how to improve it. -Matt Treinish What I like in ML is that you get notified, instead of going to a website each day to check job health, while it passes. In the end, you go there not daily, but weekly or monthly, and so the failure response is not immediate. Ihar signature.asc Description: Message signed with OpenPGP using GPGMail __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] create periodic-ci-reports mailing-list
On Wed, Apr 13, 2016 at 10:59:10AM -0400, Emilien Macchi wrote: > Hi, > > Current OpenStack Infra Periodic jobs do not send e-mails (only > periodic-stable do), so I propose to create periodic-ci-reports > mailing list [1] and to use it when our periodic jobs fail [2]. > If accepted, people who care about periodic jobs would like to > subscribe to this new ML so they can read quick feedback from > failures, thanks to e-mail filters. So a big motivation behind openstack-health was to make doing this not necessary. In practice the ML posts never get any real attention from people and things just sit. [3] So, instead of trying to create another ML here I think it would be better to figure out why openstack-health isn't working for doing this and figure out how to improve it. -Matt Treinish > > The use-case is described in [2], please use Gerrit to give feedback. > > Thanks, > > [1] https://review.openstack.org/305326 > [2] https://review.openstack.org/305278 [3] http://lists.openstack.org/pipermail/openstack-dev/2016-February/086706.html signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [all] create periodic-ci-reports mailing-list
Hi, Current OpenStack Infra Periodic jobs do not send e-mails (only periodic-stable do), so I propose to create periodic-ci-reports mailing list [1] and to use it when our periodic jobs fail [2]. If accepted, people who care about periodic jobs would like to subscribe to this new ML so they can read quick feedback from failures, thanks to e-mail filters. The use-case is described in [2], please use Gerrit to give feedback. Thanks, [1] https://review.openstack.org/305326 [2] https://review.openstack.org/305278 -- Emilien Macchi __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
