Re: Question about notification on nightly test failures
Looks fine to me. On Wed, Jan 16, 2019 at 11:44 PM Carin Meier wrote: > > The Clojure package now has turned the examples into integration tests that > we would like to have CI run. It takes about 15 min for them to complete. > > We have a PR in progress > https://github.com/apache/incubator-mxnet/pull/13624 > > I would like to propose that we run these as part of the regular CI until > the notifications for nightly builds get implemented. We will run on a > whitelist or have an exclude list so that any problematic tests can be > disabled if needed. > > Please let me know any feedback on this or concerns. > > Thanks, > Carin > > > On Tue, Jan 15, 2019 at 12:24 PM Pedro Larroy > wrote: > > > Why don't we enable a slack notifier? I think it would be useful to > > interact with notifications from slack directly, including the label > > bot for example. > > > > Pedro. > > > > On Sun, Jan 13, 2019 at 1:55 AM Carin Meier wrote: > > > > > > Thanks for the explanation Marco :) > > > > > > - Carin > > > > > > On Sat, Jan 12, 2019 at 7:43 PM Marco de Abreu > > > wrote: > > > > > > > Hi Carin, > > > > > > > > thanks for thinking about adding nightly tests to clojure, I'm sure > > this > > > > will be of big benefit! > > > > > > > > You're right, the email system is in place but we basically disabled > > the > > > > service December 2017 because it was flooding the inboxes of everybody. > > > > > > > > We've been thinking about various notification methods, but were always > > > > afraid making the notifications meaningless if they come too frequently > > > > (which is WAY better now, thanks to everybody's efforts around > > stabilizing > > > > the tests!) because people would filter them. > > > > > > > > I like the idea of a specific slack channel for the notifications. Are > > > > there any alternatives the community could think of? > > > > > > > > Best regards, > > > > Marco > > > > > > > > > > > > Am Sa., 12. Jan. 2019, 10:11 hat Carin Meier > > > > geschrieben: > > > > > > > > > The Clojure package is thinking of adding some nightly tests and I'd > > like > > > > > to understand how the notification works in case of failure. > > > > > > > > > > The code in the nightly Jenkins file > > > > > > > > > > > > > > > > https://github.com/apache/incubator-mxnet/blob/master/tests/nightly/Jenkinsfile#L135 > > > > > seems to indicate that the failure is emailed. But where does this > > email > > > > > go? > > > > > If you are a contributor, do you get notified of this? > > > > > > > > > > I don't remember seeing any notification of nightly failures, so I'm > > > > > wondering how this works and if there are any improvements to > > > > accessibility > > > > > that we can make, (like maybe posting it to a slack room)? > > > > > > > > > > Thanks, > > > > > Carin > > > > > > > > > > >
Re: Question about notification on nightly test failures
The Clojure package now has turned the examples into integration tests that we would like to have CI run. It takes about 15 min for them to complete. We have a PR in progress https://github.com/apache/incubator-mxnet/pull/13624 I would like to propose that we run these as part of the regular CI until the notifications for nightly builds get implemented. We will run on a whitelist or have an exclude list so that any problematic tests can be disabled if needed. Please let me know any feedback on this or concerns. Thanks, Carin On Tue, Jan 15, 2019 at 12:24 PM Pedro Larroy wrote: > Why don't we enable a slack notifier? I think it would be useful to > interact with notifications from slack directly, including the label > bot for example. > > Pedro. > > On Sun, Jan 13, 2019 at 1:55 AM Carin Meier wrote: > > > > Thanks for the explanation Marco :) > > > > - Carin > > > > On Sat, Jan 12, 2019 at 7:43 PM Marco de Abreu > > wrote: > > > > > Hi Carin, > > > > > > thanks for thinking about adding nightly tests to clojure, I'm sure > this > > > will be of big benefit! > > > > > > You're right, the email system is in place but we basically disabled > the > > > service December 2017 because it was flooding the inboxes of everybody. > > > > > > We've been thinking about various notification methods, but were always > > > afraid making the notifications meaningless if they come too frequently > > > (which is WAY better now, thanks to everybody's efforts around > stabilizing > > > the tests!) because people would filter them. > > > > > > I like the idea of a specific slack channel for the notifications. Are > > > there any alternatives the community could think of? > > > > > > Best regards, > > > Marco > > > > > > > > > Am Sa., 12. Jan. 2019, 10:11 hat Carin Meier > > > geschrieben: > > > > > > > The Clojure package is thinking of adding some nightly tests and I'd > like > > > > to understand how the notification works in case of failure. > > > > > > > > The code in the nightly Jenkins file > > > > > > > > > > > > https://github.com/apache/incubator-mxnet/blob/master/tests/nightly/Jenkinsfile#L135 > > > > seems to indicate that the failure is emailed. But where does this > email > > > > go? > > > > If you are a contributor, do you get notified of this? > > > > > > > > I don't remember seeing any notification of nightly failures, so I'm > > > > wondering how this works and if there are any improvements to > > > accessibility > > > > that we can make, (like maybe posting it to a slack room)? > > > > > > > > Thanks, > > > > Carin > > > > > > > >
Re: Question about notification on nightly test failures
Why don't we enable a slack notifier? I think it would be useful to interact with notifications from slack directly, including the label bot for example. Pedro. On Sun, Jan 13, 2019 at 1:55 AM Carin Meier wrote: > > Thanks for the explanation Marco :) > > - Carin > > On Sat, Jan 12, 2019 at 7:43 PM Marco de Abreu > wrote: > > > Hi Carin, > > > > thanks for thinking about adding nightly tests to clojure, I'm sure this > > will be of big benefit! > > > > You're right, the email system is in place but we basically disabled the > > service December 2017 because it was flooding the inboxes of everybody. > > > > We've been thinking about various notification methods, but were always > > afraid making the notifications meaningless if they come too frequently > > (which is WAY better now, thanks to everybody's efforts around stabilizing > > the tests!) because people would filter them. > > > > I like the idea of a specific slack channel for the notifications. Are > > there any alternatives the community could think of? > > > > Best regards, > > Marco > > > > > > Am Sa., 12. Jan. 2019, 10:11 hat Carin Meier > > geschrieben: > > > > > The Clojure package is thinking of adding some nightly tests and I'd like > > > to understand how the notification works in case of failure. > > > > > > The code in the nightly Jenkins file > > > > > > > > https://github.com/apache/incubator-mxnet/blob/master/tests/nightly/Jenkinsfile#L135 > > > seems to indicate that the failure is emailed. But where does this email > > > go? > > > If you are a contributor, do you get notified of this? > > > > > > I don't remember seeing any notification of nightly failures, so I'm > > > wondering how this works and if there are any improvements to > > accessibility > > > that we can make, (like maybe posting it to a slack room)? > > > > > > Thanks, > > > Carin > > > > >
Re: Question about notification on nightly test failures
Thanks for the explanation Marco :) - Carin On Sat, Jan 12, 2019 at 7:43 PM Marco de Abreu wrote: > Hi Carin, > > thanks for thinking about adding nightly tests to clojure, I'm sure this > will be of big benefit! > > You're right, the email system is in place but we basically disabled the > service December 2017 because it was flooding the inboxes of everybody. > > We've been thinking about various notification methods, but were always > afraid making the notifications meaningless if they come too frequently > (which is WAY better now, thanks to everybody's efforts around stabilizing > the tests!) because people would filter them. > > I like the idea of a specific slack channel for the notifications. Are > there any alternatives the community could think of? > > Best regards, > Marco > > > Am Sa., 12. Jan. 2019, 10:11 hat Carin Meier > geschrieben: > > > The Clojure package is thinking of adding some nightly tests and I'd like > > to understand how the notification works in case of failure. > > > > The code in the nightly Jenkins file > > > > > https://github.com/apache/incubator-mxnet/blob/master/tests/nightly/Jenkinsfile#L135 > > seems to indicate that the failure is emailed. But where does this email > > go? > > If you are a contributor, do you get notified of this? > > > > I don't remember seeing any notification of nightly failures, so I'm > > wondering how this works and if there are any improvements to > accessibility > > that we can make, (like maybe posting it to a slack room)? > > > > Thanks, > > Carin > > >
Re: Question about notification on nightly test failures
Hi Carin, thanks for thinking about adding nightly tests to clojure, I'm sure this will be of big benefit! You're right, the email system is in place but we basically disabled the service December 2017 because it was flooding the inboxes of everybody. We've been thinking about various notification methods, but were always afraid making the notifications meaningless if they come too frequently (which is WAY better now, thanks to everybody's efforts around stabilizing the tests!) because people would filter them. I like the idea of a specific slack channel for the notifications. Are there any alternatives the community could think of? Best regards, Marco Am Sa., 12. Jan. 2019, 10:11 hat Carin Meier geschrieben: > The Clojure package is thinking of adding some nightly tests and I'd like > to understand how the notification works in case of failure. > > The code in the nightly Jenkins file > > https://github.com/apache/incubator-mxnet/blob/master/tests/nightly/Jenkinsfile#L135 > seems to indicate that the failure is emailed. But where does this email > go? > If you are a contributor, do you get notified of this? > > I don't remember seeing any notification of nightly failures, so I'm > wondering how this works and if there are any improvements to accessibility > that we can make, (like maybe posting it to a slack room)? > > Thanks, > Carin >