Re: Bulk Closing Intermittents

[L. David Baron]

On Wednesday 2017-07-12 06:18 -0700, emor...@mozilla.com wrote:
> On Wednesday, 12 July 2017 04:09:52 UTC+1, Karl Tomlinson  wrote:
> > I assume this was integrated with OrangeFactor?
> > 
> > That is the only way I know to determine whether an intermittent
> > failure has occurred, because failures are not necessarily
> > reported to bugzilla.
> 
> This is no longer the case, see:
> https://bugzilla.mozilla.org/show_bug.cgi?id=1372277
> 
> > Is there a mechanism for tracking a failure that we intend to
> > addresss, even when it does not fail every 21 days?
> > 
> > Would that be filing another bug without the intermittent-failure
> > keyword?
> 
> I'd add the 'leave-open' keyword.

Note that this overloading of 'leave-open' means that when you
eventually fix the bug, you need to remember to remove the
'leave-open' keyword so that it gets resolved as FIXED when the
changeset is merged from autoland or mozilla-inbound into
mozilla-central.  (This is easy to forget.)

-David

-- 
턞   L. David Baron http://dbaron.org/   턂
턢   Mozilla  https://www.mozilla.org/   턂
 Before I built a wall I'd ask to know
 What I was walling in or walling out,
 And to whom I was like to give offense.
   - Robert Frost, Mending Wall (1914)


signature.asc
Description: PGP signature
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Bulk Closing Intermittents

[Emma Humphries]

Yes, my query for bugs to close filters for "leave-open".

Thanks!

-- Emma

On Wed, Jul 12, 2017 at 6:18 AM,  wrote:

> On Wednesday, 12 July 2017 04:09:52 UTC+1, Karl Tomlinson  wrote:
> > I assume this was integrated with OrangeFactor?
> >
> > That is the only way I know to determine whether an intermittent
> > failure has occurred, because failures are not necessarily
> > reported to bugzilla.
>
> This is no longer the case, see:
> https://bugzilla.mozilla.org/show_bug.cgi?id=1372277
>
> > Is there a mechanism for tracking a failure that we intend to
> > addresss, even when it does not fail every 21 days?
> >
> > Would that be filing another bug without the intermittent-failure
> > keyword?
>
> I'd add the 'leave-open' keyword.
>
> Best wishes,
>
> Ed
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Bulk Closing Intermittents

[emorley]

On Wednesday, 12 July 2017 04:09:52 UTC+1, Karl Tomlinson  wrote:
> I assume this was integrated with OrangeFactor?
> 
> That is the only way I know to determine whether an intermittent
> failure has occurred, because failures are not necessarily
> reported to bugzilla.

This is no longer the case, see:
https://bugzilla.mozilla.org/show_bug.cgi?id=1372277

> Is there a mechanism for tracking a failure that we intend to
> addresss, even when it does not fail every 21 days?
> 
> Would that be filing another bug without the intermittent-failure
> keyword?

I'd add the 'leave-open' keyword.

Best wishes,

Ed
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Bulk Closing Intermittents

[Karl Tomlinson]

I assume this was integrated with OrangeFactor?

That is the only way I know to determine whether an intermittent
failure has occurred, because failures are not necessarily
reported to bugzilla.

Is there a mechanism for tracking a failure that we intend to
addresss, even when it does not fail every 21 days?

Would that be filing another bug without the intermittent-failure
keyword?

Emma Humphries writes:

> This was the first time we bulk closed these bugs, and there will be some
> glitches. I don't consider this to be a blocker on continuing this work.
>
> Next time we do this, it won't be 5,000+ bugs. OrangeBot runs on Sundays,
> so we can do the cleanup on Monday.
>
> The long term goal is to stop using Bugzilla to record every intermittent
> test failure and only use it for test failures we intend to address.
>
> -- Emma
>
> On Mon, Jul 10, 2017 at 5:29 AM, Kartikaya Gupta  wrote:
>
>> It might be a good idea to integrate this process with the
>> OrangeFactor Robot, to avoid race conditions like what happened on bug
>> 1328486 (it was bulk-closed, and then a couple of hours later the OF
>> robot reported that there were two failures this week - but the bug
>> remained closed).
>>
>> Cheers,
>> kats
>>
>> On Fri, Jul 7, 2017 at 10:35 PM, Emma Humphries  wrote:
>> > As discussed earlier, Joel and I have kicked off a process to close
>> > intermittent test failures in Bugzilla.
>> >
>> > If a test associated with a bug does not fail in 21 days, we'll close the
>> > bug as RESOLVED:INCOMPLETE.
>> >
>> > The first batch of intermittent bugs to close has 5,130 tickets. I have a
>> > script to close these, but to close these without bug spam requires DBA
>> > intervention.
>> >
>> > I'd like to run the closures over the weekend but that's going to create a
>> > non-trivial amount of bug spam for some of you.
>> >
>> > There is a way to get rid of the bug spam!
>> >
>> > Every bug we close will have the keyword `bulk-close-intermittents` added
>> > to it.
>> >
>> > If you search for messages from `bugzilla-dae...@mozilla.org` containing
>> > `bulk-close-intermittents` you can isolate, review, and remove those
>> > messages.
>> >
>> > Thank you for your patience while we all work to get the noise out of
>> > Bugzilla so we can find the strong signals on what we must focus on to
>> > deliver Firefox 57 in November.
>> >
>> > -- Emma
>> > ___
>> > dev-platform mailing list
>> > dev-platform@lists.mozilla.org
>> > https://lists.mozilla.org/listinfo/dev-platform
>>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Bulk Closing Intermittents

[Emma Humphries]

This was the first time we bulk closed these bugs, and there will be some
glitches. I don't consider this to be a blocker on continuing this work.

Next time we do this, it won't be 5,000+ bugs. OrangeBot runs on Sundays,
so we can do the cleanup on Monday.

The long term goal is to stop using Bugzilla to record every intermittent
test failure and only use it for test failures we intend to address.

-- Emma

On Mon, Jul 10, 2017 at 5:29 AM, Kartikaya Gupta  wrote:

> It might be a good idea to integrate this process with the
> OrangeFactor Robot, to avoid race conditions like what happened on bug
> 1328486 (it was bulk-closed, and then a couple of hours later the OF
> robot reported that there were two failures this week - but the bug
> remained closed).
>
> Cheers,
> kats
>
> On Fri, Jul 7, 2017 at 10:35 PM, Emma Humphries  wrote:
> > As discussed earlier, Joel and I have kicked off a process to close
> > intermittent test failures in Bugzilla.
> >
> > If a test associated with a bug does not fail in 21 days, we'll close the
> > bug as RESOLVED:INCOMPLETE.
> >
> > The first batch of intermittent bugs to close has 5,130 tickets. I have a
> > script to close these, but to close these without bug spam requires DBA
> > intervention.
> >
> > I'd like to run the closures over the weekend but that's going to create
> a
> > non-trivial amount of bug spam for some of you.
> >
> > There is a way to get rid of the bug spam!
> >
> > Every bug we close will have the keyword `bulk-close-intermittents` added
> > to it.
> >
> > If you search for messages from `bugzilla-dae...@mozilla.org` containing
> > `bulk-close-intermittents` you can isolate, review, and remove those
> > messages.
> >
> > Thank you for your patience while we all work to get the noise out of
> > Bugzilla so we can find the strong signals on what we must focus on to
> > deliver Firefox 57 in November.
> >
> > -- Emma
> > ___
> > dev-platform mailing list
> > dev-platform@lists.mozilla.org
> > https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Bulk Closing Intermittents

[Emma Humphries]

Dylan,

Thank you for clarifying that difference. That was sloppy of me to describe
that as a DBA operation.

-- Emma

On Mon, Jul 10, 2017 at 7:25 AM, Dylan Hardison  wrote:

>
> >
> > On Jul 7, 2017, at 19:35, Emma Humphries  wrote:
> >
> > The first batch of intermittent bugs to close has 5,130 tickets. I have a
> > script to close these, but to close these without bug spam requires DBA
> > intervention.
>
> This is not a correct statement. DBAs do not modify bugzilla data.
> Instead we have some functionality that directly manipulates the data
> model, which then updates the DB.
> We have batch processes that can make these changes without sending email,
> but this is far and away from
>  having a DBA intervention.
>
> I chose to point this out for two reasons -- the first being it would
> deeply concern me, if I was an outsider, to have an important
> production app subject to a process of "the DBA goes and changes some
> tables" and secondly because it would imply that responsibility
> of the contents to the database lies with operations. It should absolutely
> never be the case that we ask a DBA to change the contents of the bugzilla
> database
> without going through the application's model/business logic layer (and
> certainly not without the involvement of the bmo module owners).
>
>
> ___
> dev-tree-management mailing list
> dev-tree-managem...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-tree-management
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Bulk Closing Intermittents

[Kartikaya Gupta]

It might be a good idea to integrate this process with the
OrangeFactor Robot, to avoid race conditions like what happened on bug
1328486 (it was bulk-closed, and then a couple of hours later the OF
robot reported that there were two failures this week - but the bug
remained closed).

Cheers,
kats

On Fri, Jul 7, 2017 at 10:35 PM, Emma Humphries  wrote:
> As discussed earlier, Joel and I have kicked off a process to close
> intermittent test failures in Bugzilla.
>
> If a test associated with a bug does not fail in 21 days, we'll close the
> bug as RESOLVED:INCOMPLETE.
>
> The first batch of intermittent bugs to close has 5,130 tickets. I have a
> script to close these, but to close these without bug spam requires DBA
> intervention.
>
> I'd like to run the closures over the weekend but that's going to create a
> non-trivial amount of bug spam for some of you.
>
> There is a way to get rid of the bug spam!
>
> Every bug we close will have the keyword `bulk-close-intermittents` added
> to it.
>
> If you search for messages from `bugzilla-dae...@mozilla.org` containing
> `bulk-close-intermittents` you can isolate, review, and remove those
> messages.
>
> Thank you for your patience while we all work to get the noise out of
> Bugzilla so we can find the strong signals on what we must focus on to
> deliver Firefox 57 in November.
>
> -- Emma
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Bulk Closing Intermittents

[Emma Humphries]

As discussed earlier, Joel and I have kicked off a process to close
intermittent test failures in Bugzilla.

If a test associated with a bug does not fail in 21 days, we'll close the
bug as RESOLVED:INCOMPLETE.

The first batch of intermittent bugs to close has 5,130 tickets. I have a
script to close these, but to close these without bug spam requires DBA
intervention.

I'd like to run the closures over the weekend but that's going to create a
non-trivial amount of bug spam for some of you.

There is a way to get rid of the bug spam!

Every bug we close will have the keyword `bulk-close-intermittents` added
to it.

If you search for messages from `bugzilla-dae...@mozilla.org` containing
`bulk-close-intermittents` you can isolate, review, and remove those
messages.

Thank you for your patience while we all work to get the noise out of
Bugzilla so we can find the strong signals on what we must focus on to
deliver Firefox 57 in November.

-- Emma
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform