Re: Critpath karma

[Adam Williamson]

On Wed, 2018-03-07 at 14:43 -0500, Randy Barlow wrote:
> On 03/06/2018 07:20 PM, Adam Williamson wrote:
> > Removing it is one choice, sure. Looking at those ideas again and
> > deciding if we want to actually go ahead and implement any of them is
> > another choice.
> 
> Cool thanks for the history there. I actually think those ideas sound
> pretty cool and I'd +1 getting them in place (though I really can't
> personally do it soon because F28 but patches welcome).
> 
> Do you think the ideas require a separate karma feedback though? What if
> we did the ideas you talked about in response to a regular -1 on a
> critpath package? I.e., just the same karma button that we use for
> normal updates, but on critpath updates the alarm bells are bigger?

There's two problems with that:

1) The critpath can be broken by non-critpath packages, because our
critpath definition is really not *close* to correct. It's just a list
we made up one day and have not done a great job of updating since. It
is not in any way 'validated'. Until recently, it didn't have bash,
setup or selinux-policy in it, for instance.

2) Critpath packages can be broken in ways that don't break the
critical path. selinux-policy is a great example there, in fact; a -1
for an selinux-policy update *could* mean "this update renders all
systems unbootable", or it *could* mean "this update broke some obscure
leaf package two people are running".

> If we do think there's a good reason to keep two kinds of karma for the
> glorious feature, perhaps we could just hide the critpath karma feedback
> buttons in the UI for now just so they don't get confused with the
> normal karma, which can lead to some issues since it doesn't really do
> anything.

That sounds reasonable to me indeed.

>  Or another easy thing I could do server side is autodetect
> someone setting critpath karma to -1 and forcing their karma to also be
> -1 when that happens.

Also seems reasonable.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Critpath karma

[Randy Barlow]

On 03/06/2018 07:20 PM, Adam Williamson wrote:
> Removing it is one choice, sure. Looking at those ideas again and
> deciding if we want to actually go ahead and implement any of them is
> another choice.

Cool thanks for the history there. I actually think those ideas sound
pretty cool and I'd +1 getting them in place (though I really can't
personally do it soon because F28 but patches welcome).

Do you think the ideas require a separate karma feedback though? What if
we did the ideas you talked about in response to a regular -1 on a
critpath package? I.e., just the same karma button that we use for
normal updates, but on critpath updates the alarm bells are bigger?

If we do think there's a good reason to keep two kinds of karma for the
glorious feature, perhaps we could just hide the critpath karma feedback
buttons in the UI for now just so they don't get confused with the
normal karma, which can lead to some issues since it doesn't really do
anything. Or another easy thing I could do server side is autodetect
someone setting critpath karma to -1 and forcing their karma to also be
-1 when that happens.



signature.asc
Description: OpenPGP digital signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Critpath karma

[Adam Williamson]

On Wed, 2018-03-07 at 08:50 +, Zbigniew Jędrzejewski-Szmek wrote:
> On Tue, Mar 06, 2018 at 04:20:26PM -0800, Adam Williamson wrote:
> > 1. Any update that is marked as 'critpath breaking' by a FAS-registered 
> > tester would be blocked from going any further in the update process
> > without manual intervention (no autopushes at all)
> 
> IIRC, that happens with all updates now: any negative karma disables
> autopush, right?

autopush to stable, yes. I believe the text at the time referred to any
movement between states as an autopush, so it was also proposing that
e.g. if an update got -1 critpath before being sent to updates-testing, 
it shouldn't go to updates-testing.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Critpath karma

[Zbigniew Jędrzejewski-Szmek]

On Tue, Mar 06, 2018 at 04:20:26PM -0800, Adam Williamson wrote:
> 1. Any update that is marked as 'critpath breaking' by a FAS-registered 
> tester would be blocked from going any further in the update process
> without manual intervention (no autopushes at all)

IIRC, that happens with all updates now: any negative karma disables
autopush, right?

Zbyszek
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Critpath karma

[Adam Williamson]

On Tue, 2018-03-06 at 08:48 -0500, Randy Barlow wrote:
> Greetings!
> 
> Does anybody here know the history and/or purpose behind Bodhi's
> critical path karma? A brief grepping of Bodhi's codebase makes me think
> it isn't really used by Bodhi for any purpose other than recording and
> displaying people's entries. I.e., it doesn't seem to be used in Bodhi's
> state machine. It also seems to be confusing for testers.
> 
> Does it serve a purpose? If not, should we remove it?

None of the replies so far was quite correct. It exists more or less
entirely because of this old post of mine:

https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/5ADUVYLOHQVOEKPPOYQYVLCBGPNMOVI7/

which wound up being quite a significant input into Bodhi 2.0's design.
Quite a few things in Bodhi 2 are there more or less "because that post
 suggested them".

The reason you're confused is we never really finished hooking up all
the ideas from that post. We put some of the necessary bits into Bodhi,
but didn't get any further than that.

So this separate feedback item exists, I think, basically because of
this idea:

"For me, one the principal benefits of such a system would be that we
could make the 'This update breaks critical path functionality'
checkbox an absolutely red flashing light, wailing siren emergency
button. I mean, you hit that thing and trucks roll from Fedora HQ,
metaphorically speaking. It would have a confirmation page which
clearly described the impact of asserting that an update broke
critpath, so we could be confident that it didn't get falsely triggered
very often. I'd suggest that:

1. Any update that is marked as 'critpath breaking' by a FAS-registered 
tester would be blocked from going any further in the update process
without manual intervention (no autopushes at all)

2. Any update marked as 'critpath breaking' by a proven tester would be
blocked from being pushed stable at all - automatically or manually -
until the PT modified the feedback or it was overridden by someone with
appropriately godlike powers (TBD, but probably not just the
maintainer)

3. Any update marked as 'critpath breaking' should probably get
announced on at least test-announce and/or devel-announce

4. Any update marked as 'critpath breaking' *after it has already been
pushed* would similarly trigger a major response: notify maintainer
very hard, notify lists, generally do stuff to make sure it gets
immediate attention"

But in the end...we put the separate feedback item into Bodhi 2's
design, then stopped. We never actually implemented all of (or any of)
the things I suggested there. So it's a bit of an odd duck at present.

Removing it is one choice, sure. Looking at those ideas again and
deciding if we want to actually go ahead and implement any of them is
another choice.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Critpath karma

[Randy Barlow]

I've filed https://github.com/fedora-infra/bodhi/issues/2194 about
considering removing this feature from Bodhi. I listed an example there
where I thought the feature was actively harmful in a particular case,
due to users not knowing that the field doesn't actually count for anything.



signature.asc
Description: OpenPGP digital signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Critpath karma

[Randy Barlow]

On 03/06/2018 10:30 AM, Dennis Gilmore wrote:
> In the past we required a proven tester to sign off on testing a
> critpath package to provide extra assurance that the update worked and
> did not break critical functionality. I do not believe that we are
> using proven testers any longer. Having some requirements on automated
> testing for critpath I think could be useful. I would suggest that we
> look at how we could reuse it to a different purpose rather than
> throwing it out.

Interesting, so it had a purpose in the past but nothing uses it
anymore. We do have requirements for critical path packages to reach a
certain level of normal karma, and we do have Greenwave to enforce
automated tests. The critpath karma field does seem like it wouldn't
serve a purpose anymore, even given automated testing.



signature.asc
Description: OpenPGP digital signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Critpath karma

[Randy Barlow]

On 03/06/2018 08:53 AM, Peter Robinson wrote:
> Critical path were (are) packages that were in blocking deliverables,
> it pre-dates the Editions, basically if it was a core blocking package
>  eg part of Workstaiton, it needed to get more testing/karma before it
> could go stable, adamw probably has the most knowledge I suspect :)

Yeah I'm familiar with the critical path packages, but I meant to focus
more on why Bodhi has a separate karma for them, in addition to the
regular karma, that basically doesn't get used.



signature.asc
Description: OpenPGP digital signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Critpath karma

[Dennis Gilmore]

El mar, 06-03-2018 a las 08:48 -0500, Randy Barlow escribió:
> Greetings!
> 
> Does anybody here know the history and/or purpose behind Bodhi's
> critical path karma? A brief grepping of Bodhi's codebase makes me
> think
> it isn't really used by Bodhi for any purpose other than recording
> and
> displaying people's entries. I.e., it doesn't seem to be used in
> Bodhi's
> state machine. It also seems to be confusing for testers.
> 
> Does it serve a purpose? If not, should we remove it?

In the past we required a proven tester to sign off on testing a
critpath package to provide extra assurance that the update worked and
did not break critical functionality. I do not believe that we are
using proven testers any longer. Having some requirements on automated
testing for critpath I think could be useful. I would suggest that we
look at how we could reuse it to a different purpose rather than
throwing it out.

Dennis

signature.asc
Description: This is a digitally signed message part
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Critpath karma

[Peter Robinson]

On Tue, Mar 6, 2018 at 1:48 PM, Randy Barlow
 wrote:
> Greetings!
>
> Does anybody here know the history and/or purpose behind Bodhi's
> critical path karma? A brief grepping of Bodhi's codebase makes me think
> it isn't really used by Bodhi for any purpose other than recording and
> displaying people's entries. I.e., it doesn't seem to be used in Bodhi's
> state machine. It also seems to be confusing for testers.
>
> Does it serve a purpose? If not, should we remove it?

Critical path were (are) packages that were in blocking deliverables,
it pre-dates the Editions, basically if it was a core blocking package
 eg part of Workstaiton, it needed to get more testing/karma before it
could go stable, adamw probably has the most knowledge I suspect :)
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Critpath karma

[Randy Barlow]

Greetings!

Does anybody here know the history and/or purpose behind Bodhi's
critical path karma? A brief grepping of Bodhi's codebase makes me think
it isn't really used by Bodhi for any purpose other than recording and
displaying people's entries. I.e., it doesn't seem to be used in Bodhi's
state machine. It also seems to be confusing for testers.

Does it serve a purpose? If not, should we remove it?



signature.asc
Description: OpenPGP digital signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org