Re: Basic criterion proposal: g-i-s shouldn't take 2 minutes to launch

[Adam Williamson]

On Fri, 2021-09-17 at 09:34 +0200, Kamil Paral wrote:
> On Thu, Sep 16, 2021 at 7:17 PM Adam Williamson 
> wrote:
> 
> > On Wed, 2021-08-25 at 11:33 -0400, Ben Cotton wrote:
> > > On Wed, Aug 25, 2021 at 11:03 AM Kamil Paral  wrote:
> > > > Or if we want to have some exact numbers, what about "it must not add
> > more than 10 seconds to a regular system boot speed".
> > > 
> > > I have no objections to that.
> > 
> > It seems Ben went ahead and merged this into the page.
> 
> 
> A long time ago, actually:
> https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org/message/F74MPPUB654AGQAQRLSUP2KUS5JPHSYP/
> 
> But I also missed his announcement, and when this discussion was renewed, I
> thought the criterion still wasn't finalized and in effect. It seems I
> wasn't the only one :o)

I wrote that mail a week or more ago, but my mail client kindly sat on
it and never sent it out till now...
-- 
Adam Williamson
Fedora QA
IRC: adamw | Twitter: adamw_ha
https://www.happyassassin.net

___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Basic criterion proposal: g-i-s shouldn't take 2 minutes to launch

[pmkel...@frontier.com]

On 9/17/21 07:29, Lukas Ruzicka wrote:

What if we replaced the "10 seconds" with something like "reasonably fast".
With different machines people could have different expectations, so
somebody could consider 10 seconds a long time, while others could take 20
for normal.
This could make more room for usual experience and expectations.

On Fri, Sep 17, 2021 at 1:13 PM Ben Cotton  wrote:



Providing for the varied experience and expectations also provides room 
 for more bug reports. Some of those reports will likely be for times 
we either can't or won't fix. Personally I like having a number. I start 
noticing a restart taking longer than expected when there are no signs 
of progress and around 10 seconds have passed. I start thinking "this is 
taking too long at around 20 seconds, and by 30 seconds I start thinking 
this needs a bug report.



Have a Great Day!

Pat (tablepc)



On Fri, Sep 17, 2021 at 3:35 AM Kamil Paral  wrote:


But I also missed his announcement, and when this discussion was

renewed, I thought the criterion still wasn't finalized and in effect. It
seems I wasn't the only one :o)



I also forgot that I had done it when the discussion picked back up.
But yeah, I'm definitely flexible on the timing. I think the reason we
went with a number was to avoid getting bogged down in what
"reasonable" meant. But the existing criterion has its own ambiguity,
so if we can make it better*, we should.

* But what does "better" mean?! :-D

--
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure





___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure


___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Basic criterion proposal: g-i-s shouldn't take 2 minutes to launch

[Lukas Ruzicka]

What if we replaced the "10 seconds" with something like "reasonably fast".
With different machines people could have different expectations, so
somebody could consider 10 seconds a long time, while others could take 20
for normal.
This could make more room for usual experience and expectations.

On Fri, Sep 17, 2021 at 1:13 PM Ben Cotton  wrote:

> On Fri, Sep 17, 2021 at 3:35 AM Kamil Paral  wrote:
> >
> > But I also missed his announcement, and when this discussion was
> renewed, I thought the criterion still wasn't finalized and in effect. It
> seems I wasn't the only one :o)
> >
> I also forgot that I had done it when the discussion picked back up.
> But yeah, I'm definitely flexible on the timing. I think the reason we
> went with a number was to avoid getting bogged down in what
> "reasonable" meant. But the existing criterion has its own ambiguity,
> so if we can make it better*, we should.
>
> * But what does "better" mean?! :-D
>
> --
> Ben Cotton
> He / Him / His
> Fedora Program Manager
> Red Hat
> TZ=America/Indiana/Indianapolis
> ___
> test mailing list -- test@lists.fedoraproject.org
> To unsubscribe send an email to test-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
> Do not reply to spam on the list, report it:
> https://pagure.io/fedora-infrastructure
>


-- 

Lukáš Růžička

FEDORA QE, RHCE

Red Hat



Purkyňova 115

612 45 Brno - Královo Pole

lruzi...@redhat.com
TRIED AND PERSONALLY TESTED, ERGO TRUSTED. 
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Basic criterion proposal: g-i-s shouldn't take 2 minutes to launch

[Ben Cotton]

On Fri, Sep 17, 2021 at 3:35 AM Kamil Paral  wrote:
>
> But I also missed his announcement, and when this discussion was renewed, I 
> thought the criterion still wasn't finalized and in effect. It seems I wasn't 
> the only one :o)
>
I also forgot that I had done it when the discussion picked back up.
But yeah, I'm definitely flexible on the timing. I think the reason we
went with a number was to avoid getting bogged down in what
"reasonable" meant. But the existing criterion has its own ambiguity,
so if we can make it better*, we should.

* But what does "better" mean?! :-D

-- 
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Basic criterion proposal: g-i-s shouldn't take 2 minutes to launch

[Kamil Paral]

On Thu, Sep 16, 2021 at 7:17 PM Adam Williamson 
wrote:

> On Wed, 2021-08-25 at 11:33 -0400, Ben Cotton wrote:
> > On Wed, Aug 25, 2021 at 11:03 AM Kamil Paral  wrote:
> > > Or if we want to have some exact numbers, what about "it must not add
> more than 10 seconds to a regular system boot speed".
> >
> > I have no objections to that.
>
> It seems Ben went ahead and merged this into the page.


A long time ago, actually:
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org/message/F74MPPUB654AGQAQRLSUP2KUS5JPHSYP/

But I also missed his announcement, and when this discussion was renewed, I
thought the criterion still wasn't finalized and in effect. It seems I
wasn't the only one :o)
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Basic criterion proposal: g-i-s shouldn't take 2 minutes to launch

[Adam Williamson]

On Wed, 2021-08-25 at 11:33 -0400, Ben Cotton wrote:
> On Wed, Aug 25, 2021 at 11:03 AM Kamil Paral  wrote:
> > Or if we want to have some exact numbers, what about "it must not add more 
> > than 10 seconds to a regular system boot speed".
> 
> I have no objections to that.

It seems Ben went ahead and merged this into the page. Personally I
think specifying "10 seconds" is a bit much - 10 seconds on what
system? If it takes 11 on my ten-year old laptop, are we really going
to block a Beta release on that? - but at least we have something. I'll
maybe propose an adjustment next week. In the mean time, I'll propose
the bug we have in F35 Beta as a blocker for review at the meeting
tomorrow. Thanks!
-- 
Adam Williamson
Fedora QA
IRC: adamw | Twitter: adamw_ha
https://www.happyassassin.net

___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Basic criterion proposal: g-i-s shouldn't take 2 minutes to launch

[Adam Williamson]

On Wed, 2021-08-25 at 17:01 +0200, Kamil Paral wrote:
> On Wed, Aug 25, 2021 at 2:30 PM Ben Cotton  wrote:
> 
> > On Wed, Aug 25, 2021 at 6:44 AM Kamil Paral  wrote:
> > > 
> > > > > > If a utility for creating user accounts and other configuration is
> > configured to launch, it must be visible within 10 seconds of the first
> > boot reaching the launch point.
> > > 
> > > I'm not exactly clear on what "the launch point" is, i.e. when I should
> > start counting.
> > 
> > Glibly, it's 10 seconds before the point where you go "huh, this
> > should have launched by now." :-)
> > 
> 
> Well then, can we have a phrasing which will not make people ask what the
> "launch point" is? If we can't define it, don't put it there, because it
> will only confuse readers.
> 
> One option was proposed by Kalev - e.g. "it must start reasonably fast and
> not wait for an unexpected/undesired timeout".
> 
> Or if we want to have some exact numbers, what about "it must not add more
> than 10 seconds to a regular system boot speed". Of course with the slight
> problem that you don't know what a regular system boot speed is, if this is
> your first installed system and you're looking at the first system boot.
> But for a regular tester this shouldn't be a problem, and if you really
> want, you can always measure and compare it with a second+ boot.

Yeah, I would be in favour of making the language as vague as we want
the evaluation to be. If it specifies an exact time, we should be able
to measure it with a stopwatch. If we can't, let's just say "within a
reasonable time" or something vague like that.
-- 
Adam Williamson
Fedora QA
IRC: adamw | Twitter: adamw_ha
https://www.happyassassin.net

___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Basic criterion proposal: g-i-s shouldn't take 2 minutes to launch

[Ben Cotton]

On Wed, Aug 25, 2021 at 11:03 AM Kamil Paral  wrote:
> Or if we want to have some exact numbers, what about "it must not add more 
> than 10 seconds to a regular system boot speed".

I have no objections to that.

-- 
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Basic criterion proposal: g-i-s shouldn't take 2 minutes to launch

[Kamil Paral]

On Wed, Aug 25, 2021 at 2:30 PM Ben Cotton  wrote:

> On Wed, Aug 25, 2021 at 6:44 AM Kamil Paral  wrote:
> >
> >> > > If a utility for creating user accounts and other configuration is
> configured to launch, it must be visible within 10 seconds of the first
> boot reaching the launch point.
> >
> > I'm not exactly clear on what "the launch point" is, i.e. when I should
> start counting.
>
> Glibly, it's 10 seconds before the point where you go "huh, this
> should have launched by now." :-)
>

Well then, can we have a phrasing which will not make people ask what the
"launch point" is? If we can't define it, don't put it there, because it
will only confuse readers.

One option was proposed by Kalev - e.g. "it must start reasonably fast and
not wait for an unexpected/undesired timeout".

Or if we want to have some exact numbers, what about "it must not add more
than 10 seconds to a regular system boot speed". Of course with the slight
problem that you don't know what a regular system boot speed is, if this is
your first installed system and you're looking at the first system boot.
But for a regular tester this shouldn't be a problem, and if you really
want, you can always measure and compare it with a second+ boot.
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Basic criterion proposal: g-i-s shouldn't take 2 minutes to launch

[Ben Cotton]

On Wed, Aug 25, 2021 at 6:44 AM Kamil Paral  wrote:
>
>> > > If a utility for creating user accounts and other configuration is 
>> > > configured to launch, it must be visible within 10 seconds of the first 
>> > > boot reaching the launch point.
>
> I'm not exactly clear on what "the launch point" is, i.e. when I should start 
> counting.

Glibly, it's 10 seconds before the point where you go "huh, this
should have launched by now." :-)

It's admittedly a bit of a fuzzy description, but I think that's okay
as a starting point. It's not like we expect testers to sit there with
a stopwatch. This will be, IMO, a subjective-ish opinion. The 10
seconds is there more as an indicator of the magnitude of acceptable
delay, not as a firm cutoff point.


-- 
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Basic criterion proposal: g-i-s shouldn't take 2 minutes to launch

[Kalev Lember]

On Wed, Aug 25, 2021 at 12:44 PM Kamil Paral  wrote:

> On Wed, Aug 25, 2021 at 12:00 AM Adam Williamson <
> adamw...@fedoraproject.org> wrote:
>
>> On Sun, 2021-03-14 at 11:13 -0400, Ben Cotton wrote:
>> > In the wake of the BZ 1924808[1] discussion in Thursday's Go/No-Go
>> > meeting[2], I am proposing an addition to the Basic Release
>> > Criteria[3]. This would go into Post-Install Requirements -> Expected
>> > installed system boot behavior -> First boot utilities (appended after
>> > the existing sentence):
>> >
>> > > If a utility for creating user accounts and other configuration is
>> configured to launch, it must be visible within 10 seconds of the first
>> boot reaching the launch point.
>>
>
> I'm not exactly clear on what "the launch point" is, i.e. when I should
> start counting.
>

Could we just say something along the lines that "Applications and the
shell must start reasonably fast and not e.g. run into a 60 second timeout
when starting." and leave it up for interpretation?

-- 
Kalev
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Basic criterion proposal: g-i-s shouldn't take 2 minutes to launch

[pmkel...@frontier.com]

My guess is that we start counting when the wall paper is up and the top 
bar is displayed.



Have a Great Day!

Pat (tablepc)


On 8/25/21 06:43, Kamil Paral wrote:

On Wed, Aug 25, 2021 at 12:00 AM Adam Williamson 
wrote:


On Sun, 2021-03-14 at 11:13 -0400, Ben Cotton wrote:

In the wake of the BZ 1924808[1] discussion in Thursday's Go/No-Go
meeting[2], I am proposing an addition to the Basic Release
Criteria[3]. This would go into Post-Install Requirements -> Expected
installed system boot behavior -> First boot utilities (appended after
the existing sentence):


If a utility for creating user accounts and other configuration is

configured to launch, it must be visible within 10 seconds of the first
boot reaching the launch point.



I'm not exactly clear on what "the launch point" is, i.e. when I should
start counting.


___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure


___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Basic criterion proposal: g-i-s shouldn't take 2 minutes to launch

[Kamil Paral]

On Wed, Aug 25, 2021 at 12:00 AM Adam Williamson 
wrote:

> On Sun, 2021-03-14 at 11:13 -0400, Ben Cotton wrote:
> > In the wake of the BZ 1924808[1] discussion in Thursday's Go/No-Go
> > meeting[2], I am proposing an addition to the Basic Release
> > Criteria[3]. This would go into Post-Install Requirements -> Expected
> > installed system boot behavior -> First boot utilities (appended after
> > the existing sentence):
> >
> > > If a utility for creating user accounts and other configuration is
> configured to launch, it must be visible within 10 seconds of the first
> boot reaching the launch point.
>

I'm not exactly clear on what "the launch point" is, i.e. when I should
start counting.
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Basic criterion proposal: g-i-s shouldn't take 2 minutes to launch

[Chris Murphy]

On Tue, Aug 24, 2021 at 4:00 PM Adam Williamson
 wrote:
>
> On Sun, 2021-03-14 at 11:13 -0400, Ben Cotton wrote:
> > In the wake of the BZ 1924808[1] discussion in Thursday's Go/No-Go
> > meeting[2], I am proposing an addition to the Basic Release
> > Criteria[3]. This would go into Post-Install Requirements -> Expected
> > installed system boot behavior -> First boot utilities (appended after
> > the existing sentence):
> >
> > > If a utility for creating user accounts and other configuration is 
> > > configured to launch, it must be visible within 10 seconds of the first 
> > > boot reaching the launch point.
> >
> > Why 10 seconds? Why not? That sort of feels like the maximum length of
> > time someone could reasonably be expected to wait. A shorter time
> > might be better.
> >
> > I don't particularly love the wording here, but I wanted to make it
> > clear that it's not 10 seconds from power on, but 10 seconds from the
> > time the boot up reaches the state where we expect gnome-initial-setup
> > or its counterparts to appear.
>
> So, this kinda stalled, but is now a live issue again because we have
> exactly the same kind of bug again:
>
> https://bugzilla.redhat.com/show_bug.cgi?id=1997310
>
> so, no-one really objected to this proposal, right? Should we just go
> ahead and implement it, or what?

Let's go ahead and implement it. Only QA types would consider waiting
2 minutes for this just to (a) time it (b) see what happens, if
anything (c) try to find logs and/or some sort of work around. All
other mortals will properly hit the power button, give it all a second
go - which of course will just hit the same problem, i.e. broken.

-- 
Chris Murphy
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Basic criterion proposal: g-i-s shouldn't take 2 minutes to launch

[Adam Williamson]

On Sun, 2021-03-14 at 11:13 -0400, Ben Cotton wrote:
> In the wake of the BZ 1924808[1] discussion in Thursday's Go/No-Go
> meeting[2], I am proposing an addition to the Basic Release
> Criteria[3]. This would go into Post-Install Requirements -> Expected
> installed system boot behavior -> First boot utilities (appended after
> the existing sentence):
> 
> > If a utility for creating user accounts and other configuration is 
> > configured to launch, it must be visible within 10 seconds of the first 
> > boot reaching the launch point.
> 
> Why 10 seconds? Why not? That sort of feels like the maximum length of
> time someone could reasonably be expected to wait. A shorter time
> might be better.
> 
> I don't particularly love the wording here, but I wanted to make it
> clear that it's not 10 seconds from power on, but 10 seconds from the
> time the boot up reaches the state where we expect gnome-initial-setup
> or its counterparts to appear.

So, this kinda stalled, but is now a live issue again because we have
exactly the same kind of bug again:

https://bugzilla.redhat.com/show_bug.cgi?id=1997310

so, no-one really objected to this proposal, right? Should we just go
ahead and implement it, or what?
-- 
Adam Williamson
Fedora QA
IRC: adamw | Twitter: adamw_ha
https://www.happyassassin.net

___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Basic criterion proposal: g-i-s shouldn't take 2 minutes to launch

[Leslie S Satenstein via test]

Hi Chris
With the beta being very very very clean, and functional, the Linux community 
has taken the date of 25March as the official release date.I found a design 
flaw,  but not a malfunction with the beta.




Regards 
 Leslie
 Leslie Satenstein
Montréal Québec, Canada

 

On Wednesday, April 14, 2021, 11:01:37 p.m. EDT, Chris Murphy 
 wrote:  
 
 On Wed, Apr 14, 2021 at 1:13 PM Brandon Nielsen  wrote:
>
> On 4/13/21 9:39 AM, Ben Cotton wrote:
> [Snip]
> >
> >> There must be no SELinux denial notifications or crash notifications on 
> >> boot of or during installation from a release-blocking live image, or at 
> >> first login after a default install of a release-blocking desktop.
> >
> > Of course, we can always adjust this criterion in the future.
> >
> [Snip]
>
> Does that make this[0] a blocker candidate?

Probably more likely this final release criterion:

All system services present after installation with one of the
release-blocking package sets must start properly, unless they require
hardware which is not present.

https://fedoraproject.org/wiki/Fedora_34_Final_Release_Criteria#System_services


> Somewhat related, is there some mapping of compose test cases to release
> criterion? Or the inverse? Something like "Failures of test case foo,
> bar, and baz may constitute a violation of release criterion qux"?

I'm pretty sure it's all contained to the Basic, Beta, and Final
release criteria pages.


-- 
Chris Murphy
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure
  ___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Basic criterion proposal: g-i-s shouldn't take 2 minutes to launch

[Chris Murphy]

On Wed, Apr 14, 2021 at 1:13 PM Brandon Nielsen  wrote:
>
> On 4/13/21 9:39 AM, Ben Cotton wrote:
> [Snip]
> >
> >> There must be no SELinux denial notifications or crash notifications on 
> >> boot of or during installation from a release-blocking live image, or at 
> >> first login after a default install of a release-blocking desktop.
> >
> > Of course, we can always adjust this criterion in the future.
> >
> [Snip]
>
> Does that make this[0] a blocker candidate?

Probably more likely this final release criterion:

All system services present after installation with one of the
release-blocking package sets must start properly, unless they require
hardware which is not present.

https://fedoraproject.org/wiki/Fedora_34_Final_Release_Criteria#System_services


> Somewhat related, is there some mapping of compose test cases to release
> criterion? Or the inverse? Something like "Failures of test case foo,
> bar, and baz may constitute a violation of release criterion qux"?

I'm pretty sure it's all contained to the Basic, Beta, and Final
release criteria pages.


-- 
Chris Murphy
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Basic criterion proposal: g-i-s shouldn't take 2 minutes to launch

[Matthew Miller]

On Wed, Apr 14, 2021 at 02:13:31PM -0500, Brandon Nielsen wrote:
> Does that make this[0] a blocker candidate?
> [0] - https://bugzilla.redhat.com/show_bug.cgi?id=1943683

Looks like it to me.

-- 
Matthew Miller

Fedora Project Leader
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Basic criterion proposal: g-i-s shouldn't take 2 minutes to launch

[Brandon Nielsen]

On 4/14/21 2:13 PM, Brandon Nielsen wrote:
[Snip]
> Somewhat related, is there some mapping of compose test cases to release
> criterion? Or the inverse? Something like "Failures of test case foo,
> bar, and baz may constitute a violation of release criterion qux"?
> [Snip]

Nevermind, I see they're cross referenced both ways, I just somehow 
never noticed before. Sorry for the noise.


___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Basic criterion proposal: g-i-s shouldn't take 2 minutes to launch

[Brandon Nielsen]

On 4/13/21 9:39 AM, Ben Cotton wrote:
[Snip]



There must be no SELinux denial notifications or crash notifications on boot of 
or during installation from a release-blocking live image, or at first login 
after a default install of a release-blocking desktop.


Of course, we can always adjust this criterion in the future.


[Snip]

Does that make this[0] a blocker candidate?

Somewhat related, is there some mapping of compose test cases to release 
criterion? Or the inverse? Something like "Failures of test case foo, 
bar, and baz may constitute a violation of release criterion qux"?


[0] - https://bugzilla.redhat.com/show_bug.cgi?id=1943683
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Basic criterion proposal: g-i-s shouldn't take 2 minutes to launch

[Ben Cotton]

As #action-ed in yesterday's QA meeting[1], I have added the proposal
as-written to the Basic release criteria[2].

> If a utility for creating user accounts and other configuration is configured 
> to launch, it must be visible within 10 seconds of the first boot reaching 
> the launch point.

The most significant feedback was that we should also block on the sad
face. As Adam noted, we can consider that under the Final criterion

> There must be no SELinux denial notifications or crash notifications on boot 
> of or during installation from a release-blocking live image, or at first 
> login after a default install of a release-blocking desktop.

Of course, we can always adjust this criterion in the future.

[1] 
https://meetbot.fedoraproject.org/teams/fedora-qa/fedora-qa.2021-04-12-15.01.log.html#l-101
[2] 
https://fedoraproject.org/w/index.php?title=Basic_Release_Criteria=history

-- 
Ben Cotton
He / Him / His
Senior Program Manager, Fedora & CentOS Stream
Red Hat
TZ=America/Indiana/Indianapolis
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Basic criterion proposal: g-i-s shouldn't take 2 minutes to launch

[Adam Williamson]

On Mon, 2021-03-15 at 09:35 -0500, Brandon Nielsen wrote:
> On 3/14/21 10:13 AM, Ben Cotton wrote:
> > In the wake of the BZ 1924808[1] discussion in Thursday's Go/No-Go
> > meeting[2], I am proposing an addition to the Basic Release
> > Criteria[3]. This would go into Post-Install Requirements -> Expected
> > installed system boot behavior -> First boot utilities (appended after
> > the existing sentence):
> > 
> > > If a utility for creating user accounts and other configuration is 
> > > configured to launch, it must be visible within 10 seconds of the first 
> > > boot reaching the launch point.
> > 
> > Why 10 seconds? Why not? That sort of feels like the maximum length of
> > time someone could reasonably be expected to wait. A shorter time
> > might be better.
> > 
> > I don't particularly love the wording here, but I wanted to make it
> > clear that it's not 10 seconds from power on, but 10 seconds from the
> > time the boot up reaches the state where we expect gnome-initial-setup
> > or its counterparts to appear.
> > 
> > 
> > [1] https://bugzilla.redhat.com/show_bug.cgi?id=1924908
> > [2] 
> > https://meetbot.fedoraproject.org/sresults/?group_id=f34-beta-go_no_go-meeting=team
> > [3] 
> > https://fedoraproject.org/wiki/Basic_Release_Criteria#Expected_installed_system_boot_behavior
> > 
> 
> I like it.
> 
> Can we also add a "and displayed without error" clause, or maybe 
> "completes with no visible error"? Something to explicitly capture the 
> "sad face" bug[0].

We actually have this already, pretty much, for Final:

"There must be no SELinux denial notifications or crash notifications
on boot of or during installation from a release-blocking live image,
or at first login after a default install of a release-blocking
desktop."

It was envisaged to cover desktop notifications, but I'd say it seems
reasonable to count it as covering the "Oh no" screen too.
-- 
Adam Williamson
Fedora QA
IRC: adamw | Twitter: adamw_ha
https://www.happyassassin.net


___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Basic criterion proposal: g-i-s shouldn't take 2 minutes to launch

[pmkel...@frontier.com]

On 3/15/21 11:57, Kevin Fenzi wrote:

On Sun, Mar 14, 2021 at 11:13:45AM -0400, Ben Cotton wrote:

In the wake of the BZ 1924808[1] discussion in Thursday's Go/No-Go
meeting[2], I am proposing an addition to the Basic Release
Criteria[3]. This would go into Post-Install Requirements -> Expected
installed system boot behavior -> First boot utilities (appended after
the existing sentence):


If a utility for creating user accounts and other configuration is configured 
to launch, it must be visible within 10 seconds of the first boot reaching the 
launch point.


Why 10 seconds? Why not? That sort of feels like the maximum length of
time someone could reasonably be expected to wait. A shorter time
might be better.

I don't particularly love the wording here, but I wanted to make it
clear that it's not 10 seconds from power on, but 10 seconds from the
time the boot up reaches the state where we expect gnome-initial-setup
or its counterparts to appear.


Sounds reasonable to me. +1

I mean, I'd be fine changing the time a little or wording, but I agree
with the general gist of it.

kevin



I agree. Speaking as someone who restarts often; especially when running 
tests. That extra time was becoming very annoying.


Have a Great Day!

Pat (tablepc)
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Basic criterion proposal: g-i-s shouldn't take 2 minutes to launch

[pmkel...@frontier.com]

On 3/15/21 10:45, Ben Cotton wrote:

On Mon, Mar 15, 2021 at 10:41 AM Brandon Nielsen  wrote:


Can we also add a "and displayed without error" clause, or maybe
"completes with no visible error"? Something to explicitly capture the
"sad face" bug[0].

[0] - https://bugzilla.redhat.com/show_bug.cgi?id=1924908


I thought about that and decided to leave it off as a Basic criterion.
I'd be fine with it for Final, but I was worried it would be too
strict for Beta, where we expect some degree of sad face from time to
time. But maybe this isn't one of those cases? If the consensus is to
include a "no errors!", I won't object.



A Beta is something that we release to the general public. I don't think 
it's good for Fedora's reputation to have the Oops sad face screen show 
up during the installation / setup.


Sure there will be bugs in a Beta. I and I think most folks expect to 
see a small number of SE alerts and/or Abrt notifications, but not 
something that Shouts "I'm Broken!"; especially during install / setup. 
Perhaps a middle position would be to say the sad face screen can't be 
displayed.


Have a Great Day!

Pat (tablepc)
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Basic criterion proposal: g-i-s shouldn't take 2 minutes to launch

[Kevin Fenzi]

On Sun, Mar 14, 2021 at 11:13:45AM -0400, Ben Cotton wrote:
> In the wake of the BZ 1924808[1] discussion in Thursday's Go/No-Go
> meeting[2], I am proposing an addition to the Basic Release
> Criteria[3]. This would go into Post-Install Requirements -> Expected
> installed system boot behavior -> First boot utilities (appended after
> the existing sentence):
> 
> > If a utility for creating user accounts and other configuration is 
> > configured to launch, it must be visible within 10 seconds of the first 
> > boot reaching the launch point.
> 
> Why 10 seconds? Why not? That sort of feels like the maximum length of
> time someone could reasonably be expected to wait. A shorter time
> might be better.
> 
> I don't particularly love the wording here, but I wanted to make it
> clear that it's not 10 seconds from power on, but 10 seconds from the
> time the boot up reaches the state where we expect gnome-initial-setup
> or its counterparts to appear.

Sounds reasonable to me. +1

I mean, I'd be fine changing the time a little or wording, but I agree
with the general gist of it. 

kevin


signature.asc
Description: PGP signature
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Basic criterion proposal: g-i-s shouldn't take 2 minutes to launch

[Brandon Nielsen]

On 3/15/21 9:45 AM, Ben Cotton wrote:

On Mon, Mar 15, 2021 at 10:41 AM Brandon Nielsen  wrote:


Can we also add a "and displayed without error" clause, or maybe
"completes with no visible error"? Something to explicitly capture the
"sad face" bug[0].

[0] - https://bugzilla.redhat.com/show_bug.cgi?id=1924908


I thought about that and decided to leave it off as a Basic criterion.
I'd be fine with it for Final, but I was worried it would be too
strict for Beta, where we expect some degree of sad face from time to
time. But maybe this isn't one of those cases? If the consensus is to
include a "no errors!", I won't object.



I expect sad face in a beta as well, but once the user gets past the 
install and setup phase and is actually using the OS.


My hope is an earlier criterion would encourage a bit more movement on 
issues that occur immediately and right in the user's face? I found it a 
little concerning how little chatter such an obvious (no steps to 
reproduce other than "install a Gnome compose") and bad looking (if 
ultimately harmless) bug caused.

___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Basic criterion proposal: g-i-s shouldn't take 2 minutes to launch

[Ben Cotton]

On Mon, Mar 15, 2021 at 10:41 AM Brandon Nielsen  wrote:
>
> Can we also add a "and displayed without error" clause, or maybe
> "completes with no visible error"? Something to explicitly capture the
> "sad face" bug[0].
>
> [0] - https://bugzilla.redhat.com/show_bug.cgi?id=1924908

I thought about that and decided to leave it off as a Basic criterion.
I'd be fine with it for Final, but I was worried it would be too
strict for Beta, where we expect some degree of sad face from time to
time. But maybe this isn't one of those cases? If the consensus is to
include a "no errors!", I won't object.

-- 
Ben Cotton
He / Him / His
Senior Program Manager, Fedora & CentOS Stream
Red Hat
TZ=America/Indiana/Indianapolis
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Basic criterion proposal: g-i-s shouldn't take 2 minutes to launch

[Brandon Nielsen]

On 3/14/21 10:13 AM, Ben Cotton wrote:

In the wake of the BZ 1924808[1] discussion in Thursday's Go/No-Go
meeting[2], I am proposing an addition to the Basic Release
Criteria[3]. This would go into Post-Install Requirements -> Expected
installed system boot behavior -> First boot utilities (appended after
the existing sentence):


If a utility for creating user accounts and other configuration is configured 
to launch, it must be visible within 10 seconds of the first boot reaching the 
launch point.


Why 10 seconds? Why not? That sort of feels like the maximum length of
time someone could reasonably be expected to wait. A shorter time
might be better.

I don't particularly love the wording here, but I wanted to make it
clear that it's not 10 seconds from power on, but 10 seconds from the
time the boot up reaches the state where we expect gnome-initial-setup
or its counterparts to appear.


[1] https://bugzilla.redhat.com/show_bug.cgi?id=1924908
[2] 
https://meetbot.fedoraproject.org/sresults/?group_id=f34-beta-go_no_go-meeting=team
[3] 
https://fedoraproject.org/wiki/Basic_Release_Criteria#Expected_installed_system_boot_behavior



I like it.

Can we also add a "and displayed without error" clause, or maybe 
"completes with no visible error"? Something to explicitly capture the 
"sad face" bug[0].


[0] - https://bugzilla.redhat.com/show_bug.cgi?id=1924908
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Basic criterion proposal: g-i-s shouldn't take 2 minutes to launch

[Ben Cotton]

In the wake of the BZ 1924808[1] discussion in Thursday's Go/No-Go
meeting[2], I am proposing an addition to the Basic Release
Criteria[3]. This would go into Post-Install Requirements -> Expected
installed system boot behavior -> First boot utilities (appended after
the existing sentence):

> If a utility for creating user accounts and other configuration is configured 
> to launch, it must be visible within 10 seconds of the first boot reaching 
> the launch point.

Why 10 seconds? Why not? That sort of feels like the maximum length of
time someone could reasonably be expected to wait. A shorter time
might be better.

I don't particularly love the wording here, but I wanted to make it
clear that it's not 10 seconds from power on, but 10 seconds from the
time the boot up reaches the state where we expect gnome-initial-setup
or its counterparts to appear.


[1] https://bugzilla.redhat.com/show_bug.cgi?id=1924908
[2] 
https://meetbot.fedoraproject.org/sresults/?group_id=f34-beta-go_no_go-meeting=team
[3] 
https://fedoraproject.org/wiki/Basic_Release_Criteria#Expected_installed_system_boot_behavior

-- 
Ben Cotton
He / Him / His
Senior Program Manager, Fedora & CentOS Stream
Red Hat
TZ=America/Indiana/Indianapolis
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure