Re: For today's agenda item - Proposed disk unmount test case

[Kamil Paral]

On Mon, Jan 20, 2020 at 1:10 PM Adam Williamson 
wrote:

> On Wed, 2020-01-08 at 09:39 -0500, pmkel...@frontier.com wrote:
> > > > https://fedoraproject.org/wiki/User:Adamwill/Draft_testcase_reboot
> > >
> > > Hey Pat, thanks for your edits. I've made some further changes to the
> same
> > > page (you can see them in history). I think the test case is good to
> go and
> > > be incorporated into our official test case set (under a more fitting
> name,
> > > e.g. "Testcase reboot unmount"), unless somebody has any more
> > > concerns/improvements?
> > >
> >
> > Looks good to me. Thanks for all of your help.
>
> I guess one remaining question is where should the test go. To me it
> makes most sense for it to be a Base test in the Base matrix:
>
> https://fedoraproject.org/wiki/Template:Base_test_matrix
>
> so to be consistent with the other tests there, its final name should
> be 'QA:Testcase_base_reboot_unmount'. Does that sound OK to everyone?
> Thanks!
>

ACK
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[Adam Williamson]

On Wed, 2020-01-08 at 09:39 -0500, pmkel...@frontier.com wrote:
> > > https://fedoraproject.org/wiki/User:Adamwill/Draft_testcase_reboot
> > 
> > Hey Pat, thanks for your edits. I've made some further changes to the same
> > page (you can see them in history). I think the test case is good to go and
> > be incorporated into our official test case set (under a more fitting name,
> > e.g. "Testcase reboot unmount"), unless somebody has any more
> > concerns/improvements?
> > 
> 
> Looks good to me. Thanks for all of your help.

I guess one remaining question is where should the test go. To me it
makes most sense for it to be a Base test in the Base matrix:

https://fedoraproject.org/wiki/Template:Base_test_matrix

so to be consistent with the other tests there, its final name should
be 'QA:Testcase_base_reboot_unmount'. Does that sound OK to everyone?
Thanks!
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[pmkel...@frontier.com]

https://fedoraproject.org/wiki/User:Adamwill/Draft_testcase_reboot


Hey Pat, thanks for your edits. I've made some further changes to the same
page (you can see them in history). I think the test case is good to go and
be incorporated into our official test case set (under a more fitting name,
e.g. "Testcase reboot unmount"), unless somebody has any more
concerns/improvements?



Looks good to me. Thanks for all of your help.


Have a Great Day!

Pat (tablepc)
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[Kamil Paral]

On Thu, Dec 19, 2019 at 7:43 PM pmkel...@frontier.com 
wrote:

> First of all: Happy Holidays! to everyone.
>
> I ended up having some free time; so I think I have the changes done you
> wanted. There is one exception. I wasn't sure if you wanted to go ahead
> with using "sudo -i" for the commands instead of logging in with "sudo
> su -" or "su -". Here's the link:
>
> https://fedoraproject.org/wiki/User:Adamwill/Draft_testcase_reboot


Hey Pat, thanks for your edits. I've made some further changes to the same
page (you can see them in history). I think the test case is good to go and
be incorporated into our official test case set (under a more fitting name,
e.g. "Testcase reboot unmount"), unless somebody has any more
concerns/improvements?
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[pmkel...@frontier.com]

On 12/17/19 09:01, Kamil Paral wrote:

On Thu, Dec 12, 2019 at 4:17 PM pmkel...@frontier.com 
wrote:


Yes, but we use the keywords from those samples in the grep pattern. So

if

the sample is outdated, the grep pattern is likely outdated as well, and

we

need to update the test case anyway. And it might be even more obvious

that

we need to update it if the sample it there (otherwise you have no idea
where the "unmounted" is coming from).


I still have the samples saved if that helps.

If you like I would be happy to pull all this together into a new
version.



Hi Pat, yes, please do, thanks.



I could also make another test case for the reboot. I just put
reboot back in because I thought one of the comments wanted it. From
what I've read it seems like we would only do the reboot test case on
the live image since the unmount messages would get wiped in the reboot.
Did I miss something. More good learning experience. :-)



My view is that we'll perform the installation, reboot to the actual
system, check for errors, reboot, check for errors again. That makes sure
the installer unmounts correctly and that the installed system unmounts
correctly.




Would we put the samples in the "box out"?



Not sure I understand. Try to do something that looks good to you :)




First of all: Happy Holidays! to everyone.

I ended up having some free time; so I think I have the changes done you 
wanted. There is one exception. I wasn't sure if you wanted to go ahead 
with using "sudo -i" for the commands instead of logging in with "sudo 
su -" or "su -". Here's the link:


https://fedoraproject.org/wiki/User:Adamwill/Draft_testcase_reboot

Have a Great Day!

Pat (tablepc)
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[Kamil Paral]

On Tue, Dec 17, 2019 at 5:39 PM pmkel...@frontier.com 
wrote:

> Okay, I'll start with Adam's version and make the changes.
>
> I am getting into end of year tasks and holidays here. I think I can
> have the new version ready by the 2nd week in January if that's okay.
>

You're not the only one having holidays :) Enjoy and see you in January.
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[pmkel...@frontier.com]

On 12/17/19 09:01, Kamil Paral wrote:

On Thu, Dec 12, 2019 at 4:17 PM pmkel...@frontier.com 
wrote:


Yes, but we use the keywords from those samples in the grep pattern. So

if

the sample is outdated, the grep pattern is likely outdated as well, and

we

need to update the test case anyway. And it might be even more obvious

that

we need to update it if the sample it there (otherwise you have no idea
where the "unmounted" is coming from).


I still have the samples saved if that helps.

If you like I would be happy to pull all this together into a new
version.



Hi Pat, yes, please do, thanks.



I could also make another test case for the reboot. I just put
reboot back in because I thought one of the comments wanted it. From
what I've read it seems like we would only do the reboot test case on
the live image since the unmount messages would get wiped in the reboot.
Did I miss something. More good learning experience. :-)



My view is that we'll perform the installation, reboot to the actual
system, check for errors, reboot, check for errors again. That makes sure
the installer unmounts correctly and that the installed system unmounts
correctly.




Would we put the samples in the "box out"?



Not sure I understand. Try to do something that looks good to you :)



Okay, I'll start with Adam's version and make the changes.

I am getting into end of year tasks and holidays here. I think I can 
have the new version ready by the 2nd week in January if that's okay.



Have a Great Day!

Pat (tablepc)
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[Kamil Paral]

On Thu, Dec 12, 2019 at 4:17 PM pmkel...@frontier.com 
wrote:

> > Yes, but we use the keywords from those samples in the grep pattern. So
> if
> > the sample is outdated, the grep pattern is likely outdated as well, and
> we
> > need to update the test case anyway. And it might be even more obvious
> that
> > we need to update it if the sample it there (otherwise you have no idea
> > where the "unmounted" is coming from).
>
> I still have the samples saved if that helps.
>
> If you like I would be happy to pull all this together into a new
> version.


Hi Pat, yes, please do, thanks.


> I could also make another test case for the reboot. I just put
> reboot back in because I thought one of the comments wanted it. From
> what I've read it seems like we would only do the reboot test case on
> the live image since the unmount messages would get wiped in the reboot.
> Did I miss something. More good learning experience. :-)
>

My view is that we'll perform the installation, reboot to the actual
system, check for errors, reboot, check for errors again. That makes sure
the installer unmounts correctly and that the installed system unmounts
correctly.


>
> Would we put the samples in the "box out"?
>

Not sure I understand. Try to do something that looks good to you :)
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[pmkel...@frontier.com]

On 12/12/19 08:28, Kamil Paral wrote:

On Wed, Dec 11, 2019 at 5:14 PM Adam Williamson 
wrote:


What's the point of running the checks from Live image as well? The local
filesystems will likely get wiped and re-created anyway. Extra failures

in

the log might just confuse us when reading the bug report.


That was in Pat's version, I just re-worded it. But yeah, this is a
reasonable point (in fact you can't check the logs after rebooting the
live image because they won't be there). I think this is a bit of an
artifact of the fact that this test case is trying to cover both 'does
it shut down / reboot correctly?' and 'do filesystems unmount
correctly?', and Pat was maybe more interested in emphasizing the
second. I think checking that the live image shuts down properly is
worthwhile, but we should reword it to reflect that's all there is to
check.



Ah, this is where the confusion comes from. I thought we've established
that this testcase is about filesystem clean unmount, and we should have a
separate test case if we want to cover reboot/poweroff. I support two
separate test cases.





(Also, if we want to keep this, it should be the first step, not the

third

one. I wouldn't expect the reader to go and read the whole test case

first,

before starting with the first step.)


Yeah, good point.


On the installed system, run a console and become root with sudo su or

su.




https://unix.stackexchange.com/questions/7013/why-do-we-use-su-and-not-just-su

*sigh* ten points


If there was output from the grep command, run the command journalctl -b

-l

journal.log


"The old options -l/--full are not useful anymore, except to undo
--no-full."
(man journalctl)


Again, taken from Pat's version. Quit blaming me. :P



Sorry, I was enjoying it a bit:-)


I don't mind; after all I'm the one still learning my way around here.




Run the following command: journalctl -b | grep -E "dirty bit|data may be

corrupt|recovery|unmounted|recovering" and see whether there is any

output.



If there was output from the grep command, run the command journalctl -b

-l

journal.log. Please file a bug report (the kernel is most likely the

correct package to file the report against) and attach the journal.log

file

to the bug report.



I'd probably add another step between those two. If there was some match,
I'd ask the user to see the full journal, find the matched line, and
inspect it in context. If it looks to be a standard operation printout,
e.g. a successful unmount operation or a message coming from some

unrelated

app and not from kernel/systemd/fsck, that message should be ignored.

Only

if it looks like a filesystem error similar to the sample errors provided
by Chris, the full log should be saved and a bug reported. Which means

I'd

like to add sample errors inside the test case, e.g. into another section
called "Filesystem errors examples". Those error samples might come in
handy in the future revisions of the test case as well, e.g. if we find

out

that the grep command is too broad.


I can see this, but I'm not *sure* about the 'sample errors', because
that's the kind of content that goes stale very quickly. I usually work
quite hard to write test cases such that they shouldn't need much
updating, because no-one ever does update them...



Yes, but we use the keywords from those samples in the grep pattern. So if
the sample is outdated, the grep pattern is likely outdated as well, and we
need to update the test case anyway. And it might be even more obvious that
we need to update it if the sample it there (otherwise you have no idea
where the "unmounted" is coming from).


I still have the samples saved if that helps.

If you like I would be happy to pull all this together into a new 
version. I could also make another test case for the reboot. I just put 
reboot back in because I thought one of the comments wanted it. From 
what I've read it seems like we would only do the reboot test case on 
the live image since the unmount messages would get wiped in the reboot. 
Did I miss something. More good learning experience. :-)


Would we put the samples in the "box out"?

Have a Great Day!

Pat (tablepc)
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[Kamil Paral]

On Wed, Dec 11, 2019 at 7:27 PM Chris Murphy 
wrote:

> I'd also drop all the preconditions like using defaults. Pretty much
> any possible installer allowed configuration that trips up this test
> case is an eyeball opener and should be tracked down.
>

+1


>
> > >
> https://unix.stackexchange.com/questions/7013/why-do-we-use-su-and-not-just-su
> >
> > *sigh* ten points
>
> How to test #1 can probably be omitted in favor of just using # in
> front of commands that are expected to be run either as root user or
> with sudo. *shrug*
>

I find it better to say "Run the following command with administrative
privileges:" (or root or superuser privileges). Anyone who doesn't know
what it means likely can't debug the issue anyway.

The second approach I do (and probably even prefer to the above, because it
is more explicit) is to prefix all such commands with sudo. Then it is
clear that you need to run them as root. And if you run them verbatim under
root directly (including sudo), there's no harm.


>
> (I'm also a fan of `sudo -i` anyway...)
>
> https://unix.stackexchange.com/questions/35338/su-vs-sudo-s-vs-sudo-i-vs-sudo-bash/35342


+1

"su -" and "sudo -i". Wipe everything else from your memory, people :-)

I'd like the test case to have the least burden on the reporter that
> also produces a useful report:
> 1. includes the -b journal (current, shows journal replay messages)
> 2. includes the -b -1 journal (previous, might show some evidence of
> why unmount was not clean)
>

Yes, that's reasonable.
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[Kamil Paral]

On Wed, Dec 11, 2019 at 5:14 PM Adam Williamson 
wrote:

> > What's the point of running the checks from Live image as well? The local
> > filesystems will likely get wiped and re-created anyway. Extra failures
> in
> > the log might just confuse us when reading the bug report.
>
> That was in Pat's version, I just re-worded it. But yeah, this is a
> reasonable point (in fact you can't check the logs after rebooting the
> live image because they won't be there). I think this is a bit of an
> artifact of the fact that this test case is trying to cover both 'does
> it shut down / reboot correctly?' and 'do filesystems unmount
> correctly?', and Pat was maybe more interested in emphasizing the
> second. I think checking that the live image shuts down properly is
> worthwhile, but we should reword it to reflect that's all there is to
> check.
>

Ah, this is where the confusion comes from. I thought we've established
that this testcase is about filesystem clean unmount, and we should have a
separate test case if we want to cover reboot/poweroff. I support two
separate test cases.


>
> > (Also, if we want to keep this, it should be the first step, not the
> third
> > one. I wouldn't expect the reader to go and read the whole test case
> first,
> > before starting with the first step.)
>
> Yeah, good point.
>
> > On the installed system, run a console and become root with sudo su or
> su.
> >
> >
> https://unix.stackexchange.com/questions/7013/why-do-we-use-su-and-not-just-su
>
> *sigh* ten points
>
> > If there was output from the grep command, run the command journalctl -b
> -l
> > > > journal.log
> >
> > "The old options -l/--full are not useful anymore, except to undo
> > --no-full."
> > (man journalctl)
>
> Again, taken from Pat's version. Quit blaming me. :P
>

Sorry, I was enjoying it a bit:-)


>
> > Run the following command: journalctl -b | grep -E "dirty bit|data may be
> > > corrupt|recovery|unmounted|recovering" and see whether there is any
> output.
> > >
> > If there was output from the grep command, run the command journalctl -b
> -l
> > > > journal.log. Please file a bug report (the kernel is most likely the
> > > correct package to file the report against) and attach the journal.log
> file
> > > to the bug report.
> > >
> >
> > I'd probably add another step between those two. If there was some match,
> > I'd ask the user to see the full journal, find the matched line, and
> > inspect it in context. If it looks to be a standard operation printout,
> > e.g. a successful unmount operation or a message coming from some
> unrelated
> > app and not from kernel/systemd/fsck, that message should be ignored.
> Only
> > if it looks like a filesystem error similar to the sample errors provided
> > by Chris, the full log should be saved and a bug reported. Which means
> I'd
> > like to add sample errors inside the test case, e.g. into another section
> > called "Filesystem errors examples". Those error samples might come in
> > handy in the future revisions of the test case as well, e.g. if we find
> out
> > that the grep command is too broad.
>
> I can see this, but I'm not *sure* about the 'sample errors', because
> that's the kind of content that goes stale very quickly. I usually work
> quite hard to write test cases such that they shouldn't need much
> updating, because no-one ever does update them...
>

Yes, but we use the keywords from those samples in the grep pattern. So if
the sample is outdated, the grep pattern is likely outdated as well, and we
need to update the test case anyway. And it might be even more obvious that
we need to update it if the sample it there (otherwise you have no idea
where the "unmounted" is coming from).
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[Chris Murphy]

On Wed, Dec 11, 2019 at 12:28 PM Adam Williamson
 wrote:
>
> On Wed, 2019-12-11 at 14:11 -0500, pmkel...@frontier.com wrote:
> > The test case is running on a new install. During the install a user was
> > set up and that user has Admin. In that situation I've never had
> > problems running journalcrl, grep, or this journalctl | grep. Perhaps we
> > should have them login to their user account.
> >
> > I've not found the need to invoke sudo, su, or sudo su.
>
> Running journalctl as regular user works, but only shows you the
> messages from that user's session. It doesn't show you the full system
> journal.

Maybe if you're in group wheel it shows you everything (?)
I'm in the habit of using sudo with it, but tried it..
[chris@flap ~]$ journalctl -b > nosudo.txt
[chris@flap ~]$ sudo journalctl -b > withsudo.txt
[chris@flap ~]$ diff nosudo.txt withsudo.txt
1c1
< -- Logs begin at Mon 2019-12-09 21:27:41 MST, end at Wed 2019-12-11
14:37:41 MST. --
---
> -- Logs begin at Mon 2019-12-09 21:27:41 MST, end at Wed 2019-12-11 14:45:12 
> MST. --
3470a3471,3474
> Dec 11 14:45:12 flap.local systemd[1357]: Starting Tracker metadata database 
> store and lookup manager...
> Dec 11 14:45:12 flap.local systemd[1357]: Started Tracker metadata database 
> store and lookup manager.
> Dec 11 14:45:12 flap.local systemd[1357]: Starting Tracker metadata 
> extractor...
> Dec 11 14:45:12 flap.local systemd[1357]: Started Tracker metadata extractor.


*shrug*

-- 
Chris Murphy
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[Adam Williamson]

On Wed, 2019-12-11 at 14:11 -0500, pmkel...@frontier.com wrote:
> The test case is running on a new install. During the install a user was 
> set up and that user has Admin. In that situation I've never had 
> problems running journalcrl, grep, or this journalctl | grep. Perhaps we 
> should have them login to their user account.
> 
> I've not found the need to invoke sudo, su, or sudo su.

Running journalctl as regular user works, but only shows you the
messages from that user's session. It doesn't show you the full system
journal.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[pmkel...@frontier.com]

The test case is running on a new install. During the install a user was 
set up and that user has Admin. In that situation I've never had 
problems running journalcrl, grep, or this journalctl | grep. Perhaps we 
should have them login to their user account.


I've not found the need to invoke sudo, su, or sudo su.

From what I've read in this chain it seems like there are references to 
one of my old drafts. Adam's draft is most recent. Are we all ready from 
the same doc?



Gave a Great Day!

Pat (tablepc)


On 12/11/19 13:58, Adam Williamson wrote:

On Wed, 2019-12-11 at 11:20 -0700, Chris Murphy wrote:

How to test #1 can probably be omitted in favor of just using # in

front of commands that are expected to be run either as root user or

with sudo. *shrug*


I never do this. I hate it. Because it causes this:

[root@adam sysprof (master *%)]# # cat /tmp/foo

commands should always, always be easy to copy/paste.


___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[Adam Williamson]

On Wed, 2019-12-11 at 11:20 -0700, Chris Murphy wrote:
> How to test #1 can probably be omitted in favor of just using # in
> 
> front of commands that are expected to be run either as root user or
> 
> with sudo. *shrug*

I never do this. I hate it. Because it causes this:

[root@adam sysprof (master *%)]# # cat /tmp/foo

commands should always, always be easy to copy/paste.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[Chris Murphy]

On Wed, Dec 11, 2019 at 9:14 AM Adam Williamson
 wrote:
> That was in Pat's version, I just re-worded it. But yeah, this is a
> reasonable point (in fact you can't check the logs after rebooting the
> live image because they won't be there). I think this is a bit of an
> artifact of the fact that this test case is trying to cover both 'does
> it shut down / reboot correctly?' and 'do filesystems unmount
> correctly?', and Pat was maybe more interested in emphasizing the
> second. I think checking that the live image shuts down properly is
> worthwhile, but we should reword it to reflect that's all there is to
> check.

I think the test case should focus on looking for the circumstantial
evidence that a previous reboot/shutdown was somehow improper.

A complete test case for all possible improper reboot/shutdown is
difficult. Little is shown on the console (Esc to hide plymouth) and
it goes by really fast. Many details of reboot/shutdown aren't logged
by default, even with systemd.log_level=debug set much of shutdown
happens after root is remounted ro and that requires special logging
(systemd debugging has a how to). In my real world example failure
case, the first indication I had that reboot went badly was a grub>
prompt :D

I suggest keeping the scope of this test case narrow, to that of an
unclean unmount.

I'd also drop all the preconditions like using defaults. Pretty much
any possible installer allowed configuration that trips up this test
case is an eyeball opener and should be tracked down.

> > https://unix.stackexchange.com/questions/7013/why-do-we-use-su-and-not-just-su
>
> *sigh* ten points

How to test #1 can probably be omitted in favor of just using # in
front of commands that are expected to be run either as root user or
with sudo. *shrug*

(I'm also a fan of `sudo -i` anyway...)
https://unix.stackexchange.com/questions/35338/su-vs-sudo-s-vs-sudo-i-vs-sudo-bash/35342

> I can see this, but I'm not *sure* about the 'sample errors', because
> that's the kind of content that goes stale very quickly. I usually work
> quite hard to write test cases such that they shouldn't need much
> updating, because no-one ever does update them...

I'd like the test case to have the least burden on the reporter that
also produces a useful report:
1. includes the -b journal (current, shows journal replay messages)
2. includes the -b -1 journal (previous, might show some evidence of
why unmount was not clean)
3. reproduce steps

I don't really care to burden them with having to look through the
journal manually. If they have to do that, the test case needs to be
tweaked.

> > I think these "expected results" can be easily omitted, because they are
> > obvious from how the test steps are written. No hard feelings, though.
>
> I'd rather keep them, just because that's how we do test cases. Also, I
> mean, it *is* useful to be explicit about which of the steps in a test
> case are the key ones and which aren't.

Expected #2 is hard to determine, short of explosions and fires,
singularities appearing...


>
> > Draft testcase reboot
> >
> > The testcase should be renamed, I think. Something like "Testcase
> > filesystem consistency" or "Testcase clean unmount" or "Testcase reboot
> > unmount".
>
> Yeah, I was actually thinking the same.

Concur.




-- 
Chris Murphy
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[Adam Williamson]

On Wed, 2019-12-11 at 14:13 +0100, Kamil Paral wrote:
> On Mon, Dec 9, 2019 at 7:05 PM Adam Williamson 
> wrote:
> 
> > On Sat, 2019-12-07 at 09:45 -0500, pmkel...@frontier.com wrote:
> > > I think I have it close to what you want. Please let me know.
> > > 
> > > Here's the link:
> > > 
> > > https://fedoraproject.org/wiki/User:Tablepc/Draft_testcase_reboot
> > > 
> > >   Have a Great Day!
> > 
> > Hey folks! So as mentioned in the meeting today, here's my draft with a
> > few revisions to Pat's draft:
> > 
> > https://fedoraproject.org/wiki/User:Adamwill/Draft_testcase_reboot
> 
> Thanks Adam. Here are some questions and nitpicks, as always:
> 
> If you are testing a live image, perform the steps below from the live
> > environment before doing the install. After this go ahead with the install.
> > Then perform the steps below from the installed system.
> > 
> 
> What's the point of running the checks from Live image as well? The local
> filesystems will likely get wiped and re-created anyway. Extra failures in
> the log might just confuse us when reading the bug report.

That was in Pat's version, I just re-worded it. But yeah, this is a
reasonable point (in fact you can't check the logs after rebooting the
live image because they won't be there). I think this is a bit of an
artifact of the fact that this test case is trying to cover both 'does
it shut down / reboot correctly?' and 'do filesystems unmount
correctly?', and Pat was maybe more interested in emphasizing the
second. I think checking that the live image shuts down properly is
worthwhile, but we should reword it to reflect that's all there is to
check.

> (Also, if we want to keep this, it should be the first step, not the third
> one. I wouldn't expect the reader to go and read the whole test case first,
> before starting with the first step.)

Yeah, good point.

> On the installed system, run a console and become root with sudo su or su.
> 
> https://unix.stackexchange.com/questions/7013/why-do-we-use-su-and-not-just-su

*sigh* ten points

> If there was output from the grep command, run the command journalctl -b -l
> > > journal.log
> 
> "The old options -l/--full are not useful anymore, except to undo
> --no-full."
> (man journalctl)

Again, taken from Pat's version. Quit blaming me. :P

> Run the following command: journalctl -b | grep -E "dirty bit|data may be
> > corrupt|recovery|unmounted|recovering" and see whether there is any output.
> > 
> If there was output from the grep command, run the command journalctl -b -l
> > > journal.log. Please file a bug report (the kernel is most likely the
> > correct package to file the report against) and attach the journal.log file
> > to the bug report.
> > 
> 
> I'd probably add another step between those two. If there was some match,
> I'd ask the user to see the full journal, find the matched line, and
> inspect it in context. If it looks to be a standard operation printout,
> e.g. a successful unmount operation or a message coming from some unrelated
> app and not from kernel/systemd/fsck, that message should be ignored. Only
> if it looks like a filesystem error similar to the sample errors provided
> by Chris, the full log should be saved and a bug reported. Which means I'd
> like to add sample errors inside the test case, e.g. into another section
> called "Filesystem errors examples". Those error samples might come in
> handy in the future revisions of the test case as well, e.g. if we find out
> that the grep command is too broad.

I can see this, but I'm not *sure* about the 'sample errors', because
that's the kind of content that goes stale very quickly. I usually work
quite hard to write test cases such that they shouldn't need much
updating, because no-one ever does update them...

> Each grep command should produce no output.
> > Running reboot should cause an orderly shutdown and restart of the
> > system.
> > 
> 
> I think these "expected results" can be easily omitted, because they are
> obvious from how the test steps are written. No hard feelings, though.

I'd rather keep them, just because that's how we do test cases. Also, I
mean, it *is* useful to be explicit about which of the steps in a test
case are the key ones and which aren't.

> Draft testcase reboot
> 
> The testcase should be renamed, I think. Something like "Testcase
> filesystem consistency" or "Testcase clean unmount" or "Testcase reboot
> unmount".

Yeah, I was actually thinking the same.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 

Re: For today's agenda item - Proposed disk unmount test case

[Kamil Paral]

On Mon, Dec 9, 2019 at 7:05 PM Adam Williamson 
wrote:

> On Sat, 2019-12-07 at 09:45 -0500, pmkel...@frontier.com wrote:
> > I think I have it close to what you want. Please let me know.
> >
> > Here's the link:
> >
> > https://fedoraproject.org/wiki/User:Tablepc/Draft_testcase_reboot
> >
> >   Have a Great Day!
>
> Hey folks! So as mentioned in the meeting today, here's my draft with a
> few revisions to Pat's draft:
>
> https://fedoraproject.org/wiki/User:Adamwill/Draft_testcase_reboot


Thanks Adam. Here are some questions and nitpicks, as always:

If you are testing a live image, perform the steps below from the live
> environment before doing the install. After this go ahead with the install.
> Then perform the steps below from the installed system.
>

What's the point of running the checks from Live image as well? The local
filesystems will likely get wiped and re-created anyway. Extra failures in
the log might just confuse us when reading the bug report.

(Also, if we want to keep this, it should be the first step, not the third
one. I wouldn't expect the reader to go and read the whole test case first,
before starting with the first step.)

On the installed system, run a console and become root with sudo su or su.
>

https://unix.stackexchange.com/questions/7013/why-do-we-use-su-and-not-just-su

If there was output from the grep command, run the command journalctl -b -l
> > journal.log
>

"The old options -l/--full are not useful anymore, except to undo
--no-full."
(man journalctl)

Run the following command: journalctl -b | grep -E "dirty bit|data may be
> corrupt|recovery|unmounted|recovering" and see whether there is any output.
>
If there was output from the grep command, run the command journalctl -b -l
> > journal.log. Please file a bug report (the kernel is most likely the
> correct package to file the report against) and attach the journal.log file
> to the bug report.
>

I'd probably add another step between those two. If there was some match,
I'd ask the user to see the full journal, find the matched line, and
inspect it in context. If it looks to be a standard operation printout,
e.g. a successful unmount operation or a message coming from some unrelated
app and not from kernel/systemd/fsck, that message should be ignored. Only
if it looks like a filesystem error similar to the sample errors provided
by Chris, the full log should be saved and a bug reported. Which means I'd
like to add sample errors inside the test case, e.g. into another section
called "Filesystem errors examples". Those error samples might come in
handy in the future revisions of the test case as well, e.g. if we find out
that the grep command is too broad.

Each grep command should produce no output.
> Running reboot should cause an orderly shutdown and restart of the
> system.
>

I think these "expected results" can be easily omitted, because they are
obvious from how the test steps are written. No hard feelings, though.

Draft testcase reboot
>

The testcase should be renamed, I think. Something like "Testcase
filesystem consistency" or "Testcase clean unmount" or "Testcase reboot
unmount".
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[Adam Williamson]

On Sat, 2019-12-07 at 09:45 -0500, pmkel...@frontier.com wrote:
> I think I have it close to what you want. Please let me know.
> 
> Here's the link:
> 
> https://fedoraproject.org/wiki/User:Tablepc/Draft_testcase_reboot
> 
>   Have a Great Day!

Hey folks! So as mentioned in the meeting today, here's my draft with a
few revisions to Pat's draft:

https://fedoraproject.org/wiki/User:Adamwill/Draft_testcase_reboot

I mostly just cleaned up the text and formatting a bit. I also added
the 'associated release criterion' boxout at the top and a boxout later
noting that you can also check for errors missed by the grep command
manually, and added another expected result (that running 'reboot'
should work).

Note I didn't include the [[Category]] block in my draft as doing so
actually puts the draft in the category, which makes it show up in
Bodhi and stuff. So I usually leave those lines out of the draft form
and add them when putting the test case in production.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[pmkel...@frontier.com]

I think I have it close to what you want. Please let me know.

Here's the link:

https://fedoraproject.org/wiki/User:Tablepc/Draft_testcase_reboot

Have a Great Day!

Pat (tablepc)


On 12/5/19 14:07, pmkel...@frontier.com wrote:


On 12/5/19 03:56, Kamil Paral wrote:



Here's the email I had in mind, containing the important journal 
messages:
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org/message/S5LQTXKFPIUKBFC7HL5SNSAV3ZRFP6WV/ 



We need to create a command or commands that will detect any of the 
failure

states (and ideally another command for any of the success states, for
confirmation).


It doesn't really matter if you grep the journal directly or save the
journal and grep the file. But it's one fewer command to grep the journal
directly. And if a failure is found, ask the user to save the journal and
report a bug.



I have the test case updated, but I ran into a problem with the test 
case template. The "|" character is apparently a reserved character in 
the template code. I couldn't get it to let me have the pipe from the 
journal to the grep. Nor could I use the -E option with the grep because 
that character is used to separate the match patterns.


 Have a Great Day!

 Pat    (tablepc)



___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/

List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org 




___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/

List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[Kamil Paral]

On Thu, Dec 5, 2019 at 8:07 PM pmkel...@frontier.com 
wrote:

>
> On 12/5/19 03:56, Kamil Paral wrote:
>
> >
> > Here's the email I had in mind, containing the important journal
> messages:
> >
> https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org/message/S5LQTXKFPIUKBFC7HL5SNSAV3ZRFP6WV/
> >
> > We need to create a command or commands that will detect any of the
> failure
> > states (and ideally another command for any of the success states, for
> > confirmation).
> >
> >
> > It doesn't really matter if you grep the journal directly or save the
> > journal and grep the file. But it's one fewer command to grep the journal
> > directly. And if a failure is found, ask the user to save the journal and
> > report a bug.
> >
>
> I have the test case updated, but I ran into a problem with the test
> case template. The "|" character is apparently a reserved character in
> the template code. I couldn't get it to let me have the pipe from the
> journal to the grep. Nor could I use the -E option with the grep because
> that character is used to separate the match patterns.
>

Those are the joys of using wiki templates :-/
You can you "|" if you enclose it in command tag (instead of
using {{command|something}} template). There's also some way how to use it
inside the {{command}} template, but I forgot it as usual.
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[pmkel...@frontier.com]

On 12/5/19 14:07, pmkel...@frontier.com wrote:


On 12/5/19 03:56, Kamil Paral wrote:



Here's the email I had in mind, containing the important journal 
messages:
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org/message/S5LQTXKFPIUKBFC7HL5SNSAV3ZRFP6WV/ 



We need to create a command or commands that will detect any of the 
failure

states (and ideally another command for any of the success states, for
confirmation).


It doesn't really matter if you grep the journal directly or save the
journal and grep the file. But it's one fewer command to grep the journal
directly. And if a failure is found, ask the user to save the journal and
report a bug.



I have the test case updated, but I ran into a problem with the test 
case template. The "|" character is apparently a reserved character in 
the template code. I couldn't get it to let me have the pipe from the 
journal to the grep. Nor could I use the -E option with the grep because 
that character is used to separate the match patterns.


 Have a Great Day!

 Pat    (tablepc)



___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/

List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org 





Sorry!

Here's the link:

https://fedoraproject.org/wiki/User:Tablepc/Draft_testcase_reboot
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[pmkel...@frontier.com]

On 12/5/19 03:56, Kamil Paral wrote:



Here's the email I had in mind, containing the important journal messages:
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org/message/S5LQTXKFPIUKBFC7HL5SNSAV3ZRFP6WV/

We need to create a command or commands that will detect any of the failure
states (and ideally another command for any of the success states, for
confirmation).


It doesn't really matter if you grep the journal directly or save the
journal and grep the file. But it's one fewer command to grep the journal
directly. And if a failure is found, ask the user to save the journal and
report a bug.



I have the test case updated, but I ran into a problem with the test 
case template. The "|" character is apparently a reserved character in 
the template code. I couldn't get it to let me have the pipe from the 
journal to the grep. Nor could I use the -E option with the grep because 
that character is used to separate the match patterns.


Have a Great Day!

Pat (tablepc)



___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org


___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[Kamil Paral]

On Tue, Dec 3, 2019 at 3:29 PM pmkel...@frontier.com 
wrote:

> On 12/3/19 06:27, Kamil Paral wrote:
> > On Mon, Dec 2, 2019 at 7:43 PM pmkel...@frontier.com <
> pmkel...@frontier.com>
> > wrote:
> >
> >> I have the update ready. I think I interpolated the discussion
> >> correctly, but probably leaned toward the "keep it simple for now" case.
> >> Please let me know if there are further changes needed. Here's the link:
> >>
> >> https://fedoraproject.org/wiki/User:Tablepc/Draft_testcase_reboot
> >
> >
> > I was hoping you would include `journalctl | grep` commands that would
> look
> > for the strings Chris mentioned, so that it's easy for people or for
> > scripts to confirm whether the filesystem was or wasn't properly
> unmounted.
> > It's still helpful to include the examples (they don't necessarily need
> to
> > be Expected Results, but it's fine there), so that people can compare the
> > full text if they want or have doubts, but those grep commands would make
> > the comparison much simpler.
> >
>
> I took a quick tour back through some the the recent e'mail, but didn't
> see any thing about grepping the journal.


Here's the email I had in mind, containing the important journal messages:
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org/message/S5LQTXKFPIUKBFC7HL5SNSAV3ZRFP6WV/

We need to create a command or commands that will detect any of the failure
states (and ideally another command for any of the success states, for
confirmation).


> From the above, I'm thinking
> it might be good to do the journalctl -b > journal.log first and then
> grep the file for the unfortunate result phrases. Then if one of the
> phrases is found in the file ask the tester to file a bug report with
> the journal file attached. Is that what you had in mind?
>

It doesn't really matter if you grep the journal directly or save the
journal and grep the file. But it's one fewer command to grep the journal
directly. And if a failure is found, ask the user to save the journal and
report a bug.
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[pmkel...@frontier.com]

On 12/3/19 06:27, Kamil Paral wrote:

On Mon, Dec 2, 2019 at 7:43 PM pmkel...@frontier.com 
wrote:


I have the update ready. I think I interpolated the discussion
correctly, but probably leaned toward the "keep it simple for now" case.
Please let me know if there are further changes needed. Here's the link:

https://fedoraproject.org/wiki/User:Tablepc/Draft_testcase_reboot



I was hoping you would include `journalctl | grep` commands that would look
for the strings Chris mentioned, so that it's easy for people or for
scripts to confirm whether the filesystem was or wasn't properly unmounted.
It's still helpful to include the examples (they don't necessarily need to
be Expected Results, but it's fine there), so that people can compare the
full text if they want or have doubts, but those grep commands would make
the comparison much simpler.



I took a quick tour back through some the the recent e'mail, but didn't 
see any thing about grepping the journal. From the above, I'm thinking 
it might be good to do the journalctl -b > journal.log first and then 
grep the file for the unfortunate result phrases. Then if one of the 
phrases is found in the file ask the tester to file a bug report with 
the journal file attached. Is that what you had in mind?




I believe the poweroff cycle can be left out, and we can ask just for a
single reboot.



I can certainly do that.


Also, can you please fix the formatting in Expected Results?



Lucas thank you for the formatting. I tried to get a better result, but 
had no luck.


Have a Great Day!

Pat (tablepc)
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[Lukas Ruzicka]

I was hoping you would include `journalctl | grep` commands that would look
> for the strings Chris mentioned, so that it's easy for people or for
> scripts to confirm whether the filesystem was or wasn't properly unmounted.
> It's still helpful to include the examples (they don't necessarily need to
> be Expected Results, but it's fine there), so that people can compare the
> full text if they want or have doubts, but those grep commands would make
> the comparison much simpler.
>

+1


>
> When asking for debug logs, tell people to save the whole journal and not
> just systemd-fsck output, i.e. `journalctl -b > journal.log`.
>
> I believe the poweroff cycle can be left out, and we can ask just for a
> single reboot.
>

+1
I believe that if the system gets properly unmounted for reboot, it does
for a poweroff, too.



>
> Also, can you please fix the formatting in Expected Results?
>

I fixed the expected results formatting. I hope you like it.




>
> Thanks.
> ___
> test mailing list -- test@lists.fedoraproject.org
> To unsubscribe send an email to test-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
>


-- 

Lukáš Růžička

FEDORA QE, RHCE

Red Hat



Purkyňova 115

612 45 Brno - Královo Pole

lruzi...@redhat.com
TRIED AND PERSONALLY TESTED, ERGO TRUSTED. 
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[Kamil Paral]

On Mon, Dec 2, 2019 at 7:43 PM pmkel...@frontier.com 
wrote:

> I have the update ready. I think I interpolated the discussion
> correctly, but probably leaned toward the "keep it simple for now" case.
> Please let me know if there are further changes needed. Here's the link:
>
> https://fedoraproject.org/wiki/User:Tablepc/Draft_testcase_reboot


I was hoping you would include `journalctl | grep` commands that would look
for the strings Chris mentioned, so that it's easy for people or for
scripts to confirm whether the filesystem was or wasn't properly unmounted.
It's still helpful to include the examples (they don't necessarily need to
be Expected Results, but it's fine there), so that people can compare the
full text if they want or have doubts, but those grep commands would make
the comparison much simpler.

When asking for debug logs, tell people to save the whole journal and not
just systemd-fsck output, i.e. `journalctl -b > journal.log`.

I believe the poweroff cycle can be left out, and we can ask just for a
single reboot.

Also, can you please fix the formatting in Expected Results?

Thanks.
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[pmkel...@frontier.com]

The first send didn't seem to work; so I'm trying again.


On 12/2/19 13:43, pmkel...@frontier.com wrote:


On 12/2/19 06:01, Kamil Paral wrote:

On Fri, Nov 29, 2019 at 7:30 PM Chris Murphy 
wrote:


On Fri, Nov 29, 2019 at 5:10 AM Kamil Paral  wrote:


Sounds reasonable to test both LiveOS and the installed system. Does it
make sense to test both installed system's reboot and poweroff, 
though? Are

there any meaningful differences?

i'm not certain.



In that case I'd probably start with the simpler version, and extended it
if we detect it's not sufficient. So one reboot after installation -> 
fsck

check -> one reboot from the installed system -> fsck check.

Pat, can you update the test case according to the changes we've 
discussed

above? Thanks.



I have the update ready. I think I interpolated the discussion 
correctly, but probably leaned toward the "keep it simple for now" case. 
Please let me know if there are further changes needed. Here's the link:


https://fedoraproject.org/wiki/User:Tablepc/Draft_testcase_reboot

 Have a Great Day!

 Pat    (tablepc)


___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[pmkel...@frontier.com]

On 12/2/19 06:01, Kamil Paral wrote:

On Fri, Nov 29, 2019 at 7:30 PM Chris Murphy 
wrote:


On Fri, Nov 29, 2019 at 5:10 AM Kamil Paral  wrote:


Sounds reasonable to test both LiveOS and the installed system. Does it

make sense to test both installed system's reboot and poweroff, though? Are
there any meaningful differences?

i'm not certain.



In that case I'd probably start with the simpler version, and extended it
if we detect it's not sufficient. So one reboot after installation -> fsck
check -> one reboot from the installed system -> fsck check.

Pat, can you update the test case according to the changes we've discussed
above? Thanks.



I have the update ready. I think I interpolated the discussion 
correctly, but probably leaned toward the "keep it simple for now" case. 
Please let me know if there are further changes needed. Here's the link:


https://fedoraproject.org/wiki/User:Tablepc/Draft_testcase_reboot

Have a Great Day!

Pat (tablepc)
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[pmkel...@frontier.com]

Kamil,

Sure thing. I was planning to work on it today.

Have a Great Day!

Pat (tablepc)


On 12/2/19 06:01, Kamil Paral wrote:

On Fri, Nov 29, 2019 at 7:30 PM Chris Murphy 
wrote:


On Fri, Nov 29, 2019 at 5:10 AM Kamil Paral  wrote:


Sounds reasonable to test both LiveOS and the installed system. Does it

make sense to test both installed system's reboot and poweroff, though? Are
there any meaningful differences?

i'm not certain.



In that case I'd probably start with the simpler version, and extended it
if we detect it's not sufficient. So one reboot after installation -> fsck
check -> one reboot from the installed system -> fsck check.

Pat, can you update the test case according to the changes we've discussed
above? Thanks.


___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org


___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[Kamil Paral]

On Fri, Nov 29, 2019 at 7:30 PM Chris Murphy 
wrote:

> On Fri, Nov 29, 2019 at 5:10 AM Kamil Paral  wrote:
> >
> > Sounds reasonable to test both LiveOS and the installed system. Does it
> make sense to test both installed system's reboot and poweroff, though? Are
> there any meaningful differences?
>
> i'm not certain.
>

In that case I'd probably start with the simpler version, and extended it
if we detect it's not sufficient. So one reboot after installation -> fsck
check -> one reboot from the installed system -> fsck check.

Pat, can you update the test case according to the changes we've discussed
above? Thanks.
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[Chris Murphy]

On Fri, Nov 29, 2019 at 5:10 AM Kamil Paral  wrote:
>
> Sounds reasonable to test both LiveOS and the installed system. Does it make 
> sense to test both installed system's reboot and poweroff, though? Are there 
> any meaningful differences?

i'm not certain.

-- 
Chris Murphy
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[Kamil Paral]

On Thu, Nov 28, 2019 at 8:29 PM Chris Murphy 
wrote:

> FAT, ext4, and XFS all have a kind of "dirty bit" set upon mount. It's
> removed when cleanly unmounted. Therefore if the file system isn't
> mounted, but the "dirty bit" is set, it can be assumed it was not
> cleanly unmounted. Both kernel code and each file system's fsck can
> detect this, and the message you see depends on which discovers the
> problem first. The subsequent messages about how this problem is
> handled, I think we can ignore. As you say, it will be variable. All
> we care about is the indicator that it was not properly unmounted.
> Here are those indicators for each file system:
>
> FAT fsck (since /etc/fstab sets EFI system partition fs_passno to 2,
> this is what's displayed for default installations)
> Nov 28 12:04:21 localhost.localdomain systemd-fsck[681]: 0x41: Dirty
> bit is set. Fs was not properly unmounted and some data may be
> corrupt.
>
> FAT kernel
> [  205.317346] FAT-fs (vdb1): Volume was not properly unmounted. Some
> data may be corrupt. Please run fsck.
>
> ext4 fsck (since /etc/fstab sets /, /boot, /home fs_passno to 1 or 2,
> this is what's displayed for default installations)
> Nov 28 12:07:21 localhost.localdomain systemd-fsck[681]: /dev/vdb2:
> recovering journal
>
> ext4 kernel
> [  316.756778] EXT4-fs (vdb2): recovery complete
>
> XFS kernel (since /etc/fstab sets / fs_passno to 0, we should only see
> this message with default installations)
> [  372.027026] XFS (vdb3): Starting recovery (logdev: internal)
>
>
> If the test case is constrained only to default installations, the
> messages to test for:
> "0x41: Dirty bit is set"
> "recovering journal"
> "XFS" and "Starting recovery"
>
> If the test case is more broad, to account for non-default additional
> volumes that may not be set in fstab or may not have fs_passno set,
> also include:
> "EXT4-fs" and "recovery complete"
> "FAT-fs" and "Volume was not properly unmounted"
>
> In each case I'm choosing the first message that indicates previously
> unclean shutdown happened. Whether fsck or kernel message, they should
> be fairly consistent in that I'm not expecting them to change multiple
> times per year.


Thanks, this is pretty extensively covered. We can put these patterns into
one big `journalctl | grep` and detect the unmount issues of the previous
boot. With this, we can easily automate it.

Who's feeling like updating the test case?



> The gotcha is, how would we know? Failure to
> automatically parse for these messages, should they change, will
> indicate a clean shutdown. *shrug*
>

If that happens and a bug appears, I guess somebody will tell us sooner or
later and we'll fix it. Parsing text output can only take us so far.


>
> >> Steps 4-7: I'm not following the purpose of these steps. What I'd like
> >> to see for step 4, is, if we get a bad result (any result 2 messages),
> >> we need to collect the journal for the prior boot: `sudo journalctl
> >> -b-1 > journal.log` and attach to a bug report; or we could maybe
> >> parse for systemd messages suggesting it didn't get everything
> >> unmounted. But offhand I don't know what those messages would be, I'd
> >> have to go dig into systemd code to find them.
> >
> >
> > I think the purpose is to verify that both reboot and poweroff shut down
> the system correctly without any filesystem issues (which means fully
> committed journals and no dirty bits set).
>
> Gotcha. Yeah, I think it's reasonable to test the LiveOS reboot as
> well as the installed system's reboot, to make sure they are both
> properly unmounting file systems.
>

Sounds reasonable to test both LiveOS and the installed system. Does it
make sense to test both installed system's reboot and poweroff, though? Are
there any meaningful differences?
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[Chris Murphy]

On Thu, Nov 28, 2019 at 12:28 PM Chris Murphy  wrote:
>
> XFS kernel (since /etc/fstab sets / fs_passno to 0, we should only see
> this message with default installations)
> [  372.027026] XFS (vdb3): Starting recovery (logdev: internal)

Rats, the part in parenthesis is confusing. Rewrite: (we should only
see this message)

This message is what we'd see pretty much no matter what, including
the Fedora Server case which defaults to XFS. Even if fs_passno were 1
or 2 for some reason, fsck.xfs is a no op,therefore we'd still see the
above message when mount is attempted.

-- 
Chris Murphy
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[Chris Murphy]

On Thu, Nov 28, 2019 at 2:30 AM Kamil Paral  wrote:
>
> On Wed, Nov 27, 2019 at 9:17 PM Chris Murphy  wrote:
>>
>> A fair point about VM testing is whether the disk cache mode affects
>> the outcome. I use unsafe because it's faster and I embrace misery. I
>> think QA bots are now mostly using unsafe because it's faster too. So
>> depending on the situation it may be true that certain corruptions are
>> expected if unsafe is used, but I *think* unsafe is only unsafe in the
>> event the host crashes or experiences a power failure. I do forced
>> power offs of VMs all the time and never lose anything, in the case of
>> ext4 and XFS, journal replay always makes the file system consistent
>> again. And journal replay in that example is expected, not a bug.
>
>
> By "that example", do you mean the story you just described, or the "bad 
> result example" from the test case?

The story, which is too verbose and also confusing, so just ignore it. :-D

> Because in that test case example, if the machine was correctly powered 
> off/rebooted, there should be no reason to reply journal or see dirty bits.

That should be true, yes.


>
>>
>>
>> How to test, step 2 and 3:
>> This only applies to FAT and ext4. XFS and Btrfs have no fsck, both
>> depend on log replay if there was an unclean shutdown. Also, there are
>> error messages for common unclean shutdowns, and error messages for
>> uncommon problems. I think we only care about the former, correct?
>
>
> I believe so. Is there a tool that could tell us whether the currently 
> mounted drives were mounted cleanly, or some error correction had to be 
> performed? Because this is quickly getting into territory where we will need 
> to provide a large amount of examples and rely on people parsing the output, 
> comparing and (mis-)judging. The wording in journal output can change any 
> time as well. And I don't really like that.

FAT, ext4, and XFS all have a kind of "dirty bit" set upon mount. It's
removed when cleanly unmounted. Therefore if the file system isn't
mounted, but the "dirty bit" is set, it can be assumed it was not
cleanly unmounted. Both kernel code and each file system's fsck can
detect this, and the message you see depends on which discovers the
problem first. The subsequent messages about how this problem is
handled, I think we can ignore. As you say, it will be variable. All
we care about is the indicator that it was not properly unmounted.
Here are those indicators for each file system:

FAT fsck (since /etc/fstab sets EFI system partition fs_passno to 2,
this is what's displayed for default installations)
Nov 28 12:04:21 localhost.localdomain systemd-fsck[681]: 0x41: Dirty
bit is set. Fs was not properly unmounted and some data may be
corrupt.

FAT kernel
[  205.317346] FAT-fs (vdb1): Volume was not properly unmounted. Some
data may be corrupt. Please run fsck.

ext4 fsck (since /etc/fstab sets /, /boot, /home fs_passno to 1 or 2,
this is what's displayed for default installations)
Nov 28 12:07:21 localhost.localdomain systemd-fsck[681]: /dev/vdb2:
recovering journal

ext4 kernel
[  316.756778] EXT4-fs (vdb2): recovery complete

XFS kernel (since /etc/fstab sets / fs_passno to 0, we should only see
this message with default installations)
[  372.027026] XFS (vdb3): Starting recovery (logdev: internal)


If the test case is constrained only to default installations, the
messages to test for:
"0x41: Dirty bit is set"
"recovering journal"
"XFS" and "Starting recovery"

If the test case is more broad, to account for non-default additional
volumes that may not be set in fstab or may not have fs_passno set,
also include:
"EXT4-fs" and "recovery complete"
"FAT-fs" and "Volume was not properly unmounted"

In each case I'm choosing the first message that indicates previously
unclean shutdown happened. Whether fsck or kernel message, they should
be fairly consistent in that I'm not expecting them to change multiple
times per year. The gotcha is, how would we know? Failure to
automatically parse for these messages, should they change, will
indicate a clean shutdown. *shrug*

>> Steps 4-7: I'm not following the purpose of these steps. What I'd like
>> to see for step 4, is, if we get a bad result (any result 2 messages),
>> we need to collect the journal for the prior boot: `sudo journalctl
>> -b-1 > journal.log` and attach to a bug report; or we could maybe
>> parse for systemd messages suggesting it didn't get everything
>> unmounted. But offhand I don't know what those messages would be, I'd
>> have to go dig into systemd code to find them.
>
>
> I think the purpose is to verify that both reboot and poweroff shut down the 
> system correctly without any filesystem issues (which means fully committed 
> journals and no dirty bits set).

Gotcha. Yeah, I think it's reasonable to test the LiveOS reboot as
well as the installed system's reboot, to make sure they are both
properly unmounting file systems.


-- 
Chris Murphy

Re: For today's agenda item - Proposed disk unmount test case

[Kamil Paral]

On Wed, Nov 27, 2019 at 9:17 PM Chris Murphy 
wrote:

> On Wed, Nov 27, 2019 at 10:35 AM pmkel...@frontier.com
>  wrote:
>
> > https://fedoraproject.org/wiki/User:Tablepc/Draft_testcase_reboot
>
> Why does it need to happen on baremetal only? Any problem discovered
> by this test case is sure to need a deep dive no matter baremetal or
> VM.


I also tend to think that this test case doesn't need to be bare metal
exclusive.


> One thing all my Btrfs effort has taught me is how underestimated
> hardware and firmware bugs are in the storage stack (and even before
> Btrfs, the ZFS folks knew about this which is why it was invented).
>
> A fair point about VM testing is whether the disk cache mode affects
> the outcome. I use unsafe because it's faster and I embrace misery. I
> think QA bots are now mostly using unsafe because it's faster too. So
> depending on the situation it may be true that certain corruptions are
> expected if unsafe is used, but I *think* unsafe is only unsafe in the
> event the host crashes or experiences a power failure. I do forced
> power offs of VMs all the time and never lose anything, in the case of
> ext4 and XFS, journal replay always makes the file system consistent
> again. And journal replay in that example is expected, not a bug.
>

By "that example", do you mean the story you just described, or the "bad
result example" from the test case? Because in that test case example, if
the machine was correctly powered off/rebooted, there should be no reason
to reply journal or see dirty bits.


>
> How to test, step 2 and 3:
> This only applies to FAT and ext4. XFS and Btrfs have no fsck, both
> depend on log replay if there was an unclean shutdown. Also, there are
> error messages for common unclean shutdowns, and error messages for
> uncommon problems. I think we only care about the former, correct?
>

I believe so. Is there a tool that could tell us whether the currently
mounted drives were mounted cleanly, or some error correction had to be
performed? Because this is quickly getting into territory where we will
need to provide a large amount of examples and rely on people parsing the
output, comparing and (mis-)judging. The wording in journal output can
change any time as well. And I don't really like that.


>
> Steps 4-7: I'm not following the purpose of these steps. What I'd like
> to see for step 4, is, if we get a bad result (any result 2 messages),
> we need to collect the journal for the prior boot: `sudo journalctl
> -b-1 > journal.log` and attach to a bug report; or we could maybe
> parse for systemd messages suggesting it didn't get everything
> unmounted. But offhand I don't know what those messages would be, I'd
> have to go dig into systemd code to find them.
>

I think the purpose is to verify that both reboot and poweroff shut down
the system correctly without any filesystem issues (which means fully
committed journals and no dirty bits set).

If we can make all of this easy to test, then we can automate it, and it's
a worthwhile effort. If this is mostly guesswork, I wonder whether we
really need such a test case. Because yes, it is something that can go
wrong (as everything can), but it's not very likely (and if it does,
affected people will probably notice soon and file bugs). And we need to be
picky when adding test cases, because we can't test anything and
everything, we need to focus on those most important or problematic bits.
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[Chris Murphy]

[  106.886984] SGI XFS with ACLs, security attributes, scrub, no debug enabled
[  106.893115] XFS (vdb1): Mounting V5 Filesystem
*[  107.017619] XFS (vdb1): Starting recovery (logdev: internal)
*[  107.023136] XFS (vdb1): Ending recovery (logdev: internal)
[  107.026215] xfs filesystem being mounted at /mnt supports
timestamps until 2038 (0x7fff)
[root@localhost-live ~]#


The * marked lines are indication of log replay due to an unclean shutdown.

--
Chris Murphy
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[Chris Murphy]

On Wed, Nov 27, 2019 at 10:35 AM pmkel...@frontier.com
 wrote:

> https://fedoraproject.org/wiki/User:Tablepc/Draft_testcase_reboot

Why does it need to happen on baremetal only? Any problem discovered
by this test case is sure to need a deep dive no matter baremetal or
VM. One thing all my Btrfs effort has taught me is how underestimated
hardware and firmware bugs are in the storage stack (and even before
Btrfs, the ZFS folks knew about this which is why it was invented).

A fair point about VM testing is whether the disk cache mode affects
the outcome. I use unsafe because it's faster and I embrace misery. I
think QA bots are now mostly using unsafe because it's faster too. So
depending on the situation it may be true that certain corruptions are
expected if unsafe is used, but I *think* unsafe is only unsafe in the
event the host crashes or experiences a power failure. I do forced
power offs of VMs all the time and never lose anything, in the case of
ext4 and XFS, journal replay always makes the file system consistent
again. And journal replay in that example is expected, not a bug.

How to test, step 2 and 3:
This only applies to FAT and ext4. XFS and Btrfs have no fsck, both
depend on log replay if there was an unclean shutdown. Also, there are
error messages for common unclean shutdowns, and error messages for
uncommon problems. I think we only care about the former, correct?

Steps 4-7: I'm not following the purpose of these steps. What I'd like
to see for step 4, is, if we get a bad result (any result 2 messages),
we need to collect the journal for the prior boot: `sudo journalctl
-b-1 > journal.log` and attach to a bug report; or we could maybe
parse for systemd messages suggesting it didn't get everything
unmounted. But offhand I don't know what those messages would be, I'd
have to go dig into systemd code to find them.

-- 
Chris Murphy
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[pmkel...@frontier.com]

As promised during yesterday's meeting, I looked at the proposed test case:
https://fedoraproject.org/wiki/User:Tablepc/Draft_testcase_reboot

I have multiple comments:
- It is trying to check two things at the same time: 1. that
reboot/shutdown works as expected 2. that filesystems are properly
unmounted. I believe there should be two separate test cases for these. So
please split these into two.
- Formatting needs be fixed to make it look like our other test case pages.
Currently it looks like a copy-paste from email without effort to use wiki
formatting.
- Basics don't need to be explained, because the required knowledge level
for performing the test case guarantees that the tester knows e.g. how to
log in. So for example the first two points can be shortened to "Switch to
a free virtual console using Ctrl+Alt+F shortcut and log in".
- We don't need to mandate a particular disk layout in test case setup. It
is more useful for different testers to have different environments, so
that they have a higher chance to detect a bug.
- I don't like checking pre-shutdown messages using halt very much. The
first problem is that with plymouth installed (the default), you won't see
the messages, but a frozen plymouth screen (unless you're quite fast and
switch it to console messages before it freezes). The second problem is
that it relies too much on user intuition in how to distinguish a success
or a failure state. There is no example of either. Additionally, do we know
for sure that a system that can't unmount filesystems will halt eventually?
I'd expect it to hang forever. I'd rather leave all the error checking for
the subsequent boot (or in the case of a hanging boot, it's obviously
broken - but we won't need a test case for it, because you'll easily see it
with regular interaction with the system).
- When checking boot logs for fsck fixes, it's important to show an example
of not just a successful case, but also of a failed case. And I seem to be
lucky today [1].
- When testing system shutdown methods, I'd only use reboot and poweroff.
Halt is very niche and shutdown is old and replaced by poweroff.

Kamil

[1]
-- Logs begin at Tue 2019-08-27 09:26:40 CEST, end at Tue 2019-11-26
14:50:14 CET. --
Nov 25 10:25:20 phoenix systemd-fsck[684]: root: recovering journal
Nov 25 10:25:20 phoenix systemd-fsck[684]: root: Clearing orphaned inode
12325283 (uid=1000, gid=1000, mode=0100644, size=641092)
Nov 25 10:25:20 phoenix systemd-fsck[684]: root: Clearing orphaned inode
12331101 (uid=1000, gid=1000, mode=0100644, size=641092)
..
Nov 25 10:25:20 phoenix systemd-fsck[684]: root: clean, 1023215/26869760
files, 46957728/107451392 blocks
Nov 25 09:25:22 phoenix systemd-fsck[877]: boot: recovering journal
Nov 25 09:25:22 phoenix systemd-fsck[878]: fsck.fat 4.1 (2017-01-24)
Nov 25 09:25:22 phoenix systemd-fsck[878]: 0x25: Dirty bit is set. Fs was
not properly unmounted and some data may be corrupt.
Nov 25 09:25:22 phoenix systemd-fsck[878]:  Automatically removing dirty
bit.
Nov 25 09:25:22 phoenix systemd-fsck[878]: Performing changes.
Nov 25 09:25:22 phoenix systemd-fsck[878]: /dev/nvme0n1p1: 34 files,
6897/51145 clusters
Nov 25 09:25:22 phoenix systemd-fsck[877]: boot: clean, 103/65536 files,
67833/262144 blocks


___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org



The revised test case is ready for review. Please let me know if this 
needs further work. Here is the link:


https://fedoraproject.org/wiki/User:Tablepc/Draft_testcase_reboot


Have a Great Day!

Pat (tablepc)
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: For today's agenda item - Proposed disk unmount test case

[Kamil Paral]

On Mon, Nov 25, 2019 at 1:39 PM pmkel...@frontier.com 
wrote:

> Here is a link to the List discussion on the subject item:
>
>
> https://lists.fedoraproject.org/archives/search?q=drive+dismount+test+case=1=test%40lists.fedoraproject.org=date-asc
>
> I don't have a current example of this happening. It was originally
> posted to the list by Alan Jenkins while F31 was still Rawhide. I had
> seen the problem too; so after a few days and no replies, I picked it
> up. The problem disappeared shortly after F31 branched. I didn't file a
> bug report; though I would have if it had continued. It just seemed like
> something important and basic that should get some attention. Especially
> since it was once a test case and had been removed from the matrix.
>

As promised during yesterday's meeting, I looked at the proposed test case:
https://fedoraproject.org/wiki/User:Tablepc/Draft_testcase_reboot

I have multiple comments:
- It is trying to check two things at the same time: 1. that
reboot/shutdown works as expected 2. that filesystems are properly
unmounted. I believe there should be two separate test cases for these. So
please split these into two.
- Formatting needs be fixed to make it look like our other test case pages.
Currently it looks like a copy-paste from email without effort to use wiki
formatting.
- Basics don't need to be explained, because the required knowledge level
for performing the test case guarantees that the tester knows e.g. how to
log in. So for example the first two points can be shortened to "Switch to
a free virtual console using Ctrl+Alt+F shortcut and log in".
- We don't need to mandate a particular disk layout in test case setup. It
is more useful for different testers to have different environments, so
that they have a higher chance to detect a bug.
- I don't like checking pre-shutdown messages using halt very much. The
first problem is that with plymouth installed (the default), you won't see
the messages, but a frozen plymouth screen (unless you're quite fast and
switch it to console messages before it freezes). The second problem is
that it relies too much on user intuition in how to distinguish a success
or a failure state. There is no example of either. Additionally, do we know
for sure that a system that can't unmount filesystems will halt eventually?
I'd expect it to hang forever. I'd rather leave all the error checking for
the subsequent boot (or in the case of a hanging boot, it's obviously
broken - but we won't need a test case for it, because you'll easily see it
with regular interaction with the system).
- When checking boot logs for fsck fixes, it's important to show an example
of not just a successful case, but also of a failed case. And I seem to be
lucky today [1].
- When testing system shutdown methods, I'd only use reboot and poweroff.
Halt is very niche and shutdown is old and replaced by poweroff.

Kamil

[1]
-- Logs begin at Tue 2019-08-27 09:26:40 CEST, end at Tue 2019-11-26
14:50:14 CET. --
Nov 25 10:25:20 phoenix systemd-fsck[684]: root: recovering journal
Nov 25 10:25:20 phoenix systemd-fsck[684]: root: Clearing orphaned inode
12325283 (uid=1000, gid=1000, mode=0100644, size=641092)
Nov 25 10:25:20 phoenix systemd-fsck[684]: root: Clearing orphaned inode
12331101 (uid=1000, gid=1000, mode=0100644, size=641092)
...
Nov 25 10:25:20 phoenix systemd-fsck[684]: root: clean, 1023215/26869760
files, 46957728/107451392 blocks
Nov 25 09:25:22 phoenix systemd-fsck[877]: boot: recovering journal
Nov 25 09:25:22 phoenix systemd-fsck[878]: fsck.fat 4.1 (2017-01-24)
Nov 25 09:25:22 phoenix systemd-fsck[878]: 0x25: Dirty bit is set. Fs was
not properly unmounted and some data may be corrupt.
Nov 25 09:25:22 phoenix systemd-fsck[878]:  Automatically removing dirty
bit.
Nov 25 09:25:22 phoenix systemd-fsck[878]: Performing changes.
Nov 25 09:25:22 phoenix systemd-fsck[878]: /dev/nvme0n1p1: 34 files,
6897/51145 clusters
Nov 25 09:25:22 phoenix systemd-fsck[877]: boot: clean, 103/65536 files,
67833/262144 blocks
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org