RE: Verification testings and processes.

[Ryan Alecci]

Thank you for your reply and the information.

Respectfully,
Ryan

Sent from Mail for Windows 10

From: Adam Williamson
Sent: Wednesday, May 24, 2017 11:36 AM
To: For testing and quality assurance of Fedora releases
Subject: Re: Verification testings and processes.

On Mon, 2017-05-22 at 17:17 -0700, Ryan wrote:
> Hello everyone,

Hi Ryan!

> I gave a shot at running a verification test. I was looking at
> Fedora_26_Branched_20170513.n.0 installation Should I be testing the
> milestones labeled as Final in the nightly builds?

Yes, absolutely! If a test is labeled as Final, that means that a
failure of the test likely blocks the Final release. But that doesn't
mean we should only run the Alpha tests on Alpha candidates and only
the Alpha+Beta tests on Beta candidates; we should run as many of the
tests as possible on all candidates, because it's much better to find
and fix *all* the important bugs early. If we can find a Final blocker
now and fix it, that's so much the better than waiting till after Beta
comes out :)

> I hope I did this the right way. Here is what I did.
> Went to Fedora_26_Branched_20170513.n.0 installation, Image sanity,
> and chose the repoclosure test case. Followed the procedures and got
> the results below:
> Added myrepo repo from /media
> Reading in repository metadata - please wait
> Checking Dependencies
> Repos looked at: 1
>myrepo
> Num Packages in Repos: 2836
> package: python-smbios-2.3.0-4.fc26.x86_64 from myrepo
>   unresolved deps: 
>  python-ctypes
> Which shows one unresolved dependency. If I am correct this would
> qualify as a automatic blocker for the final release and I should
> submit a bug report and update the test results page to show failed?

That sounds exactly right, yep! Thanks very much for running this. A
similar check is actually run on each compose automatically by releng,
and that confirms your test:

https://kojipkgs.fedoraproject.org/compose/branched/Fedora-26-20170523.n.0/logs/x86_64/repoclosure-Server.x86_64.log

I keep meaning to set up a little thing to take the releng test result
and report it to the wiki, but haven't got around to it yet.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org

___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org

RE: Verification testings and processes.

[Ryan Alecci]

Thank you for your reply and the information.

Respectfully,
Ryan


Sent from Mail for Windows 10

From: Kamil Paral
Sent: Tuesday, May 23, 2017 1:47 AM
To: For testing and quality assurance of Fedora releases
Subject: Re: Verification testings and processes.

On Tue, May 23, 2017 at 2:17 AM, Ryan <rale...@gmail.com> wrote:
Hello everyone,
I gave a shot at running a verification test. I was looking at 
Fedora_26_Branched_20170513.n.0 installation 

That's correct. There's a new page since yesterday:
https://fedoraproject.org/wiki/Test_Results:Current_Installation_Test
Once we start getting Beta RCs (release candidates), which will be hopefully 
soon, that link will point to those.
 
Should I be testing the milestones labeled as Final in the nightly builds?

All of them are useful. For Beta release, those marked with Alpha and Beta have 
to pass. For Final release, Alpha + Beta + Final have to pass.
 
I hope I did this the right way. Here is what I did.
Went to Fedora_26_Branched_20170513.n.0 installation, Image sanity, and chose 
the repoclosure test case. Followed the procedures and got the results below:
Added myrepo repo from /media
Reading in repository metadata - please wait
Checking Dependencies
Repos looked at: 1
   myrepo
Num Packages in Repos: 2836
package: python-smbios-2.3.0-4.fc26.x86_64 from myrepo
  unresolved deps: 
 python-ctypes
Which shows one unresolved dependency. If I am correct this would qualify as a 
automatic blocker for the final release and I should submit a bug report and 
update the test results page to show failed?

Yes, please do that. It should be an automatic blocker, you're correct, but 
since you're new, it's better if you use 
https://qa.fedoraproject.org/blockerbugs/propose_bug to propose the blocker, 
and we'll check whether we can reproduce it and approve it if we can.
 
I apologize I am just trying to familiarize myself with the layout and how 
things are done. So this test labeled as final for this particular build should 
have this issue resolved before it gets integrated into the main Fedora 26 
build?

It means we can release F26 Alpha/Beta with this problem, but not F26 Final 
(the final image that gets then announced to our users).
 
I am just trying to understand this process. Once I am able to edit the 
appropriate pages and sections I will post my findings on this test and submit 
the bug report. If I did this wrong and missed a few steps could someone please 
point me in the right direction.
Thank you for your patience and help.
Ryan

Thanks a lot for your help!


___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org

Re: Verification testings and processes.

[Adam Williamson]

On Mon, 2017-05-22 at 17:17 -0700, Ryan wrote:
> Hello everyone,

Hi Ryan!

> I gave a shot at running a verification test. I was looking at
> Fedora_26_Branched_20170513.n.0 installation Should I be testing the
> milestones labeled as Final in the nightly builds?

Yes, absolutely! If a test is labeled as Final, that means that a
failure of the test likely blocks the Final release. But that doesn't
mean we should only run the Alpha tests on Alpha candidates and only
the Alpha+Beta tests on Beta candidates; we should run as many of the
tests as possible on all candidates, because it's much better to find
and fix *all* the important bugs early. If we can find a Final blocker
now and fix it, that's so much the better than waiting till after Beta
comes out :)

> I hope I did this the right way. Here is what I did.
> Went to Fedora_26_Branched_20170513.n.0 installation, Image sanity,
> and chose the repoclosure test case. Followed the procedures and got
> the results below:
> Added myrepo repo from /media
> Reading in repository metadata - please wait
> Checking Dependencies
> Repos looked at: 1
>myrepo
> Num Packages in Repos: 2836
> package: python-smbios-2.3.0-4.fc26.x86_64 from myrepo
>   unresolved deps: 
>  python-ctypes
> Which shows one unresolved dependency. If I am correct this would
> qualify as a automatic blocker for the final release and I should
> submit a bug report and update the test results page to show failed?

That sounds exactly right, yep! Thanks very much for running this. A
similar check is actually run on each compose automatically by releng,
and that confirms your test:

https://kojipkgs.fedoraproject.org/compose/branched/Fedora-26-20170523.n.0/logs/x86_64/repoclosure-Server.x86_64.log

I keep meaning to set up a little thing to take the releng test result
and report it to the wiki, but haven't got around to it yet.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org

Re: Verification testings and processes.

[Kamil Paral]

On Tue, May 23, 2017 at 2:17 AM, Ryan  wrote:

> Hello everyone,
>
> I gave a shot at running a verification test. I was looking at
> Fedora_26_Branched_20170513.n.0 installation
> 
>

That's correct. There's a new page since yesterday:
https://fedoraproject.org/wiki/Test_Results:Current_Installation_Test

Once we start getting Beta RCs (release candidates), which will be
hopefully soon, that link will point to those.


> Should I be testing the milestones labeled as Final in the nightly builds?
>

All of them are useful. For Beta release, those marked with Alpha and Beta
have to pass. For Final release, Alpha + Beta + Final have to pass.


> I hope I did this the right way. Here is what I did.
>
> Went to Fedora_26_Branched_20170513.n.0 installation
> ,
> Image sanity, and chose the repoclosure test case. Followed the procedures
> and got the results below:
>
> *Added myrepo repo from /media*
> *Reading in repository metadata - please wait*
> *Checking Dependencies*
> *Repos looked at: 1*
> *   myrepo*
> *Num Packages in Repos: 2836*
> *package: python-smbios-2.3.0-4.fc26.x86_64 from myrepo*
> *  unresolved deps: *
> * python-ctypes*
>
> Which shows one unresolved dependency. If I am correct this would qualify
> as a automatic blocker for the final release and I should submit a bug
> report and update the test results page to show failed?
>

Yes, please do that. It should be an automatic blocker, you're correct, but
since you're new, it's better if you use
https://qa.fedoraproject.org/blockerbugs/propose_bug to propose the
blocker, and we'll check whether we can reproduce it and approve it if we
can.


> I apologize I am just trying to familiarize myself with the layout and how
> things are done. So this test labeled as final for this particular build
> should have this issue resolved before it gets integrated into the main
> Fedora 26 build?
>

It means we can release F26 Alpha/Beta with this problem, but not F26 Final
(the final image that gets then announced to our users).


> I am just trying to understand this process. Once I am able to edit the
> appropriate pages and sections I will post my findings on this test and
> submit the bug report. If I did this wrong and missed a few steps could
> someone please point me in the right direction.
>
> Thank you for your patience and help.
>
> Ryan
>

Thanks a lot for your help!
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org

Verification testings and processes.

[Ryan]

  
  
Hello everyone,
I gave a shot at running a verification test. I was looking at Fedora_26_Branched_20170513.n.0
installation Should I be testing the milestones labeled as
  Final in the nightly builds?

I hope I did this the right way. Here is what I did.
Went to Fedora_26_Branched_20170513.n.0
installation, Image sanity, and chose the repoclosure test
  case. Followed the procedures and got the results below:
Added myrepo repo from /media
  Reading in repository metadata - please wait
  Checking Dependencies
  Repos looked at: 1
     myrepo
  Num Packages in Repos: 2836
  package: python-smbios-2.3.0-4.fc26.x86_64 from myrepo
    unresolved deps: 
   python-ctypes
Which shows one unresolved dependency. If I am correct this would
  qualify as a automatic blocker for the final release and I should
  submit a bug report and update the test results page to show
  failed?
I apologize I am just trying to familiarize myself with the
  layout and how things are done. So this test labeled as final for
  this particular build should have this issue resolved before it
  gets integrated into the main Fedora 26 build? I am just trying to
  understand this process. Once I am able to edit the appropriate
  pages and sections I will post my findings on this test and submit
  the bug report. If I did this wrong and missed a few steps could
  someone please point me in the right direction.

Thank you for your patience and help.
Ryan



  
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org