Re: Fedora 30 Mass Rebuild
Am Donnerstag, den 14.02.2019, 14:41 +0100 schrieb Fabio Valentini: > On Thu, Feb 14, 2019 at 2:29 PM Miro Hrončok > wrote: > > On 04. 02. 19 21:48, Mohan Boddu wrote: > > > Hi all, > > > > > > Per the Fedora 30 schedule[1] we started a mass rebuild for Fedora > > > 30 > > > on Jan 31st 2019. We did a mass rebuild for Fedora 30 for the > > > changes listed in: > > > > > > https://pagure.io/releng/issue/8086 > > > > > > Mass rebuild was done in a side tag (f30-rebuild) and are now > > > getting moved over > > > to f30. > > > > > > Failures can be seen > > > https://kojipkgs.fedoraproject.org/mass-rebuild/f30-failures.html > > Some packages disappeared from the list. > > > > Fore example: python2-django1.11 > > My 4 packages (agenda, elementary-photos, gitg, and > golang-github-oschwald-maxminddb-golang) vanished as well. > > Fabio That might have happened, because I've run a dumb loop over all FTBFS packages to have them build at least a second time. It actually fixed about 180 packages by just submitting them a second time. Sry, for br3aking stuff… :( Anyways, this list still apears to be correct: https://kojipkgs.fedoraproject.org/mass-rebuild/f30-need-rebuild.html Björn signature.asc Description: This is a digitally signed message part ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 30 Mass Rebuild
On Thu, Feb 14, 2019 at 2:29 PM Miro Hrončok wrote: > > On 04. 02. 19 21:48, Mohan Boddu wrote: > > Hi all, > > > > Per the Fedora 30 schedule[1] we started a mass rebuild for Fedora 30 > > on Jan 31st 2019. We did a mass rebuild for Fedora 30 for the changes > > listed in: > > > > https://pagure.io/releng/issue/8086 > > > > Mass rebuild was done in a side tag (f30-rebuild) and are now getting moved > > over > > to f30. > > > > Failures can be seen > > https://kojipkgs.fedoraproject.org/mass-rebuild/f30-failures.html > Some packages disappeared from the list. > > Fore example: python2-django1.11 My 4 packages (agenda, elementary-photos, gitg, and golang-github-oschwald-maxminddb-golang) vanished as well. Fabio > -- > Miro Hrončok > -- > Phone: +420777974800 > IRC: mhroncok > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 30 Mass Rebuild
On 04. 02. 19 21:48, Mohan Boddu wrote: Hi all, Per the Fedora 30 schedule[1] we started a mass rebuild for Fedora 30 on Jan 31st 2019. We did a mass rebuild for Fedora 30 for the changes listed in: https://pagure.io/releng/issue/8086 Mass rebuild was done in a side tag (f30-rebuild) and are now getting moved over to f30. Failures can be seen https://kojipkgs.fedoraproject.org/mass-rebuild/f30-failures.html Some packages disappeared from the list. Fore example: python2-django1.11 -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 30 Mass Rebuild
Anderson, Charles R wrote: > On Mon, Feb 04, 2019 at 03:48:59PM -0500, Mohan Boddu wrote: >> Per the Fedora 30 schedule[1] we started a mass rebuild for Fedora 30 >> on Jan 31st 2019. We did a mass rebuild for Fedora 30 for the changes >> listed in: >> >> https://pagure.io/releng/issue/8086 >> >> Mass rebuild was done in a side tag (f30-rebuild) and are now getting >> moved over to f30. >> >> Failures can be seen >> https://kojipkgs.fedoraproject.org/mass-rebuild/f30-failures.html > > How many build failures were caused by infrastructure issues? Small sample, but of the 20 I've looked at today, 17 were of this sort. -- Rex ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 30 Mass Rebuild
On Wednesday, 06 February 2019 at 20:37, Kevin Fenzi wrote: > On 2/6/19 9:07 AM, Vít Ondruch wrote: > > Dne 06. 02. 19 v 17:58 Kevin Fenzi napsal(a): [...] > >> Our autosigning machine seems to have a failed tpm in it, which has > >> caused signing to fail to work. ;( > >> > >> We are working on getting hardware fixed, or a replacement in place. > >> > >> More updates as I know more. ;( > >> > >> This is really a anoying time for hardware to die, but of course > >> sometimes it seems like it _knows_ when that is so it can fail. > > > > > > Thx for update. > > ok. We have a new motherboard coming for the failed server, but it will > not be in until tomorrow. > > So, we shuffled things around and got a machine to use for now, and it's > been configured and is signing the f30-pending backlog again. Many > thanks to Patrick for getting things up and running again. > > As soon as the f30-pending backlog is done, I'll let it go back to > normal signing and also run the fix script to make sure we have the > latest stuff in f30 tag. > > Sorry for the troubles here, hopefully we are back on track now. Thanks for all the updates, Kevin. They are much appreciated. Kudos to you, Patrick and all the others involved in fixing this. Regards, Dominik -- Fedora https://getfedora.org | RPM Fusion http://rpmfusion.org There should be a science of discontent. People need hard times and oppression to develop psychic muscles. -- from "Collected Sayings of Muad'Dib" by the Princess Irulan ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 30 Mass Rebuild
On 2/6/19 11:49 AM, Chris Murphy wrote: > On Wed, Feb 6, 2019 at 9:59 AM Kevin Fenzi wrote: >> >> On 2/6/19 7:23 AM, Vít Ondruch wrote: >>> >>> Dne 05. 02. 19 v 21:19 Kevin Fenzi napsal(a): On 2/5/19 3:50 AM, Peter Robinson wrote: >> There seems to be an issue with moving builds over. Everything seems to >> be stuck in f30-pending and not getting tagged with f30. > That'll be the signing queue Yeah, it wasn't processing these correctly I'm afraid. I did add the mass rebuild side tag to autosign, but it was a day or two after it started so likely these are the ones before that. The autosign machine also had some issues the last few days (it lost 3 disks in a row) and we did some reboots to replace things, etc. In any case, I have a loop running telling robosign to sign and move all those f30-pending ones. It's slowly processing them down. Sadly, I don't think it will be done tonight, but we will see. >>> >>> >>> It does not look better, does it? What is the status please? >> >> Our autosigning machine seems to have a failed tpm in it, which has >> caused signing to fail to work. ;( >> >> We are working on getting hardware fixed, or a replacement in place. >> >> More updates as I know more. ;( >> >> This is really a anoying time for hardware to die, but of course >> sometimes it seems like it _knows_ when that is so it can fail. > > Is it for sure dead, or is it possible it coincided with a kernel > update? I recently saw some messages of a TPM regression on the > linux-integrity@ list, but I'm not remembering the kernel versions > affected. The affected machine was running the rhel7 kernel, and we did boot into the kernel it's been using for a long time without any change. ;( kevin signature.asc Description: OpenPGP digital signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 30 Mass Rebuild
On Wed, Feb 6, 2019 at 9:59 AM Kevin Fenzi wrote: > > On 2/6/19 7:23 AM, Vít Ondruch wrote: > > > > Dne 05. 02. 19 v 21:19 Kevin Fenzi napsal(a): > >> On 2/5/19 3:50 AM, Peter Robinson wrote: > There seems to be an issue with moving builds over. Everything seems to > be stuck in f30-pending and not getting tagged with f30. > >>> That'll be the signing queue > >> Yeah, it wasn't processing these correctly I'm afraid. > >> > >> I did add the mass rebuild side tag to autosign, but it was a day or two > >> after it started so likely these are the ones before that. > >> > >> The autosign machine also had some issues the last few days (it lost 3 > >> disks in a row) and we did some reboots to replace things, etc. > >> > >> In any case, I have a loop running telling robosign to sign and move all > >> those f30-pending ones. It's slowly processing them down. Sadly, I don't > >> think it will be done tonight, but we will see. > > > > > > It does not look better, does it? What is the status please? > > Our autosigning machine seems to have a failed tpm in it, which has > caused signing to fail to work. ;( > > We are working on getting hardware fixed, or a replacement in place. > > More updates as I know more. ;( > > This is really a anoying time for hardware to die, but of course > sometimes it seems like it _knows_ when that is so it can fail. Is it for sure dead, or is it possible it coincided with a kernel update? I recently saw some messages of a TPM regression on the linux-integrity@ list, but I'm not remembering the kernel versions affected. -- Chris Murphy ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 30 Mass Rebuild
On 2/6/19 9:07 AM, Vít Ondruch wrote: > > Dne 06. 02. 19 v 17:58 Kevin Fenzi napsal(a): >> On 2/6/19 7:23 AM, Vít Ondruch wrote: >>> Dne 05. 02. 19 v 21:19 Kevin Fenzi napsal(a): On 2/5/19 3:50 AM, Peter Robinson wrote: >> There seems to be an issue with moving builds over. Everything seems to >> be stuck in f30-pending and not getting tagged with f30. > That'll be the signing queue Yeah, it wasn't processing these correctly I'm afraid. I did add the mass rebuild side tag to autosign, but it was a day or two after it started so likely these are the ones before that. The autosign machine also had some issues the last few days (it lost 3 disks in a row) and we did some reboots to replace things, etc. In any case, I have a loop running telling robosign to sign and move all those f30-pending ones. It's slowly processing them down. Sadly, I don't think it will be done tonight, but we will see. >>> >>> It does not look better, does it? What is the status please? >> Our autosigning machine seems to have a failed tpm in it, which has >> caused signing to fail to work. ;( >> >> We are working on getting hardware fixed, or a replacement in place. >> >> More updates as I know more. ;( >> >> This is really a anoying time for hardware to die, but of course >> sometimes it seems like it _knows_ when that is so it can fail. > > > Thx for update. ok. We have a new motherboard coming for the failed server, but it will not be in until tomorrow. So, we shuffled things around and got a machine to use for now, and it's been configured and is signing the f30-pending backlog again. Many thanks to Patrick for getting things up and running again. As soon as the f30-pending backlog is done, I'll let it go back to normal signing and also run the fix script to make sure we have the latest stuff in f30 tag. Sorry for the troubles here, hopefully we are back on track now. kevin signature.asc Description: OpenPGP digital signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 30 Mass Rebuild
On Wed, 2019-02-06 at 08:58 -0800, Kevin Fenzi wrote: > This is really a anoying time for hardware to die, but of course > sometimes it seems like it _knows_ when that is so it can fail. This is the beginning of the rise of the machines. signature.asc Description: This is a digitally signed message part ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 30 Mass Rebuild
On Wed, 6 Feb 2019 at 11:15, Vít Ondruch wrote: > > > Dne 05. 02. 19 v 21:19 Kevin Fenzi napsal(a): > > On 2/5/19 3:50 AM, Peter Robinson wrote: > > There seems to be an issue with moving builds over. Everything seems to be > stuck in f30-pending and not getting tagged with f30. > > That'll be the signing queue > > Yeah, it wasn't processing these correctly I'm afraid. > > I did add the mass rebuild side tag to autosign, but it was a day or two > after it started so likely these are the ones before that. > > The autosign machine also had some issues the last few days (it lost 3 > disks in a row) and we did some reboots to replace things, etc. > > In any case, I have a loop running telling robosign to sign and move all > those f30-pending ones. It's slowly processing them down. Sadly, I don't > think it will be done tonight, but we will see. > > > It does not look better, does it? What is the status please? > The autosign hardware has developed a fault and is being worked on. -- Stephen J Smoogen. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 30 Mass Rebuild
Dne 06. 02. 19 v 17:58 Kevin Fenzi napsal(a): > On 2/6/19 7:23 AM, Vít Ondruch wrote: >> Dne 05. 02. 19 v 21:19 Kevin Fenzi napsal(a): >>> On 2/5/19 3:50 AM, Peter Robinson wrote: > There seems to be an issue with moving builds over. Everything seems to > be stuck in f30-pending and not getting tagged with f30. That'll be the signing queue >>> Yeah, it wasn't processing these correctly I'm afraid. >>> >>> I did add the mass rebuild side tag to autosign, but it was a day or two >>> after it started so likely these are the ones before that. >>> >>> The autosign machine also had some issues the last few days (it lost 3 >>> disks in a row) and we did some reboots to replace things, etc. >>> >>> In any case, I have a loop running telling robosign to sign and move all >>> those f30-pending ones. It's slowly processing them down. Sadly, I don't >>> think it will be done tonight, but we will see. >> >> It does not look better, does it? What is the status please? > Our autosigning machine seems to have a failed tpm in it, which has > caused signing to fail to work. ;( > > We are working on getting hardware fixed, or a replacement in place. > > More updates as I know more. ;( > > This is really a anoying time for hardware to die, but of course > sometimes it seems like it _knows_ when that is so it can fail. Thx for update. V. signature.asc Description: OpenPGP digital signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 30 Mass Rebuild
On 2/6/19 7:23 AM, Vít Ondruch wrote: > > Dne 05. 02. 19 v 21:19 Kevin Fenzi napsal(a): >> On 2/5/19 3:50 AM, Peter Robinson wrote: There seems to be an issue with moving builds over. Everything seems to be stuck in f30-pending and not getting tagged with f30. >>> That'll be the signing queue >> Yeah, it wasn't processing these correctly I'm afraid. >> >> I did add the mass rebuild side tag to autosign, but it was a day or two >> after it started so likely these are the ones before that. >> >> The autosign machine also had some issues the last few days (it lost 3 >> disks in a row) and we did some reboots to replace things, etc. >> >> In any case, I have a loop running telling robosign to sign and move all >> those f30-pending ones. It's slowly processing them down. Sadly, I don't >> think it will be done tonight, but we will see. > > > It does not look better, does it? What is the status please? Our autosigning machine seems to have a failed tpm in it, which has caused signing to fail to work. ;( We are working on getting hardware fixed, or a replacement in place. More updates as I know more. ;( This is really a anoying time for hardware to die, but of course sometimes it seems like it _knows_ when that is so it can fail. kevin signature.asc Description: OpenPGP digital signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 30 Mass Rebuild
Dne 05. 02. 19 v 21:19 Kevin Fenzi napsal(a): > On 2/5/19 3:50 AM, Peter Robinson wrote: >>> There seems to be an issue with moving builds over. Everything seems to be >>> stuck in f30-pending and not getting tagged with f30. >> That'll be the signing queue > Yeah, it wasn't processing these correctly I'm afraid. > > I did add the mass rebuild side tag to autosign, but it was a day or two > after it started so likely these are the ones before that. > > The autosign machine also had some issues the last few days (it lost 3 > disks in a row) and we did some reboots to replace things, etc. > > In any case, I have a loop running telling robosign to sign and move all > those f30-pending ones. It's slowly processing them down. Sadly, I don't > think it will be done tonight, but we will see. It does not look better, does it? What is the status please? V. > >>> I'm also concerned that once the signing queue starts moving, it's going to >>> tag older packages over the new builds that have happened in the mean time. > Yes, thats a danger... we will have to retag those on a case by case > basis. The script releng uses to merge side tags won't merge a build if > there is a newer one, so those should be fine, it's just if the mass > rebuild build was added and then also someone did another rawhide build. > > If you notice any cases of that, file a releng ticket and we can get it > sorted out. > > kevin > > > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org signature.asc Description: OpenPGP digital signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 30 Mass Rebuild
On 2/6/19 02:27, Kevin Fenzi wrote: On 2/5/19 3:36 PM, Kalev Lember wrote: Is there any chance releng could figure out the list of packages affected once the signing/tagging is finished and fix all of them? I can already tell that it's affecting at least 40-50 GNOME builds done in that time frame. Yes, there's a releng script to fix this very case... can run after everything is finished signing/tagging. Great, thanks Kevin! -- Kalev ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 30 Mass Rebuild
On 2/5/19 3:36 PM, Kalev Lember wrote: > > On 2/5/19 21:19, Kevin Fenzi wrote: >> Yes, thats a danger... we will have to retag those on a case by case >> basis. The script releng uses to merge side tags won't merge a build if >> there is a newer one, so those should be fine, it's just if the mass >> rebuild build was added and then also someone did another rawhide build. >> >> If you notice any cases of that, file a releng ticket and we can get it >> sorted out. > > This sounds fairly bad: everything that got built today and yesterday is > going to get masked out by the older mass rebuild packages that are > being tagged "over" the newer builds as we speak. > > Looks like that it was only the mass rebuild packages that got stuck in > the signing queue and other builds were processed normally (they moved > from f30-pending to f30), so they are going to be out of order when the > mass rebuild signing/tagging into f30 finishes. > > Is there any chance releng could figure out the list of packages > affected once the signing/tagging is finished and fix all of them? I can > already tell that it's affecting at least 40-50 GNOME builds done in > that time frame. Yes, there's a releng script to fix this very case... can run after everything is finished signing/tagging. > Also, maybe next time around it would be better to tag mass rebuild > results directly into f31, skipping f31-pending (making sure everything > is signed before doing it)? It seems just painful to go through the > -pending signing queue which takes several days to process tens of > thousands of packages and can lead to out of order tagging. The reason we tag into pending is to make sure everything is signed, but yes, we should just make sure to try and autosign the mass rebuild as it builds. ;( kevin signature.asc Description: OpenPGP digital signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 30 Mass Rebuild
On 2/5/19 21:19, Kevin Fenzi wrote: Yes, thats a danger... we will have to retag those on a case by case basis. The script releng uses to merge side tags won't merge a build if there is a newer one, so those should be fine, it's just if the mass rebuild build was added and then also someone did another rawhide build. If you notice any cases of that, file a releng ticket and we can get it sorted out. This sounds fairly bad: everything that got built today and yesterday is going to get masked out by the older mass rebuild packages that are being tagged "over" the newer builds as we speak. Looks like that it was only the mass rebuild packages that got stuck in the signing queue and other builds were processed normally (they moved from f30-pending to f30), so they are going to be out of order when the mass rebuild signing/tagging into f30 finishes. Is there any chance releng could figure out the list of packages affected once the signing/tagging is finished and fix all of them? I can already tell that it's affecting at least 40-50 GNOME builds done in that time frame. Also, maybe next time around it would be better to tag mass rebuild results directly into f31, skipping f31-pending (making sure everything is signed before doing it)? It seems just painful to go through the -pending signing queue which takes several days to process tens of thousands of packages and can lead to out of order tagging. Kalev ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 30 Mass Rebuild
On Tue, 2019-02-05 at 12:06 +, Anderson, Charles R wrote: > On Mon, Feb 04, 2019 at 03:48:59PM -0500, Mohan Boddu wrote: > > Per the Fedora 30 schedule[1] we started a mass rebuild for Fedora > > 30 > > on Jan 31st 2019. We did a mass rebuild for Fedora 30 for the > > changes > > listed in: > > > > https://pagure.io/releng/issue/8086 > > > > Mass rebuild was done in a side tag (f30-rebuild) and are now > > getting moved > > over to f30. > > > > Failures can be seen > > https://kojipkgs.fedoraproject.org/mass-rebuild/f30-failures.html > > How many build failures were caused by infrastructure issues? > > https://kojipkgs.fedoraproject.org//work/tasks/5509/32465509/root.log An idea. Releng team could run a script to detect all builds that fails in root.log and resubmit it. I also have a couple of fails in root.log which aren't a real FTBFS . Cheers, > DEBUG util.py:492: zstd armv7hl 1.3.8-1.fc30 build 261 k > DEBUG util.py:492: Transaction Summary > DEBUG util.py:492: > = > === > DEBUG util.py:492: Install 145 Packages > DEBUG util.py:492: Total download size: 57 M > DEBUG util.py:492: Installed size: 218 M > DEBUG util.py:492: Downloading Packages: > DEBUG util.py:490: BUILDSTDERR: Error: Error downloading packages: > DEBUG util.py:490: BUILDSTDERR: Cannot download > toplink/packages/util-linux/2.33.1/2.fc30/armv7hl/util-linux-2.33.1- > 2.fc30.armv7hl.rpm: All mirrors were tried > DEBUG util.py:634: Child return code was: 1 > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: > https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org -- Sérgio M. B. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 30 Mass Rebuild
On 2/5/19 3:50 AM, Peter Robinson wrote: >> There seems to be an issue with moving builds over. Everything seems to be >> stuck in f30-pending and not getting tagged with f30. > > That'll be the signing queue Yeah, it wasn't processing these correctly I'm afraid. I did add the mass rebuild side tag to autosign, but it was a day or two after it started so likely these are the ones before that. The autosign machine also had some issues the last few days (it lost 3 disks in a row) and we did some reboots to replace things, etc. In any case, I have a loop running telling robosign to sign and move all those f30-pending ones. It's slowly processing them down. Sadly, I don't think it will be done tonight, but we will see. > >> I'm also concerned that once the signing queue starts moving, it's going to >> tag older packages over the new builds that have happened in the mean time. Yes, thats a danger... we will have to retag those on a case by case basis. The script releng uses to merge side tags won't merge a build if there is a newer one, so those should be fine, it's just if the mass rebuild build was added and then also someone did another rawhide build. If you notice any cases of that, file a releng ticket and we can get it sorted out. kevin signature.asc Description: OpenPGP digital signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 30 Mass Rebuild
On Mon, Feb 04, 2019 at 03:48:59PM -0500, Mohan Boddu wrote: > Per the Fedora 30 schedule[1] we started a mass rebuild for Fedora 30 > on Jan 31st 2019. We did a mass rebuild for Fedora 30 for the changes > listed in: > > https://pagure.io/releng/issue/8086 > > Mass rebuild was done in a side tag (f30-rebuild) and are now getting moved > over to f30. > > Failures can be seen > https://kojipkgs.fedoraproject.org/mass-rebuild/f30-failures.html How many build failures were caused by infrastructure issues? https://kojipkgs.fedoraproject.org//work/tasks/5509/32465509/root.log DEBUG util.py:492: zstd armv7hl 1.3.8-1.fc30 build 261 k DEBUG util.py:492: Transaction Summary DEBUG util.py:492: DEBUG util.py:492: Install 145 Packages DEBUG util.py:492: Total download size: 57 M DEBUG util.py:492: Installed size: 218 M DEBUG util.py:492: Downloading Packages: DEBUG util.py:490: BUILDSTDERR: Error: Error downloading packages: DEBUG util.py:490: BUILDSTDERR: Cannot download toplink/packages/util-linux/2.33.1/2.fc30/armv7hl/util-linux-2.33.1-2.fc30.armv7hl.rpm: All mirrors were tried DEBUG util.py:634: Child return code was: 1 ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 30 Mass Rebuild
> There seems to be an issue with moving builds over. Everything seems to be > stuck in f30-pending and not getting tagged with f30. That'll be the signing queue > I'm also concerned that once the signing queue starts moving, it's going to > tag older packages over the new builds that have happened in the mean time. > > Pete > > > 05.02.2019, 07:45, "Mohan Boddu" : > > Hi all, > > Per the Fedora 30 schedule[1] we started a mass rebuild for Fedora 30 > on Jan 31st 2019. We did a mass rebuild for Fedora 30 for the changes listed > in: > > https://pagure.io/releng/issue/8086 > > Mass rebuild was done in a side tag (f30-rebuild) and are now getting moved > over to f30. > > Failures can be seen > https://kojipkgs.fedoraproject.org/mass-rebuild/f30-failures.html > > Things still needing rebuilt > https://kojipkgs.fedoraproject.org/mass-rebuild/f30-need-rebuild.html > > 19097 builds have been tagged into f30, there s currently 1787 failed > builds that need to be addressed by the package maintainers. > > FTBFS bugs will be filed shortly. > > Please be sure to let releng know if you see any bugs in the > reporting. You can contact releng in #fedora-releng on freenode, by > dropping an email to our list[2] or filing an issue in pagure[3] > > Regards, > Mohan Boddu. > > [1] https://fedoraproject.org/wiki/Releases/30/Schedule > [2] > https://lists.fedoraproject.org/admin/lists/rel-eng.lists.fedoraproject.org/ > [3] https://pagure.io/releng/ > , > > ___ > devel-announce mailing list -- devel-annou...@lists.fedoraproject.org > To unsubscribe send an email to devel-announce-le...@lists.fedoraproject.org > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel-annou...@lists.fedoraproject.org > > , > > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 30 Mass Rebuild
No. If count as many times as many maintainers it has it will be about 3300 fails. вт, 5 февр. 2019 г. в 14:32, Vít Ondruch : > > > Dne 04. 02. 19 v 21:48 Mohan Boddu napsal(a): > > Hi all, > > Per the Fedora 30 schedule[1] we started a mass rebuild for Fedora 30 > on Jan 31st 2019. We did a mass rebuild for Fedora 30 for the changes listed > in: > > https://pagure.io/releng/issue/8086 > > Mass rebuild was done in a side tag (f30-rebuild) and are now getting moved > over to f30. > > Failures can be seen > https://kojipkgs.fedoraproject.org/mass-rebuild/f30-failures.html > > Things still needing rebuilt > https://kojipkgs.fedoraproject.org/mass-rebuild/f30-need-rebuild.html > > 19097 builds have been tagged into f30, there s currently 1787 failed > > > I might be wrong but think the number of failed packages is incorrect. It > seems that every package is counted as many times as many maintainers it has. > > > Vít > > > builds that need to be addressed by the package maintainers. > > FTBFS bugs will be filed shortly. > > Please be sure to let releng know if you see any bugs in the > reporting. You can contact releng in #fedora-releng on freenode, by > dropping an email to our list[2] or filing an issue in pagure[3] > > Regards, > Mohan Boddu. > > [1] https://fedoraproject.org/wiki/Releases/30/Schedule > [2] > https://lists.fedoraproject.org/admin/lists/rel-eng.lists.fedoraproject.org/ > [3] https://pagure.io/releng/ > > ___ > devel-announce mailing list -- devel-annou...@lists.fedoraproject.org > To unsubscribe send an email to devel-announce-le...@lists.fedoraproject.org > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel-annou...@lists.fedoraproject.org > > > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 30 Mass Rebuild
Dne 04. 02. 19 v 21:48 Mohan Boddu napsal(a): > Hi all, > > Per the Fedora 30 schedule[1] we started a mass rebuild for Fedora 30 > on Jan 31st 2019. We did a mass rebuild for Fedora 30 for the changes > listed in: > > https://pagure.io/releng/issue/8086 > > Mass rebuild was done in a side tag (f30-rebuild) and are now getting > moved over to f30. > > Failures can be seen > https://kojipkgs.fedoraproject.org/mass-rebuild/f30-failures.html > > Things still needing rebuilt > https://kojipkgs.fedoraproject.org/mass-rebuild/f30-need-rebuild.html > > 19097 builds have been tagged into f30, there s currently 1787 failed I might be wrong but think the number of failed packages is incorrect. It seems that every package is counted as many times as many maintainers it has. Vít > builds that need to be addressed by the package maintainers. > > FTBFS bugs will be filed shortly. > > Please be sure to let releng know if you see any bugs in the > reporting. You can contact releng in #fedora-releng on freenode, by > dropping an email to our list[2] or filing an issue in pagure[3] > > Regards, > Mohan Boddu. > > [1] https://fedoraproject.org/wiki/Releases/30/Schedule > [2] > https://lists.fedoraproject.org/admin/lists/rel-eng.lists.fedoraproject.org/ > [3] https://pagure.io/releng/ > > ___ > devel-announce mailing list -- devel-annou...@lists.fedoraproject.org > To unsubscribe send an email to devel-announce-le...@lists.fedoraproject.org > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel-annou...@lists.fedoraproject.org > > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 30 Mass Rebuild
There seems to be an issue with moving builds over. Everything seems to be stuck in f30-pending and not getting tagged with f30. I'm also concerned that once the signing queue starts moving, it's going to tag older packages over the new builds that have happened in the mean time. Pete 05.02.2019, 07:45, "Mohan Boddu" :Hi all, Per the Fedora 30 schedule[1] we started a mass rebuild for Fedora 30on Jan 31st 2019. We did a mass rebuild for Fedora 30 for the changes listed in: https://pagure.io/releng/issue/8086 Mass rebuild was done in a side tag (f30-rebuild) and are now getting moved over to f30. Failures can be seenhttps://kojipkgs.fedoraproject.org/mass-rebuild/f30-failures.html Things still needing rebuilthttps://kojipkgs.fedoraproject.org/mass-rebuild/f30-need-rebuild.html 19097 builds have been tagged into f30, there s currently 1787 failedbuilds that need to be addressed by the package maintainers. FTBFS bugs will be filed shortly. Please be sure to let releng know if you see any bugs in thereporting. You can contact releng in #fedora-releng on freenode, bydropping an email to our list[2] or filing an issue in pagure[3] Regards,Mohan Boddu. [1] https://fedoraproject.org/wiki/Releases/30/Schedule[2] https://lists.fedoraproject.org/admin/lists/rel-eng.lists.fedoraproject.org/[3] https://pagure.io/releng/,___devel-announce mailing list -- devel-annou...@lists.fedoraproject.orgTo unsubscribe send an email to devel-announce-le...@lists.fedoraproject.orgFedora Code of Conduct: https://getfedora.org/code-of-conduct.htmlList Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelinesList Archives: https://lists.fedoraproject.org/archives/list/devel-annou...@lists.fedoraproject.org,___devel mailing list -- devel@lists.fedoraproject.orgTo unsubscribe send an email to devel-le...@lists.fedoraproject.orgFedora Code of Conduct: https://getfedora.org/code-of-conduct.htmlList Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelinesList Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Fedora 30 Mass Rebuild
Hi all, Per the Fedora 30 schedule[1] we started a mass rebuild for Fedora 30 on Jan 31st 2019. We did a mass rebuild for Fedora 30 for the changes listed in: https://pagure.io/releng/issue/8086 Mass rebuild was done in a side tag (f30-rebuild) and are now getting moved over to f30. Failures can be seen https://kojipkgs.fedoraproject.org/mass-rebuild/f30-failures.html Things still needing rebuilt https://kojipkgs.fedoraproject.org/mass-rebuild/f30-need-rebuild.html 19097 builds have been tagged into f30, there s currently 1787 failed builds that need to be addressed by the package maintainers. FTBFS bugs will be filed shortly. Please be sure to let releng know if you see any bugs in the reporting. You can contact releng in #fedora-releng on freenode, by dropping an email to our list[2] or filing an issue in pagure[3] Regards, Mohan Boddu. [1] https://fedoraproject.org/wiki/Releases/30/Schedule [2] https://lists.fedoraproject.org/admin/lists/rel-eng.lists.fedoraproject.org/ [3] https://pagure.io/releng/ ___ devel-announce mailing list -- devel-annou...@lists.fedoraproject.org To unsubscribe send an email to devel-announce-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel-annou...@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 30 Mass Rebuild
Hi all, Fedora 30 Mass Rebuild will start in few minutes. We are preparing for the final steps and will run mass rebuild once everything is in place. On Wed, Jan 30, 2019 at 4:02 AM Mohan Boddu wrote: > Hi all, > > Per the Fedora 30 schedule[1] we are supposed to start the mass rebuild > on Jan 30th 2019, but due to known bug with gcc it got pushed by a day and > will start on Jan 31st 2019[2]. We are doing a mass rebuild for Fedora 30 for > all the changes listed in > > https://pagure.io/releng/issue/8086 > > This is a heads up that it will be done in a side tag and moved over > when completed. We will be running scripts to output failure stats. > please be sure to let releng know if you see any bugs in the reporting. > > You can contact releng in #fedora-releng on freenode. > > Failures can be seen > > https://kojipkgs.fedoraproject.org/mass-rebuild/f30-failures.html > > Things still needing rebuilt > > https://kojipkgs.fedoraproject.org/mass-rebuild/f30-need-rebuild.html > > [1] https://fedoraproject.org/wiki/Releases/30/Schedule > [2] https://pagure.io/fesco/issue/2066 > > Regards > ___ devel-announce mailing list -- devel-annou...@lists.fedoraproject.org To unsubscribe send an email to devel-announce-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel-annou...@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fedora 30 Mass Rebuild
Hi all, Fedora 30 Mass Rebuild will start in few minutes. We are preparing for the final steps and will run mass rebuild once everything is in place. On Wed, Jan 30, 2019 at 4:02 AM Mohan Boddu wrote: > Hi all, > > Per the Fedora 30 schedule[1] we are supposed to start the mass rebuild > on Jan 30th 2019, but due to known bug with gcc it got pushed by a day and > will start on Jan 31st 2019[2]. We are doing a mass rebuild for Fedora 30 for > all the changes listed in > > https://pagure.io/releng/issue/8086 > > This is a heads up that it will be done in a side tag and moved over > when completed. We will be running scripts to output failure stats. > please be sure to let releng know if you see any bugs in the reporting. > > You can contact releng in #fedora-releng on freenode. > > Failures can be seen > > https://kojipkgs.fedoraproject.org/mass-rebuild/f30-failures.html > > Things still needing rebuilt > > https://kojipkgs.fedoraproject.org/mass-rebuild/f30-need-rebuild.html > > [1] https://fedoraproject.org/wiki/Releases/30/Schedule > [2] https://pagure.io/fesco/issue/2066 > > Regards > ___ devel-announce mailing list -- devel-announce@lists.fedoraproject.org To unsubscribe send an email to devel-announce-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel-announce@lists.fedoraproject.org
Fedora 30 Mass Rebuild
Hi all, Per the Fedora 30 schedule[1] we are supposed to start the mass rebuild on Jan 30th 2019, but due to known bug with gcc it got pushed by a day and will start on Jan 31st 2019[2]. We are doing a mass rebuild for Fedora 30 for all the changes listed in https://pagure.io/releng/issue/8086 This is a heads up that it will be done in a side tag and moved over when completed. We will be running scripts to output failure stats. please be sure to let releng know if you see any bugs in the reporting. You can contact releng in #fedora-releng on freenode. Failures can be seen https://kojipkgs.fedoraproject.org/mass-rebuild/f30-failures.html Things still needing rebuilt https://kojipkgs.fedoraproject.org/mass-rebuild/f30-need-rebuild.html [1] https://fedoraproject.org/wiki/Releases/30/Schedule [2] https://pagure.io/fesco/issue/2066 Regards ___ devel-announce mailing list -- devel-announce@lists.fedoraproject.org To unsubscribe send an email to devel-announce-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel-announce@lists.fedoraproject.org
Fedora 30 Mass Rebuild
Hi all, Per the Fedora 30 schedule[1] we are supposed to start the mass rebuild on Jan 30th 2019, but due to known bug with gcc it got pushed by a day and will start on Jan 31st 2019[2]. We are doing a mass rebuild for Fedora 30 for all the changes listed in https://pagure.io/releng/issue/8086 This is a heads up that it will be done in a side tag and moved over when completed. We will be running scripts to output failure stats. please be sure to let releng know if you see any bugs in the reporting. You can contact releng in #fedora-releng on freenode. Failures can be seen https://kojipkgs.fedoraproject.org/mass-rebuild/f30-failures.html Things still needing rebuilt https://kojipkgs.fedoraproject.org/mass-rebuild/f30-need-rebuild.html [1] https://fedoraproject.org/wiki/Releases/30/Schedule [2] https://pagure.io/fesco/issue/2066 Regards ___ devel-announce mailing list -- devel-annou...@lists.fedoraproject.org To unsubscribe send an email to devel-announce-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel-annou...@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
