Re: Fw: Re: OUTAGE: Koji system 2019-01-11 -> 2019-01-14
Neal Gompa writes: > > * Buildbot (Python 3) > * Jenkins (Java) > * Vespene (Python 3) Just FYI: vespene has been recently discontinued: https://medium.com/@michaeldehaan/discontinuing-vespene-17fd9c0f3079 (also iirc it was licensed under Apache license + commons clause, which would make inclusion in Fedora problematic, but that's no longer of concern anyway) > * GoCD (Java + Ruby) > * Zuul (Python) > > Of the four listed above, only the first two are packaged in Fedora. > Buildbot is up-to-date in Fedora, but Jenkins lags behind considerably > and needs love. > signature.asc Description: PGP signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fw: Re: OUTAGE: Koji system 2019-01-11 -> 2019-01-14
Neal Gompa wrote: > Sorry, I was referring to systems like the OverDrive 1000[1], which > are PC form-factor ARM systems. > > I have one of these myself, and they're very handy for doing > reasonable development of software on ARM. > > [1]: https://softiron.com/development-tools/overdrive-1000/ At $599, this is NOT a low-cost device, and most definitely NOT in the "impulse buy range". And you can get comparable x86 machines (4 cores, 8 GiB RAM, 1 TB HDD) for ~$200 less (for less than $400). Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fw: Re: OUTAGE: Koji system 2019-01-11 -> 2019-01-14
On Mon, Jan 14, 2019 at 02:11:38PM +0100, Vít Ondruch wrote:
>
> Dne 14. 01. 19 v 13:59 Daniel P. Berrangé napsal(a):
> > On Sun, Jan 13, 2019 at 04:38:37PM -0800, Kevin Fenzi wrote:
> >> On 1/13/19 4:11 PM, Kevin Kofler wrote:
> >>> On Fri, 7 Dec 2018 at 14:51, Stephen John Smoogen wrote:
> During this time the s390 builders will not be available and all
> builds will be queued up until they are available and will not
> complete.
> >>> The good news is that the builders appear to be already back up now, a
> >>> day
> >>> before schedule.
> >> Yeah, the outage included extra time in case there were issues, but
> >> overall things went smoothly.
> > Unfortunately things don't actually appear to be operating smoothly this
> > morning. fedpkg is failing to start koji builds with messages such as:
> >
> > $ fedpkg build
> > Could not execute build: ('Connection aborted.', RemoteDisconnected('Remote
> > end closed connection without response'))
>
>
> At least this appears to be more due to:
>
> https://bugzilla.redhat.com/show_bug.cgi?id=1609298
>
> But might be related to the others as well. At least last week, I had
> issues with submitting update (although I am not sure if PDC was
> mentioned in that context)
Thanks for the pointer - i don't have the koji-1.16.1-3.fc29 update
mentioned in that BZ installed, so will try that.
Regards,
Daniel
--
|: https://berrange.com -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fw: Re: OUTAGE: Koji system 2019-01-11 -> 2019-01-14
Dne 14. 01. 19 v 13:59 Daniel P. Berrangé napsal(a):
> On Sun, Jan 13, 2019 at 04:38:37PM -0800, Kevin Fenzi wrote:
>> On 1/13/19 4:11 PM, Kevin Kofler wrote:
>>> On Fri, 7 Dec 2018 at 14:51, Stephen John Smoogen wrote:
During this time the s390 builders will not be available and all
builds will be queued up until they are available and will not
complete.
>>> The good news is that the builders appear to be already back up now, a day
>>> before schedule.
>> Yeah, the outage included extra time in case there were issues, but
>> overall things went smoothly.
> Unfortunately things don't actually appear to be operating smoothly this
> morning. fedpkg is failing to start koji builds with messages such as:
>
> $ fedpkg build
> Could not execute build: ('Connection aborted.', RemoteDisconnected('Remote
> end closed connection without response'))
At least this appears to be more due to:
https://bugzilla.redhat.com/show_bug.cgi?id=1609298
But might be related to the others as well. At least last week, I had
issues with submitting update (although I am not sure if PDC was
mentioned in that context)
Vít
>
> $ fedpkg build
> Could not execute build: The following error occurred while trying to get the
> active release branches in PDC: {"detail":"The database encountered an
> internal error or misconfiguration and was unable to complete your request."}
>
> it is non-deterministic though - the first build I did this morning
> started first time, but second time around I got these errors and
> had to retry 20+ times before it eventually succeeded in starting.
>
>
> Koji task monitoring then failed before the build completed:
>
> Could not execute build: ('Connection aborted.', RemoteDisconnected('Remote
> end closed connection without response'))
>
> though that has happened randomly & frequently for years now, so probably
> unrelated.
>
> fedpkg also failed to create an update:
>
> $ fedpkg update
> Could not execute update: Could not generate update request: Unable to
> create update. Bodhi failed to get a resource from PDC at the following URL
> "https://pdc.fedoraproject.org/rest_api/v1/component-branches/?active=true_path=true=global_component=f29_size=100=rpm_component=entangle;.
> The status code was "500".
> A copy of the filled in template is saved as bodhi.template.last
>
> I tried a few times, and gave up and used the web UI which worked.
>
> Regards,
> Daniel
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fw: Re: OUTAGE: Koji system 2019-01-11 -> 2019-01-14
On Sun, Jan 13, 2019 at 04:38:37PM -0800, Kevin Fenzi wrote:
> On 1/13/19 4:11 PM, Kevin Kofler wrote:
> > On Fri, 7 Dec 2018 at 14:51, Stephen John Smoogen wrote:
> >> During this time the s390 builders will not be available and all
> >> builds will be queued up until they are available and will not
> >> complete.
> >
> > The good news is that the builders appear to be already back up now, a day
> > before schedule.
>
> Yeah, the outage included extra time in case there were issues, but
> overall things went smoothly.
Unfortunately things don't actually appear to be operating smoothly this
morning. fedpkg is failing to start koji builds with messages such as:
$ fedpkg build
Could not execute build: ('Connection aborted.', RemoteDisconnected('Remote end
closed connection without response'))
$ fedpkg build
Could not execute build: The following error occurred while trying to get the
active release branches in PDC: {"detail":"The database encountered an internal
error or misconfiguration and was unable to complete your request."}
it is non-deterministic though - the first build I did this morning
started first time, but second time around I got these errors and
had to retry 20+ times before it eventually succeeded in starting.
Koji task monitoring then failed before the build completed:
Could not execute build: ('Connection aborted.', RemoteDisconnected('Remote
end closed connection without response'))
though that has happened randomly & frequently for years now, so probably
unrelated.
fedpkg also failed to create an update:
$ fedpkg update
Could not execute update: Could not generate update request: Unable to create
update. Bodhi failed to get a resource from PDC at the following URL
"https://pdc.fedoraproject.org/rest_api/v1/component-branches/?active=true_path=true=global_component=f29_size=100=rpm_component=entangle;.
The status code was "500".
A copy of the filled in template is saved as bodhi.template.last
I tried a few times, and gave up and used the web UI which worked.
Regards,
Daniel
--
|: https://berrange.com -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fw: Re: OUTAGE: Koji system 2019-01-11 -> 2019-01-14
It must produce the same results as real hw. Or mock parameter --forcearch must be removed. пн, 14 янв. 2019 г. в 12:02, Dan Horák : > > On Mon, 14 Jan 2019 10:48:59 +0300 > Vascom wrote: > > > How about switch build rpms for exotic and slow arches (arm, s390...) > > arm is slow, but not exotic > s390 is exotic, but not slow > > > from hardware to cross-compile on x86_64? > > mock can do it via qemu-user-static > > emulation has functional deficiencies, so it won't produce the same > results as real hw > > > Dan > > > > > пн, 14 янв. 2019 г. в 10:37, Dan Horák : > > > > > > On Sun, 13 Jan 2019 20:52:30 -0500 > > > Neal Gompa wrote: > > > > > > > On Sun, Jan 13, 2019 at 8:50 PM Neal Gompa > > > > wrote: > > > > > > > > > > * Buildbot (Python 3) > > > > > * Jenkins (Java) > > > > > * Vespene (Python 3) > > > > > * GoCD (Java + Ruby) > > > > > * Zuul (Python) > > > > > > > > > > Of the four listed above, only the first two are packaged in > > > > > Fedora. Buildbot is up-to-date in Fedora, but Jenkins lags > > > > > behind considerably and needs love. > > > > > > > > > > > > > Also, regrettably, I can't count. :) > > > > > > > > Yes, there's actually five options there. Zuul is actually a > > > > pretty bad option because it requires Ansible playbooks, which is > > > > overkill for 99% of projects. > > > > > > jenkins and buildbot are already being used by various upstream on > > > our s390x infra. They work fine and more upstream are welcome to > > > come to us. I was thinking rather about lack of things like Travis, > > > where the upstreams don't have to care about the CI infrastructure > > > (aka CI as a service). > > > > > > > > > Dan > > > ___ > > > devel mailing list -- devel@lists.fedoraproject.org > > > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > > > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > > > List Guidelines: > > > https://fedoraproject.org/wiki/Mailing_list_guidelines List > > > Archives: > > > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > > ___ > > devel mailing list -- devel@lists.fedoraproject.org > > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > > List Guidelines: > > https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: > > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fw: Re: OUTAGE: Koji system 2019-01-11 -> 2019-01-14
On Mon, 14 Jan 2019 10:48:59 +0300 Vascom wrote: > How about switch build rpms for exotic and slow arches (arm, s390...) arm is slow, but not exotic s390 is exotic, but not slow > from hardware to cross-compile on x86_64? > mock can do it via qemu-user-static emulation has functional deficiencies, so it won't produce the same results as real hw Dan > > пн, 14 янв. 2019 г. в 10:37, Dan Horák : > > > > On Sun, 13 Jan 2019 20:52:30 -0500 > > Neal Gompa wrote: > > > > > On Sun, Jan 13, 2019 at 8:50 PM Neal Gompa > > > wrote: > > > > > > > > * Buildbot (Python 3) > > > > * Jenkins (Java) > > > > * Vespene (Python 3) > > > > * GoCD (Java + Ruby) > > > > * Zuul (Python) > > > > > > > > Of the four listed above, only the first two are packaged in > > > > Fedora. Buildbot is up-to-date in Fedora, but Jenkins lags > > > > behind considerably and needs love. > > > > > > > > > > Also, regrettably, I can't count. :) > > > > > > Yes, there's actually five options there. Zuul is actually a > > > pretty bad option because it requires Ansible playbooks, which is > > > overkill for 99% of projects. > > > > jenkins and buildbot are already being used by various upstream on > > our s390x infra. They work fine and more upstream are welcome to > > come to us. I was thinking rather about lack of things like Travis, > > where the upstreams don't have to care about the CI infrastructure > > (aka CI as a service). > > > > > > Dan > > ___ > > devel mailing list -- devel@lists.fedoraproject.org > > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > > List Guidelines: > > https://fedoraproject.org/wiki/Mailing_list_guidelines List > > Archives: > > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: > https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fw: Re: OUTAGE: Koji system 2019-01-11 -> 2019-01-14
How about switch build rpms for exotic and slow arches (arm, s390...) from hardware to cross-compile on x86_64? mock can do it via qemu-user-static пн, 14 янв. 2019 г. в 10:37, Dan Horák : > > On Sun, 13 Jan 2019 20:52:30 -0500 > Neal Gompa wrote: > > > On Sun, Jan 13, 2019 at 8:50 PM Neal Gompa wrote: > > > > > > * Buildbot (Python 3) > > > * Jenkins (Java) > > > * Vespene (Python 3) > > > * GoCD (Java + Ruby) > > > * Zuul (Python) > > > > > > Of the four listed above, only the first two are packaged in Fedora. > > > Buildbot is up-to-date in Fedora, but Jenkins lags behind > > > considerably and needs love. > > > > > > > Also, regrettably, I can't count. :) > > > > Yes, there's actually five options there. Zuul is actually a pretty > > bad option because it requires Ansible playbooks, which is overkill > > for 99% of projects. > > jenkins and buildbot are already being used by various upstream on our > s390x infra. They work fine and more upstream are welcome to come to > us. I was thinking rather about lack of things like Travis, where the > upstreams don't have to care about the CI infrastructure (aka CI as a > service). > > > Dan > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fw: Re: OUTAGE: Koji system 2019-01-11 -> 2019-01-14
On Sun, 13 Jan 2019 20:52:30 -0500 Neal Gompa wrote: > On Sun, Jan 13, 2019 at 8:50 PM Neal Gompa wrote: > > > > * Buildbot (Python 3) > > * Jenkins (Java) > > * Vespene (Python 3) > > * GoCD (Java + Ruby) > > * Zuul (Python) > > > > Of the four listed above, only the first two are packaged in Fedora. > > Buildbot is up-to-date in Fedora, but Jenkins lags behind > > considerably and needs love. > > > > Also, regrettably, I can't count. :) > > Yes, there's actually five options there. Zuul is actually a pretty > bad option because it requires Ansible playbooks, which is overkill > for 99% of projects. jenkins and buildbot are already being used by various upstream on our s390x infra. They work fine and more upstream are welcome to come to us. I was thinking rather about lack of things like Travis, where the upstreams don't have to care about the CI infrastructure (aka CI as a service). Dan ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fw: Re: OUTAGE: Koji system 2019-01-11 -> 2019-01-14
> Well, those low-cost devices are typically: > * usable only for development/hacking/toying around, due to their form > factor (bare board), peripherals and/or (lack of) computing power, Certainly not, there are many ARM devices with processors that are well over the 2 GHz clock mark, have several gigabytes of RAM and lots of other goodies for under $100 USD. And they're certainly not just "bare boards", but full products. Just the boards are even less! > * extremely slow at everything including building packages (just lo> ok at > how slow the relatively fast Koji ARM builders are; low-cost devices are a > lot worse than that), They're not speed demons, but they're certainly fast. > * so bare that you have to spend a multiple of the purchase price of the > device to add needed development equipment (RAM, storage, I/O > peripherals, etc.), I don't know what you're referring to here, but if it's the ARM device I mentioned, see my first response.. > so I am not convinced that they are really useful targets for developers. Installing Fedora on an ARM device is a great first step into alternative architectures, and could even be a gateway to embedded system development. -- John M. Harris, Jr. Splentity https://splentity.com/ signature.asc Description: This is a digitally signed message part. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fw: Re: OUTAGE: Koji system 2019-01-11 -> 2019-01-14
On Sun, Jan 13, 2019 at 10:08 PM Kevin Kofler wrote: > > Neal Gompa wrote: > > Well, strictly speaking, this is really only a problem for IBM > > architectures (ppc64le and s390x) because there's no economical way > > for anyone to be able to care for them. For both ARM architectures we > > support, there are a number of low-cost (in the impulse buy range, > > even!) systems that people can acquire to do local development. For > > the upcoming RISC-V port, it's practically guaranteed that we're going > > to see similar low-cost hardware so that people can be exposed to the > > platform develop relatively soon after the Fedora RISC-V port is > > mainlined. > > Well, those low-cost devices are typically: > * usable only for development/hacking/toying around, due to their form > factor (bare board), peripherals and/or (lack of) computing power, > * extremely slow at everything including building packages (just look at how > slow the relatively fast Koji ARM builders are; low-cost devices are a lot > worse than that), > * so bare that you have to spend a multiple of the purchase price of the > device to add needed development equipment (RAM, storage, I/O peripherals, > etc.), > so I am not convinced that they are really useful targets for developers. > Sorry, I was referring to systems like the OverDrive 1000[1], which are PC form-factor ARM systems. I have one of these myself, and they're very handy for doing reasonable development of software on ARM. [1]: https://softiron.com/development-tools/overdrive-1000/ -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fw: Re: OUTAGE: Koji system 2019-01-11 -> 2019-01-14
Neal Gompa wrote: > Well, strictly speaking, this is really only a problem for IBM > architectures (ppc64le and s390x) because there's no economical way > for anyone to be able to care for them. For both ARM architectures we > support, there are a number of low-cost (in the impulse buy range, > even!) systems that people can acquire to do local development. For > the upcoming RISC-V port, it's practically guaranteed that we're going > to see similar low-cost hardware so that people can be exposed to the > platform develop relatively soon after the Fedora RISC-V port is > mainlined. Well, those low-cost devices are typically: * usable only for development/hacking/toying around, due to their form factor (bare board), peripherals and/or (lack of) computing power, * extremely slow at everything including building packages (just look at how slow the relatively fast Koji ARM builders are; low-cost devices are a lot worse than that), * so bare that you have to spend a multiple of the purchase price of the device to add needed development equipment (RAM, storage, I/O peripherals, etc.), so I am not convinced that they are really useful targets for developers. Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fw: Re: OUTAGE: Koji system 2019-01-11 -> 2019-01-14
On Sun, Jan 13, 2019 at 8:50 PM Neal Gompa wrote: > > * Buildbot (Python 3) > * Jenkins (Java) > * Vespene (Python 3) > * GoCD (Java + Ruby) > * Zuul (Python) > > Of the four listed above, only the first two are packaged in Fedora. > Buildbot is up-to-date in Fedora, but Jenkins lags behind considerably > and needs love. > Also, regrettably, I can't count. :) Yes, there's actually five options there. Zuul is actually a pretty bad option because it requires Ansible playbooks, which is overkill for 99% of projects. -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fw: Re: OUTAGE: Koji system 2019-01-11 -> 2019-01-14
On Sun, Jan 13, 2019 at 5:42 AM Dan Horák wrote: > > On Sun, 13 Jan 2019 11:05:33 +0100 > Miro Hrončok wrote: > > > On 12. 01. 19 19:47, Kevin Fenzi wrote: > > > On 1/12/19 5:59 AM, Miro Hrončok wrote: > > >> On 12. 01. 19 14:19, Kevin Kofler wrote: > > >>> Dan Horák wrote: > > This is a reminder that this begins this Friday and will > > probably be in place for 4 days. If there are increased > > downtimes, we will update the data when we know it. > > >>> > > >>> 4 whole DAYS of Koji outage are absolutely unacceptable! > > >>> > > >>> This just shows how bad an idea it was to add those exotic > > >>> secondary architectures to the primary Koji and to make them > > >>> blocking for all builds. > > >>> We really need to go back to building secondary architectures on > > >>> secondary > > >>> Koji instances, so that they do not hold the entire Fedora > > >>> hostage for days! > > >> > > >> While I don't necessarily agree with the tone or wording, Kevin > > >> has a point here. > > > > > > I don't agree with the tone, wording or point. :) > > > > Ah, it seems my e-mail was a little pointless without providing more > > thoughts. Sorry about that. > > > > > When there were secondary arch koji's it took a number of people > > > full time to nurse along koji shadow. Often things would land in > > > primary, secondary would find a bug later and there would need to > > > be a lot of coordination and back and forth before things were > > > fixed in both places. > > > > > > Now all those folks can concentrate on the bugs as they happen in > > > primary koji, fix them faster and everyone wins. > > > > I agree with this. That clearly reduces a lot of work that would > > otherwise be carried by a group of alternate arch fighters. Also, the > > situation is more easily to read etc. > > > > No doubt that having everything in one Koji is beneficial. > > > > > If there were constant outages you might have more of a point, but > > > this is the only multi-day one I can think of, it's over a weekend, > > > noarch packages can keep building and if there's a security or > > > urgent issue we can just Exclude s390x until it's back up. > > > > It's however not just about outages. What bothers me about s390x is: > > > > * From time to time there are situations where the build waits for > > a s390x builder. > > as they wait for other arch builders too > This is actually a really bad situation. If it weren't for the fact Koji tags all resources in and regenerates the internal repos with all the arches at once, I'd suggest we should consider making it possible for each arch to independently tag in and merge into the buildroot repos. There are a number of problems with this suggestion based on how Koji currently works today. It's regrettable that it was so much more work to maintain mostly unusable architectures in shadow Koji instances, because the direct exposure has mostly had a negative effect on the packager experience, since most packagers can't do anything with the architecture and have no knowledge for fixing it. > > * Upstreams don't have access to a s390x CI service and are often > > unable to debug s390x problems without (very slow) virtualization. > > we are aware of the problem, for all non-x86 arches, but there is no > simple solution > Well, strictly speaking, this is really only a problem for IBM architectures (ppc64le and s390x) because there's no economical way for anyone to be able to care for them. For both ARM architectures we support, there are a number of low-cost (in the impulse buy range, even!) systems that people can acquire to do local development. For the upcoming RISC-V port, it's practically guaranteed that we're going to see similar low-cost hardware so that people can be exposed to the platform develop relatively soon after the Fedora RISC-V port is mainlined. The "simple" solution is to develop some way for affordable access to the equipment so that it can be cared for by more people. That's probably a job for IBM and OpenPOWER, but it's something _somebody_ needs to figure out. > > * With ppc64 removal, s390x is the only Big Endian arch in the pool > > (while ppc64 was easier to get to via CentOS CI). > > > > I just think that we are burning a lot of resources without a clear > > visible benefit. Don't get me wrong, I am no architectures expert and > > I don't intent to pretend I am. It's just that all the s390x bugs > > I've seen for packages I happen to help taking care of are from other > > Fedora maintainers who fight FTBFS for their own packages etc., never > > from any actual users. > > > > s390x just feels to me like "that weird thing that often breaks and > > nobody really cares about". I realize this view is very narrow and is > > mostly based on feelings rather than facts, however it just how it > > feels. > > > > Hence I consider it unfortunate that s390x can block the whole > > distro, even if it's just for a couple days. > > is it really such a big problem,
Re: Fw: Re: OUTAGE: Koji system 2019-01-11 -> 2019-01-14
On 1/13/19 4:11 PM, Kevin Kofler wrote: > On Fri, 7 Dec 2018 at 14:51, Stephen John Smoogen wrote: >> During this time the s390 builders will not be available and all >> builds will be queued up until they are available and will not >> complete. > > The good news is that the builders appear to be already back up now, a day > before schedule. Yeah, the outage included extra time in case there were issues, but overall things went smoothly. >The bad news is that the sentence quoted above is not > completely true, since the queued up builds (all?) failed when the builders > got back up because they reported readiness before being really ready and > thus produced "Connection refused" tracebacks. Therefore, they must be > resubmitted. Yes, sorry about that... the host there that has a varnish cache for packages came up before the network came all the way back up, and varnish couldn't resolve/reach it's backend so it failed. :( I'll try and come up with a fix for this. kevin signature.asc Description: OpenPGP digital signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fw: Re: OUTAGE: Koji system 2019-01-11 -> 2019-01-14
On Fri, 7 Dec 2018 at 14:51, Stephen John Smoogen wrote: > During this time the s390 builders will not be available and all > builds will be queued up until they are available and will not > complete. The good news is that the builders appear to be already back up now, a day before schedule. The bad news is that the sentence quoted above is not completely true, since the queued up builds (all?) failed when the builders got back up because they reported readiness before being really ready and thus produced "Connection refused" tracebacks. Therefore, they must be resubmitted. Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fw: Re: OUTAGE: Koji system 2019-01-11 -> 2019-01-14
On Sun, 13 Jan 2019 at 11:09, Kevin Kofler wrote: > Kevin Fenzi wrote: > > When there were secondary arch koji's it took a number of people full > > time to nurse along koji shadow. Often things would land in primary, > > secondary would find a bug later and there would need to be a lot of > > coordination and back and forth before things were fixed in both places. > > > > Now all those folks can concentrate on the bugs as they happen in > > primary koji, fix them faster and everyone wins. > > Well, of course it is less work for the architecture maintainers, because > they offloaded almost all the work onto us package maintainers, who have > to > fix the build on architectures we don't give a darn about because they > block > our entire builds. And also wait for them to complete, though sadly the > main > offender there (armv7hl) has even been declared a primary architecture. > > We could all get our work done much faster if the primary Koji were only > x86_64 and everything else (including ARM) were built asynchronously and > taken care of by the handful people that actually cares about the > architecture. > > > If there were constant outages you might have more of a point, but this > > is the only multi-day one I can think of, > > One multi-day outage is one too many. > > Then your expectations are never going to be met by our Infrastructure and would be better off by designing and building your own. Fedora has had and will have multiple day outages. This week it was the s390. Next time it might be our storage or a cloud outage or a network problem or it might be we have to take everything down for a week in order to try and build something less likely to constantly disappoint you. However it is going to happen. Either get used to it or build the failproof alternative that is better. -- Stephen J Smoogen. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fw: Re: OUTAGE: Koji system 2019-01-11 -> 2019-01-14
Kevin Fenzi wrote: > When there were secondary arch koji's it took a number of people full > time to nurse along koji shadow. Often things would land in primary, > secondary would find a bug later and there would need to be a lot of > coordination and back and forth before things were fixed in both places. > > Now all those folks can concentrate on the bugs as they happen in > primary koji, fix them faster and everyone wins. Well, of course it is less work for the architecture maintainers, because they offloaded almost all the work onto us package maintainers, who have to fix the build on architectures we don't give a darn about because they block our entire builds. And also wait for them to complete, though sadly the main offender there (armv7hl) has even been declared a primary architecture. We could all get our work done much faster if the primary Koji were only x86_64 and everything else (including ARM) were built asynchronously and taken care of by the handful people that actually cares about the architecture. > If there were constant outages you might have more of a point, but this > is the only multi-day one I can think of, One multi-day outage is one too many. The outage is delaying the fix for showstopper bugs such as: https://bugzilla.redhat.com/show_bug.cgi?id=1664181 (a package that is essentially unusable, that we will be able to fix only on Tuesday, after the outage). > it's over a weekend, The weekend is the only time where most of us unpaid volunteers have time to take care of our packages. Fedora is not just Red Hat employees! In addition, the outage is scheduled to last practically the whole of Monday in Europe too. (The declared end of Monday 22:00 UTC is Monday 23:00 CET. And from EET eastwards, the outage lasts the whole of Monday too.) > noarch packages can keep building and if there's a security or urgent > issue we can just Exclude s390x until it's back up. But such bogus ExcludeArch is going to create significant extra work for the package maintainers and/or the architecture maintainers to undo later. So it is at most a solution for critical security issues. > In short, I think having the other arches in the one koji is a win for > everyone. And I have to respectfully disagree. It means more work for almost everyone (there are disproportionally more package maintainers than architecture maintainers), slower builds, and additional single points of failure (like the one single S/390 mainframe that is apparently the physical host for ALL the s390x "builders"). Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fw: Re: OUTAGE: Koji system 2019-01-11 -> 2019-01-14
On 13. 01. 19 11:41, Dan Horák wrote: On Sun, 13 Jan 2019 11:05:33 +0100 Miro Hrončok wrote: On 12. 01. 19 19:47, Kevin Fenzi wrote: On 1/12/19 5:59 AM, Miro Hrončok wrote: On 12. 01. 19 14:19, Kevin Kofler wrote: Dan Horák wrote: This is a reminder that this begins this Friday and will probably be in place for 4 days. If there are increased downtimes, we will update the data when we know it. 4 whole DAYS of Koji outage are absolutely unacceptable! This just shows how bad an idea it was to add those exotic secondary architectures to the primary Koji and to make them blocking for all builds. We really need to go back to building secondary architectures on secondary Koji instances, so that they do not hold the entire Fedora hostage for days! While I don't necessarily agree with the tone or wording, Kevin has a point here. I don't agree with the tone, wording or point. :) Ah, it seems my e-mail was a little pointless without providing more thoughts. Sorry about that. When there were secondary arch koji's it took a number of people full time to nurse along koji shadow. Often things would land in primary, secondary would find a bug later and there would need to be a lot of coordination and back and forth before things were fixed in both places. Now all those folks can concentrate on the bugs as they happen in primary koji, fix them faster and everyone wins. I agree with this. That clearly reduces a lot of work that would otherwise be carried by a group of alternate arch fighters. Also, the situation is more easily to read etc. No doubt that having everything in one Koji is beneficial. If there were constant outages you might have more of a point, but this is the only multi-day one I can think of, it's over a weekend, noarch packages can keep building and if there's a security or urgent issue we can just Exclude s390x until it's back up. It's however not just about outages. What bothers me about s390x is: * From time to time there are situations where the build waits for a s390x builder. as they wait for other arch builders too I guess what I wanted to say is that it's s390x more often than others. I mean, I usually wait for arm as it takes long time to finish, and s390x actually builds quite fast, however it is s390x that's often "blue" (no pun intended). * Upstreams don't have access to a s390x CI service and are often unable to debug s390x problems without (very slow) virtualization. we are aware of the problem, for all non-x86 arches, but there is no simple solution Good. I never suspected this would be easy. * With ppc64 removal, s390x is the only Big Endian arch in the pool (while ppc64 was easier to get to via CentOS CI). I just think that we are burning a lot of resources without a clear visible benefit. Don't get me wrong, I am no architectures expert and I don't intent to pretend I am. It's just that all the s390x bugs I've seen for packages I happen to help taking care of are from other Fedora maintainers who fight FTBFS for their own packages etc., never from any actual users. s390x just feels to me like "that weird thing that often breaks and nobody really cares about". I realize this view is very narrow and is mostly based on feelings rather than facts, however it just how it feels. Hence I consider it unfortunate that s390x can block the whole distro, even if it's just for a couple days. is it really such a big problem, during a weekend? It's not a big problem. I've never said it's big. I said I consider it unfortunate that it can block the distro (aka not this particular outage but rather the possibility). Chances are any arch can block the entire distro, however I guess getting a spare s309x mainframe if something happens would be a bit harder than getting builders for other arches. But this is empty talk. All I wanted to say that "I consider it unfortunate", **not** that "we must go and change it now" or something like that. I'm not saying all this to start a flame or to blame one poor architecture, I'd genuinely love to know the answers to: * Where can upstreams get a nice CI as a service for s390x? [1] nowhere yet, but how it is different from other non-x86 arches? Upstream can run their CI on our infrastructure, a number of them already does. BTW is there any open-source "CI as a service" software that could be just deployed (on any arch)? I don't know. Does Jenkins work on any arch? * What are the users of Linux on z and what packages/apps/tools are they interested about? Do they run desktop environments, should I e.g. bother fixing a bug in a desktop GUI application that controls a 3D printer? Or is it just OK to Exclude s390x on such apps? etc. they probably won't control 3D printers, but the overall goal is to make s390x looks as any other arch, thus we have for example virtio-gpu, so for a user there won't be a difference when running their desktop environment. In general having an heterogeneous
Re: Fw: Re: OUTAGE: Koji system 2019-01-11 -> 2019-01-14
On Sun, 13 Jan 2019 11:05:33 +0100 Miro Hrončok wrote: > On 12. 01. 19 19:47, Kevin Fenzi wrote: > > On 1/12/19 5:59 AM, Miro Hrončok wrote: > >> On 12. 01. 19 14:19, Kevin Kofler wrote: > >>> Dan Horák wrote: > This is a reminder that this begins this Friday and will > probably be in place for 4 days. If there are increased > downtimes, we will update the data when we know it. > >>> > >>> 4 whole DAYS of Koji outage are absolutely unacceptable! > >>> > >>> This just shows how bad an idea it was to add those exotic > >>> secondary architectures to the primary Koji and to make them > >>> blocking for all builds. > >>> We really need to go back to building secondary architectures on > >>> secondary > >>> Koji instances, so that they do not hold the entire Fedora > >>> hostage for days! > >> > >> While I don't necessarily agree with the tone or wording, Kevin > >> has a point here. > > > > I don't agree with the tone, wording or point. :) > > Ah, it seems my e-mail was a little pointless without providing more > thoughts. Sorry about that. > > > When there were secondary arch koji's it took a number of people > > full time to nurse along koji shadow. Often things would land in > > primary, secondary would find a bug later and there would need to > > be a lot of coordination and back and forth before things were > > fixed in both places. > > > > Now all those folks can concentrate on the bugs as they happen in > > primary koji, fix them faster and everyone wins. > > I agree with this. That clearly reduces a lot of work that would > otherwise be carried by a group of alternate arch fighters. Also, the > situation is more easily to read etc. > > No doubt that having everything in one Koji is beneficial. > > > If there were constant outages you might have more of a point, but > > this is the only multi-day one I can think of, it's over a weekend, > > noarch packages can keep building and if there's a security or > > urgent issue we can just Exclude s390x until it's back up. > > It's however not just about outages. What bothers me about s390x is: > > * From time to time there are situations where the build waits for > a s390x builder. as they wait for other arch builders too > * Upstreams don't have access to a s390x CI service and are often > unable to debug s390x problems without (very slow) virtualization. we are aware of the problem, for all non-x86 arches, but there is no simple solution > * With ppc64 removal, s390x is the only Big Endian arch in the pool > (while ppc64 was easier to get to via CentOS CI). > > I just think that we are burning a lot of resources without a clear > visible benefit. Don't get me wrong, I am no architectures expert and > I don't intent to pretend I am. It's just that all the s390x bugs > I've seen for packages I happen to help taking care of are from other > Fedora maintainers who fight FTBFS for their own packages etc., never > from any actual users. > > s390x just feels to me like "that weird thing that often breaks and > nobody really cares about". I realize this view is very narrow and is > mostly based on feelings rather than facts, however it just how it > feels. > > Hence I consider it unfortunate that s390x can block the whole > distro, even if it's just for a couple days. is it really such a big problem, during a weekend? > I'm not saying all this to start a flame or to blame one poor > architecture, I'd genuinely love to know the answers to: > > * Where can upstreams get a nice CI as a service for s390x? [1] nowhere yet, but how it is different from other non-x86 arches? Upstream can run their CI on our infrastructure, a number of them already does. BTW is there any open-source "CI as a service" software that could be just deployed (on any arch)? > * What are the users of Linux on z and what packages/apps/tools are > they interested about? Do they run desktop environments, should I > e.g. bother fixing a bug in a desktop GUI application that controls a > 3D printer? Or is it just OK to Exclude s390x on such apps? etc. they probably won't control 3D printers, but the overall goal is to make s390x looks as any other arch, thus we have for example virtio-gpu, so for a user there won't be a difference when running their desktop environment. In general having an heterogeneous environment is useful because it increases the overall quality pointing to buggy user code, toolchain bugs, etc. > * Where do we actually gain users of Fedora/s390x? When I search > for Linux s309x, I get to our Wiki (after several Ubuntu and IBM > links) [2], but that page is probably not very user targeted and > seems a bit outdated. The users are primarily interested in the enterprise products for s390x, but they are well aware that Fedora shows them what will appear in the next version of enterprise products. Or they simply need to run newer stuff than enterprise version provide. And yes, our wiki would appreciate more love :-)
Re: Fw: Re: OUTAGE: Koji system 2019-01-11 -> 2019-01-14
On 12. 01. 19 19:47, Kevin Fenzi wrote: On 1/12/19 5:59 AM, Miro Hrončok wrote: On 12. 01. 19 14:19, Kevin Kofler wrote: Dan Horák wrote: This is a reminder that this begins this Friday and will probably be in place for 4 days. If there are increased downtimes, we will update the data when we know it. 4 whole DAYS of Koji outage are absolutely unacceptable! This just shows how bad an idea it was to add those exotic secondary architectures to the primary Koji and to make them blocking for all builds. We really need to go back to building secondary architectures on secondary Koji instances, so that they do not hold the entire Fedora hostage for days! While I don't necessarily agree with the tone or wording, Kevin has a point here. I don't agree with the tone, wording or point. :) Ah, it seems my e-mail was a little pointless without providing more thoughts. Sorry about that. When there were secondary arch koji's it took a number of people full time to nurse along koji shadow. Often things would land in primary, secondary would find a bug later and there would need to be a lot of coordination and back and forth before things were fixed in both places. Now all those folks can concentrate on the bugs as they happen in primary koji, fix them faster and everyone wins. I agree with this. That clearly reduces a lot of work that would otherwise be carried by a group of alternate arch fighters. Also, the situation is more easily to read etc. No doubt that having everything in one Koji is beneficial. If there were constant outages you might have more of a point, but this is the only multi-day one I can think of, it's over a weekend, noarch packages can keep building and if there's a security or urgent issue we can just Exclude s390x until it's back up. It's however not just about outages. What bothers me about s390x is: * From time to time there are situations where the build waits for a s390x builder. * Upstreams don't have access to a s390x CI service and are often unable to debug s390x problems without (very slow) virtualization. * With ppc64 removal, s390x is the only Big Endian arch in the pool (while ppc64 was easier to get to via CentOS CI). I just think that we are burning a lot of resources without a clear visible benefit. Don't get me wrong, I am no architectures expert and I don't intent to pretend I am. It's just that all the s390x bugs I've seen for packages I happen to help taking care of are from other Fedora maintainers who fight FTBFS for their own packages etc., never from any actual users. s390x just feels to me like "that weird thing that often breaks and nobody really cares about". I realize this view is very narrow and is mostly based on feelings rather than facts, however it just how it feels. Hence I consider it unfortunate that s390x can block the whole distro, even if it's just for a couple days. I'm not saying all this to start a flame or to blame one poor architecture, I'd genuinely love to know the answers to: * Where can upstreams get a nice CI as a service for s390x? [1] * What are the users of Linux on z and what packages/apps/tools are they interested about? Do they run desktop environments, should I e.g. bother fixing a bug in a desktop GUI application that controls a 3D printer? Or is it just OK to Exclude s390x on such apps? etc. * Where do we actually gain users of Fedora/s390x? When I search for Linux s309x, I get to our Wiki (after several Ubuntu and IBM links) [2], but that page is probably not very user targeted and seems a bit outdated. [1] https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/5FH2DUYXQEOJQZPDA7KSHVBRANKXV3X2/ [2] https://fedoraproject.org/wiki/Architectures/s390x -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fw: Re: OUTAGE: Koji system 2019-01-11 -> 2019-01-14
On 1/12/19 5:59 AM, Miro Hrončok wrote: > On 12. 01. 19 14:19, Kevin Kofler wrote: >> Dan Horák wrote: >>> This is a reminder that this begins this Friday and will probably be in >>> place for 4 days. If there are increased downtimes, we will update the >>> data when we know it. >> >> 4 whole DAYS of Koji outage are absolutely unacceptable! >> >> This just shows how bad an idea it was to add those exotic secondary >> architectures to the primary Koji and to make them blocking for all >> builds. >> We really need to go back to building secondary architectures on >> secondary >> Koji instances, so that they do not hold the entire Fedora hostage for >> days! > > While I don't necessarily agree with the tone or wording, Kevin has a > point here. I don't agree with the tone, wording or point. :) When there were secondary arch koji's it took a number of people full time to nurse along koji shadow. Often things would land in primary, secondary would find a bug later and there would need to be a lot of coordination and back and forth before things were fixed in both places. Now all those folks can concentrate on the bugs as they happen in primary koji, fix them faster and everyone wins. If there were constant outages you might have more of a point, but this is the only multi-day one I can think of, it's over a weekend, noarch packages can keep building and if there's a security or urgent issue we can just Exclude s390x until it's back up. In short, I think having the other arches in the one koji is a win for everyone. kevin signature.asc Description: OpenPGP digital signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fw: Re: OUTAGE: Koji system 2019-01-11 -> 2019-01-14
On 12. 01. 19 14:19, Kevin Kofler wrote: Dan Horák wrote: This is a reminder that this begins this Friday and will probably be in place for 4 days. If there are increased downtimes, we will update the data when we know it. 4 whole DAYS of Koji outage are absolutely unacceptable! This just shows how bad an idea it was to add those exotic secondary architectures to the primary Koji and to make them blocking for all builds. We really need to go back to building secondary architectures on secondary Koji instances, so that they do not hold the entire Fedora hostage for days! While I don't necessarily agree with the tone or wording, Kevin has a point here. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Fw: Re: OUTAGE: Koji system 2019-01-11 -> 2019-01-14
Dan Horák wrote: > This is a reminder that this begins this Friday and will probably be in > place for 4 days. If there are increased downtimes, we will update the > data when we know it. 4 whole DAYS of Koji outage are absolutely unacceptable! This just shows how bad an idea it was to add those exotic secondary architectures to the primary Koji and to make them blocking for all builds. We really need to go back to building secondary architectures on secondary Koji instances, so that they do not hold the entire Fedora hostage for days! Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Fw: Re: OUTAGE: Koji system 2019-01-11 -> 2019-01-14
Date: Tue, 8 Jan 2019 15:03:57 -0500 From: Stephen John Smoogen To: Fedora Infrastructure , Development discussions related to Fedora , devel-annou...@lists.fedoraproject.org, annou...@lists.fedoraproject.org Subject: Re: OUTAGE: Koji system 2019-01-11 -> 2019-01-14 This is a reminder that this begins this Friday and will probably be in place for 4 days. If there are increased downtimes, we will update the data when we know it. On Fri, 7 Dec 2018 at 14:51, Stephen John Smoogen wrote: > Planned Outage - Koji Services - 2019-01-11 22:00 UTC > > There will be an koji outage starting at 2019-01-11 22:00 UTC, which > will last approximately 4 days. > > To convert UTC to your local time, take a look at > http://fedoraproject.org/wiki/Infrastructure/UTCHowto or run: > > date -d '2019-01-11 22:00UTC' > > Reason for outage: > > The facility that houses the Fedora s390 server will have a major > power outage starting 2019-01-11 22:00 UTC and ending 2019-01-14 22:00 > UTC. During this time the s390 builders will not be available and all > builds will be queued up until they are available and will not > complete. > > Affected Services: > > koji and related services which make composes and builds > > Ticket Link: > > https://pagure.io/fedora-infrastructure/issue/7429 > > Please join #fedora-admin or #fedora-noc on irc.freenode.net or add > comments to the ticket for this outage above. > > -- > Stephen J Smoogen. > -- Stephen J Smoogen. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
