I just got off the phone with Rackspace and the maintenance has finally been completed. I've re-enabled Jenkins, cleaned up some of the timer based jobs so we had a smaller queue and it's now working through verifies and merges :)
Everything is looking good. -Andy- On 11/26/2016 08:41 AM, Andrew Grimberg wrote: > So... they started earlier than expected as there was a miscommunication > on the time. As such, maintenance is now happening! I'll send out a note > when everything is finished. > > -Andy- > > On 11/22/2016 11:58 AM, Andrew Grimberg wrote: >> Greetings folks, >> >> This is just a reminder notice that this maintenance window is happening >> this coming Saturday and to also note that it has been expanded by 1.5 >> hours on the front end since there is a hardware issue that needs to be >> addressed before the OpenStack upgrade happens. >> >> Times have been updated in the subject and original body accordingly. >> >> -Andy- >> >> On 11/09/2016 11:07 AM, Andrew Grimberg wrote: >>> What: Rackspace will be performing an upgrade of OpenStack on the >>> private cloud that OpenDaylight uses for CI. An entire environment >>> shutdown will be needed due to the nature of the upgrade path >>> >>> When: November 26, 2016 (2016-11-26) from 10:30 - 18:00 PT (2016-11-26 >>> 18:00 - 2016-11-27 01:00 UTC) >>> >>> Why: We're still early enough in the Carbon cycle and there are >>> presently SRs that are scheduled right at that time that taking an >>> extended outage shouldn't be a problem. Since the private cloud is >>> presently running on the Kilo release of OpenStack it's time to start >>> getting it updated to newer versions. >>> >>> Rackspace will be performing system level updates as well as upgrading >>> OpenStack from Kilo to Liberty. An upgrade from Liberty to Mitaka will >>> be scheduled at a later time >>> >>> Impact: Since our private cloud environment is so small it is not >>> feasible to do a rolling cloud restart at this time. In particular, >>> since we have systems that use cinder storage and our current cinder >>> cluster is very small, we cannot do rolling upgrade / restarts on the >>> storage nodes which means that any instances that are using that storage >>> _must_ be turned off during the updates. Since the nodes that are the >>> most strongly impacted are Gerrit, Nexus and Jenkins we're taking the >>> advice of Rackspace and will be shutting down all CI services when >>> Rackspace gets to the phase of the updates requiring system reboots. >>> >>> What the community will see during the window are some instability in >>> getting to Gerrit as the hot updates are being applied. Eventually all >>> services will be brought offline as the hardware gets rebooted. Jenkins >>> will be placed into maintenance / shutdown mode near the start of the >>> window so that any running jobs will have an opportunity to finish but >>> new ones will not start. >>> >>> Notices will be sent to the mailing lists and to #opendaylight on >>> Freenode at the start and end of the maintenance. >>> >>> -Andy- >>> >> > -- Andrew J Grimberg Systems Administrator Release Engineering Team Lead The Linux Foundation
signature.asc
Description: OpenPGP digital signature
_______________________________________________ infrastructure mailing list infrastructure@lists.opendaylight.org https://lists.opendaylight.org/mailman/listinfo/infrastructure
