Hi Everyone, Rackspace Public cloud appears to be routing again since around 11:00 AM EST and we're able to connect to our public cloud nodes. Unfortunately Jenkins master was stuck this morning due to the large queue size caused by CSIT jobs being unable to run so we had to give it a kick to reset everything and aborted a few jobs. Unfortunately the queue was lost so any jobs that did not run will need to be requeued but so far all the csit jobs that started appear to be successfully running the jobs.
I will ask Rackspace for a RCA. We did notice an update from them this morning about an hour before things started working [1] regarding network related issues in their DFW cloud space which we're using for our public cloud. Regards, Thanh [1] https://status.rackspace.com/index/viewincidents?group= 19&start=1470315969&guid=3185 On Wed, Aug 3, 2016 at 10:53 PM, Thanh Ha <[email protected]> wrote: > Just an update, we reached out to Rackspace about our continual routing > issue to the public cloud as we can still not route to the public cloud > servers that Jenkins is spinning up. They are looking into it and we will > post an update as we hear back from Rackspace. > > Thanh > > > On Wed, Aug 3, 2016 at 5:35 PM, Thanh Ha <[email protected]> > wrote: > >> Hi Everyone, >> >> According to the rackspace status page [0] they are performance some >> maintenance in our public cloud firewall. This unfortunately appears to be >> affecting our ability to spin up systems in the public cloud so all CSIT >> jobs are currently failing. This is affecting both releng and sandbox since >> they both spin up csit instances in the same public cloud. >> >> The private cloud is still operating as usual as far as we can tell so >> verify/merge jobs should continue to work as expected. I've decided to put >> the sandbox into shutdown mode so that new csit jobs don't attempt to >> start. We will leave the releng silo as is since we want to let verify and >> merge jobs continue to work but csit jobs will fail in the meantime. >> >> We'll continue to monitor the system and update when Rackspace's >> maintenance window is over. >> >> Regards, >> Thanh >> >> [0] https://status.rackspace.com/ >> > >
_______________________________________________ infrastructure mailing list [email protected] https://lists.opendaylight.org/mailman/listinfo/infrastructure
