2011/2/14 Magnus Hagander <mag...@hagander.net>: > On Mon, Feb 14, 2011 at 11:46, Cédric Villemain > <cedric.villemain.deb...@gmail.com> wrote: >> 2011/2/14 Stefan Kaltenbrunner <ste...@kaltenbrunner.cc>: >>> On 02/14/2011 10:09 AM, Magnus Hagander wrote: >>>> On Mon, Feb 14, 2011 at 07:13, Stefan Kaltenbrunner >>>> <ste...@kaltenbrunner.cc> wrote: >>>>> On 02/14/2011 01:27 AM, Tom Lane wrote: >>>>>> >>>>>> Magnus Hagander<mag...@hagander.net> writes: >>>>>>> >>>>>>> Unfortunately, one of the worst-case scenarios appears to have >>>>>>> happened - a machine did not come back up after a reboot. >>>>>>> ... >>>>>>> We'll get back to you with more information as soon as we have it. >>>>>> >>>>>> I didn't see any followup to this? >>>>> >>>>> yeah - the hosting company managed to reboot the box for us which brought >>>>> it >>>>> back to life in the middle of the night (with both magnus and me asleep). >>>> >>>> Indeed. But the good news is that once it came back up, the VM with >>>> the git server started ok :-) >>>> >>>> >>>>>> gitmaster seems to be responding as of now, is it safe to push? >>>>> >>>>> yes it is - however we will need to schedule another maintenance window >>>>> soon >>>>> to finish the stuff we actually wanted to do. >>>> >>>> So, after some discussion with Stefan, we (well, I guess I) decided we >>>> should just go ahead and declare the maintenance window not closed >>>> yet, and finish off the upgrade right now :-) Given that the majority >>>> of our commits don't happen now, we'll hopefully have it done by the >>>> time the US folks wake up again. >>>> >>>> So, maintenance window again, starting now, and we'll let you know as >>>> soon as we're done. And we're definitely hoping for the machine to >>>> come back up properly this time :-) >>> >>> and it did not... We are trying to figure out what the actual problem >>> here really is because it seems to boot just fine when powercycled just >>> not with a software initiated reboot. >>> We will notify once we have more information... >>> >> >> Does it make sense to get some console link or ipmi set up for those >> crucial parts of the infrastructure ? > > This is production servers, of course they are equipped with remove consoles. > > However, these consoles are only accessible from the hosting companys > internal company network or VPN, so we cannot access them directly. > > It is something we are discussing with them...
ok. Not the top priority here I believe, but those kind of crisis period usually help (to have it set up quickly, as the topic is hot ) Thank you for your time and effort spent, -- Cédric Villemain 2ndQuadrant http://2ndQuadrant.fr/ PostgreSQL : Expertise, Formation et Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
