It went down hard today but I think fortunately I uncovered the issue. The vm had hundreds of stuck puppet threads that slowly consumed all the ram until ubuntu started kill -9'ing the pid. The slow death of swap and ram likely explains a lot of the timeouts recently until everything finally caved completely. There's a cron job to kill the zombie puppets so hopefully we don't see that anymore.
On Wed, Jun 18, 2014 at 10:34 AM, Igor Fedorenko <i...@ifedorenko.com> wrote: > I got 503 page several times during the day and evening yesterday. > > -- > Regards, > Igor > > > On 2014-06-18, 9:13, Jason van Zyl wrote: >> >> Two times during the one hour window that I was trying to release that the >> service was not available. It was an Apache page but I assume Nexus wasn't >> running behind it or not responding. >> >> On Jun 18, 2014, at 8:57 AM, Brian Fox <bri...@infinity.nu> wrote: >> >>> I haven't received any alerts that it's been down although there are >>> sporadic reports of timeouts. Did you receive a timeout, 502 or >>> something else? >>> >>> Nexus is on shared vm hosts and shared disks and I suspect that some >>> other guest is occasionally bursting and screwing up the io >>> throughput. >>> >>> On Tue, Jun 17, 2014 at 11:42 AM, Jason van Zyl <ja...@takari.io> wrote: >>>> >>>> Can we discuss what kind of hardware Nexus is on? It's gone down again >>>> while I'm trying to release and it's very annoying. >>>> >>>> On Jun 17, 2014, at 9:46 AM, Jason van Zyl <ja...@takari.io> wrote: >>>> >>>>> Thanks for bringing it back whoever kicked it. >>>>> >>>>> On Jun 17, 2014, at 9:39 AM, Jason van Zyl <ja...@takari.io> wrote: >>>>> >>>>>> Is Nexus down for scheduled maintenance or did it just fall over? >>>>>> >>>>>> I'm trying to cut the 3.2.2 release and Nexus appears dead: >>>>>> >>>>>> https://repository.apache.org >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Jason >>>>>> >>>>>> ---------------------------------------------------------- >>>>>> Jason van Zyl >>>>>> Founder, Apache Maven >>>>>> http://twitter.com/jvanzyl >>>>>> http://twitter.com/takari_io >>>>>> --------------------------------------------------------- >>>>>> >>>>>> believe nothing, no matter where you read it, >>>>>> or who has said it, >>>>>> not even if i have said it, >>>>>> unless it agrees with your own reason >>>>>> and your own common sense. >>>>>> >>>>>> -- Buddha >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Jason >>>>> >>>>> ---------------------------------------------------------- >>>>> Jason van Zyl >>>>> Founder, Apache Maven >>>>> http://twitter.com/jvanzyl >>>>> http://twitter.com/takari_io >>>>> --------------------------------------------------------- >>>>> >>>>> You are never dedicated to something you have complete confidence in. >>>>> No one is fanatically shouting that the sun is going to rise tomorrow. >>>>> They know it is going to rise tomorrow. When people are fanatically >>>>> dedicated to political or religious faiths or any other kind of >>>>> dogmas or goals, it's always because these dogmas or >>>>> goals are in doubt. >>>>> >>>>> -- Robert Pirzig, Zen and the Art of Motorcycle Maintenance >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>> >>>> Thanks, >>>> >>>> Jason >>>> >>>> ---------------------------------------------------------- >>>> Jason van Zyl >>>> Founder, Apache Maven >>>> http://twitter.com/jvanzyl >>>> http://twitter.com/takari_io >>>> --------------------------------------------------------- >>>> >>>> The modern conservative is engaged in one of man's oldest exercises in >>>> moral philosophy; that is, >>>> the search for a superior moral justification for selfishness. >>>> >>>> -- John Kenneth Galbraith >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >>> For additional commands, e-mail: dev-h...@maven.apache.org >>> >> >> Thanks, >> >> Jason >> >> ---------------------------------------------------------- >> Jason van Zyl >> Founder, Apache Maven >> http://twitter.com/jvanzyl >> http://twitter.com/takari_io >> --------------------------------------------------------- >> >> Be not afraid of growing slowly, be only afraid of standing still. >> >> -- Chinese Proverb >> >> >> >> >> >> >> >> >> >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
