Hello, On Tue, 16 Apr 2013 06:19:51 +0530 Vishal Bhoj <[email protected]> wrote:
[] > > > This error is related to infrastructure. I am not sure if this > > > will be resolved if the publishing is updated. > > > > ChannelClosedException as quoted below means an EC2 instance got > > terminated (or otherwise "lost") behind Jenkins' back. Generally, > > this issue is a known non-deterministic failure issue, and bound to > > happen from time to time due to the nature of EC2 (complex big > > system, has non-zero stream of errors). > > > > We really need to find a solution for this. We are running into this > error on a regular basis nowadays. Yes, I see that 2 builds I tried yesterday didn't succeed either, with the same error. Builds of that job look weird, because they're still in compile phase after 3.5hrs after the start - that's too long. At last 2 builds were killed at almost the same time, but it's not build timeout (set at 4:45mins, looks differently), not EC2 monitoring script (not active now, never killed running builds, only zombie instances). I'm cc:ing Phillip just in case if he may know of anything which may kill EC2 in the "old" EC2 Linaro account in 3.5hrs? I still think the likely cause though is master overload due to publishing issues, and would like to keep working on resolving that first. I have good results so far - "copycat" build on a sandbox finished with less than 2min: https://ec2-107-20-93-222.compute-1.amazonaws.com/jenkins/job/pfalcon_galaxynexus-linaro/9/ That's 1/2 of all work needed tho, going to deploy needed parts on production and continue with it. > > > > > The fact that it happened 3 times is worrying though. We appear to > > have had another zombie slave storm over weekend, and on top of > > that, we have > > https://bugs.launchpad.net/linaro-android-infrastructure/+bug/1164273 , > > which causes builds to prolong too much, to hit any EC2-related > > issues with higher probability. I'm working on that issue as top > > priority. > > > > In the meantime, I restarted > > https://android-build.linaro.org/builds/~linaro-android/vexpress-linaro-mp/ > > and will watch it to make sure we have fresh succeeding build. > > > > > > > > > On 15 April 2013 10:55, Naresh Kamboju <[email protected]> > > > wrote: > > > > > > > Hi, > > > > > > > > I would like to inform you that following builds failed. > > > > > > > > > > > > > > https://android-build.linaro.org/builds/~linaro-android/vexpress-linaro-mp/#build=257 > > > > > > > > > > https://android-build.linaro.org/builds/~linaro-android/vexpress-linaro-mp/#build=256 > > > > > > > > > > https://android-build.linaro.org/builds/~linaro-android/vexpress-linaro-mp/#build=255 > > > > > > > > Build log snapshot: > > > > ----------------------- > > > > Caused by: hudson.remoting.ChannelClosedException: channel is > > > > already closed > > > > Caused by: hudson.remoting.Channel$OrderlyShutdown > > > > Caused by: Command close created at > > > > > > > > FATAL: hudson.remoting.RequestAbortedException: > > > > hudson.remoting.Channel$OrderlyShutdown > > > > > > > > build complete log: > > > > ----------------------- > > > > > > > > > > https://android-build.linaro.org/jenkins/job/linaro-android_vexpress-linaro-mp/257/consoleText > > > > > > > > Best regards > > > > Naresh Kamboju > > > > > > > > > > > > -- > > Best Regards, > > Paul > > > > Linaro.org | Open source software for ARM SoCs > > Follow Linaro: http://www.facebook.com/pages/Linaro > > http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog > > -- Best Regards, Paul Linaro.org | Open source software for ARM SoCs Follow Linaro: http://www.facebook.com/pages/Linaro http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog _______________________________________________ linaro-validation mailing list [email protected] http://lists.linaro.org/mailman/listinfo/linaro-validation
