Hi, On Fri, May 30, 2014 at 1:05 PM, <dev-digest-h...@hive.apache.org> wrote:
> Subject: Re: Fixing Trunk tests and getting stable nightly build on b.a.o > Thanks for asking. Wanted to add, due to high hardware requirements of > Hive build (>20 hours and the memory like you see) and the limited Apache > hardware resources at the time, we had moved the Hive builds to our > dedicated Amazon EC2 cluster (currently sponsored by Cloudera, open to > other sponsorship), currently of 6 EC2 spot instances of size c1.xlarge. > I noticed that this is linked to from the Hive front page bigtop01.cloudera.org:8080/view/Hive/ which is dead... or which times out. > For this, we have written a parallel test framework that divides the test > suite among the nodes as described here: > https://cwiki.apache.org/confluence/display/Hive/Hive+PTest2+Infrastructure > . > It has given pretty good throughput. Brock did the original move, and he > or others might have more context of old hive builds running on Apache. > > I think it is always a good idea to set this up on Apache hardware, but > only issue is whether it can meet the requirements, as we have a really > huge volume of incoming pre-commit builds that need to be run. > > Yeah this is an issue for sure. Honestly, I don't have a solution for these types of resources being run on Apache builds infrastructure. We have a similar situation with a goraci [0] test suite for Apache Gora... the only option we have is to run this (when we can) on outside infrastructure. Unfortunately we don't have someone kind enough like Cloudera to run this on though. Lewis [0] https://github.com/keith-turner/goraci
