I have a suggestion. Since Pig is also available on Amazon's EMR, can we get some kind folks at Amazon to donate computing resources. It's a win-win - Pig is validated against their infrastructure and the community benefits too.
Any thoughts from the Amazon folks on the mailing list? Thanks, Santhosh -----Original Message----- From: Alan Gates [mailto:[email protected]] Sent: Thursday, September 01, 2011 10:29 PM To: [email protected] Subject: Re: Moving to new e2e harness for end-to-end testing One of my goals is to find a way to run public nightly builds for Pig. Since it needs a cluster it's not clear to me Apache's infrastructure is the best choice. I'm also investigating OSU's supercell (http://supercell.osuosl.org/) facilities. Other recommendations are welcome. I'd also like to get it set up so that there is a patch build, where contributors can submit a patch and a list of tests they think should be run, and have it run for them. For now I am running the tests nightly on a local Jenkins and will report any errors when I see them. And there's EC2 for running your own tests if you don't have a cluster. But this isn't a viable long term solution. Alan. On Sep 1, 2011, at 6:49 PM, Dmitriy Ryaboy wrote: > Alan, > Great work. > Any plans for hooking this up to the apache Jenkins instance? > > D > > On Thu, Sep 1, 2011 at 5:50 PM, Alan Gates <[email protected]> wrote: > >> I have gotten the end-to-end test harness to the point where it runs >> basically all the existing tests and where it can be run from ant. >> It can be run on either an existing cluster or in Amazon's EC2. >> There are instructions on how to run it in both settings at >> https://cwiki.apache.org/confluence/display/PIG/HowToTest. Please >> try it out to see if it works in your environment. >> >> As proposed in >> https://cwiki.apache.org/confluence/display/PIG/PigTestProposal, I >> would like to start migrating the junit tests that are really >> end-to-end tests (ie the ones that use MiniCluster) to the e2e >> harness. I'll start with the very long running ones and work down to the >> shorter ones. >> >> Also, as we contribute new patches these should include end-to-end >> tests that will run in the new harness rather than in junit. True >> unit tests should still, of course, be done in junit. >> >> Alan.
