On 19/08/16 14:20, Ellison Anne Williams wrote: > Also, AWS and GCP are options - they won't integrate into travis, but they > are a (relatively) easy way to run through the distributed test suite. > > I had thought about posting some instructions (at some point) for 'How to > Run the Pirk Distributed Tests on AWS/GCP/etc' to help new folks get up to > speed quickly. Of course, AWS and GCP both have detailed instructions, but > they take time to wade through. Would that be helpful?
I've had a play with AWS and got a Spark cluster defined and started -- but our instructions for running Pirk distributed tests [1] don't really give enough information on how to send it work to do. It'll take me a while to figure it out from the code, so if you can share any properties etc that would be helpful. [1] http://pirk.incubator.apache.org/for_developers#testing Thanks, Tim > On Fri, Aug 19, 2016 at 9:09 AM, Darin Johnson <[email protected]> > wrote: > >> I've built full integration tests with hadoop-minicluster before. They're >> a pain to setup but aren't bad to maintain once done and could be >> integrated into travis-ci. >> >> On Fri, Aug 19, 2016 at 9:02 AM, Tim Ellison <[email protected]> >> wrote: >> >>> On 18/08/16 17:12, Ellison Anne Williams wrote: >>>> As a friendly public service announcement - please make sure that you >> run >>>> the distributed test suite before you accept a PR (or at least, before >>>> accepting a PR that touches anything affecting the tests). >>> >>> Mea culpa. >>> >>> My usual working practice is: >>> - hack, hack, hack >>> - run mvn clean test locally >>> - commit to new local branch >>> - push to my github fork >>> - wait until Travis declares it tested ok >>> - open the PR, expect the PR to pass the Travis checks >>> >>> Now I agree that I should also be doing the distributed tests; and even >>> more so as I work my way up the Pirk stack into the distributed code. >>> >>> What I really want is the equivalent of a Travis check for the stuff I'm >>> doing, and the PRs I'm reviewing. Any thoughts about how we can achieve >>> that as I try to figure out how I can run the distributed tests? >>> >>> Regards, >>> Tim >>> >>> >>> >> >
