Other projects like Spark, flink, Oryx that are first class Hadoop citizens are setup to run integration tests that involve kafka, spinning up a mini spark cluster etc... and run the same on Travis.
We can target that for next release. On Fri, Aug 19, 2016 at 9:20 AM, Ellison Anne Williams < [email protected]> wrote: > Also, AWS and GCP are options - they won't integrate into travis, but they > are a (relatively) easy way to run through the distributed test suite. > > I had thought about posting some instructions (at some point) for 'How to > Run the Pirk Distributed Tests on AWS/GCP/etc' to help new folks get up to > speed quickly. Of course, AWS and GCP both have detailed instructions, but > they take time to wade through. Would that be helpful? > > On Fri, Aug 19, 2016 at 9:09 AM, Darin Johnson <[email protected]> > wrote: > > > I've built full integration tests with hadoop-minicluster before. > They're > > a pain to setup but aren't bad to maintain once done and could be > > integrated into travis-ci. > > > > On Fri, Aug 19, 2016 at 9:02 AM, Tim Ellison <[email protected]> > > wrote: > > > > > On 18/08/16 17:12, Ellison Anne Williams wrote: > > > > As a friendly public service announcement - please make sure that you > > run > > > > the distributed test suite before you accept a PR (or at least, > before > > > > accepting a PR that touches anything affecting the tests). > > > > > > Mea culpa. > > > > > > My usual working practice is: > > > - hack, hack, hack > > > - run mvn clean test locally > > > - commit to new local branch > > > - push to my github fork > > > - wait until Travis declares it tested ok > > > - open the PR, expect the PR to pass the Travis checks > > > > > > Now I agree that I should also be doing the distributed tests; and even > > > more so as I work my way up the Pirk stack into the distributed code. > > > > > > What I really want is the equivalent of a Travis check for the stuff > I'm > > > doing, and the PRs I'm reviewing. Any thoughts about how we can > achieve > > > that as I try to figure out how I can run the distributed tests? > > > > > > Regards, > > > Tim > > > > > > > > > > > >
