To expand on the RunnableOnService test suggestion, here [1] is the commit from the Spark runner. You will get a lot more information if you can port this for your runner than you would from an example end-to-end test.
Note that this just pulls in the tests from the core SDK. For testing with other I/O connectors, you'll add them to the dependenciesToScan. [1] https://github.com/apache/incubator-beam/commit/4254749bf103c4bb6f68e316768c0aa46d9f7df0 On Tue, Jun 21, 2016 at 4:06 PM, Lukasz Cwik <[email protected]> wrote: > There is a start to getting more e2e like integration tests going with the > first being WordCount. > > https://github.com/apache/incubator-beam/blob/master/examples/java/src/test/java/org/apache/beam/examples/WordCountIT.java > You could add WindowedWordCountIT.java which will be launched with the > proper configuration of the Apex runner pom.xml > > I would also suggest that you take a look at the @RunnableOnService tests > which are a comprehensive validation suite of ~200ish tests that test > everything from triggers to side inputs. It requires some pom changes and > creating a test runner which is able to setup an apex environment. > > Furthermore, we could really use an addition to the Beam wiki about testing > and how runners write tests/execute tests/... > > Some relevant links: > Older presentation about getting cross runner tests going: > > https://docs.google.com/presentation/d/1uTb7dx4-Y2OM_B0_3XF_whwAL2FlDTTuq2QzP9sJ4Mg/edit#slide=id.g127d614316_19_39 > > Examples of test runners: > > https://github.com/apache/incubator-beam/blob/master/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/testing/TestDataflowRunner.java > > https://github.com/apache/incubator-beam/blob/master/runners/flink/runner/src/main/java/org/apache/beam/runners/flink/TestFlinkRunner.java > > https://github.com/apache/incubator-beam/blob/master/runners/spark/src/main/java/org/apache/beam/runners/spark/TestSparkRunner.java > > Section of pom dedicated to enabling runnable on service tests: > > https://github.com/apache/incubator-beam/blob/master/runners/spark/pom.xml#L54 > > On Tue, Jun 21, 2016 at 2:21 PM, Thomas Weise <[email protected]> > wrote: > > > Hi, > > > > As part of the Apex runner, we have a few unit tests for the supported > > transformations. Next, I would like to test the WindowedWordCount > example. > > > > Is there an example of configuring this pipeline for another runner? Is > it > > recommended to supply such configuration as a JUnit test? What is the > > general (repeatable?) approach to exercise different runners with the set > > of example pipelines? > > > > Thanks, > > Thomas > > >
