+Max and Kenn explicitly It looks like the Flink pipeline runner currently depends on the DataflowPipelineOptions, which may add some complicated runtime dependencies.
https://github.com/apache/incubator-beam/blob/master/runners/flink/runner/src/main/java/org/apache/beam/runners/flink/FlinkPipelineOptions.java#L21 It seems conceivable that this could cause issues like this. Dan On Mon, May 9, 2016 at 12:51 PM, amir bahmanyari <[email protected]> wrote: > "because currently it links in the original Dataflow runners by default" > Which jar(s) should be on the classpath to resolve "the original Dataflow > runners"? > In case of using FlinkPipelineRunner, I am getting this exception that > clearly references Dataflow runners but the Dataflow runners classes are > not fond. > Thanks for your help. > > Caused by: java.lang.NoClassDefFoundError: > org/apache/beam/runners/dataflow/*DataflowPipelineRunner* > at org.apache.beam.runners.flink.FlinkPipelineRunner.fromOptions( > *FlinkPipelineRunner*.java:82) > ... 20 more > > > > ------------------------------ > *From:* Frances Perry <[email protected]> > *To:* [email protected] > *Sent:* Monday, May 9, 2016 7:32 AM > *Subject:* Re: Direct Runner Example > > The whole goal of Beam is that you won't need to change your pipeline code > to swap between runners. So like JB said, you should look in the examples > <https://github.com/apache/incubator-beam/tree/master/examples> module. > The idea is that you can use the --runner option to select from any runner > currently on your classpath. (Note that the Flink runner currently has its > own copy for legacy reasons -- we'll be removing that.) > > So for example, you can run with the direct runner like this: > > $ mvn compile exec:java -pl examples/java > -Dexec.mainClass=org.apache.beam.examples.WordCount > -Dexec.args="--runner=DirectPipelineRunner --output=output" > > (We still need to fix the pom a bit to be runner-agnostic, because > currently it links in the original Dataflow runners by default.) > > You can also take a look at this Word Count Walkthrough > <https://cloud.google.com/dataflow/examples/wordcount-example> that we'll > be porting from Dataflow to Beam soon. > > Frances > > > > On Mon, May 9, 2016 at 4:36 AM, Jean-Baptiste Onofré <[email protected]> > wrote: > > Hi > > You have a word count sample in the examples module. > > Regards > JB > > > -------- Original message -------- > From: Punit Naik <[email protected]> > Date: 09/05/2016 12:56 (GMT+01:00) > To: [email protected] > Subject: Direct Runner Example > > Can I get a wordcount direct runner example (batch)? > > -- > Thank You > > Regards > > Punit Naik > > > > >
