"because currently it links in the original Dataflow runners by default"Which
jar(s) should be on the classpath to resolve "the original Dataflow runners"?In
case of using FlinkPipelineRunner, I am getting this exception that clearly
references Dataflow runners but the Dataflow runners classes are not
fond.Thanks for your help.
Caused by: java.lang.NoClassDefFoundError:
org/apache/beam/runners/dataflow/DataflowPipelineRunner at
org.apache.beam.runners.flink.FlinkPipelineRunner.fromOptions(FlinkPipelineRunner.java:82)
... 20 more
From: Frances Perry <[email protected]>
To: [email protected]
Sent: Monday, May 9, 2016 7:32 AM
Subject: Re: Direct Runner Example
The whole goal of Beam is that you won't need to change your pipeline code to
swap between runners. So like JB said, you should look in the examples module.
The idea is that you can use the --runner option to select from any runner
currently on your classpath. (Note that the Flink runner currently has its own
copy for legacy reasons -- we'll be removing that.)
So for example, you can run with the direct runner like this:
$ mvn compile exec:java -pl examples/java
-Dexec.mainClass=org.apache.beam.examples.WordCount
-Dexec.args="--runner=DirectPipelineRunner --output=output"
(We still need to fix the pom a bit to be runner-agnostic, because currently it
links in the original Dataflow runners by default.)
You can also take a look at this Word Count Walkthrough that we'll be porting
from Dataflow to Beam soon.
Frances
On Mon, May 9, 2016 at 4:36 AM, Jean-Baptiste Onofré <[email protected]> wrote:
Hi
You have a word count sample in the examples module.
Regards JB
-------- Original message --------
From: Punit Naik <[email protected]>
Date: 09/05/2016 12:56 (GMT+01:00)
To: [email protected]
Subject: Direct Runner Example
Can I get a wordcount direct runner example (batch)?
--
Thank You
Regards
Punit Naik
