Thanks Ismael.I thought/was looking for: compile once and run the same binary
for different Runner(s) passing it from the command line.Is this a realistic
assumption?The source code is extremely helpful to get on the same page as far
as style, logging etc.Have a great day.
From: Ismaël Mejía <[email protected]>
To: [email protected]
Sent: Tuesday, July 26, 2016 12:28 AM
Subject: Re: Example: pass Runner at command line
Hello,
You can look at the repository for GDELT based examples that JB and me are
working on, they include maven profiles to run the Pipelines in most of the
existing runners (not Gearpump yet but soon). You can take those profiles as an
example. I just changed the README so you can see how to execute them with
Maven.
https://github.com/jbonofre/beam-samples
Ismaël
On Tue, Jul 26, 2016 at 12:30 AM, amir bahmanyari <[email protected]> wrote:
Thanks Emanuele,Yes, I know these examples exist. I thought there has been one
put together that addresses the Runner agnostic coding specifically without
doing any extra work.A true "unified" example.Did you solve your State issue? I
had the same questions sometime ago.For now, I use Redis to persist run-time
state. Kinda poor man's way :-) works for now, but doesn't scale as I want
it.Cheers
From: Emanuele Cesena <[email protected]>
To: [email protected]; amir bahmanyari <[email protected]>
Sent: Monday, July 25, 2016 3:18 PM
Subject: Re: Example: pass Runner at command line
Hi Amir,
If you’re looking for a runner-independent example, you can find some in
examples/.
If you’re looking for runner-independent code that works on Flink, I think you
should still wait a few iterations.
Or, you can take code that refers to Flink, and remove the dependencies
yourself.
For instance, if you start from this example:
https://github.com/ecesena/beam-starter/blob/master/src/main/java/com/dataradiant/beam/examples/WordCount.java
You can remove the setRunner by passing -Drunner=FlinkRunner in pom.xml
You can remove FlinkPipelineOptions but you have to implement yourself some of
the getter/setter required by Flink, e.g. jobName:
https://github.com/apache/incubator-beam/blob/master/runners/flink/runner/src/main/java/org/apache/beam/runners/flink/FlinkPipelineOptions.java
Hope this helps,
> On Jul 25, 2016, at 2:59 PM, amir bahmanyari <[email protected]> wrote:
>
> Hi Colleagues,
> Is there a simple genetic example where the Runner is passed at the command
> line, the Beam code sets it in the generic Beam Options.set Runner(), and
> Pipeline.create() is?
> No mention of ANY specific Runner in the code like FlinkPipelineOptions .
>
> Thanks.
> Amir-
--
Emanuele Cesena, Data Eng.
http://www.shopkick.com
Il corpo non ha ideali
