Hi Amir, We need more information to tell you what the problem is. One of the issues about the example pom of the old flink-dataflow runner is that it filters out all "org.apache.flink" classes. Some of these classes are actually not provided by the Flink cluster environment, e.g. connectors like Kafka. So you could try to remove the Shade plugin. This will increase your Jar size. Another problem people run often into is that they use different versions of Flink, e.g. two versions of the Scala library. That does not work.
Could you please provide a stack trace? Otherwise we have to guess what the problem is :) Thanks, Max On Mon, May 9, 2016 at 9:17 PM, amir bahmanyari <[email protected]> wrote: > Hi Colleagues, > I have been trying to build and run a Beam FlinkPipelineRunner app in a > Flink Cluster for sometime. > I followed the bottom procedure at the bottom of this link where it says how > to run it in a Flink Cluster. > dataArtisans/flink-dataflow > > dataArtisans/flink-dataflow > > flink-dataflow - Google Dataflow Runner for Apache Flink > > > My app gets builds ok, at least it seems like that. > But, when I run it in the Flink Cluster, I have been getting zillions of > different types of exceptions and its non ending. > Is there a custom pom.xml, or a guideline to develop this app so it just > builds and runs like the example in the above link? > > This is how I run it in Flink Cluster: > > ./flink run > /opt/analytics/apache/beamproject/incubator-beam-master/examples/java/src/main/java/org/apache/beam/examples/beamflink-bench/target/beamflink-bench-1.0-SNAPSHOT.jar > --topic lrdata --bootstrap.servers myhost:9092 --zookeeper.connect kirk:2181 > --group.id myGroup. > > These are the dependencies I currently have in my pom.xml. > Thanks so much for your help. Appreciate your kind attention. > > <dependencies> > <dependency> > <groupId>com.google.oauth-client</groupId> > <artifactId>google-oauth-client</artifactId> > <version>1.22.0</version> > </dependency> > <dependency> > <groupId>org.apache.beam</groupId> > <artifactId>flink-runner_2.10</artifactId> > <version>0.1.0-incubating-SNAPSHOT</version> > </dependency> > <dependency> > <groupId>com.apache.kafka</groupId> > <artifactId>kafka-0.1.0</artifactId> > <version>0.1.0-incubating-SNAPSHOT</version> > </dependency> > <dependency> > <groupId>com.apache.beam</groupId> > <artifactId>direct-runner-0.1.0</artifactId> > <version>0.1.0-incubating-SNAPSHOT</version> > </dependency> > <dependency> > <groupId>com.google.collections</groupId> > <artifactId>google-collections</artifactId> > <version>1.0-rc2</version> > </dependency> > > <dependency> > <groupId>junit</groupId> > <artifactId>junit</artifactId> > <version>3.8.1</version> > <scope>test</scope> > </dependency> > </dependencies> >
