vcastell-tibco opened a new issue, #29660:
URL: https://github.com/apache/beam/issues/29660
I am running into some issues and would like comments on how to proceed or
resolve the issue.
I have been successful in running the Flink examples against a Flink Cluster
running on Kubernetes. I am trying to run similar examples from Apache Beam. I
am able to the following using Apache Beam 2.5.0 or 2.5.2 against Flink 1.16.3
but I am trying to use Flink 1.17.1 or later but I cannot submit job against
Flink cluster.
Package Apache Beam Wordcount example with Flink runner using command “$ mvn
package -Pflink-runner”?
Push the application to Flink Cluster using below convention which is
documented in Beam docs.
mvn exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
-Pflink-runner \
-Dexec.args=“--runner=FlinkRunner \
--inputFile=/path/to/pom.xml \
--output=/path/to/counts \
--flinkMaster=<flink master url> \
--filesToStage=target/word-count-beam-bundled-0.1.jar”
If I do above steps using Beam 2.50.0 or 2.52.0 against Flink 1.16.3 then it
works fine. Job is submitted and runs successfully. If I use the same version
of Beam but a higher version of Flink then the job fails to submit. I am
getting errors with the dependencies which are packaged with application. It
looks like the additional jars packaged are not compatible with the Flink
runtime. Can someone please let me know what combination of Beam is required
with Flink 1.17.x or higher? Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
