Re: Does anyone have an example of Bazel working with Flink?

2020-06-18 Thread Austin Cawley-Edwards
Ok, no worries Aaron, that's still good advice :) One last question - are you using JAR-based or image-based deployments? The only real problem using Flink & Bazel and a JAR-based deployment from our experience is removing the Flink libs present in the deploy environment from the deploy jar, and

Re: Does anyone have an example of Bazel working with Flink?

2020-06-18 Thread Aaron Levin
Hi Austin, In our experience, `rules_scala` and `rules_java` are enough for us at this point. It's entirely possible I'm not thinking far enough into the future, though, so don't take our lack of investment as a sign it's not worth investing in :) Best, Aaron Levin On Thu, Jun 18, 2020 at

Re: Does anyone have an example of Bazel working with Flink?

2020-06-18 Thread Austin Cawley-Edwards
Great to hear Dan! @Aaron - would you/ your team be interested in a `rules_flink` project? I'm still fairly new to Bazel and know enough to contribute, but could definitely use guidance on design as well. Best, Austin On Mon, Jun 15, 2020 at 11:07 PM Dan Hill wrote: > Thanks for the replies!

Re: Does anyone have an example of Bazel working with Flink?

2020-06-15 Thread Dan Hill
Thanks for the replies! I was able to use the provided answers to get a setup working (maybe not the most efficiently). The main change I made was to switch to including the deploy jar in the image (rather than the default one). I'm open to contributing to a "rules_flink" project. I don't know

Re: Does anyone have an example of Bazel working with Flink?

2020-06-13 Thread Till Rohrmann
Hi Dan, if you want to run a Flink job without specifying the main class via `bin/flink run --class org.a.b.Foobar` then you have to add a MANIFEST.MF file to your jar under META-INF and this file needs to contain `Main-Class: org.a.b.Foobar`. Cheers, Till On Fri, Jun 12, 2020 at 12:30 AM

Re: Does anyone have an example of Bazel working with Flink?

2020-06-11 Thread Austin Cawley-Edwards
Hey all, Adding to Aaron's response, we use Bazel to build our Flink apps. We've open-sourced some of our setup here[1] though a bit outdated. There are definitely rough edges/ probably needs a good deal of work to fit other setups. We have written a wrapper around the `java_library` and

Re: Does anyone have an example of Bazel working with Flink?

2020-06-11 Thread Aaron Levin
Hi Dan, We use Bazel to compile our Flink applications. We're using "rules_scala" ( https://github.com/bazelbuild/rules_scala) to manage the dependencies and produce jars. We haven't had any issues. However, I have found that sometimes it's difficult to figure out exactly what Flink target or

Does anyone have an example of Bazel working with Flink?

2020-06-11 Thread Dan Hill
I took the Flink playground and I'm trying to swap out Maven for Bazel. I got to the point where I'm hitting the following error. I want to diff my code with an existing, working setup. Thanks! - Dan client_1| org.apache.flink.client.program.ProgramInvocationException: Neither