Ok, no worries Aaron, that's still good advice :) One last question - are you using JAR-based or image-based deployments? The only real problem using Flink & Bazel and a JAR-based deployment from our experience is removing the Flink libs present in the deploy environment from the deploy jar, and still having them present when we want to do local debugging/ integration testing. Also possible that we're just not using Bazel entirely correctly!
Thank you! Austin On Thu, Jun 18, 2020 at 12:32 PM Aaron Levin <aaronle...@stripe.com> wrote: > Hi Austin, > > In our experience, `rules_scala` and `rules_java` are enough for us at > this point. > > It's entirely possible I'm not thinking far enough into the future, > though, so don't take our lack of investment as a sign it's not worth > investing in :) > > Best, > > Aaron Levin > > On Thu, Jun 18, 2020 at 10:27 AM Austin Cawley-Edwards < > austin.caw...@gmail.com> wrote: > >> Great to hear Dan! >> >> @Aaron - would you/ your team be interested in a `rules_flink` project? >> I'm still fairly new to Bazel and know enough to contribute, but could >> definitely use guidance on design as well. >> >> Best, >> Austin >> >> On Mon, Jun 15, 2020 at 11:07 PM Dan Hill <quietgol...@gmail.com> wrote: >> >>> Thanks for the replies! I was able to use the provided answers to get a >>> setup working (maybe not the most efficiently). The main change I made was >>> to switch to including the deploy jar in the image (rather than the default >>> one). >>> >>> I'm open to contributing to a "rules_flink" project. I don't know >>> enough yet to help design it. >>> >>> On Sat, Jun 13, 2020 at 4:39 AM Till Rohrmann <trohrm...@apache.org> >>> wrote: >>> >>>> Hi Dan, >>>> >>>> if you want to run a Flink job without specifying the main class via >>>> `bin/flink run --class org.a.b.Foobar` then you have to add a MANIFEST.MF >>>> file to your jar under META-INF and this file needs to contain `Main-Class: >>>> org.a.b.Foobar`. >>>> >>>> Cheers, >>>> Till >>>> >>>> On Fri, Jun 12, 2020 at 12:30 AM Austin Cawley-Edwards < >>>> austin.caw...@gmail.com> wrote: >>>> >>>>> Hey all, >>>>> >>>>> Adding to Aaron's response, we use Bazel to build our Flink apps. >>>>> We've open-sourced some of our setup here[1] though a bit outdated. There >>>>> are definitely rough edges/ probably needs a good deal of work to fit >>>>> other >>>>> setups. We have written a wrapper around the `java_library` and >>>>> `java_binary` and could do the same for `rules_scala`, though we just >>>>> started using Bazel last November and have a lot to learn in terms of best >>>>> practices there. >>>>> >>>>> If you're interested in contributing to a `rules_flink` project, I >>>>> would be as well! >>>>> >>>>> Best, >>>>> Austin >>>>> >>>>> [1]: >>>>> https://github.com/fintechstudios/vp-operator-demo-ff-virtual-2020 >>>>> >>>>> On Thu, Jun 11, 2020 at 6:14 PM Aaron Levin <aaronle...@stripe.com> >>>>> wrote: >>>>> >>>>>> Hi Dan, >>>>>> >>>>>> We use Bazel to compile our Flink applications. We're using >>>>>> "rules_scala" (https://github.com/bazelbuild/rules_scala) to manage >>>>>> the dependencies and produce jars. We haven't had any issues. However, I >>>>>> have found that sometimes it's difficult to figure out exactly what Flink >>>>>> target or dependency my application needs. >>>>>> >>>>>> Unfortunately I'm not sure what issue you're seeing here. I would >>>>>> guess either your flink application wasn't compiled into the jar >>>>>> you're executing. If you can paste the bazel target used to generate your >>>>>> jar and how you're launching the application, that will be helpful >>>>>> for diagnosis. >>>>>> >>>>>> On Thu, Jun 11, 2020 at 5:21 PM Dan Hill <quietgol...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> I took the Flink playground and I'm trying to swap out Maven for >>>>>>> Bazel. I got to the point where I'm hitting the following error. I >>>>>>> want >>>>>>> to diff my code with an existing, working setup. >>>>>>> >>>>>>> Thanks! - Dan >>>>>>> >>>>>>> >>>>>>> client_1 | >>>>>>> org.apache.flink.client.program.ProgramInvocationException: >>>>>>> Neither a 'Main-Class', nor a 'program-class' entry was found in the jar >>>>>>> file. >>>>>>> >>>>>>> client_1 | at >>>>>>> org.apache.flink.client.program.PackagedProgram.getEntryPointClassNameFromJar(PackagedProgram.java:596) >>>>>>> >>>>>>> client_1 | at >>>>>>> org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:190) >>>>>>> >>>>>>> client_1 | at >>>>>>> org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:128) >>>>>>> >>>>>>> client_1 | at >>>>>>> org.apache.flink.client.cli.CliFrontend.buildProgram(CliFrontend.java:862) >>>>>>> >>>>>>> client_1 | at >>>>>>> org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:204) >>>>>>> >>>>>>> client_1 | at >>>>>>> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1050) >>>>>>> >>>>>>> client_1 | at >>>>>>> org.apache.flink.client.cli.CliFrontend.lambda$main$11(CliFrontend.java:1126) >>>>>>> >>>>>>> client_1 | at >>>>>>> org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30) >>>>>>> >>>>>>> client_1 | at >>>>>>> org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1126) >>>>>>> >>>>>>
