Hi Arvid! What you describe sounds like a great use case. I am not aware of an integration of Flink with KNIME. Triggering progamms from other programs should work through the Client & PackagedProgram classes-
- https://github.com/apache/incubator-flink/blob/master/flink-clients/src/main/java/org/apache/flink/client/program/Client.java - https://github.com/apache/incubator-flink/blob/master/flink-clients/src/main/java/org/apache/flink/client/program/PackagedProgram.java Triggering programs through HTTPS REST should be possible with a thin layer that wraps RPC calls / actor messages in HTTP requests. Out-of-the-box images for Amazon do not exist with any current version of the system. So far, elastic MapReduce with YARN has proved a simpler setup than any VM we had prepared before. How would you skip the SSH key setup with prepared images? Greetings, Stephan On Mon, Nov 10, 2014 at 3:02 PM, Arvid Heise <[email protected]> wrote: > Dear Flinkler, > > For my current project, we want to outsource some performance critical > parts of a complex KNIME workflow to Flink. Is there already a way to > trigger a Flink workflow from KNIME? If not, we will probably provide a > straight-forward way to execute Flink (Scala) programs from KNIME within > this month. The overall goal is to upload the data from the KNIME workflow > to s3, execute Flink on this data, and retrieve the output from s3. > > Since my employer (BfR) has a rather strict firewall setup, I will also add > a feature request for a REST API of the job manager similar to s3 ( > http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-post-example.html), > so > that we can completely rely on HTTP(S). > > Lastly, are there any plans for providing out-of-the-box VMs for Amazon > MapReduce? I saw the blog post > > https://flink.incubator.apache.org/news/2014/02/18/amazon-elastic-mapreduce-cloud-yarn.html > , which would suffice, but still requires quite a bit manual work (with > SSH, which does not work here). > > Feel free to split the topics into separate threads if you want to discuss > them individually. > > Best, > > Arvid >
