Re: On integrating Flink with Apache NiFi

2015-09-23 Thread Fabian Hueske
That depends on your requirements. A common interface is usually less feature rich than the abstracted processing engines. For example, Cascading does not support iterations or streaming. If you are fine with the features (and limitations) of the common interface, your approach might make sense. C

Re: On integrating Flink with Apache NiFi

2015-09-23 Thread Flavio Pompermaier
Hi Kostas, my question is related to the fact that now Flink is available as Cascading engine. So, if I had to write a general purpose dataflow UI I think I'd use Cascading or Dataflow, so that I could draw my pipeline with the UI and write the execution code using just one API. Obviously this int

Re: On integrating Flink with Apache NiFi

2015-09-22 Thread Kostas Tzoumas
I had a discussion with Joe from the NiFi community, and they are interested in contributing a connector between NiFi and Flink. I created a JIRA issue for that: https://issues.apache.org/jira/browse/FLINK-2740 I believe that this is the easiest and most useful integration point to begin with, as

Re: On integrating Flink with Apache NiFi

2015-09-22 Thread Flavio Pompermaier
I saw that now cascading supports Flink..so maybe you could think in programming a cascading abstraction to have also spark and tez compatibility for free!what do you think? On 22 Sep 2015 11:17, "Christian Kreutzfeldt" wrote: > Hi Slim, > > thanks for sharing the presentation. Like Flavio I see

Re: On integrating Flink with Apache NiFi

2015-09-22 Thread Christian Kreutzfeldt
Hi Slim, thanks for sharing the presentation. Like Flavio I see the (UI) capabilities provided by Apache NiFi as very helpful for (enterprise) customers. At the Otto Group we are currently think about how to track data through a stream based system like we do in the batch world using staging laye

Re: On integrating Flink with Apache NiFi

2015-09-19 Thread Flavio Pompermaier
As a user that would be very helpful! On 19 Sep 2015 12:34, "Slim Baltagi" wrote: > Hi Flink experts! > > I came across Apache Nifi https://nifi.apache.org > https://www.youtube.com/watch?v=sQCgtCoZyFQ > In the Nifi project, there is an open JIRA > issue:https://issues.apache.org/jira/browse/NIFI