I should add that Flink also works well for relational use cases (see the examples). However, It does not offer a SQL interface which means that a query must be implemented as a "handcrafted" data flow consisting of operators such as filter, join, project, and group.
Adding a SQL interface (such as Spark) should be possible without changing a lot of the internals. However, nobody in the community is currently putting a focus on that, AFAIK. 2014-10-22 11:14 GMT+02:00 Fabian Hueske <[email protected]>: > Sure, no problem :-) > > Drill is a SQL engine and therefore in the same league as Apache Hive, > Apache Tajo, or Cloudera's Impala. > > Flink (and Spark) focus on use cases that exceed pure SQL (+ a few UDFs) > such as Graph processing, Machine Learning, and very custom data flows. > > Best, Fabian > > 2014-10-22 10:58 GMT+02:00 Flavio Pompermaier <[email protected]>: > >> Sorry for the intrusion but you mentioned the differences between Spark >> and Flink..what is not clear to me if Apache Drill is a less architectured >> version of Flink or something very similar..could someone give me a >> clarification on this? >> >> Best, >> Flavio >> >> On Wed, Oct 22, 2014 at 9:26 AM, Mohit Singh <[email protected]> wrote: >> >>> Great. >>> And So am I. :) >>> Looking forward to learn things and contribute back. >>> >>> On Wed, Oct 22, 2014 at 12:22 AM, Fabian Hueske <[email protected]> >>> wrote: >>> >>>> Done! >>>> I'm very happy that you're joining the community :-) >>>> >>>> Welcome! >>>> Fabian >>>> >>>> 2014-10-22 9:11 GMT+02:00 Mohit Singh <[email protected]>: >>>> >>>>> Hi Fabian, >>>>> Yeah, that would be great :) >>>>> Thanks >>>>> >>>>> On Wed, Oct 22, 2014 at 12:06 AM, Fabian Hueske <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Mohit, >>>>>> >>>>>> I saw you created a JIRA issue for the Hadoop mapreduce function >>>>>> wrappers. >>>>>> Shall I assign the issue to you, so everybody knows you are working >>>>>> on it? >>>>>> >>>>>> Best, Fabian >>>>>> >>>>>> 2014-10-22 0:14 GMT+02:00 Fabian Hueske <[email protected]>: >>>>>> >>>>>>> Hi Mohit, >>>>>>> >>>>>>> that would be a good issue to start with. >>>>>>> Unfortunately, I assigned the issue to myself an started working on >>>>>>> it. However, I am not done yet. >>>>>>> If you like, you can pick up the issue and either continue with what >>>>>>> I did so far [1] or start all over. >>>>>>> Just let me know... >>>>>>> >>>>>>> Another option would be to extend the Hadoop Compatibility Layer. >>>>>>> Right now, we have wrappers for Hadoop's mapred-API function (Mapper, >>>>>>> Reducer), but not for the mapreduce-API functions [2]. Having wrappers >>>>>>> for >>>>>>> mapreduce-API functions would also be cool. There is no JIRA for this >>>>>>> issue >>>>>>> yet. >>>>>>> >>>>>>> And then there are of course plenty of other issues ;-) >>>>>>> >>>>>>> Cheers, Fabian >>>>>>> >>>>>>> [1] https://github.com/fhueske/incubator-flink/tree/multiCharDelim >>>>>>> [2] >>>>>>> http://flink.incubator.apache.org/docs/0.7-incubating/hadoop_compatibility.html >>>>>>> >>>>>>> >>>>>>> >>>>>>> 2014-10-21 23:56 GMT+02:00 Mohit Singh <[email protected]>: >>>>>>> >>>>>>>> Maybe I can start here: >>>>>>>> https://issues.apache.org/jira/browse/FLINK-1168 ?? >>>>>>>> >>>>>>>> On Tue, Oct 21, 2014 at 2:26 AM, Ufuk Celebi <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hey Mohit, >>>>>>>>> >>>>>>>>> On 21 Oct 2014, at 02:11, Mohit Singh <[email protected]> wrote: >>>>>>>>> >>>>>>>>> > Cool... Thanks for the update.. >>>>>>>>> > What would be a good way to start contributing into flink. >>>>>>>>> > I am comfortable with java but not so much on scala end but I >>>>>>>>> would love to pick it up as I go.. >>>>>>>>> > But basically, is there a good starting place for start pitching >>>>>>>>> in and contribute? >>>>>>>>> >>>>>>>>> Great to hear that you are interested in contributing. :) It is >>>>>>>>> not necessary to know Scala, because the core runtime is written in >>>>>>>>> Java >>>>>>>>> for the most part. >>>>>>>>> >>>>>>>>> As first steps I would suggest the following: >>>>>>>>> - Have a look at the contribution guide here [1]. >>>>>>>>> - We try to assign all issues to system components [2]. This might >>>>>>>>> help you to get an overview about which parts of the system are >>>>>>>>> interesting >>>>>>>>> to you. >>>>>>>>> >>>>>>>>> Does this help as a starting point? We can then discuss specific >>>>>>>>> issues here on the mailing list or in the respective issue. >>>>>>>>> >>>>>>>>> – Ufuk >>>>>>>>> >>>>>>>>> [1] http://flink.incubator.apache.org/how-to-contribute.html >>>>>>>>> [2] >>>>>>>>> https://issues.apache.org/jira/browse/FLINK?selectedTab=com.atlassian.jira.jira-projects-plugin:components-panel >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Mohit >>>>>>>> >>>>>>>> "When you want success as badly as you want the air, then you will >>>>>>>> get it. There is no other secret of success." >>>>>>>> -Socrates >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Mohit >>>>> >>>>> "When you want success as badly as you want the air, then you will get >>>>> it. There is no other secret of success." >>>>> -Socrates >>>>> >>>> >>>> >>> >>> >>> -- >>> Mohit >>> >>> "When you want success as badly as you want the air, then you will get >>> it. There is no other secret of success." >>> -Socrates >>> >> >> >
