+ Falcon group Great comment Chris.
Biren. On Dec 4, 2015, at 12:51 PM, Chris Nauroth <[email protected]<mailto:[email protected]>> wrote: I'll just add a bit to Biren's response by saying that I personally find Falcon compelling as a user. I was a user of Hadoop before I became a developer on Hadoop. As a user, I found that most of my engineering effort went into figuring out how to get data into Hadoop and then figuring out how to get job results back out to other external systems. I wrote a lot of custom one-off code to do this for different projects. Eventually, I helped build a somewhat Falcon-like system to simplify the process of onboarding new data sets into my Hadoop cluster. Falcon has a richer feature set though compared to that system I helped build. If I was still in my prior role, I'd be giving Falcon a serious evaluation as a replacement. --Chris Nauroth From: Biren Saini <[email protected]<mailto:[email protected]>> Date: Friday, December 4, 2015 at 6:25 AM To: praveenesh kumar <[email protected]<mailto:[email protected]>>, Chris Nauroth <[email protected]<mailto:[email protected]>> Cc: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: RE: Falcon usecases Praveenesh, I am the Governance SME lead at Hortonworks which includes Falcon. Like Chris said falcon mailing list is a better group for this question but here is the crux of what you are looking for - We have many clients (including F100) who have been using Falcon in production very successfully. There are a ton on features in the roadmap that I am looking forward to. Falcon plays a critical role in the overall data governance story for Hadoop. Check out - hortonworks.com/hadoop/falcon<http://hortonworks.com/hadoop/falcon> for overview of the tool and more details. Here is a tutorial that will get you started - http://hortonworks.com/hadoop-tutorial/defining-processing-data-end-end-data-pipeline-apache-falcon/ Sample data pipeline built using Falcon in my github repo - https://github.com/sainib/hadoop-data-pipeline For any more follow up questions - please try the falcon distribution. Biren. -------- Original message -------- From: praveenesh kumar <[email protected]<mailto:[email protected]>> Date:12/04/2015 6:43 AM (GMT-05:00) To: Chris Nauroth <[email protected]<mailto:[email protected]>> Cc: [email protected]<mailto:[email protected]> Subject: Re: Falcon usecases Thanks Chris for pointing me to the mailing list and HDP support forums. However my question is more general and generic that is why I thought of putting it here. All I am trying to understand from anyone in the hadoop community who has encountered Falcon before to understand how the community is responding towards it. Does anyone using it or trying to use it. I can understand that falcon mailing list currently doesn't support user mailing list that is why I thought of putting this question here rather than subscribing to one more mailing list. @Chris - What is the reason HDP is backing it and delivering it in the HDP distribution? Do you see any future/current client use cases which kinds of highlighting its necessities. FYI - I am trying to be working on falcon for past 2 weeks and trying to understand it much better from the industry point of view, hence asking this to understand whether I am on a right path or its still a long way to go before falcon can be used as a production tool. On Wed, Dec 2, 2015 at 5:26 PM, Chris Nauroth <[email protected]<mailto:[email protected]>> wrote: Hello Prav, You might have better luck getting a response to this question by directly asking the Falcon community. I don't see a user@ mailing list for Falcon, but I do see a dev@ list. More details are here: http://falcon.apache.org/mail-lists.html For questions related specifically to HDP Sandbox, you'll likely get more help from Hortonworks support forums. (This is generally true for any vendor product that differentiates from the Apache distro.) I hope this helps. --Chris Nauroth From: praveenesh kumar <[email protected]<mailto:[email protected]>> Date: Wednesday, December 2, 2015 at 10:01 AM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Falcon usecases Hello hadoopers Just curious to understand what is the current state of falcon.. How much it is currently being adopted in the industry.. Anyone even using it other than the creators? There is not much information on the internet about falcon examples and use cases but then it is coming along in HDP distribution. Hence this question on understanding what are the best engineering/deployment principles around it ? I personally tried the GUI and it doesn't seems to be working properly on HDP sandbox 2.3.2, but that is another question to dig later. Before that I wanted to understand the current adoption of Falcon around big data industry. Anyone with any insights, please share..!! Regards Prav
