Typo : Oozie version is 4.2 and RM HA testing is still in progress. On Sun, Oct 25, 2015 at 9:25 AM, pragya mittal <[email protected]> wrote:
> Hi Venkat, > > From past three months, Inmobi has been using Oozie in uber mode. > Currently we are using falcon-0.7 with oozie-3.2 (along with several > patches specific to our use case). Following use cases have been tested and > are active in our production environment. > > 1. NN HA/RM HA > 2. Oozie ubermode for launcher job > 3. Multi cluster with prism > 4. Sanity of falcon features with oozie(in uber mode). > > After falcon-0.8 release we plan to go forward with Oozie HA feature. > > Please let me know if you have any more concerns. > > Regards, > Pragya Mittal > > On Sun, Oct 25, 2015 at 8:15 AM, Srikanth Sundarrajan <[email protected] > > wrote: > >> Am guessing venkat you meant to refer FALCON-1555. Good check list to go >> through for each release. >> >> Specifically on >> >> 6. Multi cluster managed with Oozie (with single Falcon instance) >> >> Since we haven't particularly done anything by isolating the classes and >> their dependencies for each cluster with independent class loader, I think >> this would work only as long as all the clusters are compatible. If things >> were to change say with hadoop-3 for ex, this may no longer work. >> >> Regards >> Srikanth Sundarrajan >> >> > From: [email protected] >> > Date: Sun, 25 Oct 2015 05:29:20 +0530 >> > Subject: Re: Oozie uber mode and other modes of running the software >> > To: [email protected] >> > >> > Venkat, >> > >> > Sounds good, let's discuss it in the next sync up. For the benefit of >> > everyone on the mailing list, I will summarise our use cases. Yes, we >> > (Inmobi) use Oozie's uber mode in our production environment(IIRC we are >> > using it since Apache Falcon 0.7). We do have several thousand jobs >> running >> > per day(several at minutely frequencies) and a heavily utilised >> cluster, so >> > this makes sense for us. We do test all Oozie releases, though sometimes >> > our Oozie releases are not exactly aligned with Oozie's general release >> - >> > it might contain some extra patches, might be from trunk etc. and our >> > testing might be specific to features being used at Inmobi. In fact last >> > time Pragya Mittal did the testing for both Apache Falcon 0.7 and Oozie. >> > She can add more on this. >> > >> > List of scenarios looks comprehensive. Just curious, what storage >> backends >> > are other users using for titan? >> > >> > >> > Cheers >> > Ajay Yadava >> > >> > On Sat, Oct 24, 2015 at 9:20 PM, Venkat Ranganathan < >> > [email protected]> wrote: >> > >> > > FALCON-1551 was interesting for two reasons - that we are still >> caught by >> > > the FS caching issue (this should have been dealt by most of the >> projects >> > > one time or another), but the use of Oozie uber mode - I am wondering >> if we >> > > have done uber mode testing generally - Is this the mode that is used >> in >> > > your environments – It makes sense when thousands of Oozie jobs are >> run – >> > > saving one container. I would like to understand more about your >> > > experience with uber mode if you have – may be in the next Falcon >> sync up. >> > > >> > > I think we have create a list of modes that we expect the product to >> run >> > > before we introduce changes generally – Given that there are >> different sets >> > > of execution modes that we want to test. Please add more on the >> scenarios >> > > that I am most likely missing. Let us discuss this also if we have >> time. >> > > >> > > 1. Security (Kerberos and Wire encryption) >> > > 2. NN HA/RM HA >> > > 3. Oozie HA >> > > 4. Oozie ubermode for launcher job >> > > 5. Multi cluster with prism >> > > 6. Multi cluster managed with Oozie (with single Falcon instance) >> > > 7. Hadoop/Hive/Oozie versions and the implications (S3 and Azure >> blob >> > > storage went out of hadoop-common in 2.6.0+ and the implications). >> > > >> > > >> > > Thanks >> > > >> > > Venkat >> > > >> > > >> > >> > -- >> > _____________________________________________________________ >> > The information contained in this communication is intended solely for >> the >> > use of the individual or entity to whom it is addressed and others >> > authorized to receive it. It may contain confidential or legally >> privileged >> > information. If you are not the intended recipient you are hereby >> notified >> > that any disclosure, copying, distribution or taking any action in >> reliance >> > on the contents of this information is strictly prohibited and may be >> > unlawful. If you have received this communication in error, please >> notify >> > us immediately by responding to this email and then delete it from your >> > system. The firm is neither liable for the proper and complete >> transmission >> > of the information contained in this communication nor for any delay in >> its >> > receipt. >> >> > >
