Am guessing venkat you meant to refer FALCON-1555. Good check list to go through for each release.
Specifically on >> 6. Multi cluster managed with Oozie (with single Falcon instance) Since we haven't particularly done anything by isolating the classes and their dependencies for each cluster with independent class loader, I think this would work only as long as all the clusters are compatible. If things were to change say with hadoop-3 for ex, this may no longer work. Regards Srikanth Sundarrajan > From: [email protected] > Date: Sun, 25 Oct 2015 05:29:20 +0530 > Subject: Re: Oozie uber mode and other modes of running the software > To: [email protected] > > Venkat, > > Sounds good, let's discuss it in the next sync up. For the benefit of > everyone on the mailing list, I will summarise our use cases. Yes, we > (Inmobi) use Oozie's uber mode in our production environment(IIRC we are > using it since Apache Falcon 0.7). We do have several thousand jobs running > per day(several at minutely frequencies) and a heavily utilised cluster, so > this makes sense for us. We do test all Oozie releases, though sometimes > our Oozie releases are not exactly aligned with Oozie's general release - > it might contain some extra patches, might be from trunk etc. and our > testing might be specific to features being used at Inmobi. In fact last > time Pragya Mittal did the testing for both Apache Falcon 0.7 and Oozie. > She can add more on this. > > List of scenarios looks comprehensive. Just curious, what storage backends > are other users using for titan? > > > Cheers > Ajay Yadava > > On Sat, Oct 24, 2015 at 9:20 PM, Venkat Ranganathan < > [email protected]> wrote: > > > FALCON-1551 was interesting for two reasons - that we are still caught by > > the FS caching issue (this should have been dealt by most of the projects > > one time or another), but the use of Oozie uber mode - I am wondering if we > > have done uber mode testing generally - Is this the mode that is used in > > your environments – It makes sense when thousands of Oozie jobs are run – > > saving one container. I would like to understand more about your > > experience with uber mode if you have – may be in the next Falcon sync up. > > > > I think we have create a list of modes that we expect the product to run > > before we introduce changes generally – Given that there are different sets > > of execution modes that we want to test. Please add more on the scenarios > > that I am most likely missing. Let us discuss this also if we have time. > > > > 1. Security (Kerberos and Wire encryption) > > 2. NN HA/RM HA > > 3. Oozie HA > > 4. Oozie ubermode for launcher job > > 5. Multi cluster with prism > > 6. Multi cluster managed with Oozie (with single Falcon instance) > > 7. Hadoop/Hive/Oozie versions and the implications (S3 and Azure blob > > storage went out of hadoop-common in 2.6.0+ and the implications). > > > > > > Thanks > > > > Venkat > > > > > > -- > _____________________________________________________________ > The information contained in this communication is intended solely for the > use of the individual or entity to whom it is addressed and others > authorized to receive it. It may contain confidential or legally privileged > information. If you are not the intended recipient you are hereby notified > that any disclosure, copying, distribution or taking any action in reliance > on the contents of this information is strictly prohibited and may be > unlawful. If you have received this communication in error, please notify > us immediately by responding to this email and then delete it from your > system. The firm is neither liable for the proper and complete transmission > of the information contained in this communication nor for any delay in its > receipt.
