Toby, your 2.5 release doesn't seem right. Latest 2.5 release is 2.5.3. here are the changes for Oozie in 2.5 http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_release-notes/content/patch_oozie.html
And http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_release-notes/content/fixed_issues.html I'm at Summit, stop by our booth and say Hi. On Apr 5, 2017 4:28 PM, "Venkat Ranganathan" <[email protected]> wrote: > Toby > > For HDP specific questions, please visit https://community.hortonworks.com > > Thanks > > Venkat > ________________________________________ > From: Toby Evans <[email protected]> > Sent: Wednesday, April 05, 2017 6:00 AM > To: [email protected] > Subject: Re: Immutable vs mutable parameters in Oozie 4.2.0 > > Ok, will do, currently at Munich for DataSummit, on it when i get back > > Cheers > > Toby > > On Wed, 5 Apr 2017, 09:27 Andras Piros, <[email protected]> wrote: > > Hi Toby, > > thanks for your interest on using Oozie! > > In order I can help you further, can you please: > > - add *Apache GitHub <https://github.com/apache/oozie>* git hashes of > Oozie versions for each HDP version > - add a small example of two workflows, one sets a parameter, the other > reads the changed parameter - that was working in HDP versions 2.3 and > 2.4, > but not anymore in 2.5? > > Regards, > > Andras > > On Tue, Apr 4, 2017 at 4:38 PM, Toby Evans <[email protected]> wrote: > > > Hi there, > > > > We've got a big Oozie pipeline, which I'm having a little difficulty > > upgrading from Hortonworks 2.3 > 2.4 > 2.5 > > > > HDP Oozie > > 2.3 4.2.0.2.3.2.0-2950 > > 2.4 4.2.0.2.4.2.0-258 > > 2.5 4.2.0.2.5.0.0-1245 > > > > > > So, we ingest and process data from a range of sources, eg databases, > FTP, > > SQL dbs etc. These are all different, but share so many common features > > that we have grown various heavily templated workflows. Each of our > > workflows has a set of parameters (eg date range, source type, > > hasCustomStep) all that sort of thing, all stored in "profile" workflows. > > We first call our "profile" workflow, which sets up the various > parameters > > and then calls the template workflow, which then fills in the values > > supplied from the profile. > > > > This has run fine for a couple of years in HDP 2.3, seems OK in 2.4, but > > definitely doesn't in 2.5. The problem is that once a parameter has been > > set in a workflow chain, then it stays set. I understand that you don't > > want to be changing a parameter inside the same workflow, and we've > > refactored so that we don't have a parameter value passed to a workflow, > > which then changes the parameter value then calls another sub-workflow. > > However, we're getting caught out by parameters being set by a workflow > and > > then not being reset much further down the pipeline when used by a > > different "profile", which should be isolated but clearly isn't > > > > Has there been a code change between the versions, and is there a config > > setting that allows us to switch off cross-workflow parameter > immutability? > > > > thanks > > > > Toby > > >
