My coordinators all have the dataset initial instance hardcoded to a couple years ago when I started working with oozie. As far as I know this doesn't pose any real issues.
-paul -----Original Message----- From: anil gupta [mailto:[email protected]] Sent: Monday, January 05, 2015 2:30 PM To: [email protected] Subject: Re: Dataset uri-template "${YEAR}/${MONTH}/${DAY}/${HOUR}" in PST timezone? Yup, initial instance was the issue. After making sure that initial instance is atleast 8 hour before starttime. Now, "missing deps" has a value. Thanks a lot, Paul! But now, i need to make sure that that initial instance is 8 hours before starttime. Is there any syntax in oozie to express this condition? Otherwise, i will need to do some manual stuff. How do you do it in your apps? Sorry, to ask so many things. I am still learning Oozie. On Mon, Jan 5, 2015 at 2:15 PM, Paul Chavez <[email protected]> wrote: > I have not used a negative value that large but most of my > coordinators use that EL function with either -8 or -32 (or -7 and -28 > depending on time of year ;) > > The first thing that comes to mind is to make sure your dataset > initial instance start date is older than the first instance you will > try to reference. You cannot reference an instane before the dataset > initial-instance value. > > -paul > > -----Original Message----- > From: anil gupta [mailto:[email protected]] > Sent: Monday, January 05, 2015 2:10 PM > To: [email protected] > Subject: Re: Dataset uri-template "${YEAR}/${MONTH}/${DAY}/${HOUR}" in > PST timezone? > > Nice trick, Paul. I tried doing > <instance>${coord:current(-96)}</instance> > in my data-in element. > *Unfortunately, the oozie job starts erroring out with the following > error:* > > > > > > > > *2015-01-05 13:50:58,853 INFO CoordActionNotificationXCommand:539 - > USER[-] GROUP[-] TOKEN[-] APP[-] > JOB[0003458-141103151054529-oozie-oozi-C] > ACTION[0003458-141103151054529-oozie-oozi-C@1] STARTED Coordinator > Notification actionId=0003458-141103151054529-oozie-oozi-C@1 : > WAITING2015-01-05 13:50:58,854 INFO > CoordActionNotificationXCommand:539 - USER[-] GROUP[-] TOKEN[-] APP[-] > JOB[0003458-141103151054529-oozie-oozi-C] > ACTION[0003458-141103151054529-oozie-oozi-C@1] No Notification URL is > defined. Therefore nothing to notify for job > 0003458-141103151054529-oozie-oozi-C action ID > 0003458-141103151054529-oozie-oozi-C@12015-01-05 13:50:58,854 INFO > CoordActionNotificationXCommand:539 - USER[-] GROUP[-] TOKEN[-] APP[-] > JOB[0003458-141103151054529-oozie-oozi-C] > ACTION[0003458-141103151054529-oozie-oozi-C@1] ENDED Coordinator > Notification actionId=0003458-141103151054529-oozie-oozi-C@12015-01-05 > 13:50:58,857 INFO CoordActionInputCheckXCommand:539 - USER[-] > GROUP[-] TOKEN[-] APP[-] JOB[0003458-141103151054529-oozie-oozi-C] > ACTION[0003458-141103151054529-oozie-oozi-C@1] > [0003458-141103151054529-oozie-oozi-C@1]::ActionInputCheck:: Action is > in WAITING state.2015-01-05 13:50:58,858 INFO > CoordActionInputCheckXCommand:539 - USER[-] GROUP[-] TOKEN[-] APP[-] > JOB[0003458-141103151054529-oozie-oozi-C] > ACTION[0003458-141103151054529-oozie-oozi-C@1] > [0003458-141103151054529-oozie-oozi-C@1]::CoordActionInputCheck:: > Missing > deps: 2015-01-05 13:50:58,859 INFO CoordActionInputCheckXCommand:539 > - USER[-] GROUP[-] TOKEN[-] APP[-] > JOB[0003458-141103151054529-oozie-oozi-C] > ACTION[0003458-141103151054529-oozie-oozi-C@1] > [0003458-141103151054529-oozie-oozi-C@1]::ActionInputCheck:: In > checkResolvedUris...2015-01-05 13:50:59,117 WARN > ParameterVerifier:542 - USER[-] GROUP[-] TOKEN[-] APP[-] > JOB[0003458-141103151054529-oozie-oozi-C] > ACTION[0003458-141103151054529-oozie-oozi-C@1] The application does > not define formal parameters in its XML definition2015-01-05 > 13:50:59,268 INFO > CoordActionUpdateXCommand:539 - USER[-] GROUP[-] TOKEN[-] APP[-] > JOB[0003458-141103151054529-oozie-oozi-C] > ACTION[0003458-141103151054529-oozie-oozi-C@1] Updating Coordintaor > action id :0003458-141103151054529-oozie-oozi-C@1 status to FAILED, > pending = 0* > > > *When i used current(0), **"Missing deps:" also has a value(ex: * > > */user/oozie//2015/01/05/20/_READY ).It seems like oozie is not liking > current(-96). Sounds like a bug to me now. Have you really tried using > a negative value like that?* Thanks, Anil Gupta > > On Mon, Jan 5, 2015 at 1:31 PM, Paul Chavez <[email protected]> wrote: > > > When you reference the data set for your data-in or data-out > > elements, use the EL function coord:current() to reference the right > > instance. I also use Pacific time so my coordinators use data-in > > elements that look > like this: > > > > <data-in name="CleanResult" dataset="Results"> > > <instance>${coord:current(-32)}</instance> > > </data-in> > > > > In that example the coordinator is run on 15 minute intervals and I > > want 8 hours 'behind' UTC so 4*8 = 32 is how many instances to go 'back'. > > > > Note that this requires resubmittal of the coordinator when the time > > changes, there is a time zone offset EL function that could be used > > but it is broken in the oozie version I have but apparently fixed as > > of v4.0. ( > > https://issues.apache.org/jira/browse/OOZIE-1573) > > > > Hope that helps, > > Paul > > > > > > -----Original Message----- > > From: anil gupta [mailto:[email protected]] > > Sent: Monday, January 05, 2015 1:17 PM > > To: [email protected] > > Subject: Dataset uri-template "${YEAR}/${MONTH}/${DAY}/${HOUR}" in > > PST timezone? > > > > Hi > > > > I have a dataset "A" that has an uri > > /user/oozie/${YEAR}/${MONTH}/${DAY}/${HOUR}. > > <dataset name="inv" frequency="5" initial-instance="${startDate}" > > timezone="America/Los_Angeles"> > > > > <uri-template>/user/oozie/${YEAR}/${MONTH}/${DAY}/${HOUR}</uri-template> > > <done-flag>_READY</done-flag> > > </dataset> > > > > When the coordinator is resolving time for > > "${YEAR}/${MONTH}/${DAY}/${HOUR}", its resolving it in UTC. I want > > the coordinator to consider America/Los_Angeles time while building > > the folder path. > > Can you please let me know how to do this? > > > > Here is the definition of coordinator app: > > <coordinator-app name="proc-coordinator" > > timezone="America/Los_Angeles" > > frequency="5" > > start="${startDate}" > > end="${endDate}" > > xmlns="uri:oozie:coordinator:0.2"> > > > > > > > > -- > > Thanks & Regards, > > Anil Gupta > > > > > > -- > Thanks & Regards, > Anil Gupta > -- Thanks & Regards, Anil Gupta
