[
https://issues.apache.org/jira/browse/OOZIE-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Kanter resolved OOZIE-2018.
----------------------------------
Resolution: Fixed
> Coordinator materialization problems with cron syntax
> -----------------------------------------------------
>
> Key: OOZIE-2018
> URL: https://issues.apache.org/jira/browse/OOZIE-2018
> Project: Oozie
> Issue Type: Bug
> Components: coordinator
> Affects Versions: trunk, 4.0.0
> Reporter: Robert Kanter
>
> Suppose you submit the following coordinator job:
> {code:xml}
> <coordinator-app name="DailySleep"
> frequency="*/2 * * * *"
> start="2013-06-01T00:00Z" end="2013-06-05T00:00Z"
> timezone="America/Los_Angeles"
> xmlns="uri:oozie:coordinator:0.2"
> >
> <controls>
> <timeout>-1</timeout>
> <concurrency>1</concurrency>
> <execution>FIFO</execution>
> <throttle>2</throttle>
> </controls>
> <datasets>
> <dataset name="sleep_time" frequency="${coord:days(1)}"
> initial-instance="2012-05-31T00:00Z"
> timezone="America/Los_Angeles">
> <uri-template>${DAY}</uri-template>
> <done-flag></done-flag>
> </dataset>
> </datasets>
> <action>
> <workflow>
> <app-path>${wf_application_path}</app-path>
> <configuration>
> <property>
> <name>REDUCER_SLEEP_TIME</name>
> <value>120000</value>
> </property>
> <property>
> <name>oozie.use.system.libpath</name>
> <value>true</value>
> </property>
> </configuration>
> </workflow>
> </action>
> </coordinator-app>
> {code}
> Where {{$\{wf_application_path}}} points to a workflow that simply runs a
> sleep MR job for 2 mins.
> Notice that the above coordinator job is set to run with a frequency of {{*/2
> * * * *}}, which means every 2 minutes, and the throttle is 2.
> When you run this job, you’ll see a few anomalies:
> # Other than the first action, each action is materialized twice. The action
> numbering works fine, but you’ll see two actions for each Nominal Time. You
> can see this in the job info below.
> # You can’t see this in the job info below, but while it’s running, there are
> actually 3 jobs READY at the same time, when there should be only 2 (because
> throttle was set to 2)
> # OOZIE-1680 added an oozie-site config property
> {{oozie.service.coord.check.maximum.frequency=true}} which is supposed to
> block jobs with frequencies faster than 5 minutes; it didn’t stop this
> coordinator
> Points 1 and 2 above are likely the same problem. Point 3 is somewhat
> trivial.
> Here’s the job info (I killed the job before it finished, and I cut out
> non-relevent info to make it easier to read):
> {noformat}
> ---------------------------------------------------------------------------------------------------------------------------------------
> ID External ID
> Created Nominal Time
> ---------------------------------------------------------------------------------------------------------------------------------------
> 0000005-140922161548481-oozie-oozi-C@1
> 0000006-140922161548481-oozie-oozi-W 2014-09-22 23:34:38 GMT
> 2013-06-01 00:00:00 GMT
> ---------------------------------------------------------------------------------------------------------------------------------------
> 0000005-140922161548481-oozie-oozi-C@2
> 0000007-140922161548481-oozie-oozi-W 2014-09-22 23:34:38 GMT
> 2013-06-01 00:02:00 GMT
> ---------------------------------------------------------------------------------------------------------------------------------------
> 0000005-140922161548481-oozie-oozi-C@3
> 0000008-140922161548481-oozie-oozi-W 2014-09-22 23:36:11 GMT
> 2013-06-01 00:02:00 GMT
> ---------------------------------------------------------------------------------------------------------------------------------------
> 0000005-140922161548481-oozie-oozi-C@4
> 0000009-140922161548481-oozie-oozi-W 2014-09-22 23:36:11 GMT
> 2013-06-01 00:04:00 GMT
> ---------------------------------------------------------------------------------------------------------------------------------------
> 0000005-140922161548481-oozie-oozi-C@5
> 0000005-140922161548481-oozie-oozi-C 2014-09-22 23:41:11 GMT
> 2013-06-01 00:04:00 GMT
> ---------------------------------------------------------------------------------------------------------------------------------------
> 0000005-140922161548481-oozie-oozi-C@6
> 0000005-140922161548481-oozie-oozi-C 2014-09-22 23:41:11 GMT
> 2013-06-01 00:06:00 GMT
> ---------------------------------------------------------------------------------------------------------------------------------------
> {noformat}
> I tried the same coordinator job, but used the old frequency syntax
> ({{$\{coord:minutes(2)}}}, and even though we don’t recommend a 2 min
> frequency, it actually worked correctly (once I set
> {{oozie.service.coord.check.maximum.frequency=false}} of course). So this
> appears to be a problem with the cron syntax. If ({{$\{coord:minutes(2)}}}
> didn’t work either, then I’d say it’s just once of the quirks of too high a
> frequency, but that’s not the case here.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
