[
https://issues.apache.org/jira/browse/OOZIE-1336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey updated OOZIE-1336:
--------------------------
Description:
Our servers do live in Europe/Moscow timezone.
The problem is that oozie coordinator is always 4 hours before nominal time of
datasets.
Here is the sample of coordinator workflow:
{code:title=coord.xml|borderStyle=solid}
<coordinator-app name="Url-rating-coordinator" frequency="60"
start="${httpCoordStartTime}" end="2113-01-01T00:02Z"
timezone="${httpCoordTimeZone}" xmlns="uri:oozie:coordinator:0.3">
<controls>
<timeout>2</timeout>
<concurrency>1</concurrency>
</controls>
<datasets>
<dataset name="urlRatingInputDataset" frequency="${coord:hours(1)}"
initial-instance="${httpCoordStartTime}" timezone="${httpCoordTimeZone}">
<uri-template>${nameNode}/staging/landing/source/protei/http/${YEAR}/${MONTH}/${DAY}/${HOUR}</uri-template>
<done-flag></done-flag>
</dataset>
<dataset name="urlRatingOutputDataset" frequency="${coord:hours(1)}"
initial-instance="${httpCoordStartTime}" timezone="${httpCoordTimeZone}">
<uri-template>${nameNode}/masterdata/source/protei/http/archive/${YEAR}/${MONTH}/${DAY}/${HOUR}</uri-template>
<done-flag></done-flag>
</dataset>
</datasets>
{code}
Here are the parameters:
httpCoordStartTime=2013-04-21T16:02Z
httpCoordTimeZone=GMT+04:00
Imagine: Right now is 16-22 in Moscow.
The input dataset folder is ready to be processed:
{code:xml}
/staging/landing/source/protei/http/${YEAR}/${MONTH}/${DAY}/16
{code}
I do submit oozie coordinator with mentioned parameters.
I do expect that Oozie will take input folder
{code:xml}
/staging/landing/source/protei/http/${YEAR}/${MONTH}/${DAY}/16 and process
{code}
it. But it doesn't do it. It says:
{code:xml}
2013-04-21 16:22:45,868 INFO
org.apache.oozie.command.coord.CoordSubmitXCommand: USER[hdfs] GROUP[-] TOKEN[]
APP[Url-rating-coordinator] JOB[0000334-130410234028321-oozie-oozi-C] ACTION[-]
ENDED Coordinator Submit jobId=0000334-130410234028321-oozie-oozi-C
2013-04-21 16:22:45,972 WARN
org.apache.oozie.command.coord.CoordMaterializeTransitionXCommand: USER[hdfs]
GROUP[-] TOKEN[] APP[Url-rating-coordinator]
JOB[0000334-130410234028321-oozie-oozi-C] ACTION[-] E1100: Command precondition
does not hold before execution, [CoordMaterializeTransitionXCommand for
jobId=0000334-130410234028321-oozie-oozi-C job's start time is not reached yet
- nothing to materialize], Error Code: E1100
{code}
Ok, I try to change params to (roll start time 4 hours back):
httpCoordStartTime=2013-04-21T12:02Z
httpCoordTimeZone=GMT+04:00
Ooize happily starts to process input folder
{code:xml}
/staging/landing/source/protei/http/${YEAR}/${MONTH}/${DAY}/12
{code}
The same situation, it always for hours before the time I want it to pocess the
data.
Is it a bug?
was:
Our servers do live in Europe/Moscow timezone.
The problem is that oozie coordinator is always 4 hours before nominal time of
datasets.
Here is the sample of coordinator workflow:
<coordinator-app name="Url-rating-coordinator" frequency="60"
start="${httpCoordStartTime}" end="2113-01-01T00:02Z"
timezone="${httpCoordTimeZone}" xmlns="uri:oozie:coordinator:0.3">
<controls>
<timeout>2</timeout>
<concurrency>1</concurrency>
</controls>
<datasets>
<dataset name="urlRatingInputDataset" frequency="${coord:hours(1)}"
initial-instance="${httpCoordStartTime}" timezone="${httpCoordTimeZone}">
<uri-template>${nameNode}/staging/landing/source/protei/http/${YEAR}/${MONTH}/${DAY}/${HOUR}</uri-template>
<done-flag></done-flag>
</dataset>
<dataset name="urlRatingOutputDataset" frequency="${coord:hours(1)}"
initial-instance="${httpCoordStartTime}" timezone="${httpCoordTimeZone}">
<uri-template>${nameNode}/masterdata/source/protei/http/archive/${YEAR}/${MONTH}/${DAY}/${HOUR}</uri-template>
<done-flag></done-flag>
</dataset>
</datasets>
Here are the parameters:
httpCoordStartTime=2013-04-21T16:02Z
httpCoordTimeZone=GMT+04:00
Imagine: Right now is 16-22 in Moscow.
The input dataset folder is ready to be processed:
/staging/landing/source/protei/http/${YEAR}/${MONTH}/${DAY}/16
I do submit oozie coordinator with mentioned parameters.
I do expect that Oozie will take input folder
/staging/landing/source/protei/http/${YEAR}/${MONTH}/${DAY}/16 and process it.
But it doesn't do it. It says:
2013-04-21 16:22:45,868 INFO
org.apache.oozie.command.coord.CoordSubmitXCommand: USER[hdfs] GROUP[-] TOKEN[]
APP[Url-rating-coordinator] JOB[0000334-130410234028321-oozie-oozi-C] ACTION[-]
ENDED Coordinator Submit jobId=0000334-130410234028321-oozie-oozi-C
2013-04-21 16:22:45,972 WARN
org.apache.oozie.command.coord.CoordMaterializeTransitionXCommand: USER[hdfs]
GROUP[-] TOKEN[] APP[Url-rating-coordinator]
JOB[0000334-130410234028321-oozie-oozi-C] ACTION[-] E1100: Command precondition
does not hold before execution, [CoordMaterializeTransitionXCommand for
jobId=0000334-130410234028321-oozie-oozi-C job's start time is not reached yet
- nothing to materialize], Error Code: E1100
Ok, I try to change params to (roll start time 4 hours back):
httpCoordStartTime=2013-04-21T12:02Z
httpCoordTimeZone=GMT+04:00
Ooize happily starts to process input folder
/staging/landing/source/protei/http/${YEAR}/${MONTH}/${DAY}/12
The same situation, it always for hours before the time I want it to pocess the
data.
Is it a bug?
> Can't force Oozie to work in local time
> ---------------------------------------
>
> Key: OOZIE-1336
> URL: https://issues.apache.org/jira/browse/OOZIE-1336
> Project: Oozie
> Issue Type: Bug
> Components: coordinator
> Affects Versions: 3.3.2
> Reporter: Sergey
> Labels: GMT, UTC
>
> Our servers do live in Europe/Moscow timezone.
> The problem is that oozie coordinator is always 4 hours before nominal time
> of datasets.
> Here is the sample of coordinator workflow:
> {code:title=coord.xml|borderStyle=solid}
> <coordinator-app name="Url-rating-coordinator" frequency="60"
> start="${httpCoordStartTime}" end="2113-01-01T00:02Z"
> timezone="${httpCoordTimeZone}" xmlns="uri:oozie:coordinator:0.3">
> <controls>
> <timeout>2</timeout>
> <concurrency>1</concurrency>
> </controls>
> <datasets>
> <dataset name="urlRatingInputDataset" frequency="${coord:hours(1)}"
> initial-instance="${httpCoordStartTime}" timezone="${httpCoordTimeZone}">
>
> <uri-template>${nameNode}/staging/landing/source/protei/http/${YEAR}/${MONTH}/${DAY}/${HOUR}</uri-template>
> <done-flag></done-flag>
> </dataset>
> <dataset name="urlRatingOutputDataset" frequency="${coord:hours(1)}"
> initial-instance="${httpCoordStartTime}" timezone="${httpCoordTimeZone}">
>
> <uri-template>${nameNode}/masterdata/source/protei/http/archive/${YEAR}/${MONTH}/${DAY}/${HOUR}</uri-template>
> <done-flag></done-flag>
> </dataset>
> </datasets>
> {code}
> Here are the parameters:
> httpCoordStartTime=2013-04-21T16:02Z
> httpCoordTimeZone=GMT+04:00
> Imagine: Right now is 16-22 in Moscow.
> The input dataset folder is ready to be processed:
> {code:xml}
> /staging/landing/source/protei/http/${YEAR}/${MONTH}/${DAY}/16
> {code}
> I do submit oozie coordinator with mentioned parameters.
> I do expect that Oozie will take input folder
> {code:xml}
> /staging/landing/source/protei/http/${YEAR}/${MONTH}/${DAY}/16 and process
> {code}
> it. But it doesn't do it. It says:
> {code:xml}
> 2013-04-21 16:22:45,868 INFO
> org.apache.oozie.command.coord.CoordSubmitXCommand: USER[hdfs] GROUP[-]
> TOKEN[] APP[Url-rating-coordinator] JOB[0000334-130410234028321-oozie-oozi-C]
> ACTION[-] ENDED Coordinator Submit jobId=0000334-130410234028321-oozie-oozi-C
> 2013-04-21 16:22:45,972 WARN
> org.apache.oozie.command.coord.CoordMaterializeTransitionXCommand: USER[hdfs]
> GROUP[-] TOKEN[] APP[Url-rating-coordinator]
> JOB[0000334-130410234028321-oozie-oozi-C] ACTION[-] E1100: Command
> precondition does not hold before execution,
> [CoordMaterializeTransitionXCommand for
> jobId=0000334-130410234028321-oozie-oozi-C job's start time is not reached
> yet - nothing to materialize], Error Code: E1100
> {code}
> Ok, I try to change params to (roll start time 4 hours back):
> httpCoordStartTime=2013-04-21T12:02Z
> httpCoordTimeZone=GMT+04:00
> Ooize happily starts to process input folder
> {code:xml}
> /staging/landing/source/protei/http/${YEAR}/${MONTH}/${DAY}/12
> {code}
> The same situation, it always for hours before the time I want it to pocess
> the data.
> Is it a bug?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
