Hi, Please think everything from UTC point of view. Your data set directory should be based on UTC as well. Only conversion that you will do when you specify the "start-time" which should be your start time time (based on your TZ) - 4 hours.
Now think from UTC perspective. For example, coordinator action at 7:00 UTC (11:00 your time) should use 6:00 UTC data as current(-1). I would request not to use current(-3). Again data directory should be based on UTC as well. I know these are confusing sometime. Regards, Mohammad On Sunday, October 13, 2013 6:09 AM, Serega Sheypak <[email protected]> wrote: I've changed <input-events> <data-in name="action_stg_in" dataset="action_stg"> <instance>*${coord:current(-1)}*</instance> </data-in> </input-events> <output-events> <data-out name="action_md_out" dataset="action_md"> <instance>*${coord:current(-1)}*</instance> </data-out> </output-events> to <input-events> <data-in name="action_stg_in" dataset="action_stg"> <instance>*${coord:current(3)}*</instance> </data-in> </input-events> <output-events> <data-out name="action_md_out" dataset="action_md"> <instance>*${coord:current(3)}*</instance> </data-out> </output-events> Now it works as expected, but this *${coord:current(3)} looks like magic number... Why 3, not -1 or 123? Because server has GMT+04:00 timezone?* 2013/10/13 Serega Sheypak <[email protected]> > Thank you very much. I did it the way you say, now I have other problem. ^( > > I want my coordinator to run each hour and take data from previous hour. > > The problem is that hour difference =5 hours. > > Imagine current time is 14:31 > My coordinator materializes each hour on third minute and takes dataset > from previous hour (some/hdfs/path/2103/10/12/*13*) > > But it materalized for hour 09 and does nothing else. I do understand why > difference is 5 hours = GMT+04:00 + 1 hour from corodinator settings (take > previous hour) > > I don't want 5 hour lag, I want 1 hour lag.What do I do wrong? > > The settings fot timezones are UTC: > > The part of coorinator is: > > <datasets> > <dataset name="action_stg" > frequency="${coord:hours(1)}" > initial-instance="2013-01-01T00:02Z" > timezone="${timezone}"> > > <uri-template>${nameNode}/staging/landing/stream/action/${YEAR}/${MONTH}/${DAY}/${HOUR}</uri-template> > <done-flag></done-flag> > </dataset> > <dataset name="action_md" > frequency="${coord:hours(1)}" > initial-instance="2013-01-01T00:02Z" > timezone="${timezone}"> > > <uri-template>${nameNode}/masterdata/stream/action/${YEAR}/${MONTH}/${DAY}/${HOUR}</uri-template> > <done-flag></done-flag> > </dataset> > </datasets> > > <input-events> > <data-in name="action_stg_in" dataset="action_stg"> > <instance>${coord:current(-1)}</instance> > </data-in> > </input-events> > > <output-events> > <data-out name="action_md_out" dataset="action_md"> > <instance>${coord:current(-1)}</instance> > </data-out> > </output-events> > > > > > > > 2013/10/13 Mohammad Islam <[email protected]> > >> Fully agreed with Robert. >> Reiterating .. >> Please use UTC in all cases. And define all time values (such as start >> time, end time , initial time) based on UTC. For your example, 00:03 local >> time is previous day 20:03 UTC time. >> >> "timezone"parameter doesn't have any impact in this case -- only for DST. >> >> Regards, >> Mohammad >> >> >> >> On Friday, October 11, 2013 6:21 PM, Robert Kanter <[email protected]> >> wrote: >> >> Hi Serega, >> >> Oozie always runs everything in "oozie.processing.timezone", which >> defaults >> to UTC. The "timezone" in the coordinator is a little misleading as it >> doesn't actually change the timezone; only the daylight savings time rules >> from this timezone are used. >> >> We typically recommend users to leave the "oozie.processing.timezone" at >> UTC and to do the math for setting the times in your coordinator (like >> Mohammad said). >> >> >> - Robert >> >> >> >> On Thu, Oct 10, 2013 at 2:56 AM, Serega Sheypak <[email protected] >> >wrote: >> >> > Yesterday I've set >> > oozie.processing.timezone = UTC (back, it's default value) >> > and set startTime=20:03 of previous day. >> > timezone for coordinator was: GMT+0400, GMT+0000, UTC >> > >> > Yes. It works. It really starts as 00:03 of the next day. >> > But really "timezone" value (GMT+0400, GMT+0000, UTC) didn't change >> > something. Oozie always adds +4 hours. >> > >> > Can you please explain this trick? >> > I just want to run coordinator each day at 00:03 why do I have to >> perform >> > such non evident hour calculations for each coordinator? >> > >> > Please explain. I don't understand the reason of complicity. Thank you. >> > >> > >> > >> > 2013/10/10 Mohammad Islam <[email protected]> >> > >> > > Can you please set two timezone values ( oozie.processing.timezone >> > > and timezone) to UTC? >> > > Basically don't specify any timezone value other than UTC. >> > > >> > > Then try with the following start times : startTime=20:03 <previous >> > > days>/startTime=00:03/startTime=04:03. >> > > Please let me know what you got from this three cases. >> > > >> > > After getting your response, i can explain to you (if needed). >> > > >> > > Regards, >> > > Mohammad >> > > >> > > >> > > >> > > >> > > ________________________________ >> > > From: Serega Sheypak <[email protected]> >> > > To: [email protected] >> > > Sent: Wednesday, October 9, 2013 12:06 PM >> > > Subject: HowTo make Oozie respect timezone? >> > > >> > > >> > > Hi, I completely don't understand how timezones work in oozie. >> > > Oozie server is in GMT+0400 >> > > Oozie timezone property: oozie.processing.timezone=UTC >> > > >> > > I want to run daily coordinator at 00:02 each day >> > > What do I have to set in coordinator fields? >> > > >> > > Right know I try to set >> > > startTime=00:03 >> > > timeZone=GMT+0400 >> > > >> > > And Oozie starts coordinator at 04:03. >> > > I've tried t set timezone of coordinator to UTC, GMT-0400, nothing >> helps. >> > > >> > > Anyway oozie starts coordinator at 04:03 >> > > What do I do wrong? >> > > >> > >> > >
