[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16461119#comment-16461119 ] Hadoop QA commented on OOZIE-2494: -- Testing JIRA OOZIE-2494 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:red}-1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:red}-1{color} the patch contains 12 line(s) longer than 132 characters .{color:green}+1{color} the patch adds/modifies 1 testcase(s) {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} the patch does not seem to introduce new Javadoc warnings {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:green}+1{color} There are no new bugs found in total. . {color:green}+1{color} There are no new bugs found in [examples]. . {color:green}+1{color} There are no new bugs found in [webapp]. . {color:green}+1{color} There are no new bugs found in [core]. . {color:green}+1{color} There are no new bugs found in [tools]. . {color:green}+1{color} There are no new bugs found in [server]. . {color:green}+1{color} There are no new bugs found in [docs]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive2]. . {color:green}+1{color} There are no new bugs found in [sharelib/pig]. . {color:green}+1{color} There are no new bugs found in [sharelib/streaming]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive]. . {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. . {color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. . {color:green}+1{color} There are no new bugs found in [sharelib/oozie]. . {color:green}+1{color} There are no new bugs found in [sharelib/distcp]. . {color:green}+1{color} There are no new bugs found in [sharelib/spark]. . {color:green}+1{color} There are no new bugs found in [client]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:red}-1 TESTS{color} .Tests run: 2125 .Tests failed: 1 .Tests errors: 1 .The patch failed the following testcases: testCoordActionRecoveryServiceForWaitingRegisterPartition(org.apache.oozie.service.TestRecoveryService) .Tests failing with errors: testConnectionRetry(org.apache.oozie.service.TestJMSAccessorService) .{color:orange}Tests failed at first run:{color} TestJavaActionExecutor#testCredentialsSkip TestOozieDBCLI#testOozieDBCLI .For the complete list of flaky tests, see TEST-SUMMARY-FULL files. {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:red}*-1 Overall result, please check the reported -1(s)*{color} The full output of the test-patch run is available at . https://builds.apache.org/job/PreCommit-OOZIE-Build/502/ > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0, 5.0.0 >Reporter: Dennis Pallett >Assignee: Julia Kinga Marton >Priority: Blocker > Attachments: CoordinatorFunctionalSpec.html, CronExpressionPOC.java, > OOZIE-2494-001.patch, OOZIE-2494-002.patch, OOZIE-2494-003.patch, > OOZIE-2494-004.patch, OOZIE-2494-005.patch, OOZIE-2494-006.patch, > OOZIE-2494-007.patch, OOZIE-2494-008.patch, OOZIE-2494-009.patch, > OOZIE-2494-010.patch, testActionMaterWithDST3.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16461021#comment-16461021 ] Hadoop QA commented on OOZIE-2494: -- Testing JIRA OOZIE-2494 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:red}-1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:red}-1{color} the patch contains 11 line(s) longer than 132 characters .{color:green}+1{color} the patch adds/modifies 1 testcase(s) {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} the patch does not seem to introduce new Javadoc warnings {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:green}+1{color} There are no new bugs found in total. . {color:green}+1{color} There are no new bugs found in [examples]. . {color:green}+1{color} There are no new bugs found in [webapp]. . {color:green}+1{color} There are no new bugs found in [core]. . {color:green}+1{color} There are no new bugs found in [tools]. . {color:green}+1{color} There are no new bugs found in [server]. . {color:green}+1{color} There are no new bugs found in [docs]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive2]. . {color:green}+1{color} There are no new bugs found in [sharelib/pig]. . {color:green}+1{color} There are no new bugs found in [sharelib/streaming]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive]. . {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. . {color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. . {color:green}+1{color} There are no new bugs found in [sharelib/oozie]. . {color:green}+1{color} There are no new bugs found in [sharelib/distcp]. . {color:green}+1{color} There are no new bugs found in [sharelib/spark]. . {color:green}+1{color} There are no new bugs found in [client]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:red}-1 TESTS{color} .Tests run: 2125 .Tests failed: 1 .Tests errors: 1 .The patch failed the following testcases: testCoordActionRecoveryServiceForWaitingRegisterPartition(org.apache.oozie.service.TestRecoveryService) .Tests failing with errors: testConnectionRetry(org.apache.oozie.service.TestJMSAccessorService) .{color:orange}Tests failed at first run:{color} TestJavaActionExecutor#testCredentialsSkip TestOozieDBCLI#testOozieDBCLI .For the complete list of flaky tests, see TEST-SUMMARY-FULL files. {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:red}*-1 Overall result, please check the reported -1(s)*{color} The full output of the test-patch run is available at . https://builds.apache.org/job/PreCommit-OOZIE-Build/501/ > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0, 5.0.0 >Reporter: Dennis Pallett >Assignee: Julia Kinga Marton >Priority: Blocker > Attachments: CoordinatorFunctionalSpec.html, CronExpressionPOC.java, > OOZIE-2494-001.patch, OOZIE-2494-002.patch, OOZIE-2494-003.patch, > OOZIE-2494-004.patch, OOZIE-2494-005.patch, OOZIE-2494-006.patch, > OOZIE-2494-007.patch, OOZIE-2494-008.patch, OOZIE-2494-009.patch, > OOZIE-2494-010.patch, testActionMaterWithDST3.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16461023#comment-16461023 ] Hadoop QA commented on OOZIE-2494: -- PreCommit-OOZIE-Build started > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0, 5.0.0 >Reporter: Dennis Pallett >Assignee: Julia Kinga Marton >Priority: Blocker > Attachments: CoordinatorFunctionalSpec.html, CronExpressionPOC.java, > OOZIE-2494-001.patch, OOZIE-2494-002.patch, OOZIE-2494-003.patch, > OOZIE-2494-004.patch, OOZIE-2494-005.patch, OOZIE-2494-006.patch, > OOZIE-2494-007.patch, OOZIE-2494-008.patch, OOZIE-2494-009.patch, > OOZIE-2494-010.patch, testActionMaterWithDST3.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: el-LA > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 00:10 GMT 2016-03-13 17:10 PDT > {noformat} > As you can see, @2’s nominal time is adjusted to an hour earlier in the > "el-LA" job, but not in the "el-UTC" job. > However, when running a similar set of jobs, but using cron syntax [({{10 1 > 1/1 * *}}|http://crontab.guru/#10_1_1/1_*_*], which indicates 1:10 every day > of every month), this isn’t the case: > {noformat} > Job Name: cron-UTC > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: cron-LA > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > As you can see, @2’s nominal time are the same in both the "cron-UTC" and > "cron-LA" jobs. The "cron-LA" job should have the same nominal time as the > "el-LA" job from earlier. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16460921#comment-16460921 ] Hadoop QA commented on OOZIE-2494: -- PreCommit-OOZIE-Build started > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0, 5.0.0 >Reporter: Dennis Pallett >Assignee: Julia Kinga Marton >Priority: Blocker > Attachments: CoordinatorFunctionalSpec.html, CronExpressionPOC.java, > OOZIE-2494-001.patch, OOZIE-2494-002.patch, OOZIE-2494-003.patch, > OOZIE-2494-004.patch, OOZIE-2494-005.patch, OOZIE-2494-006.patch, > OOZIE-2494-007.patch, OOZIE-2494-008.patch, OOZIE-2494-009.patch, > testActionMaterWithDST3.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: el-LA > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 00:10 GMT 2016-03-13 17:10 PDT > {noformat} > As you can see, @2’s nominal time is adjusted to an hour earlier in the > "el-LA" job, but not in the "el-UTC" job. > However, when running a similar set of jobs, but using cron syntax [({{10 1 > 1/1 * *}}|http://crontab.guru/#10_1_1/1_*_*], which indicates 1:10 every day > of every month), this isn’t the case: > {noformat} > Job Name: cron-UTC > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: cron-LA > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > As you can see, @2’s nominal time are the same in both the "cron-UTC" and > "cron-LA" jobs. The "cron-LA" job should have the same nominal time as the > "el-LA" job from earlier. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16460877#comment-16460877 ] Hadoop QA commented on OOZIE-2494: -- Testing JIRA OOZIE-2494 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:red}-1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:red}-1{color} the patch contains 11 line(s) longer than 132 characters .{color:green}+1{color} the patch adds/modifies 1 testcase(s) {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} the patch does not seem to introduce new Javadoc warnings {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:green}+1{color} There are no new bugs found in total. . {color:green}+1{color} There are no new bugs found in [webapp]. . {color:green}+1{color} There are no new bugs found in [core]. . {color:green}+1{color} There are no new bugs found in [tools]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive2]. . {color:green}+1{color} There are no new bugs found in [sharelib/distcp]. . {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. . {color:green}+1{color} There are no new bugs found in [sharelib/streaming]. . {color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. . {color:green}+1{color} There are no new bugs found in [sharelib/oozie]. . {color:green}+1{color} There are no new bugs found in [sharelib/pig]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive]. . {color:green}+1{color} There are no new bugs found in [sharelib/spark]. . {color:green}+1{color} There are no new bugs found in [client]. . {color:green}+1{color} There are no new bugs found in [examples]. . {color:green}+1{color} There are no new bugs found in [docs]. . {color:green}+1{color} There are no new bugs found in [server]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:red}-1 TESTS{color} .Tests run: 2125 .Tests failed: 1 .Tests errors: 0 .The patch failed the following testcases: testHiveAction(org.apache.oozie.action.hadoop.TestHiveActionExecutor) .Tests failing with errors: .{color:orange}Tests failed at first run:{color} TestJavaActionExecutor#testCredentialsSkip TestCoordActionsKillXCommand#testActionKillCommandActionNumbers TestOozieDBCLI#testOozieDBCLI .For the complete list of flaky tests, see TEST-SUMMARY-FULL files. {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:red}*-1 Overall result, please check the reported -1(s)*{color} The full output of the test-patch run is available at . https://builds.apache.org/job/PreCommit-OOZIE-Build/500/ > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0, 5.0.0 >Reporter: Dennis Pallett >Assignee: Julia Kinga Marton >Priority: Blocker > Attachments: CronExpressionPOC.java, OOZIE-2494-001.patch, > OOZIE-2494-002.patch, OOZIE-2494-003.patch, OOZIE-2494-004.patch, > OOZIE-2494-005.patch, OOZIE-2494-006.patch, OOZIE-2494-007.patch, > OOZIE-2494-008.patch, OOZIE-2494-009.patch, testActionMaterWithDST3.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT >
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16460765#comment-16460765 ] Julia Kinga Marton commented on OOZIE-2494: --- The failed tests are unrelated. The long lines are from the twiki file. The javac warnings are valid. I will correct them. > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0, 5.0.0 >Reporter: Dennis Pallett >Assignee: Julia Kinga Marton >Priority: Blocker > Attachments: CronExpressionPOC.java, OOZIE-2494-001.patch, > OOZIE-2494-002.patch, OOZIE-2494-003.patch, OOZIE-2494-004.patch, > OOZIE-2494-005.patch, OOZIE-2494-006.patch, OOZIE-2494-007.patch, > OOZIE-2494-008.patch, OOZIE-2494-009.patch, testActionMaterWithDST3.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: el-LA > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 00:10 GMT 2016-03-13 17:10 PDT > {noformat} > As you can see, @2’s nominal time is adjusted to an hour earlier in the > "el-LA" job, but not in the "el-UTC" job. > However, when running a similar set of jobs, but using cron syntax [({{10 1 > 1/1 * *}}|http://crontab.guru/#10_1_1/1_*_*], which indicates 1:10 every day > of every month), this isn’t the case: > {noformat} > Job Name: cron-UTC > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: cron-LA > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > As you can see, @2’s nominal time are the same in both the "cron-UTC" and > "cron-LA" jobs. The "cron-LA" job should have the same nominal time as the > "el-LA" job from earlier. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16460757#comment-16460757 ] Hadoop QA commented on OOZIE-2494: -- Testing JIRA OOZIE-2494 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:red}-1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:red}-1{color} the patch contains 11 line(s) longer than 132 characters .{color:green}+1{color} the patch adds/modifies 1 testcase(s) {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} the patch does not seem to introduce new Javadoc warnings {color:red}-1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:red}-1{color} the patch seems to introduce 8 new javac warning(s) {color:green}+1{color} There are no new bugs found in total. . {color:green}+1{color} There are no new bugs found in [docs]. . {color:green}+1{color} There are no new bugs found in [sharelib/pig]. . {color:green}+1{color} There are no new bugs found in [sharelib/oozie]. . {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive2]. . {color:green}+1{color} There are no new bugs found in [sharelib/distcp]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive]. . {color:green}+1{color} There are no new bugs found in [sharelib/spark]. . {color:green}+1{color} There are no new bugs found in [sharelib/streaming]. . {color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. . {color:green}+1{color} There are no new bugs found in [webapp]. . {color:green}+1{color} There are no new bugs found in [examples]. . {color:green}+1{color} There are no new bugs found in [client]. . {color:green}+1{color} There are no new bugs found in [tools]. . {color:green}+1{color} There are no new bugs found in [server]. . {color:green}+1{color} There are no new bugs found in [core]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:green}+1 TESTS{color} .Tests run: 2125 .{color:orange}Tests failed at first run:{color} TestJavaActionExecutor#testCredentialsSkip TestHiveActionExecutor#testHiveAction TestOozieDBCLI#testOozieDBCLI .For the complete list of flaky tests, see TEST-SUMMARY-FULL files. {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:red}*-1 Overall result, please check the reported -1(s)*{color} The full output of the test-patch run is available at . https://builds.apache.org/job/PreCommit-OOZIE-Build/499/ > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0, 5.0.0 >Reporter: Dennis Pallett >Assignee: Julia Kinga Marton >Priority: Blocker > Attachments: CronExpressionPOC.java, OOZIE-2494-001.patch, > OOZIE-2494-002.patch, OOZIE-2494-003.patch, OOZIE-2494-004.patch, > OOZIE-2494-005.patch, OOZIE-2494-006.patch, OOZIE-2494-007.patch, > OOZIE-2494-008.patch, OOZIE-2494-009.patch, testActionMaterWithDST3.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: el-LA > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 20
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16460756#comment-16460756 ] Hadoop QA commented on OOZIE-2494: -- PreCommit-OOZIE-Build started > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0, 5.0.0 >Reporter: Dennis Pallett >Assignee: Julia Kinga Marton >Priority: Blocker > Attachments: CronExpressionPOC.java, OOZIE-2494-001.patch, > OOZIE-2494-002.patch, OOZIE-2494-003.patch, OOZIE-2494-004.patch, > OOZIE-2494-005.patch, OOZIE-2494-006.patch, OOZIE-2494-007.patch, > OOZIE-2494-008.patch, OOZIE-2494-009.patch, testActionMaterWithDST3.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: el-LA > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 00:10 GMT 2016-03-13 17:10 PDT > {noformat} > As you can see, @2’s nominal time is adjusted to an hour earlier in the > "el-LA" job, but not in the "el-UTC" job. > However, when running a similar set of jobs, but using cron syntax [({{10 1 > 1/1 * *}}|http://crontab.guru/#10_1_1/1_*_*], which indicates 1:10 every day > of every month), this isn’t the case: > {noformat} > Job Name: cron-UTC > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: cron-LA > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > As you can see, @2’s nominal time are the same in both the "cron-UTC" and > "cron-LA" jobs. The "cron-LA" job should have the same nominal time as the > "el-LA" job from earlier. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16460745#comment-16460745 ] Andras Piros commented on OOZIE-2494: - Thanks for the contribution [~kmarton]! +1 (pending Jenkins) > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0, 5.0.0 >Reporter: Dennis Pallett >Assignee: Julia Kinga Marton >Priority: Blocker > Attachments: CronExpressionPOC.java, OOZIE-2494-001.patch, > OOZIE-2494-002.patch, OOZIE-2494-003.patch, OOZIE-2494-004.patch, > OOZIE-2494-005.patch, OOZIE-2494-006.patch, OOZIE-2494-007.patch, > OOZIE-2494-008.patch, OOZIE-2494-009.patch, testActionMaterWithDST3.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: el-LA > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 00:10 GMT 2016-03-13 17:10 PDT > {noformat} > As you can see, @2’s nominal time is adjusted to an hour earlier in the > "el-LA" job, but not in the "el-UTC" job. > However, when running a similar set of jobs, but using cron syntax [({{10 1 > 1/1 * *}}|http://crontab.guru/#10_1_1/1_*_*], which indicates 1:10 every day > of every month), this isn’t the case: > {noformat} > Job Name: cron-UTC > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: cron-LA > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > As you can see, @2’s nominal time are the same in both the "cron-UTC" and > "cron-LA" jobs. The "cron-LA" job should have the same nominal time as the > "el-LA" job from earlier. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16460628#comment-16460628 ] Hadoop QA commented on OOZIE-2494: -- PreCommit-OOZIE-Build started > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0 >Reporter: Dennis Pallett >Assignee: Julia Kinga Marton >Priority: Blocker > Attachments: CronExpressionPOC.java, OOZIE-2494-001.patch, > OOZIE-2494-002.patch, OOZIE-2494-003.patch, OOZIE-2494-004.patch, > OOZIE-2494-005.patch, OOZIE-2494-006.patch, OOZIE-2494-007.patch, > OOZIE-2494-008.patch, testActionMaterWithDST3.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: el-LA > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 00:10 GMT 2016-03-13 17:10 PDT > {noformat} > As you can see, @2’s nominal time is adjusted to an hour earlier in the > "el-LA" job, but not in the "el-UTC" job. > However, when running a similar set of jobs, but using cron syntax [({{10 1 > 1/1 * *}}|http://crontab.guru/#10_1_1/1_*_*], which indicates 1:10 every day > of every month), this isn’t the case: > {noformat} > Job Name: cron-UTC > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: cron-LA > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > As you can see, @2’s nominal time are the same in both the "cron-UTC" and > "cron-LA" jobs. The "cron-LA" job should have the same nominal time as the > "el-LA" job from earlier. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16440562#comment-16440562 ] Hadoop QA commented on OOZIE-2494: -- Testing JIRA OOZIE-2494 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:green}+1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:green}+1{color} the patch does not introduce any line longer than 132 .{color:green}+1{color} the patch adds/modifies 1 testcase(s) {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} the patch does not seem to introduce new Javadoc warnings {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:green}+1{color} There are no new bugs found in total. . {color:green}+1{color} There are no new bugs found in [examples]. . {color:green}+1{color} There are no new bugs found in [webapp]. . {color:green}+1{color} There are no new bugs found in [core]. . {color:green}+1{color} There are no new bugs found in [tools]. . {color:green}+1{color} There are no new bugs found in [server]. . {color:green}+1{color} There are no new bugs found in [docs]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive2]. . {color:green}+1{color} There are no new bugs found in [sharelib/pig]. . {color:green}+1{color} There are no new bugs found in [sharelib/streaming]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive]. . {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. . {color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. . {color:green}+1{color} There are no new bugs found in [sharelib/oozie]. . {color:green}+1{color} There are no new bugs found in [sharelib/distcp]. . {color:green}+1{color} There are no new bugs found in [sharelib/spark]. . {color:green}+1{color} There are no new bugs found in [client]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:red}-1 TESTS{color} .Tests run: 2115 .Tests failed: 1 .Tests errors: 1 .The patch failed the following testcases: testCoordActionRecoveryServiceForWaitingRegisterPartition(org.apache.oozie.service.TestRecoveryService) .Tests failing with errors: testConnectionRetry(org.apache.oozie.service.TestJMSAccessorService) .{color:orange}Tests failed at first run:{color} TestJavaActionExecutor#testCredentialsSkip TestOozieDBCLI#testOozieDBCLI .For the complete list of flaky tests, see TEST-SUMMARY-FULL files. {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:red}*-1 Overall result, please check the reported -1(s)*{color} The full output of the test-patch run is available at . https://builds.apache.org/job/PreCommit-OOZIE-Build/486/ > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0 >Reporter: Dennis Pallett >Assignee: Julia Kinga Marton >Priority: Blocker > Attachments: CronExpressionPOC.java, OOZIE-2494-001.patch, > OOZIE-2494-002.patch, OOZIE-2494-003.patch, OOZIE-2494-004.patch, > OOZIE-2494-005.patch, OOZIE-2494-006.patch, OOZIE-2494-007.patch, > testActionMaterWithDST3.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} >
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16440471#comment-16440471 ] Hadoop QA commented on OOZIE-2494: -- PreCommit-OOZIE-Build started > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0 >Reporter: Dennis Pallett >Assignee: Julia Kinga Marton >Priority: Blocker > Attachments: CronExpressionPOC.java, OOZIE-2494-001.patch, > OOZIE-2494-002.patch, OOZIE-2494-003.patch, OOZIE-2494-004.patch, > OOZIE-2494-005.patch, OOZIE-2494-006.patch, OOZIE-2494-007.patch, > testActionMaterWithDST3.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: el-LA > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 00:10 GMT 2016-03-13 17:10 PDT > {noformat} > As you can see, @2’s nominal time is adjusted to an hour earlier in the > "el-LA" job, but not in the "el-UTC" job. > However, when running a similar set of jobs, but using cron syntax [({{10 1 > 1/1 * *}}|http://crontab.guru/#10_1_1/1_*_*], which indicates 1:10 every day > of every month), this isn’t the case: > {noformat} > Job Name: cron-UTC > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: cron-LA > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > As you can see, @2’s nominal time are the same in both the "cron-UTC" and > "cron-LA" jobs. The "cron-LA" job should have the same nominal time as the > "el-LA" job from earlier. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16439337#comment-16439337 ] Hadoop QA commented on OOZIE-2494: -- Testing JIRA OOZIE-2494 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:green}+1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:green}+1{color} the patch does not introduce any line longer than 132 .{color:green}+1{color} the patch adds/modifies 1 testcase(s) {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} the patch does not seem to introduce new Javadoc warnings {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:green}+1{color} There are no new bugs found in total. . {color:green}+1{color} There are no new bugs found in [examples]. . {color:green}+1{color} There are no new bugs found in [webapp]. . {color:green}+1{color} There are no new bugs found in [core]. . {color:green}+1{color} There are no new bugs found in [tools]. . {color:green}+1{color} There are no new bugs found in [server]. . {color:green}+1{color} There are no new bugs found in [docs]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive2]. . {color:green}+1{color} There are no new bugs found in [sharelib/pig]. . {color:green}+1{color} There are no new bugs found in [sharelib/streaming]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive]. . {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. . {color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. . {color:green}+1{color} There are no new bugs found in [sharelib/oozie]. . {color:green}+1{color} There are no new bugs found in [sharelib/distcp]. . {color:green}+1{color} There are no new bugs found in [sharelib/spark]. . {color:green}+1{color} There are no new bugs found in [client]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:red}-1 TESTS{color} .Tests run: 2115 .Tests failed: 1 .Tests errors: 1 .The patch failed the following testcases: testCoordActionRecoveryServiceForWaitingRegisterPartition(org.apache.oozie.service.TestRecoveryService) .Tests failing with errors: testConnectionRetry(org.apache.oozie.service.TestJMSAccessorService) .{color:orange}Tests failed at first run:{color} TestJavaActionExecutor#testCredentialsSkip TestOozieDBCLI#testOozieDBCLI .For the complete list of flaky tests, see TEST-SUMMARY-FULL files. {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:red}*-1 Overall result, please check the reported -1(s)*{color} The full output of the test-patch run is available at . https://builds.apache.org/job/PreCommit-OOZIE-Build/484/ > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0 >Reporter: Dennis Pallett >Assignee: Julia Kinga Marton >Priority: Blocker > Attachments: CronExpressionPOC.java, OOZIE-2494-001.patch, > OOZIE-2494-002.patch, OOZIE-2494-003.patch, OOZIE-2494-004.patch, > OOZIE-2494-005.patch, OOZIE-2494-006.patch, testActionMaterWithDST3.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16439252#comment-16439252 ] Julia Kinga Marton commented on OOZIE-2494: --- I have corrected the long line. The test failures are not related to the patch. However they are passing locally. > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0 >Reporter: Dennis Pallett >Assignee: Julia Kinga Marton >Priority: Blocker > Attachments: CronExpressionPOC.java, OOZIE-2494-001.patch, > OOZIE-2494-002.patch, OOZIE-2494-003.patch, OOZIE-2494-004.patch, > OOZIE-2494-005.patch, OOZIE-2494-006.patch, testActionMaterWithDST3.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: el-LA > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 00:10 GMT 2016-03-13 17:10 PDT > {noformat} > As you can see, @2’s nominal time is adjusted to an hour earlier in the > "el-LA" job, but not in the "el-UTC" job. > However, when running a similar set of jobs, but using cron syntax [({{10 1 > 1/1 * *}}|http://crontab.guru/#10_1_1/1_*_*], which indicates 1:10 every day > of every month), this isn’t the case: > {noformat} > Job Name: cron-UTC > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: cron-LA > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > As you can see, @2’s nominal time are the same in both the "cron-UTC" and > "cron-LA" jobs. The "cron-LA" job should have the same nominal time as the > "el-LA" job from earlier. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16439253#comment-16439253 ] Hadoop QA commented on OOZIE-2494: -- PreCommit-OOZIE-Build started > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0 >Reporter: Dennis Pallett >Assignee: Julia Kinga Marton >Priority: Blocker > Attachments: CronExpressionPOC.java, OOZIE-2494-001.patch, > OOZIE-2494-002.patch, OOZIE-2494-003.patch, OOZIE-2494-004.patch, > OOZIE-2494-005.patch, OOZIE-2494-006.patch, testActionMaterWithDST3.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: el-LA > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 00:10 GMT 2016-03-13 17:10 PDT > {noformat} > As you can see, @2’s nominal time is adjusted to an hour earlier in the > "el-LA" job, but not in the "el-UTC" job. > However, when running a similar set of jobs, but using cron syntax [({{10 1 > 1/1 * *}}|http://crontab.guru/#10_1_1/1_*_*], which indicates 1:10 every day > of every month), this isn’t the case: > {noformat} > Job Name: cron-UTC > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: cron-LA > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > As you can see, @2’s nominal time are the same in both the "cron-UTC" and > "cron-LA" jobs. The "cron-LA" job should have the same nominal time as the > "el-LA" job from earlier. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16437415#comment-16437415 ] Hadoop QA commented on OOZIE-2494: -- Testing JIRA OOZIE-2494 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:red}-1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:red}-1{color} the patch contains 1 line(s) longer than 132 characters .{color:green}+1{color} the patch adds/modifies 1 testcase(s) {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} the patch does not seem to introduce new Javadoc warnings {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:green}+1{color} There are no new bugs found in total. . {color:green}+1{color} There are no new bugs found in [docs]. . {color:green}+1{color} There are no new bugs found in [examples]. . {color:green}+1{color} There are no new bugs found in [tools]. . {color:green}+1{color} There are no new bugs found in [core]. . {color:green}+1{color} There are no new bugs found in [server]. . {color:green}+1{color} There are no new bugs found in [client]. . {color:green}+1{color} There are no new bugs found in [sharelib/distcp]. . {color:green}+1{color} There are no new bugs found in [sharelib/oozie]. . {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. . {color:green}+1{color} There are no new bugs found in [sharelib/streaming]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive]. . {color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. . {color:green}+1{color} There are no new bugs found in [sharelib/spark]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive2]. . {color:green}+1{color} There are no new bugs found in [sharelib/pig]. . {color:green}+1{color} There are no new bugs found in [webapp]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:red}-1 TESTS{color} .Tests run: 2115 .Tests failed: 17 .Tests errors: 12 .The patch failed the following testcases: testPartitionDependency(org.apache.oozie.service.TestPartitionDependencyManagerEhcache) testCoordActionRecoveryServiceForWaitingRegisterPartition(org.apache.oozie.service.TestRecoveryService) testCoordinatorActionSelectors(org.apache.oozie.jms.TestJMSJobEventListener) testCoordinatorActionSelectorsNegative(org.apache.oozie.jms.TestJMSJobEventListener) testWorkflowJobSelectors(org.apache.oozie.jms.TestJMSJobEventListener) testOnCoordinatorJobSuccessEvent(org.apache.oozie.jms.TestJMSJobEventListener) testOnWorkflowJobSuspendEvent(org.apache.oozie.jms.TestJMSJobEventListener) testWorkflowJobSelectorsAnd(org.apache.oozie.jms.TestJMSJobEventListener) testOnCoordinatorJobFailureEvent(org.apache.oozie.jms.TestJMSJobEventListener) testConnectionDrop(org.apache.oozie.jms.TestJMSJobEventListener) testWorkflowJobSelectorsNegative(org.apache.oozie.jms.TestJMSJobEventListener) testWorkflowJobSelectorsOr(org.apache.oozie.jms.TestJMSJobEventListener) testOnWorkflowJobSuccessEvent(org.apache.oozie.jms.TestJMSJobEventListener) testOnCoordinatorActionWaitingEvent(org.apache.oozie.jms.TestJMSJobEventListener) testOnWorkflowJobFailureEvent(org.apache.oozie.jms.TestJMSJobEventListener) testOnCoordinatorActionStartEvent(org.apache.oozie.jms.TestJMSJobEventListener) testOnWorkflowJobStartedEvent(org.apache.oozie.jms.TestJMSJobEventListener) .Tests failing with errors: testDropEventTypeMessage(org.apache.oozie.jms.TestHCatMessageHandler) testCacheUpdateByMessage(org.apache.oozie.jms.TestHCatMessageHandler) testOnSLADurationMetEvent(org.apache.oozie.jms.TestJMSSLAEventListener) testOnSLAStartMetEvent(org.apache.oozie.jms.TestJMSSLAEventListener) testSLAJobSelectorsNegative(org.apache.oozie.jms.TestJMSSLAEventListener) testOnSLAEndMetEvent(org.apache.oozie.jms.TestJMSSLAEventListener) testOnSLAStartMissEvent(org.apache.oozie.jms.TestJMSSLAEventListener) testSLAJobSelectors(org.apache.oozie.jms.TestJMSSLAEventListener) testOnSLAEndMissEvent(org.apache.oozie.jms.TestJMSSLAEventListener) testOnSLADurationMissEvent(org.apache.oozie.jms.TestJMSSLAEventListener) testConnectionRetry(org.apache.oozie.service.TestJMSAccessorService) testThreadLocalSession(org.apache.oozie.jms.TestDefaultConnectionContext)
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16437307#comment-16437307 ] Hadoop QA commented on OOZIE-2494: -- PreCommit-OOZIE-Build started > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0 >Reporter: Dennis Pallett >Assignee: Julia Kinga Marton >Priority: Blocker > Attachments: CronExpressionPOC.java, OOZIE-2494-001.patch, > OOZIE-2494-002.patch, OOZIE-2494-003.patch, OOZIE-2494-004.patch, > OOZIE-2494-005.patch, testActionMaterWithDST3.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: el-LA > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 00:10 GMT 2016-03-13 17:10 PDT > {noformat} > As you can see, @2’s nominal time is adjusted to an hour earlier in the > "el-LA" job, but not in the "el-UTC" job. > However, when running a similar set of jobs, but using cron syntax [({{10 1 > 1/1 * *}}|http://crontab.guru/#10_1_1/1_*_*], which indicates 1:10 every day > of every month), this isn’t the case: > {noformat} > Job Name: cron-UTC > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: cron-LA > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > As you can see, @2’s nominal time are the same in both the "cron-UTC" and > "cron-LA" jobs. The "cron-LA" job should have the same nominal time as the > "el-LA" job from earlier. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16433998#comment-16433998 ] Julia Kinga Marton commented on OOZIE-2494: --- Thank you [~andras.piros], I have created a patch with the corrections for the cron case, in order to have the same behaviour as in case of EL functions. After this changes will be accepted, I am planing to update the documentation as well, with some DST related examples. > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0 >Reporter: Dennis Pallett >Assignee: Julia Kinga Marton >Priority: Blocker > Attachments: CronExpressionPOC.java, OOZIE-2494-001.patch, > OOZIE-2494-002.patch, OOZIE-2494-003.patch, OOZIE-2494-004.patch, > OOZIE-2494-005.patch, testActionMaterWithDST3.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: el-LA > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 00:10 GMT 2016-03-13 17:10 PDT > {noformat} > As you can see, @2’s nominal time is adjusted to an hour earlier in the > "el-LA" job, but not in the "el-UTC" job. > However, when running a similar set of jobs, but using cron syntax [({{10 1 > 1/1 * *}}|http://crontab.guru/#10_1_1/1_*_*], which indicates 1:10 every day > of every month), this isn’t the case: > {noformat} > Job Name: cron-UTC > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: cron-LA > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > As you can see, @2’s nominal time are the same in both the "cron-UTC" and > "cron-LA" jobs. The "cron-LA" job should have the same nominal time as the > "el-LA" job from earlier. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16433564#comment-16433564 ] Andras Piros commented on OOZIE-2494: - [~kmarton] thanks for the heads up! I'd suggest following approach: * only address Cron like syntax (so no EL DST corrections in this JIRA like {{$\{coord:hours\(n\)\}}} or {{$\{coord:minutes\(n\)\}}}). If you like you can open another JIRA for that one. To me it seems the community is fairly satisfied w/ the solution applied there so far * for Cron syntax of any kind: ** for daily {{frequency}}, go with the suggestion of the JIRA description having the same functionality as EL {{$\{coord:days\(1\)\}}} ** for hourly {{frequency}}, have the same functionality as EL {{$\{coord:hours\(1\)\}}} ** for {{frequency}} by the minute, have the same functionality as EL {{$\{coord:minutes\(1\)\}}} > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0 >Reporter: Dennis Pallett >Assignee: Julia Kinga Marton >Priority: Blocker > Attachments: CronExpressionPOC.java, OOZIE-2494-001.patch, > OOZIE-2494-002.patch, OOZIE-2494-003.patch, OOZIE-2494-004.patch, > testActionMaterWithDST3.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: el-LA > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 00:10 GMT 2016-03-13 17:10 PDT > {noformat} > As you can see, @2’s nominal time is adjusted to an hour earlier in the > "el-LA" job, but not in the "el-UTC" job. > However, when running a similar set of jobs, but using cron syntax [({{10 1 > 1/1 * *}}|http://crontab.guru/#10_1_1/1_*_*], which indicates 1:10 every day > of every month), this isn’t the case: > {noformat} > Job Name: cron-UTC > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: cron-LA > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > As you can see, @2’s nominal time are the same in both the "cron-UTC" and > "cron-LA" jobs. The "cron-LA" job should have the same nominal time as the > "el-LA" job from earlier. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16433546#comment-16433546 ] Julia Kinga Marton commented on OOZIE-2494: --- The problem from this issue is about the DST changes with a daily frequency. But what about the higher frequencies? For example hourly ones? If we apply DST correction I would apply it in every case, but in case of hourly frequencies or even higher ones, we might have some troubles. For example in case of an hourly frequency what is the expected behaviour? To have an action for let`s say 1 am, 2 am, 3am in local time (eq Los Angeles), which means 8am, 10am, 11am in GMT, so an hour will be skipped when DST ends, or doubled when a DST starts, or in this case is accepted if in local time 2am will be skipped or doubled? Currently in case of EL DST corrections are not applied in case of {code:java} hours(n) or minutes(n).{code} > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0 >Reporter: Dennis Pallett >Assignee: Julia Kinga Marton >Priority: Blocker > Attachments: CronExpressionPOC.java, OOZIE-2494-001.patch, > OOZIE-2494-002.patch, OOZIE-2494-003.patch, OOZIE-2494-004.patch, > testActionMaterWithDST3.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: el-LA > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 00:10 GMT 2016-03-13 17:10 PDT > {noformat} > As you can see, @2’s nominal time is adjusted to an hour earlier in the > "el-LA" job, but not in the "el-UTC" job. > However, when running a similar set of jobs, but using cron syntax [({{10 1 > 1/1 * *}}|http://crontab.guru/#10_1_1/1_*_*], which indicates 1:10 every day > of every month), this isn’t the case: > {noformat} > Job Name: cron-UTC > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: cron-LA > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > As you can see, @2’s nominal time are the same in both the "cron-UTC" and > "cron-LA" jobs. The "cron-LA" job should have the same nominal time as the > "el-LA" job from earlier. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16428169#comment-16428169 ] Julia Kinga Marton commented on OOZIE-2494: --- I have checked the coordinator functional specification and if the user don't want any DST shifting, he/she has the option to not fill the timezone or to use UTC in the xml. As [~rkanter] and [~jphelps] suggested, I will make here the DST adjustment for the nominal time and I will create a separate issue for the improvement. > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0 >Reporter: Dennis Pallett >Assignee: Julia Kinga Marton >Priority: Blocker > Attachments: CronExpressionPOC.java, OOZIE-2494-001.patch, > OOZIE-2494-002.patch, OOZIE-2494-003.patch, OOZIE-2494-004.patch, > testActionMaterWithDST3.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: el-LA > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 00:10 GMT 2016-03-13 17:10 PDT > {noformat} > As you can see, @2’s nominal time is adjusted to an hour earlier in the > "el-LA" job, but not in the "el-UTC" job. > However, when running a similar set of jobs, but using cron syntax [({{10 1 > 1/1 * *}}|http://crontab.guru/#10_1_1/1_*_*], which indicates 1:10 every day > of every month), this isn’t the case: > {noformat} > Job Name: cron-UTC > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: cron-LA > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > As you can see, @2’s nominal time are the same in both the "cron-UTC" and > "cron-LA" jobs. The "cron-LA" job should have the same nominal time as the > "el-LA" job from earlier. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427449#comment-16427449 ] Peter Bacsko commented on OOZIE-2494: - I think the problem is twofold. First we have the issue described in this JIRA is that whevever there is a DST change, it's not reflected on the materialized time. On the other hand, properly following DST changes can also introduce weird side-effects: if we have a WF that runs at 2AM every day, then once a year it will be skipped during the 1:59:59 --> 3:00:00 transition and will run twice during the 2:59:59 --> 2:00:00 transition. For some people this will be normal, but I believe most customers wouldn't even consider this scenario when they set up their coordinators, so we have to be careful when we modify the semantics of WF execution. > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0 >Reporter: Dennis Pallett >Assignee: Julia Kinga Marton >Priority: Blocker > Attachments: CronExpressionPOC.java, OOZIE-2494-001.patch, > OOZIE-2494-002.patch, OOZIE-2494-003.patch, OOZIE-2494-004.patch, > testActionMaterWithDST3.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: el-LA > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 00:10 GMT 2016-03-13 17:10 PDT > {noformat} > As you can see, @2’s nominal time is adjusted to an hour earlier in the > "el-LA" job, but not in the "el-UTC" job. > However, when running a similar set of jobs, but using cron syntax [({{10 1 > 1/1 * *}}|http://crontab.guru/#10_1_1/1_*_*], which indicates 1:10 every day > of every month), this isn’t the case: > {noformat} > Job Name: cron-UTC > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: cron-LA > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > As you can see, @2’s nominal time are the same in both the "cron-UTC" and > "cron-LA" jobs. The "cron-LA" job should have the same nominal time as the > "el-LA" job from earlier. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427423#comment-16427423 ] Jason Phelps commented on OOZIE-2494: - [~kmarton], [~andras.piros] >From my point of view, I have not seen any customers who relied on, or would >prefer the current setup where the materialized time shifts one hour either >way when DST starts or stops. At the very least, most customers just live with >it since that is how it works, while several have not expected that the >materializing time would shift by an hour. I agree with [~rkanter] that for this JIRA, it should fix the current issue where it skips certain jobs (I.e. a daily job scheduled between 1 AM and 2 AM will get skipped when DST ends), with a new feature to add a configurable timezone in a separate JIRA I would +1 an option to allow a configurable timezone ( based on normal JAVA timezones, i.e. {{America/Los_Angeles) }}that would match the behavior of the EL function {{0 3 * * *}} would remain 3AM in that timezone regardless of whether DST shifts or not) with the default behavior being the existing UTC. > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0 >Reporter: Dennis Pallett >Assignee: Julia Kinga Marton >Priority: Blocker > Attachments: CronExpressionPOC.java, OOZIE-2494-001.patch, > OOZIE-2494-002.patch, OOZIE-2494-003.patch, OOZIE-2494-004.patch, > testActionMaterWithDST3.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: el-LA > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 00:10 GMT 2016-03-13 17:10 PDT > {noformat} > As you can see, @2’s nominal time is adjusted to an hour earlier in the > "el-LA" job, but not in the "el-UTC" job. > However, when running a similar set of jobs, but using cron syntax [({{10 1 > 1/1 * *}}|http://crontab.guru/#10_1_1/1_*_*], which indicates 1:10 every day > of every month), this isn’t the case: > {noformat} > Job Name: cron-UTC > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: cron-LA > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > As you can see, @2’s nominal time are the same in both the "cron-UTC" and > "cron-LA" jobs. The "cron-LA" job should have the same nominal time as the > "el-LA" job from earlier. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16426656#comment-16426656 ] Julia Kinga Marton commented on OOZIE-2494: --- For me is not very clear what is the expected behaviour? For example if there is a coordinator with an hourly frequency and we have a DST switch: * having a correction and having each year 2 actions with other time difference than 60 mins (once with 0 mins and once with 120 mins), but having it at the same hour in the defined timezone (how actually the EL functions works) *or* * not applying any correction and having the actions with exactly 60 mins difference, but having some shift in the local time (how actually the cron expression works). I can imagine that there are users who prefer the first option, and there are others who would prefer the second one. Whichever option we will decide for, I think it would be good to have the same output for the EL expression and cron syntax usage. > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0 >Reporter: Dennis Pallett >Assignee: Julia Kinga Marton >Priority: Blocker > Attachments: CronExpressionPOC.java, OOZIE-2494-001.patch, > OOZIE-2494-002.patch, OOZIE-2494-003.patch, OOZIE-2494-004.patch, > testActionMaterWithDST3.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: el-LA > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 00:10 GMT 2016-03-13 17:10 PDT > {noformat} > As you can see, @2’s nominal time is adjusted to an hour earlier in the > "el-LA" job, but not in the "el-UTC" job. > However, when running a similar set of jobs, but using cron syntax [({{10 1 > 1/1 * *}}|http://crontab.guru/#10_1_1/1_*_*], which indicates 1:10 every day > of every month), this isn’t the case: > {noformat} > Job Name: cron-UTC > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: cron-LA > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > As you can see, @2’s nominal time are the same in both the "cron-UTC" and > "cron-LA" jobs. The "cron-LA" job should have the same nominal time as the > "el-LA" job from earlier. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16426618#comment-16426618 ] Andras Piros commented on OOZIE-2494: - [~rkanter] a fair point to maintain backwards compatibility. Do you think users who rely on current bug in their coordinators (cron syntax not honoring DST) won't get disappointed if we just fix it? > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0 >Reporter: Dennis Pallett >Assignee: Julia Kinga Marton >Priority: Blocker > Attachments: CronExpressionPOC.java, OOZIE-2494-001.patch, > OOZIE-2494-002.patch, OOZIE-2494-003.patch, OOZIE-2494-004.patch, > testActionMaterWithDST3.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: el-LA > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 00:10 GMT 2016-03-13 17:10 PDT > {noformat} > As you can see, @2’s nominal time is adjusted to an hour earlier in the > "el-LA" job, but not in the "el-UTC" job. > However, when running a similar set of jobs, but using cron syntax [({{10 1 > 1/1 * *}}|http://crontab.guru/#10_1_1/1_*_*], which indicates 1:10 every day > of every month), this isn’t the case: > {noformat} > Job Name: cron-UTC > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: cron-LA > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > As you can see, @2’s nominal time are the same in both the "cron-UTC" and > "cron-LA" jobs. The "cron-LA" job should have the same nominal time as the > "el-LA" job from earlier. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16426163#comment-16426163 ] Robert Kanter commented on OOZIE-2494: -- I'm fine with adding a way to tell Oozie to actually use a different timezone if we can do that correctly (and I'm sure a lot of users would like this), but it's also important to remain consistent so we don't break anything. So I'd suggest that we fix the original issue here (i.e. the cron syntax not handling DST as expected) and then we can file a "New Feature" Jira to add a way to use other timezones. > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0 >Reporter: Dennis Pallett >Assignee: Julia Kinga Marton >Priority: Blocker > Attachments: CronExpressionPOC.java, OOZIE-2494-001.patch, > OOZIE-2494-002.patch, OOZIE-2494-003.patch, OOZIE-2494-004.patch, > testActionMaterWithDST3.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: el-LA > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 00:10 GMT 2016-03-13 17:10 PDT > {noformat} > As you can see, @2’s nominal time is adjusted to an hour earlier in the > "el-LA" job, but not in the "el-UTC" job. > However, when running a similar set of jobs, but using cron syntax [({{10 1 > 1/1 * *}}|http://crontab.guru/#10_1_1/1_*_*], which indicates 1:10 every day > of every month), this isn’t the case: > {noformat} > Job Name: cron-UTC > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: cron-LA > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > As you can see, @2’s nominal time are the same in both the "cron-UTC" and > "cron-LA" jobs. The "cron-LA" job should have the same nominal time as the > "el-LA" job from earlier. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16425387#comment-16425387 ] Julia Kinga Marton commented on OOZIE-2494: --- One approach for DST handling would be something implemented in OOZIE-2726, but with including a DST corrections, even if the timezone is ignored, is not so elegant and it may lead to further misunderstandings. A more elegant solution (proposed earlier by [~pbacsko]) would be to allow the user to choose if he want to use the default processing timezone (how is it working now), or if he would like to use another timezone., by adding an extra element in the xml. I would go for this xml extension option, but I am curious about other opinions as well. [~rkanter] what is your opinion about this approach? > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0 >Reporter: Dennis Pallett >Assignee: Julia Kinga Marton >Priority: Blocker > Attachments: CronExpressionPOC.java, OOZIE-2494-001.patch, > OOZIE-2494-002.patch, OOZIE-2494-003.patch, OOZIE-2494-004.patch, > testActionMaterWithDST3.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: el-LA > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 00:10 GMT 2016-03-13 17:10 PDT > {noformat} > As you can see, @2’s nominal time is adjusted to an hour earlier in the > "el-LA" job, but not in the "el-UTC" job. > However, when running a similar set of jobs, but using cron syntax [({{10 1 > 1/1 * *}}|http://crontab.guru/#10_1_1/1_*_*], which indicates 1:10 every day > of every month), this isn’t the case: > {noformat} > Job Name: cron-UTC > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: cron-LA > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > As you can see, @2’s nominal time are the same in both the "cron-UTC" and > "cron-LA" jobs. The "cron-LA" job should have the same nominal time as the > "el-LA" job from earlier. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16425359#comment-16425359 ] Julia Kinga Marton commented on OOZIE-2494: --- With the fix of OOZIE-2726 a small functionality change is included for the following scenario: * between the start and end date there is one DST start * the nominal time of the last action should be maximum 1h before the end time. * this last action will not be created after the OOZIE-2726, because of the DST correction, the nominal time will exceed the end time. (is almost the same with the opposite scenario, when +1 action is cerated) > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0 >Reporter: Dennis Pallett >Assignee: Julia Kinga Marton >Priority: Blocker > Attachments: CronExpressionPOC.java, OOZIE-2494-001.patch, > OOZIE-2494-002.patch, OOZIE-2494-003.patch, OOZIE-2494-004.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: el-LA > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 00:10 GMT 2016-03-13 17:10 PDT > {noformat} > As you can see, @2’s nominal time is adjusted to an hour earlier in the > "el-LA" job, but not in the "el-UTC" job. > However, when running a similar set of jobs, but using cron syntax [({{10 1 > 1/1 * *}}|http://crontab.guru/#10_1_1/1_*_*], which indicates 1:10 every day > of every month), this isn’t the case: > {noformat} > Job Name: cron-UTC > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: cron-LA > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > As you can see, @2’s nominal time are the same in both the "cron-UTC" and > "cron-LA" jobs. The "cron-LA" job should have the same nominal time as the > "el-LA" job from earlier. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15327907#comment-15327907 ] Hadoop QA commented on OOZIE-2494: -- Testing JIRA OOZIE-2494 Cleaning local git workspace {color:red}-1{color} Patch failed to apply to head of branch > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0 >Reporter: Dennis Pallett >Assignee: Peter Bacsko >Priority: Blocker > Attachments: CronExpressionPOC.java, OOZIE-2494-001.patch, > OOZIE-2494-002.patch, OOZIE-2494-003.patch, OOZIE-2494-004.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: el-LA > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 00:10 GMT 2016-03-13 17:10 PDT > {noformat} > As you can see, @2’s nominal time is adjusted to an hour earlier in the > "el-LA" job, but not in the "el-UTC" job. > However, when running a similar set of jobs, but using cron syntax [({{10 1 > 1/1 * *}}|http://crontab.guru/#10_1_1/1_*_*], which indicates 1:10 every day > of every month), this isn’t the case: > {noformat} > Job Name: cron-UTC > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: cron-LA > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > As you can see, @2’s nominal time are the same in both the "cron-UTC" and > "cron-LA" jobs. The "cron-LA" job should have the same nominal time as the > "el-LA" job from earlier. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15327881#comment-15327881 ] Peter Bacsko commented on OOZIE-2494: - Please ignore the provided patches, they are not functionally correct! I attached a POC instead which demonstrates a more proper approach. In theory it handles all cases well. > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0 >Reporter: Dennis Pallett >Assignee: Peter Bacsko >Priority: Blocker > Attachments: CronExpressionPOC.java, OOZIE-2494-001.patch, > OOZIE-2494-002.patch, OOZIE-2494-003.patch, OOZIE-2494-004.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: el-LA > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 00:10 GMT 2016-03-13 17:10 PDT > {noformat} > As you can see, @2’s nominal time is adjusted to an hour earlier in the > "el-LA" job, but not in the "el-UTC" job. > However, when running a similar set of jobs, but using cron syntax [({{10 1 > 1/1 * *}}|http://crontab.guru/#10_1_1/1_*_*], which indicates 1:10 every day > of every month), this isn’t the case: > {noformat} > Job Name: cron-UTC > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: cron-LA > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > As you can see, @2’s nominal time are the same in both the "cron-UTC" and > "cron-LA" jobs. The "cron-LA" job should have the same nominal time as the > "el-LA" job from earlier. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15299308#comment-15299308 ] Hadoop QA commented on OOZIE-2494: -- Testing JIRA OOZIE-2494 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:green}+1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:green}+1{color} the patch does not introduce any line longer than 132 .{color:green}+1{color} the patch does adds/modifies 1 testcase(s) {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} the patch does not seem to introduce new Javadoc warnings {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:green}+1 TESTS{color} .Tests run: 1781 {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:green}*+1 Overall result, good!, no -1s*{color} The full output of the test-patch run is available at . https://builds.apache.org/job/oozie-trunk-precommit-build/2900/ > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0 >Reporter: Dennis Pallett >Assignee: Peter Bacsko >Priority: Blocker > Attachments: OOZIE-2494-001.patch, OOZIE-2494-002.patch, > OOZIE-2494-003.patch, OOZIE-2494-004.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: el-LA > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 00:10 GMT 2016-03-13 17:10 PDT > {noformat} > As you can see, @2’s nominal time is adjusted to an hour earlier in the > "el-LA" job, but not in the "el-UTC" job. > However, when running a similar set of jobs, but using cron syntax [({{10 1 > 1/1 * *}}|http://crontab.guru/#10_1_1/1_*_*], which indicates 1:10 every day > of every month), this isn’t the case: > {noformat} > Job Name: cron-UTC > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: cron-LA > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > As you can see, @2’s nominal time are the same in both the "cron-UTC" and > "cron-LA" jobs. The "cron-LA" job should have the same nominal time as the > "el-LA" job from earlier. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15297743#comment-15297743 ] Hadoop QA commented on OOZIE-2494: -- Testing JIRA OOZIE-2494 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:red}-1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:green}+1{color} the patch does not introduce any line longer than 132 .{color:red}-1{color} the patch does not add/modify any testcase {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} the patch does not seem to introduce new Javadoc warnings {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:red}-1 TESTS{color} .Tests run: 1780 .Tests failed: 1 .Tests errors: 0 .The patch failed the following testcases: . testIDGeneration(org.apache.oozie.service.TestZKUUIDService) {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:red}*-1 Overall result, please check the reported -1(s)*{color} The full output of the test-patch run is available at . https://builds.apache.org/job/oozie-trunk-precommit-build/2897/ > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0 >Reporter: Dennis Pallett >Assignee: Peter Bacsko >Priority: Blocker > Attachments: OOZIE-2494-001.patch, OOZIE-2494-002.patch, > OOZIE-2494-003.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: el-LA > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 00:10 GMT 2016-03-13 17:10 PDT > {noformat} > As you can see, @2’s nominal time is adjusted to an hour earlier in the > "el-LA" job, but not in the "el-UTC" job. > However, when running a similar set of jobs, but using cron syntax [({{10 1 > 1/1 * *}}|http://crontab.guru/#10_1_1/1_*_*], which indicates 1:10 every day > of every month), this isn’t the case: > {noformat} > Job Name: cron-UTC > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: cron-LA > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > As you can see, @2’s nominal time are the same in both the "cron-UTC" and > "cron-LA" jobs. The "cron-LA" job should have the same nominal time as the > "el-LA" job from earlier. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15297625#comment-15297625 ] Hadoop QA commented on OOZIE-2494: -- Testing JIRA OOZIE-2494 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:red}-1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:red}-1{color} the patch contains 3 line(s) with trailing spaces .{color:green}+1{color} the patch does not introduce any line longer than 132 .{color:red}-1{color} the patch does not add/modify any testcase {color:red}-1 RAT{color} .{color:red}-1{color} the patch seems to introduce 1 new RAT warning(s) {color:green}+1 JAVADOC{color} .{color:green}+1{color} the patch does not seem to introduce new Javadoc warnings {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:red}-1 TESTS{color} .Tests run: 1780 .Tests failed: 1 .Tests errors: 0 .The patch failed the following testcases: . testKill(org.apache.oozie.command.coord.TestAbandonedCoordChecker) {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:red}*-1 Overall result, please check the reported -1(s)*{color} The full output of the test-patch run is available at . https://builds.apache.org/job/oozie-trunk-precommit-build/2896/ > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0 >Reporter: Dennis Pallett >Assignee: Peter Bacsko >Priority: Blocker > Attachments: OOZIE-2494-001.patch, OOZIE-2494-002.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: el-LA > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 00:10 GMT 2016-03-13 17:10 PDT > {noformat} > As you can see, @2’s nominal time is adjusted to an hour earlier in the > "el-LA" job, but not in the "el-UTC" job. > However, when running a similar set of jobs, but using cron syntax [({{10 1 > 1/1 * *}}|http://crontab.guru/#10_1_1/1_*_*], which indicates 1:10 every day > of every month), this isn’t the case: > {noformat} > Job Name: cron-UTC > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: cron-LA > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > As you can see, @2’s nominal time are the same in both the "cron-UTC" and > "cron-LA" jobs. The "cron-LA" job should have the same nominal time as the > "el-LA" job from earlier. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294605#comment-15294605 ] Hadoop QA commented on OOZIE-2494: -- Testing JIRA OOZIE-2494 Cleaning local git workspace {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:green}+1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:green}+1{color} the patch does not introduce any line longer than 132 .{color:green}+1{color} the patch does adds/modifies 1 testcase(s) {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} the patch does not seem to introduce new Javadoc warnings {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:green}+1 BACKWARDS_COMPATIBILITY{color} .{color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations .{color:green}+1{color} the patch does not modify JPA files {color:red}-1 TESTS{color} .Tests run: 1780 .Tests failed: 4 .Tests errors: 1 .The patch failed the following testcases: . testActionMaterWithDST1(org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand) . testActionMaterWithDST2(org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand) . testMaxMatThrottleNotPicked(org.apache.oozie.service.TestCoordMaterializeTriggerService) . testBasicSubmitWithCronFrequency(org.apache.oozie.command.coord.TestCoordSubmitXCommand) {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:red}*-1 Overall result, please check the reported -1(s)*{color} The full output of the test-patch run is available at . https://builds.apache.org/job/oozie-trunk-precommit-build/2892/ > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0 >Reporter: Dennis Pallett >Assignee: Peter Bacsko >Priority: Blocker > Attachments: OOZIE-2494-001.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: el-LA > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 00:10 GMT 2016-03-13 17:10 PDT > {noformat} > As you can see, @2’s nominal time is adjusted to an hour earlier in the > "el-LA" job, but not in the "el-UTC" job. > However, when running a similar set of jobs, but using cron syntax [({{10 1 > 1/1 * *}}|http://crontab.guru/#10_1_1/1_*_*], which indicates 1:10 every day > of every month), this isn’t the case: > {noformat} > Job Name: cron-UTC > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: cron-LA > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > As you can see, @2’s nominal time are the same in both the "cron-UTC" an
[jira] [Commented] (OOZIE-2494) Cron syntax not handling DST properly
[ https://issues.apache.org/jira/browse/OOZIE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294361#comment-15294361 ] Peter Bacsko commented on OOZIE-2494: - Attached first version of the patch. Some notes: 1) I talked with Robert. The way Oozie works is the following: timestamp is always interpreted in UTC (so-called "Oozie processing time zone"). Timezone is only used for determining proper DST. 2) CronExpression has a setTimeZone() method. However I didn't get it work because it produced some funky results that I didn't understand so I chose a different approach. 3) After calculating nextTime, we check if we went through a DST shift in either direction. If we did, then we either add or subtract the DST offset (which is supposed to be 1h in for all zones, but we determine it dynamically just to be safe). To do: 1) Tests with edge cases 2) Some Java docs > Cron syntax not handling DST properly > - > > Key: OOZIE-2494 > URL: https://issues.apache.org/jira/browse/OOZIE-2494 > Project: Oozie > Issue Type: Bug > Components: coordinator >Affects Versions: 4.2.0 >Reporter: Dennis Pallett >Assignee: Peter Bacsko >Priority: Blocker > Attachments: OOZIE-2494-001.patch > > > When specifying a coordinator frequency, you can also specify a "timezone". > While the frequency is always calculated in UTC, the timezone’s DST rules are > still applied. We can see this in the following two Coordinators, which ran > across the DST shift (March 13 2016 at 2am) for the America/Los_Angeles > timezone. The "el-UTC" job has "UTC" as the timezone, while the "el-LA" job > has “America/Los_Angeles” as the timezone. Both jobs have a frequency of > {{$\{coord:days(1)\}}}. > {noformat} > Job Name: el-UTC > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: el-LA > Start Time : 2016-03-13 01:10 GMT | 2016-03-12 17:10 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 00:10 GMT 2016-03-13 17:10 PDT > {noformat} > As you can see, @2’s nominal time is adjusted to an hour earlier in the > "el-LA" job, but not in the "el-UTC" job. > However, when running a similar set of jobs, but using cron syntax [({{10 1 > 1/1 * *}}|http://crontab.guru/#10_1_1/1_*_*], which indicates 1:10 every day > of every month), this isn’t the case: > {noformat} > Job Name: cron-UTC > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > {noformat} > Job Name: cron-LA > Start Time : 2016-03-13 01:08 GMT | 2016-03-12 17:08 PST > - > ID Nominal Time (UTC) Nominal Time (LA) > @1 2016-03-13 01:10 GMT 2016-03-12 17:10 PST > @2 2016-03-14 01:10 GMT 2016-03-13 18:10 PDT > {noformat} > As you can see, @2’s nominal time are the same in both the "cron-UTC" and > "cron-LA" jobs. The "cron-LA" job should have the same nominal time as the > "el-LA" job from earlier. -- This message was sent by Atlassian JIRA (v6.3.4#6332)