date:20181114

[jira] [Commented] (OOZIE-3380) TestCoordMaterializeTransitionXCommand failure after DST change date

2018-11-14 Thread Julia Kinga Marton (JIRA)



[ 
https://issues.apache.org/jira/browse/OOZIE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686383#comment-16686383
 ] 

Julia Kinga Marton commented on OOZIE-3380:
---

Thank you [~asalamon74] for the contribution. 

+1, committed to master

> TestCoordMaterializeTransitionXCommand failure after DST change date
> 
>
> Key: OOZIE-3380
> URL: https://issues.apache.org/jira/browse/OOZIE-3380
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3380-01.patch, OOZIE-3380-02.patch
>
>
> TestCoordMaterializeTransitionXCommand.testMaterializationLookup failed for 
> OOZIE-3377 and OOZIE-3378. It also fails for the trunk:
> {noformat}
> junit.framework.AssertionFailedError: 
> Expected :Mon Nov 05 17:21:58 CET 2018
> Actual   :Sun Nov 04 17:21:58 CET 2018
>  
>   at junit.framework.Assert.fail(Assert.java:57)
>   at junit.framework.Assert.failNotEquals(Assert.java:329)
>   at junit.framework.Assert.assertEquals(Assert.java:78)
>   at junit.framework.Assert.assertEquals(Assert.java:86)
>   at junit.framework.TestCase.assertEquals(TestCase.java:253)
>   at 
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.testMaterializationLookup(TestCoordMaterializeTransitionXCommand.java:691)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at junit.framework.TestCase.runTest(TestCase.java:176)
>   at junit.framework.TestCase.runBare(TestCase.java:141)
>   at junit.framework.TestResult$1.protect(TestResult.java:122)
>   at junit.framework.TestResult.runProtected(TestResult.java:142)
>   at junit.framework.TestResult.run(TestResult.java:125)
>   at junit.framework.TestCase.run(TestCase.java:129)
>   at junit.framework.TestSuite.runTest(TestSuite.java:255)
>   at junit.framework.TestSuite.run(TestSuite.java:250)
>   at 
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
>   at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
>   at 
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
>   at 
> com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
>   at 
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
>   at 
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
> {noformat}
> This test uses the following dates for testing:
> {noformat}
> startTime = new Date(new Date().getTime() - TIME_IN_DAY * 3);
> endTime = new Date(startTime.getTime() + TIME_IN_DAY * 3);   
> Date next = new Date(startTime.getTime() + TIME_IN_DAY * 3);
> {noformat}
> start time is before the DST change date, end time is after the DST change 
> date. If I shift the interval by two days (so start and end are both after 
> the DST change date) the test works correctly.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3380) TestCoordMaterializeTransitionXCommand failure after DST change date

2018-11-14 Thread Julia Kinga Marton (JIRA)



[ 
https://issues.apache.org/jira/browse/OOZIE-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686360#comment-16686360
 ] 

Julia Kinga Marton commented on OOZIE-3380:
---

I have created a follow-up issue (OOZIE-3383) for the TimeUnit related issue, 
so this issue can be considered done.

> TestCoordMaterializeTransitionXCommand failure after DST change date
> 
>
> Key: OOZIE-3380
> URL: https://issues.apache.org/jira/browse/OOZIE-3380
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3380-01.patch, OOZIE-3380-02.patch
>
>
> TestCoordMaterializeTransitionXCommand.testMaterializationLookup failed for 
> OOZIE-3377 and OOZIE-3378. It also fails for the trunk:
> {noformat}
> junit.framework.AssertionFailedError: 
> Expected :Mon Nov 05 17:21:58 CET 2018
> Actual   :Sun Nov 04 17:21:58 CET 2018
>  
>   at junit.framework.Assert.fail(Assert.java:57)
>   at junit.framework.Assert.failNotEquals(Assert.java:329)
>   at junit.framework.Assert.assertEquals(Assert.java:78)
>   at junit.framework.Assert.assertEquals(Assert.java:86)
>   at junit.framework.TestCase.assertEquals(TestCase.java:253)
>   at 
> org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand.testMaterializationLookup(TestCoordMaterializeTransitionXCommand.java:691)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at junit.framework.TestCase.runTest(TestCase.java:176)
>   at junit.framework.TestCase.runBare(TestCase.java:141)
>   at junit.framework.TestResult$1.protect(TestResult.java:122)
>   at junit.framework.TestResult.runProtected(TestResult.java:142)
>   at junit.framework.TestResult.run(TestResult.java:125)
>   at junit.framework.TestCase.run(TestCase.java:129)
>   at junit.framework.TestSuite.runTest(TestSuite.java:255)
>   at junit.framework.TestSuite.run(TestSuite.java:250)
>   at 
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
>   at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
>   at 
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
>   at 
> com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
>   at 
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
>   at 
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
> {noformat}
> This test uses the following dates for testing:
> {noformat}
> startTime = new Date(new Date().getTime() - TIME_IN_DAY * 3);
> endTime = new Date(startTime.getTime() + TIME_IN_DAY * 3);   
> Date next = new Date(startTime.getTime() + TIME_IN_DAY * 3);
> {noformat}
> start time is before the DST change date, end time is after the DST change 
> date. If I shift the interval by two days (so start and end are both after 
> the DST change date) the test works correctly.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (OOZIE-3383) Refactor TimeUnit usage

2018-11-14 Thread Julia Kinga Marton (JIRA)

Julia Kinga Marton created OOZIE-3383:
-

 Summary: Refactor TimeUnit usage
 Key: OOZIE-3383
 URL: https://issues.apache.org/jira/browse/OOZIE-3383
 Project: Oozie
  Issue Type: Improvement
Affects Versions: trunk
Reporter: Julia Kinga Marton


In Oozie we have four TimeUnit (two of them are called Timeunit) classes.

Check if they can be replaced by  
*[java.util.concurrent.TimeUnit|https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/TimeUnit.html]*
 and do some code cleanup around this TimeUnit usage.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [VOTE] Release Oozie 5.1.0 (candidate 1)

2018-11-14 Thread Andras Piros

Thanks Gp,

extending the vote by two weeks (Fri Nov 23, 12pm PDT) to see what others
find.

Andras

On Tue, Nov 13, 2018 at 2:01 PM Peter Cseh 
wrote:

> Hi!
>
> Tested with Hadoop2.7.2
> - signature, checksum is fine
> - rat report is ok
> - distro builds
> - tried some of the examples (MR, Hive, Spark), they are running fine.
>
> +1
>
>
> On Tue, Nov 13, 2018 at 12:59 PM Andras Piros
>  wrote:
>
> > Thanks Kinga for checking out RC1, and good to hear it passed from your
> > perspective.
> >
> > Regards,
> >
> > Andras
> >
> > On Tue, Nov 13, 2018 at 12:36 PM Julia Kinga Marton
> >  wrote:
> >
> > > Hi,
> > >
> > > I made some testing on the RC, and it seems OK.
> > >
> > > Here is what I checked:
> > > - sha512
> > > - gpg signature
> > > - rat report
> > > - built Oozie against Hadoop-2.6.0
> > > - ran a few examples (wf and coordinator), clicked the Web UI, ran some
> > CLI
> > > commands
> > > - checked Fluent Job API by running Java and Shell examples
> > > - looked briefly at the documentation
> > >
> > > +1 from my side (non-binding)
> > >
> > > Best regards,
> > > Kinga
> > >
> > > On Mon, Nov 5, 2018 at 8:44 PM Andras Piros  wrote:
> > >
> > > > Hi there,
> > > >
> > > > I have created a build for Oozie 5.1.0, release candidate 1.
> > > >
> > > > This release introduces following major changes and fixes:
> > > > - Fluent Job API
> > > > - Git action
> > > > - reduced heap waste
> > > > - switched documentation from TWiki to Markdown
> > > > - callable queue improvements
> > > > - Spark action fixes
> > > > - SLA improvements
> > > > - third party dependency revamp
> > > > - fixed CVE-2018-11799
> > > >
> > > > Keys to verify the signature of the release artifact are available at
> > > >
> > > >  http://www.apache.org/dist/oozie/KEYS
> > > >
> > > > Please download, test, and try it out:
> > > >
> > > > http://people.apache.org/~andras/oozie-5.1.0-rc1/
> > > >
> > > > The release candidate, sha512 signature, gpg signature, and rat
> report
> > > can
> > > > all be found
> > > > at the above address.
> > > >
> > > > The vote is open until Friday, November 9, 2018 at 12:00am PDT.
> > > >
> > > > Thanks,
> > > >
> > > > Andras Piros
> > > >
> > >
> >
>
>
> --
> *Peter Cseh *| Software Engineer
> cloudera.com 
>
> [image: Cloudera] 
>
> [image: Cloudera on Twitter]  [image:
> Cloudera on Facebook]  [image: Cloudera
> on LinkedIn] 
> --
>

[jira] [Updated] (OOZIE-3384) [tests] TestWorkflowActionRetryInfoXCommand#testRetryConsoleUrlForked() is flaky

2018-11-14 Thread Andras Piros (JIRA)



 [ 
https://issues.apache.org/jira/browse/OOZIE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros updated OOZIE-3384:

Component/s: tests

> [tests] TestWorkflowActionRetryInfoXCommand#testRetryConsoleUrlForked() is 
> flaky
> 
>
> Key: OOZIE-3384
> URL: https://issues.apache.org/jira/browse/OOZIE-3384
> Project: Oozie
>  Issue Type: Sub-task
>  Components: tests
>Reporter: Andras Piros
>Priority: Major
>
> {code:java}
> junit.framework.AssertionFailedError: Expected :2 Actual :1  difference> at junit.framework.Assert.fail(Assert.java:57) at
> ...
> org.apache.oozie.command.wf.TestWorkflowActionRetryInfoXCommand.validateRetryConsoleUrl(TestWorkflowActionRetryInfoXCommand.java:172)
>  at 
> org.apache.oozie.command.wf.TestWorkflowActionRetryInfoXCommand.testRetryConsoleUrlForked(TestWorkflowActionRetryInfoXCommand.java:125)
>  at
>  ...
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (OOZIE-3384) [tests] TestWorkflowActionRetryInfoXCommand#testRetryConsoleUrlForked() is flaky

2018-11-14 Thread Andras Piros (JIRA)

Andras Piros created OOZIE-3384:
---

 Summary: [tests] 
TestWorkflowActionRetryInfoXCommand#testRetryConsoleUrlForked() is flaky
 Key: OOZIE-3384
 URL: https://issues.apache.org/jira/browse/OOZIE-3384
 Project: Oozie
  Issue Type: Sub-task
Reporter: Andras Piros


{code:java}
junit.framework.AssertionFailedError: Expected :2 Actual :1  at junit.framework.Assert.fail(Assert.java:57) at
...
org.apache.oozie.command.wf.TestWorkflowActionRetryInfoXCommand.validateRetryConsoleUrl(TestWorkflowActionRetryInfoXCommand.java:172)
 at 
org.apache.oozie.command.wf.TestWorkflowActionRetryInfoXCommand.testRetryConsoleUrlForked(TestWorkflowActionRetryInfoXCommand.java:125)
 at
 ...
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3381) [coordinator] Enhance logging of CoordElFunctions

2018-11-14 Thread Andras Piros (JIRA)



[ 
https://issues.apache.org/jira/browse/OOZIE-3381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686507#comment-16686507
 ] 

Andras Piros commented on OOZIE-3381:
-

Test failure is unrelated and caused by OOZIE-3384. [~kmarton] can you please 
review the patch? Thanks!

> [coordinator] Enhance logging of CoordElFunctions
> -
>
> Key: OOZIE-3381
> URL: https://issues.apache.org/jira/browse/OOZIE-3381
> Project: Oozie
>  Issue Type: Task
>  Components: coordinator
>Affects Versions: 5.1.0
>Reporter: Andras Piros
>Assignee: Andras Piros
>Priority: Major
> Attachments: OOZIE-3381.001.patch
>
>
> When using [{{$\{coord:latest\(n\)}}} coordinator EL 
> function|https://oozie.apache.org/docs/5.0.0/CoordinatorFunctionalSpec.html#a6.6.6._coord:latestint_n_EL_Function_for_Synchronous_Datasets]
>  inside an input dataset dependency, it's often the case that more 
> information is needed how many HDFS URIs are being checked for each 
> {{}}.
> Right now we don't have this information. While debugging and fine tuning 
> parameters like {{dataset frequency}}, {{initial-instance}}, and {{data-in}} 
> {{instance}}, it would be very useful to know how many HDFS roundtrips are 
> issues by the current settings {{CoordELFunctions#coord_latestRange_sync()}} 
> and {{CoordELFunctions#coord_futureRange_sync()}} having called 
> {{DFSClient#exists()}}. We need appropriate logging there.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] Subscription: Oozie Patch Available

2018-11-14 Thread jira

Issue Subscription
Filter: Oozie Patch Available (96 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-3381  [coordinator] Enhance logging of CoordElFunctions
https://issues.apache.org/jira/browse/OOZIE-3381
OOZIE-3379  Auth token cache file name should include Oozie URL
https://issues.apache.org/jira/browse/OOZIE-3379
OOZIE-3375  Can't use empty  in coordinator
https://issues.apache.org/jira/browse/OOZIE-3375
OOZIE-3367  Using && in EL expressions in oozie bundle.xml files generates 
parse errors
https://issues.apache.org/jira/browse/OOZIE-3367
OOZIE-3366  Update workflow status and subworkflow status on suspend command
https://issues.apache.org/jira/browse/OOZIE-3366
OOZIE-3364  Rerunning Oozie bundle jobs starts the coordinators in 
indeterminate order
https://issues.apache.org/jira/browse/OOZIE-3364
OOZIE-3362  When killed, SSH action should kill the spawned processes on target 
host
https://issues.apache.org/jira/browse/OOZIE-3362
OOZIE-3326  Sqoop Action should support tez delegation tokens for hive-import
https://issues.apache.org/jira/browse/OOZIE-3326
OOZIE-3320  Oozie ShellAction should support absolute bash file path
https://issues.apache.org/jira/browse/OOZIE-3320
OOZIE-3319  Log SSH action callback error output
https://issues.apache.org/jira/browse/OOZIE-3319
OOZIE-3301  Update NOTICE file
https://issues.apache.org/jira/browse/OOZIE-3301
OOZIE-3274  Remove slf4j
https://issues.apache.org/jira/browse/OOZIE-3274
OOZIE-3266  Coord action rerun support RERUN_SKIP_NODES option
https://issues.apache.org/jira/browse/OOZIE-3266
OOZIE-3265  properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to 
appear together
https://issues.apache.org/jira/browse/OOZIE-3265
OOZIE-3256  refactor OozieCLI class
https://issues.apache.org/jira/browse/OOZIE-3256
OOZIE-3249  [tools] Instrumentation log parser
https://issues.apache.org/jira/browse/OOZIE-3249
OOZIE-3218  Oozie Sqoop action with command splits the select clause into 
multiple parts due to delimiter being space
https://issues.apache.org/jira/browse/OOZIE-3218
OOZIE-3199  Let system property restriction configurable
https://issues.apache.org/jira/browse/OOZIE-3199
OOZIE-3196  Authorization: restrict world readability by user
https://issues.apache.org/jira/browse/OOZIE-3196
OOZIE-3194  Oozie should set proper permissions to sharelib after upload
https://issues.apache.org/jira/browse/OOZIE-3194
OOZIE-3186  Oozie is unable to use configuration linked using jceks://file/...
https://issues.apache.org/jira/browse/OOZIE-3186
OOZIE-3179  Adding a configurable config-default.xml location to a workflow
https://issues.apache.org/jira/browse/OOZIE-3179
OOZIE-3170  Oozie Diagnostic Bundle tool fails with NPE due to missing service 
class
https://issues.apache.org/jira/browse/OOZIE-3170
OOZIE-3137  Add support for log4j2 in HiveMain
https://issues.apache.org/jira/browse/OOZIE-3137
OOZIE-3135  Configure log4j2 in SqoopMain
https://issues.apache.org/jira/browse/OOZIE-3135
OOZIE-3120  maven-assembly-plugin fails when bumped from 2.2.1
https://issues.apache.org/jira/browse/OOZIE-3120
OOZIE-3091  Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: 
org/apache/avro/mapred/AvroWrapper"
https://issues.apache.org/jira/browse/OOZIE-3091
OOZIE-3071  Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 
than Spark 2.2.0
https://issues.apache.org/jira/browse/OOZIE-3071
OOZIE-3063  Sanitizing variables that are part of openjpa.ConnectionProperties
https://issues.apache.org/jira/browse/OOZIE-3063
OOZIE-3062  Set HADOOP_CONF_DIR for spark action
https://issues.apache.org/jira/browse/OOZIE-3062
OOZIE-2952  Fix Findbugs warnings in oozie-sharelib-oozie
https://issues.apache.org/jira/browse/OOZIE-2952
OOZIE-2949  Escape quotes whitespaces in Sqoop  field
https://issues.apache.org/jira/browse/OOZIE-2949
OOZIE-2927  Append new line character for Hive2 query using query tag
https://issues.apache.org/jira/browse/OOZIE-2927
OOZIE-2834  ParameterVerifier logging non-useful warning for workflow definition
https://issues.apache.org/jira/browse/OOZIE-2834
OOZIE-2833  when using uber mode the regex pattern used in the 
extractHeapSizeMB method does not allow heap sizes specified in bytes.
https://issues.apache.org/jira/browse/OOZIE-2833
OOZIE-2812  SparkConfigurationService should support loading configurations 
from multiple Spark versions
https://issues.apache.org/jira/browse/OOZIE-2812
OOZIE-2795  Create lib directory or symlink for Oozie CLI during packaging
https://issues.apache.org/jira/browse/OOZIE-2795

[jira] [Commented] (OOZIE-3380) TestCoordMaterializeTransitionXCommand failure after DST change date

[jira] [Commented] (OOZIE-3380) TestCoordMaterializeTransitionXCommand failure after DST change date

[jira] [Created] (OOZIE-3383) Refactor TimeUnit usage

Re: [VOTE] Release Oozie 5.1.0 (candidate 1)

[jira] [Updated] (OOZIE-3384) [tests] TestWorkflowActionRetryInfoXCommand#testRetryConsoleUrlForked() is flaky

[jira] [Created] (OOZIE-3384) [tests] TestWorkflowActionRetryInfoXCommand#testRetryConsoleUrlForked() is flaky

[jira] [Commented] (OOZIE-3381) [coordinator] Enhance logging of CoordElFunctions

[jira] Subscription: Oozie Patch Available

8 matches

Site Navigation

Mail list logo

Footer information